VOOZH about

URL: https://docs.litellm.ai/docs/providers/volcano

⇱ Volcano Engine (Volcengine) | liteLLM

Skip to main content

https://www.volcengine.com/docs/82379/1263482

tip

We support ALL Volcengine models including Chat and Embeddings, just set model=volcengine/<any-model-on-volcengine> as a prefix when sending litellm requests

API Key​

# env variable
os.environ['VOLCENGINE_API_KEY']
# or
os.environ['ARK_API_KEY']

Sample Usage​

from litellm import completion
import os

os.environ['VOLCENGINE_API_KEY']=""
response = completion(
 model="volcengine/<OUR_ENDPOINT_ID>",
 messages=[
{
"role":"user",
"content":"What's the weather like in Boston today in Fahrenheit?",
}
],
 temperature=0.2,# optional
 top_p=0.9,# optional
 frequency_penalty=0.1,# optional
 presence_penalty=0.1,# optional
 max_tokens=10,# optional
 stop=["\n\n"],# optional
)
print(response)

Sample Usage - Streaming​

from litellm import completion
import os

os.environ['VOLCENGINE_API_KEY']=""
response = completion(
 model="volcengine/<OUR_ENDPOINT_ID>",
 messages=[
{
"role":"user",
"content":"What's the weather like in Boston today in Fahrenheit?",
}
],
 stream=True,
 temperature=0.2,# optional
 top_p=0.9,# optional
 frequency_penalty=0.1,# optional
 presence_penalty=0.1,# optional
 max_tokens=10,# optional
 stop=["\n\n"],# optional
)

for chunk in response:
print(chunk)

Sample Usage - Embedding​

from litellm import embedding
import os

os.environ['VOLCENGINE_API_KEY']=""
response = embedding(
 model="volcengine/doubao-embedding-text-240715",
input=["hello world","good morning"]
)
print(response)

Supported Embedding Models​

  • doubao-embedding-large (2048 dimensions)
  • doubao-embedding-large-text-250515 (2048 dimensions)
  • doubao-embedding-large-text-240915 (4096 dimensions)
  • doubao-embedding (2560 dimensions)
  • doubao-embedding-text-240715 (2560 dimensions)

Embedding Parameters​

from litellm import embedding

response = embedding(
 model="volcengine/doubao-embedding-text-240715",
input=["sample text"],
 encoding_format="float",# optional: "float" (default), "base64"
 user="user-123",# optional: user identifier for tracking
)

Supported Models - 💥 ALL Volcengine Models Supported!​

We support ALL volcengine models for both chat completions and embeddings:

  • Chat Models: Set volcengine/<OUR_ENDPOINT_ID> as a prefix when sending completion requests
  • Embedding Models: Use the specific model names listed above (e.g., volcengine/doubao-embedding-text-240715)

Sample Usage - LiteLLM Proxy​

Config.yaml setting​

model_list:
# Chat model
-model_name: volcengine-model
litellm_params:
model: volcengine/<OUR_ENDPOINT_ID>
api_key: os.environ/VOLCENGINE_API_KEY
# Embedding model
-model_name: volcengine-embedding
litellm_params:
model: volcengine/doubao-embedding-text-240715
api_key: os.environ/VOLCENGINE_API_KEY

Send Request​

Chat Completion​

curl --location 'http://localhost:4000/chat/completions' \
 --header 'Authorization: Bearer sk-1234' \
 --header 'Content-Type: application/json' \
 --data '{
 "model": "volcengine-model",
 "messages": [
 {
 "role": "user",
 "content": "here is my api key. openai_api_key=sk-1234"
 }
 ]
}'

Embedding​

curl --location 'http://localhost:4000/embeddings' \
 --header 'Authorization: Bearer sk-1234' \
 --header 'Content-Type: application/json' \
 --data '{
 "model": "volcengine-embedding",
 "input": ["hello world", "good morning"]
}'