LiteLLM supports OpenAI text completion models
Required API Keys
import os
os.environ["OPENAI_API_KEY"]="your-api-key"
Usage
import os
from litellm import completion
os.environ["OPENAI_API_KEY"]="your-api-key"
# openai call
response = completion(
model ="gpt-3.5-turbo-instruct",
messages=[{"content":"Hello, how are you?","role":"user"}]
)
Usage - LiteLLM Proxy Server
Here's how to call OpenAI models with the LiteLLM Proxy Server
1. Save key in your environment
export OPENAI_API_KEY=""
2. Start the proxy
- config.yaml
- config.yaml - proxy all OpenAI models
- CLI
model_list:
-model_name: gpt-3.5-turbo
litellm_params:
model: openai/gpt-3.5-turbo # The `openai/` prefix will call openai.chat.completions.create
api_key: os.environ/OPENAI_API_KEY
-model_name: gpt-3.5-turbo-instruct
litellm_params:
model: text-completion-openai/gpt-3.5-turbo-instruct # The `text-completion-openai/` prefix will call openai.completions.create
api_key: os.environ/OPENAI_API_KEY
Use this to add all openai models with one API Key. WARNING: This will not do any load balancing
This means requests to gpt-4, gpt-3.5-turbo , gpt-4-turbo-preview will all go through this route
model_list:
-model_name:"*"# all requests where model not in your config go to this deployment
litellm_params:
model: openai/* # set `openai/` to use the openai route
api_key: os.environ/OPENAI_API_KEY
$ litellm --model gpt-3.5-turbo-instruct
# Server running on http://0.0.0.0:4000
3. Test it
- Curl Request
- OpenAI v1.0.0+
- Langchain
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-3.5-turbo-instruct",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'
import openai
client = openai.OpenAI(
api_key="anything",
base_url="http://0.0.0.0:4000"
)
# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(model="gpt-3.5-turbo-instruct", messages =[
{
"role":"user",
"content":"this is a test request, write a short poem"
}
])
print(response)
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import(
ChatPromptTemplate,
HumanMessagePromptTemplate,
SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage
chat = ChatOpenAI(
openai_api_base="http://0.0.0.0:4000",# set openai_api_base to the LiteLLM Proxy
model ="gpt-3.5-turbo-instruct",
temperature=0.1
)
messages =[
SystemMessage(
content="You are a helpful assistant that im using to make a test request to."
),
HumanMessage(
content="test from litellm. tell me why it's amazing in 1 sentence"
),
]
response = chat(messages)
print(response)
OpenAI Text Completion Models / Instruct Models
| Model Name | Function Call |
|---|---|
| gpt-3.5-turbo-instruct | response = completion(model="gpt-3.5-turbo-instruct", messages=messages) |
| gpt-3.5-turbo-instruct-0914 | response = completion(model="gpt-3.5-turbo-instruct-0914", messages=messages) |
| text-davinci-003 | response = completion(model="text-davinci-003", messages=messages) |
| ada-001 | response = completion(model="ada-001", messages=messages) |
| curie-001 | response = completion(model="curie-001", messages=messages) |
| babbage-001 | response = completion(model="babbage-001", messages=messages) |
| babbage-002 | response = completion(model="babbage-002", messages=messages) |
| davinci-002 | response = completion(model="davinci-002", messages=messages) |