litellm 1.89.2

LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.

Jump to LiteLLM Proxy (LLM Gateway) Docs
Jump to Supported LLM Providers

Why LiteLLM

Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:

Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
Drop-in OpenAI compatibility — swap providers without rewriting your code
Production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box
8ms P95 latency at 1k RPS (benchmarks)

OSS Adopters

Netflix

👁 OpenAI Agents SDK

Features

Supported Providers (Website Supported Models | Docs)

Provider	`/chat/completions`	`/messages`	`/responses`	`/embeddings`	`/image/generations`	`/audio/transcriptions`	`/audio/speech`	`/moderations`	`/batches`
Abliteration (`abliteration`)	✅
AI/ML API (`aiml`)	✅	✅	✅	✅	✅
AI21 (`ai21`)	✅	✅	✅
AI21 Chat (`ai21_chat`)	✅	✅	✅
Aleph Alpha	✅	✅	✅
Amazon Nova	✅	✅	✅
Anthropic (`anthropic`)	✅	✅	✅	✅
Anthropic Text (`anthropic_text`)	✅	✅	✅	✅
Anyscale	✅	✅	✅
AssemblyAI (`assemblyai`)	✅	✅	✅	✅
Auto Router (`auto_router`)	✅	✅	✅
AWS - Bedrock (`bedrock`)	✅	✅	✅	✅	✅
AWS - Sagemaker (`sagemaker`)	✅	✅	✅	✅
Azure (`azure`)	✅	✅	✅	✅	✅	✅	✅	✅	✅
Azure AI (`azure_ai`)	✅	✅	✅	✅	✅	✅	✅	✅	✅
Azure Text (`azure_text`)	✅	✅	✅	✅	✅	✅	✅
Baseten (`baseten`)	✅	✅	✅
Bytez (`bytez`)	✅	✅	✅
Cerebras (`cerebras`)	✅	✅	✅
Clarifai (`clarifai`)	✅	✅	✅
Cloudflare AI Workers (`cloudflare`)	✅	✅	✅
Codestral (`codestral`)	✅	✅	✅
Cohere (`cohere`)	✅	✅	✅	✅	✅
Cohere Chat (`cohere_chat`)	✅	✅	✅
CometAPI (`cometapi`)	✅	✅	✅	✅
CompactifAI (`compactifai`)	✅	✅	✅
Custom (`custom`)	✅	✅	✅
Custom OpenAI (`custom_openai`)	✅	✅	✅	✅	✅	✅	✅
Dashscope (`dashscope`)	✅	✅	✅	✅	✅
Databricks (`databricks`)	✅	✅	✅
DataRobot (`datarobot`)	✅	✅	✅
Deepgram (`deepgram`)	✅	✅	✅	✅
DeepInfra (`deepinfra`)	✅	✅	✅
Deepseek (`deepseek`)	✅	✅	✅
ElevenLabs (`elevenlabs`)	✅	✅	✅	✅	✅
Empower (`empower`)	✅	✅	✅
Fal AI (`fal_ai`)	✅	✅	✅	✅
Featherless AI (`featherless_ai`)	✅	✅	✅
Fireworks AI (`fireworks_ai`)	✅	✅	✅
FriendliAI (`friendliai`)	✅	✅	✅
Galadriel (`galadriel`)	✅	✅	✅
GitHub Copilot (`github_copilot`)	✅	✅	✅	✅
GitHub Models (`github`)	✅	✅	✅
Google - PaLM	✅	✅	✅
Google - Vertex AI (`vertex_ai`)	✅	✅	✅	✅	✅
Google AI Studio - Gemini (`gemini`)	✅	✅	✅
GradientAI (`gradient_ai`)	✅	✅	✅
Groq AI (`groq`)	✅	✅	✅
Heroku (`heroku`)	✅	✅	✅
Hosted VLLM (`hosted_vllm`)	✅	✅	✅
Huggingface (`huggingface`)	✅	✅	✅	✅	✅
Hyperbolic (`hyperbolic`)	✅	✅	✅
IBM - Watsonx.ai (`watsonx`)	✅	✅	✅	✅
Infinity (`infinity`)	✅
Jina AI (`jina_ai`)	✅
Lambda AI (`lambda_ai`)	✅	✅	✅
Lemonade (`lemonade`)	✅	✅	✅
LiteLLM Proxy (`litellm_proxy`)	✅	✅	✅	✅	✅
Llamafile (`llamafile`)	✅	✅	✅
LM Studio (`lm_studio`)	✅	✅	✅
Maritalk (`maritalk`)	✅	✅	✅
Meta - Llama API (`meta_llama`)	✅	✅	✅
Mistral AI API (`mistral`)	✅	✅	✅	✅
Moonshot (`moonshot`)	✅	✅	✅
Morph (`morph`)	✅	✅	✅
Nebius AI Studio (`nebius`)	✅	✅	✅	✅
NLP Cloud (`nlp_cloud`)	✅	✅	✅
Novita AI (`novita`)	✅	✅	✅
Nscale (`nscale`)	✅	✅	✅
Nvidia NIM (`nvidia_nim`)	✅	✅	✅
OCI (`oci`)	✅	✅	✅
Ollama (`ollama`)	✅	✅	✅	✅
Ollama Chat (`ollama_chat`)	✅	✅	✅
Oobabooga (`oobabooga`)	✅	✅	✅	✅	✅	✅	✅
OpenAI (`openai`)	✅	✅	✅	✅	✅	✅	✅	✅	✅
OpenAI-like (`openai_like`)	✅
OpenRouter (`openrouter`)	✅	✅	✅
OVHCloud AI Endpoints (`ovhcloud`)	✅	✅	✅
Perplexity AI (`perplexity`)	✅	✅	✅
Petals (`petals`)	✅	✅	✅
Predibase (`predibase`)	✅	✅	✅
Recraft (`recraft`)	✅
Replicate (`replicate`)	✅	✅	✅
Sagemaker Chat (`sagemaker_chat`)	✅	✅	✅
Sambanova (`sambanova`)	✅	✅	✅
Snowflake (`snowflake`)	✅	✅	✅
Text Completion Codestral (`text-completion-codestral`)	✅	✅	✅
Text Completion OpenAI (`text-completion-openai`)	✅	✅	✅	✅	✅	✅	✅
Together AI (`together_ai`)	✅	✅	✅
Topaz (`topaz`)	✅	✅	✅
Triton (`triton`)	✅	✅	✅
V0 (`v0`)	✅	✅	✅
Vercel AI Gateway (`vercel_ai_gateway`)	✅	✅	✅
VLLM (`vllm`)	✅	✅	✅
Volcengine (`volcengine`)	✅	✅	✅
Voyage AI (`voyage`)	✅
WandB Inference (`wandb`)	✅	✅	✅
Watsonx Text (`watsonx_text`)	✅	✅	✅
xAI (`xai`)	✅	✅	✅
Xinference (`xinference`)	✅

Read the Docs

Get Started

You can use LiteLLM through either the Proxy Server or Python SDK. Both give you a unified interface to access multiple LLMs (100+ LLMs). Choose the option that best fits your needs:

	LiteLLM AI Gateway	LiteLLM Python SDK
Use Case	Central service (LLM Gateway) to access multiple LLMs	Use LiteLLM directly in your Python code
Who Uses It?	Gen AI Enablement / ML Platform Teams	Developers building LLM projects
Key Features	Centralized API gateway with authentication and authorization, multi-tenant cost tracking and spend management per project/user, per-project customization (logging, guardrails, caching), virtual keys for secure access control, admin dashboard UI for monitoring and management	Direct Python library integration in your codebase, Router with retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - Router, application-level load balancing and cost tracking, exception handling with OpenAI-compatible errors, observability callbacks (Lunary, MLflow, Langfuse, etc.)

Stable Release: Use docker images with the -stable tag. These have undergone 12 hour load tests, before being published. More information about the release cycle here

Support for more providers. Missing a provider or LLM Platform, raise a feature request.

Run in Developer Mode

Services

Setup .env file in root
Run dependent services docker-compose up db prometheus

Backend

(In root) create virtual environment python -m venv .venv
Activate virtual environment source .venv/bin/activate
Install dependencies uv sync --all-extras --group proxy-dev
uv run prisma generate
prisma generate
Start proxy backend python litellm/proxy/proxy_cli.py

Frontend

Navigate to ui/litellm-dashboard
Install dependencies npm install
Run npm run dev to start the dashboard

Verify Docker Image Signatures

All LiteLLM Docker images published to GHCR are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosignverify\
--keyhttps://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub\
ghcr.io/berriai/litellm:<release-tag>

Verify using a release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosignverify\
--keyhttps://raw.githubusercontent.com/BerriAI/litellm/<release-tag>/cosign.pub\
ghcr.io/berriai/litellm:<release-tag>

Replace <release-tag> with the version you are deploying (e.g. v1.83.0-stable).

Enterprise

For companies that need better security, user management and professional support

Get an Enterprise License Talk to founders

This covers:

✅ Features under the LiteLLM Commercial License:
✅ Feature Prioritization
✅ Custom Integrations
✅ Professional Support - Dedicated discord + slack
✅ Custom SLAs
✅ Secure access with Single Sign-On

Contributing

We welcome contributions to LiteLLM! Whether you're fixing bugs, adding features, or improving documentation, we appreciate your help.

Quick Start for Contributors

This requires uv to be installed.

gitclonehttps://github.com/BerriAI/litellm.git
cdlitellm
makeinstall-dev# Install development dependencies
makeformat# Format your code
makelint# Run all linting checks
maketest-unit# Run unit tests
makeformat-check# Check formatting only

For detailed contributing guidelines, see CONTRIBUTING.md.

📖 Contributing to documentation? The LiteLLM docs have moved to a separate repository: BerriAI/litellm-docs. Please open doc PRs there. Docs are served at docs.litellm.ai.

Code Quality / Linting

LiteLLM follows the Google Python Style Guide.

Our automated checks include:

Black for code formatting
Ruff for linting and code quality
MyPy for type checking
Circular import detection
Import safety checks

All these checks must pass before your PR can be merged.

Support / talk with founders

Schedule Demo 👋
Community Discord 💭
Community Slack 💭
Our emails ✉️ ishaan@berri.ai / krrish@berri.ai

Algorithm	Hash digest
SHA256	`b2534d69568eed026310f4e006407db2d46494eb629bd1e71eb9603ec146540d`
MD5	`944188d1adf07a26183032ba2d230c8f`
BLAKE2b-256	`c129865d38325f9c424daf0bcc7ab61908cf51a87862b3a0dfebd059b60dac97`

Algorithm	Hash digest
SHA256	`07e8e43b1a70fe919021376742897d18ffe7577ccfbb84632c949670f9abdc03`
MD5	`02b02cd14a5aea0d46af41b4cb90c4b9`
BLAKE2b-256	`32c851c93e8d017e7af02600171b5b6cc3805b07507acb2b94de7235ce764015`

URL: https://pypi.org/project/litellm/

⇱ litellm · PyPI

litellm 1.89.2

Navigation

Verified details

Owner

Unverified details

Project links

Meta

Project description

🚅 LiteLLM

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

👁 PyPI Version 👁 GitHub Stars 👁 Y Combinator W23 👁 Whatsapp 👁 Discord 👁 Slack 👁 CodSpeed

What is LiteLLM