VOOZH about

URL: https://pypi.org/project/bentoml/

⇱ bentoml · PyPI

Skip to main content

bentoml 1.4.39

pip install bentoml

Latest release

Released:

BentoML: The easiest way to serve AI apps and models

Navigation

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: BentoML Team
  • Tags BentoML , Compound AI Systems , LLMOps , MLOps , Model Deployment , Model Inference , Model Serving
  • Requires: Python >=3.9
  • Provides-Extra: all , aws , grpc , grpc-channelz , grpc-reflection , io , io-image , io-pandas , monitor-otlp , tracing , tracing-jaeger , tracing-otlp , tracing-zipkin , triton , unsloth

Project description

Unified Model Serving Framework

🍱 Build model inference APIs and multi-model serving systems with any open-source or custom AI models. 👉 Join our forum!

👁 License: Apache-2.0
👁 Releases
👁 CI
👁 Twitter

What is BentoML?

BentoML is a Python library for building online serving systems optimized for AI apps and model inference.

  • 🍱 Easily build APIs for Any AI/ML Model. Turn any model inference script into a REST API server with just a few lines of code and standard Python type hints.
  • 🐳 Docker Containers made simple. No more dependency hell! Manage your environments, dependencies and model versions with a simple config file. BentoML automatically generates Docker images, ensures reproducibility, and simplifies how you deploy to different environments.
  • 🧭 Maximize CPU/GPU utilization. Build high performance inference APIs leveraging built-in serving optimization features like dynamic batching, model parallelism, multi-stage pipeline and multi-model inference-graph orchestration.
  • 👩‍💻 Fully customizable. Easily implement your own APIs or task queues, with custom business logic, model inference and multi-model composition. Supports any ML framework, modality, and inference runtime.
  • 🚀 Ready for Production. Develop, run and debug locally. Seamlessly deploy to production with Docker containers or BentoCloud.

Getting started

Install BentoML:

# Requires Python≥3.9
pip install -U bentoml

Define APIs in a service.py file.

importbentoml

@bentoml.service(
 image=bentoml.images.Image(python_version="3.11").python_packages("torch", "transformers"),
)
classSummarization:
 def__init__(self) -> None:
 importtorch
 fromtransformersimport pipeline

 device = "cuda" if torch.cuda.is_available() else "cpu"
 self.pipeline = pipeline('summarization', device=device)

 @bentoml.api(batchable=True)
 defsummarize(self, texts: list[str]) -> list[str]:
 results = self.pipeline(texts)
 return [item['summary_text'] for item in results]

💻 Run locally

Install PyTorch and Transformers packages to your Python virtual environment.

pipinstalltorchtransformers# additional dependencies for local run

Run the service code locally (serving at http://localhost:3000 by default):

bentomlserve

You should expect to see the following output.

[INFO] [cli] Starting production HTTP BentoServer from "service:Summarization" listening on http://localhost:3000 (Press CTRL+C to quit)
[INFO] [entry_service:Summarization:1] Service Summarization initialized

Now you can run inference from your browser at http://localhost:3000 or with a Python script:

importbentoml

with bentoml.SyncHTTPClient('http://localhost:3000') as client:
 summarized_text: str = client.summarize([bentoml.__doc__])[0]
 print(f"Result: {summarized_text}")

🐳 Deploy using Docker

Run bentoml build to package necessary code, models, dependency configs into a Bento - the standardized deployable artifact in BentoML:

bentomlbuild

Ensure Docker is running. Generate a Docker container image for deployment:

bentomlcontainerizesummarization:latest

Run the generated image:

dockerrun--rm-p3000:3000summarization:latest

☁️ Deploy on BentoCloud

BentoCloud provides compute infrastructure for rapid and reliable GenAI adoption. It helps speed up your BentoML development process leveraging cloud compute resources, and simplify how you deploy, scale and operate BentoML in production.

Sign up for BentoCloud for personal access; for enterprise use cases, contact our team.

# After signup, run the following command to create an API token:
bentomlcloudlogin

# Deploy from current directory:
bentomldeploy

👁 bentocloud-ui

For detailed explanations, read the Hello World example.

Examples

Check out the full list for more sample code and usage.

Advanced topics

See Documentation for more tutorials and guides.

Community

Get involved and join our Community Forum 💬, where thousands of AI/ML engineers help each other, contribute to the project, and talk about building AI products.

To report a bug or suggest a feature request, use GitHub Issues.

Contributing

There are many ways to contribute to the project:

Thanks to all of our amazing contributors!

👁 Image

Usage tracking and feedback

The BentoML framework collects anonymous usage data that helps our community improve the product. Only BentoML's internal API calls are being reported. This excludes any sensitive information, such as user code, model data, model names, or stack traces. Here's the code used for usage tracking. You can opt-out of usage tracking by the --do-not-track CLI option:

bentoml[command]--do-not-track

Or by setting the environment variable:

exportBENTOML_DO_NOT_TRACK=True

License

Apache License 2.0

Project details

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: BentoML Team
  • Tags BentoML , Compound AI Systems , LLMOps , MLOps , Model Deployment , Model Inference , Model Serving
  • Requires: Python >=3.9
  • Provides-Extra: all , aws , grpc , grpc-channelz , grpc-reflection , io , io-image , io-pandas , monitor-otlp , tracing , tracing-jaeger , tracing-otlp , tracing-zipkin , triton , unsloth

Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bentoml-1.4.39.tar.gz (988.8 kB view details)

Uploaded Source

Built Distribution

Filter files by name, interpreter, ABI, and platform.

If you're not sure about the file name format, learn more about wheel file names.

Copy a direct link to the current filters

bentoml-1.4.39-py3-none-any.whl (1.2 MB view details)

Uploaded Python 3

File details

Details for the file bentoml-1.4.39.tar.gz.

File metadata

  • Download URL: bentoml-1.4.39.tar.gz
  • Upload date:
  • Size: 988.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for bentoml-1.4.39.tar.gz
Algorithm Hash digest
SHA256 e92381702fc13ac926fd41c48fee6721f5c50f6f9bbed6c8cfc3b9346b6b0d41
MD5 826869d4c0bb7ed4003dcc281dae03b9
BLAKE2b-256 b1547ee2d823b653ce048dd1280dbf122485509b982f155133ae77d9f3ea53fc

See more details on using hashes here.

Provenance

The following attestation bundles were made for bentoml-1.4.39.tar.gz:

Publisher: release.yml on bentoml/BentoML

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file bentoml-1.4.39-py3-none-any.whl.

File metadata

  • Download URL: bentoml-1.4.39-py3-none-any.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for bentoml-1.4.39-py3-none-any.whl
Algorithm Hash digest
SHA256 5243c7725ec59a88e6121380bac54f1ac70916d5b15eaa1665d09ce0969f637b
MD5 d68fe66de3a864924c0d28f773b9dd61
BLAKE2b-256 72280716a70ebc31954cebc10dcd2c04abf48532c592396829605cc15400b041

See more details on using hashes here.

Provenance

The following attestation bundles were made for bentoml-1.4.39-py3-none-any.whl:

Publisher: release.yml on bentoml/BentoML

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

👁 Image
AWS Cloud computing and Security Sponsor 👁 Image
Datadog Monitoring 👁 Image
Depot Continuous Integration 👁 Image
Fastly CDN 👁 Image
Google Download Analytics 👁 Image
Pingdom Monitoring 👁 Image
Sentry Error logging 👁 Image
StatusPage Status page