π Image
π Image
π Image
π Image
π Image
Scale Python across 1,000 computers in 1 second.
Burla is an open-source cloud platform for Python developers. It only has one function:
from burla import remote_parallel_map my_inputs = list(range(1000)) def my_function(x): print(f"[#{x}] running on separate computer") remote_parallel_map(my_function, my_inputs)
This runs my_function on 1,000 VMs in the cloud, in < 1 second:
π Burla terminal demo showing remote_parallel_map running on 1,000 computers
The simplest way to build scalable data-pipelines.
Burla scales up to 10,000 CPUs in a single function call, supports GPUs, and custom containers.
Load data in parallel from cloud storage, then write results in parallel from thousands of VMs at once.
remote_parallel_map(process, [...]) remote_parallel_map(aggregate, [...], func_cpu=64) remote_parallel_map(predict, [...], func_gpu="A100")
This creates a pipeline like:
π Burla data pipeline animation
Monitor progress in the dashboard:
Cancel bad runs, filter logs to watch individual inputs, or monitor output files in the UI.
How it works:
With Burla, running code in the cloud feels the same as coding on your laptop:
return_values = remote_parallel_map(my_function, my_inputs)
When functions are run with remote_parallel_map:
- Anything they print appears locally (and inside Burla's dashboard).
- Any exceptions are thrown locally.
- Any packages or local modules they use are (very quickly) cloned on remote machines.
- Code starts running in under one second, even with millions of inputs or thousands of machines.
Features:
-
π¦ Automatic Package Sync
Burla automatically (and very quickly) clones your Python packages on every remote machine where code is executed. -
π Custom Containers
Easily run code in any Docker container. Public or private, just paste an image URI in the settings, then hit start. -
π Network Filesystem
Need to get big data into or out of the cluster? Burla automatically mounts a cloud storage bucket to a folder in every container. -
βοΈ Variable Hardware Per-Function
Thefunc_cpuandfunc_ramargs make it possible to assign big hardware to some functions, and less to others.
Try Burla for Free, using 1,000 CPUs!
- Sign in using your Google or Microsoft account.
- Run the quickstart in this Google Colab notebook (takes less than 2 minutes):
Examples
- Process 2.4TB of Parquet files in 76s with 10,000 CPUs
- Hyperparameter tune XGBoost using 1,000 CPUs
- Genome alignments using 1,300 CPUs
Learn more at docs.burla.dev.
