Sail MCP Server for Spark SQL
@lakehq
a year ago
Sail is an open-source computation framework that serves as a drop-in replacement for Apache Spark (SQL and DataFrame API) in both single-host and distributed settings. The built-in MCP server in Sail exposes tools for LLM agents to register datasets and execute Spark SQL queries.
Overview
what is Sail?
Sail is a unified platform designed for stream processing, batch processing, and compute-intensive workloads, including AI tasks. It serves as a drop-in replacement for Spark SQL and the Spark DataFrame API, functioning in both single-host and distributed environments.
how to use Sail?
To use Sail, install it via pip with pip install "pysail[spark]", or build it from source for optimized performance. Start the Sail server using command line, Python API, or deploy it on Kubernetes for distributed processing.
key features of Sail?
- Unified processing for stream, batch, and AI workloads.
- Drop-in replacement for Spark SQL and DataFrame API.
- Supports local and distributed server setups.
- Easy integration with PySpark.
use cases of Sail?
- Real-time data analytics and processing.
- Batch processing of large datasets.
- AI model training and inference in a distributed environment.
FAQ from Sail?
- Is Sail compatible with existing Spark applications?
Yes! Sail is designed to be a drop-in replacement for Spark SQL and DataFrame API.
- Can I run Sail on Kubernetes?
Yes! Sail can be deployed on Kubernetes for distributed processing.
- What support options are available for Sail?
LakeSail offers flexible enterprise support options for Sail.
Server Config
{
"mcpServers": {
"sail": {
"command": "sail",
"args": [
"spark",
"mcp-server",
"--transport",
"stdio"
]
}
}
}