Meltano Singer SDK¶

Quick Start¶

Create your first tap in seconds:

cookiecutterhttps://github.com/meltano/sdk--directory="cookiecutter/tap-template"
cdtap-myapi# or whatever you named it
uvsync

Build a Simple REST API Tap¶

Create a data extractor in just a few lines of code:

fromsinger_sdkimport Tap, Stream
fromsinger_sdk.streamsimport RESTStream
importsinger_sdk.typingasth

classUsersStream(RESTStream):
"""Users stream."""
 name = "users"
 url_base = "https://api.example.com"
 path = "/users"
 primary_keys = ["id"]
 records_jsonpath = "$.data[*]"

 schema = th.PropertiesList(
 th.Property("id", th.IntegerType),
 th.Property("name", th.StringType),
 th.Property("email", th.StringType),
 ).to_dict()

classMyTap(Tap):
"""My custom tap."""
 name = "tap-myapi"
 config_jsonschema = th.PropertiesList(
 th.Property("api_url", th.StringType, required=True),
 th.Property("api_key", th.StringType, required=True, secret=True),
 ).to_dict()

 defdiscover_streams(self):
 return [UsersStream(self)]

if __name__ == "__main__":
 MyTap.cli()

Build a Simple Target¶

Create a data loader to write records to any destination:

fromsinger_sdkimport Target
fromsinger_sdk.sinksimport RecordSink

classMySink(RecordSink):
"""My custom sink."""

 defprocess_record(self, record: dict, context: dict) -> None:
"""Process a single record."""
 # Your custom logic here
 print(f"Processing: {record}")

classMyTarget(Target):
"""My custom target."""
 name = "target-mydb"
 config_jsonschema = th.PropertiesList(
 th.Property("host", th.StringType, required=True),
 th.Property("port", th.IntegerType, default=5432),
 th.Property("database", th.StringType, required=True),
 th.Property("username", th.StringType, required=True),
 th.Property("password", th.StringType, required=True, secret=True),
 ).to_dict()
 default_sink_class = MySink

if __name__ == "__main__":
 MyTarget.cli()

Key Features¶

• Less Code: Focus on your business logic, not boilerplate

• Auto-Compliant: Singer Spec compliance built-in

• Battle-Tested: Powers millions of pipeline runs monthly

• Rich Ecosystem: 600+ existing connectors on Meltano Hub

• Advanced Features: Authentication, pagination, incremental sync, batching, and more

Core Classes¶

The SDK provides these main building blocks:

Taps (Data Extractors)

• Tap - Base class for all data extractors

• SQLTap - For database sources

• RESTStream - For REST APIs

• GraphQLStream - For GraphQL APIs

• SQLStream - For SQL databases

Targets (Data Loaders)

• Target - Base class for all data loaders

• SQLTarget - For database destinations

• RecordSink - Process one record at a time

• BatchSink - Process records in batches

Additional Tools

• Built-in authentication classes (OAuth, JWT, API Key, etc.)

• Automatic pagination handling

• Schema validation and typing

• Incremental replication support

What’s a Tap vs Target?¶

• Tap: Extracts data from a source (API, database, file)

• Target: Loads data to a destination (database, warehouse, file)

They work together in ELT pipelines: tap-source | target-destination

Next Steps¶

• Getting Started - Complete development guide

• SDK Code Samples - Copy-paste examples

• In-depth Guides - In-depth tutorials

• Meltano Tutorial - End-to-end walkthrough

Built by Meltano and the Singer Community¶

The SDK is built with love by the Meltano core team and contributors, with contributions from developers across the Singer open source community. SDK-based plugins can easily be integrated with Meltano, but they can also be used in any data pipeline tool that supports the Singer Spec.

The SDK project is 100% open source, licensed under the Apache 2.0 permissive license. Please visit our code repo here in GitHub, where you can read the source code, log an issue or feature request, and contribute back to the SDK. We also regularly discuss SDK topics in our Meltano Slack community, within the #singer-tap-development and #singer-target-development Slack channels.

Index and Search¶

• Index

• Search Page