Cloud Native Ecosystem Containers Databases Edge Computing Infrastructure as Code Linux Microservices Open Source Networking Storage

AI AI Engineering API Management Backend development Data Frontend Development Large Language Models Security Software Development WebAssembly

AI Operations CI/CD Cloud Services DevOps Kubernetes Observability Operations Platform Engineering

C++ Developer tools Go Java JavaScript Programming Languages Python Rust TypeScript

Data platform provider AirByte has released an open source Python library to help speed the process for Python programmers to connect data sources with other destinations that would use that data.

PyAirbyte provides over 250 data connectors for bridging different applications via API. It also works with open source orchestration tools such as Dagster. On the LLM side, a connector can work with LangChain for the creation of applications using large language models (LLMs).

According to the documentation, PyAirbyte “is meant to be used in situations where setting up an Airbyte server or cloud account is not possible or desirable, for example in a Jupyter notebook or when iterating on early prototypes on a developer’s workstation,” “A majority of existing data pipelines are written in Python today,” said Michel Tricot, co-founder and CEO, Airbyte, in a statement. “We’ve seen growing demand for this capability, which provides a natural fit into the coding workflow and only requires running one command.” 👁 Image
PyAirByte works with Python 3.9 and later versions. An installation can be done from the Python command line: Among the 290 connectors are those for:

• Amazon Ads
• Amazon Seller Partner
• Amazon SQS
• Auth0
• AWS CloudTrail
• Azure Blob Storage
• Azure Table Storage
• GitHub
• GitLab

Thanks to its simplicity, Python is widely used as a programming language in the data science community and is integrated across many data analysis tools, such as SQL databases, Hadoop, and Spark.

Airbyte itself specializes in synchronizing data across applications, APIs, and databases to data warehouses, lakes, and other destinations, through offerings such as Airbyte Open Source, Airbyte Self-Managed, Airbyte Cloud, and Powered by Airbyte. The open source community supporting the platform through connectors has garnered over 800 contributors. Other data pipeline enhancements from the company include the Airbyte API and Terraform Provider, To learn more, register for the webinar to be held on March 14.

AWS, GitLab, and Microsoft are sponsors of The New Stack.

The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy.