VOOZH about

URL: https://www.cdata.com/kb/tech/elasticsearch-ssis-setup.rst

⇱ Getting Started with the CData SSIS Components for Elasticsearch

Getting Started with the CData SSIS Components for Elasticsearch

Complete guide to installing, licensing, and connecting with the CData SSIS Components for Elasticsearch.

This guide explains everything you need to get started with the CData SSIS Components for Elasticsearch. You'll learn how to install the components, activate your license, configure your first SSIS connection manager, and build a data flow task to move and transform Elasticsearch data in your SQL Server Integration Services workflows.

About Elasticsearch Data Integration

Accessing and integrating live data from Elasticsearch has never been easier with CData. Customers rely on CData connectivity to:

  • Access both the SQL endpoints and REST endpoints, optimizing connectivity and offering more options when it comes to reading and writing Elasticsearch data.
  • Connect to virtually every Elasticsearch instance starting with v2.2 and Open Source Elasticsearch subscriptions.
  • Always receive a relevance score for the query results without explicitly requiring the SCORE() function, simplifying access from 3rd party tools and easily seeing how the query results rank in text relevance.
  • Search through multiple indices, relying on Elasticsearch to manage and process the query and results instead of the client machine.

Users frequently integrate Elasticsearch data with analytics tools such as Crystal Reports, Power BI, and Excel, and leverage our tools to enable a single, federated access layer to all of their data sources, including Elasticsearch.

For more information on CData's Elasticsearch solutions, check out our Knowledge Base article: CData Elasticsearch Driver Features & Differentiators.


Getting Started


Installation & Licensing

System Requirements

  • Windows: Windows 10/11 or Windows Server 2016+
  • Visual Studio: Visual Studio 2015 or later
  • SQL Server: SQL Server 2014, 2016, 2017, 2019, or 2022

Installing the SSIS Components

  1. Download the SSIS Components installer for Elasticsearch from your CData account or the evaluation download page
  2. Run the installer and follow the installation wizard
  3. The installer automatically registers the Connection Manager, Source, and Destination components with Visual Studio
  4. When prompted, activate your license using the product key sent to you by the CData Orders Team:
    XXXX-XXXX-XXXX-XXXX-XXXX
    • Note: To run a trial, choose the Trial Key option.

Enabling SSIS in Visual Studio 2022

If you are using Visual Studio 2022, the SQL Server Integration Services Projects extension must be installed.

  1. In Visual Studio, select Extensions > Manage Extensions
  2. Search for SQL Server Integration Services Projects 2022
  3. Click Install
  4. Close Visual Studio and run the downloaded Microsoft.DataTools.IntegrationServices.exe installer
  5. Reopen Visual Studio. The Integration Services Project template will now appear when creating a new project

Activating Your License

During installation, you are prompted to activate the SSIS Component license. If you need to update or change activation:

License Activation

The installer automatically prompts you to add your license. During installation, you can choose to:

  • Use your existing subscription license key, or
  • Enter your trial license

To activate a full subscription license, contact the CData Orders Team and request your product key at [email protected].

Enter the license key in the installer when prompted. Once activated, the components will be licensed and ready to use inside Visual Studio without any additional steps.

Runtime Licensing

When deploying SSIS packages, a Runtime Key (RTK) can also be used:

  • Set the RTK property in the Connection Manager before deployment

Common Licensing Questions

Can I use my license on multiple machines?
Yes, depending on your subscription tier. Contact [email protected] for details.

I lost my license key. How do I retrieve it?
Email [email protected] with your order number, and we'll resend your license key.

How do I transfer my license to another machine?

Yes. When transferring the license to a different machine, you will need to submit a License Transfer Request on our site linked below:

https://www.cdata.com/lic/transfer/

After the License Transfer Request is submitted and successfully processed, an activation will be added to your Product Key and you will be able to activate the full license on the other machine. Once this process is finished, the license on the previous machine will be invalid.

You may also view and upgrade licenses in the self-service portal at portal.cdata.com.

Connection Configuration

Once the components are installed and licensed, you can configure a connection to Elasticsearch using an SSIS Connection Manager. This Connection Manager stores all authentication and connection properties used by the Source and Destination components.

Creating a Connection Manager

  1. In the bottom Connection Managers panel of your SSIS package, right-click and select New Connection 👁 Selecting new connection
  2. Select CData SSIS Components for Elasticsearch from the list 👁 Adding Connection Manager
  3. Click Add to open the Connection Manager UI
  4. Enter the required authentication properties (OAuth, API token, client credentials, etc.) depending on your Elasticsearch

    5. Configuring Connection Properties

    Set the Server and Port connection properties to connect. To authenticate, set the User and Password properties, PKI (public key infrastructure) properties, or both. To use PKI, set the SSLClientCert, SSLClientCertType, SSLClientCertSubject, and SSLClientCertPassword properties.

    The data provider uses X-Pack Security for TLS/SSL and authentication. To connect over TLS/SSL, prefix the Server value with 'https://'. Note: TLS/SSL and client authentication must be enabled on X-Pack to use PKI.

    Once the data provider is connected, X-Pack will then perform user authentication and grant role permissions based on the realms you have configured.

  5. Click Test Connection to confirm connectivity 👁 Testing connection

Building an SSIS Data Flow

With a Connection Manager created, you can now pull data from Elasticsearch or push data into it using SSIS data flow tasks.

Creating a Data Flow Task

  1. In the Control Flow tab, drag a Data Flow Task onto the design surface 👁 Dropping data flow task
  2. Double-click the task to open the Data Flow workspace

Using the Source Component

  1. In the SSIS Toolbox, drag the CData Elasticsearch Source component into the Data Flow 👁 Dropping source component
  2. Double-click it to open the Source Editor
  3. Select the CData Elasticsearch Connection Manager you created 👁 Selecting CM and table
  4. Choose a table or view to extract records from
  5. Click OK to save your configuration

Using the Destination Component

  1. Drag a SQL Server Destination onto the canvas 👁 Dragging destination component
  2. Double-click it to open the Destination Editor 👁 Adding new CM
  3. Select an existing table or click New to auto-generate a table based on the Source schema 👁 Creating new table
  4. Connect the Source output to the Destination input and map the columns as needed
  5. At this point you have created a data flow task for replicating your Elasticsearch data to a SQL Server database 👁 Completing data flow task

Testing Your Data Flow

  1. Return to the Control Flow tab
  2. Click Start Debugging
  3. Monitor the progress indicators
  4. Review row counts and ensure data is loading as expected

Common Connection Issues

Authentication Failed

Solution: Verify OAuth settings, client IDs, secrets, or token permissions for your Elasticsearch. Contact [email protected] for OAuth troubleshooting.

Cannot Reach Server

Solution: Check firewall, proxy, and VPN configurations. Contact [email protected] for specific port requirements.

Table Not Found

Solution: Confirm you selected the correct schema or database when querying Elasticsearch.

What's Next

Now that you have installed, licensed, and configured the SSIS Components, here are scenarios you can use to explore our SSIS tools:

Get Support

If you need assistance at any point:

FAQs

Installation & Licensing

  • Do I need administrator rights to install the SSIS Components?
    Yes, administrator rights are required to install components for use across Visual Studio.
  • Do I need an RTK to deploy to Azure Data Factory?
    Yes. Set the RTK property in the Connection Manager before publishing.

Connecting

  • Can I use multiple Elasticsearch accounts?
    Create separate Connection Managers for each account.
  • Can I connect through a proxy?
    Yes. Configure proxy settings in the Connection Manager properties.
  • How do I test my connection?
    Click Test Connection in the Connection Manager UI.

Performance & Troubleshooting

  • Why is my data flow slow?
    Add filters, limit rows, and ensure batching settings are configured in the Source component.
  • How do I enable logging?
    Add the following to your connection manager:
    • Logfile: /path/to/logfile.log
    • Verbosity: 3

    Be prepared to securely upload the log file upon request when reaching out to [email protected] for troubleshooting analysis.

    For questions not covered in this FAQ, contact [email protected].

Ready to get started?

Download a free trial of the Elasticsearch SSIS Component to get started:

 Download Now

Learn more:

👁 Elasticsearch Icon
Elasticsearch SSIS Components

Powerful SSIS Source & Destination Components that allows you to easily connect SQL Server with live Elasticsearch document databases through SSIS Workflows.

Use the Elasticsearch Data Flow Components to synchronize with Elasticsearch. Perfect for data synchronization, local back-ups, workflow automation, and more!