Query Scrapfly Data in DataGrip

DataGrip is a database IDE that allows SQL developers to query, create, and manage databases. When paired with the CData API Driver for JDBC, DataGrip can work with live Scrapfly data. This article shows how to establish a connection to Scrapfly data in DataGrip.

Create a New Driver Definition for Scrapfly

The steps below describe how to create a new Data Source in DataGrip for Scrapfly.

1. In DataGrip, click File -> New > Project and name the project 👁 Creating a new DataGrip project.
   
2. In the Database Explorer, click the plus icon () and select Driver. 👁 Adding a new Driver.
   
3. In the Driver tab:
   • Set Name to a user-friendly name (e.g. "CData Scrapfly Driver")
   • Set Driver Files to the appropriate JAR file. To add the file, click the plus (), select "Add Files," navigate to the "lib" folder in the driver's installation directory and select the JAR file (e.g. cdata.jdbc.api.jar).
   • Set Class to cdata.jdbc.api.API.jar
4. Click "Apply" then "OK" to save the Connection 👁 A configured Driver (Salesforce is shown).
   

Configure a Connection to Scrapfly

1. Once the connection is saved, click the plus (), then "Data Source" then "CData Scrapfly Driver" to create a new Scrapfly Data Source.
2. In the new window, configure the connection to Scrapfly with a JDBC URL.

   Built-in Connection String Designer

   For assistance in constructing the JDBC URL, use the connection string designer built into the Scrapfly JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

    java -jar cdata.jdbc.api.jar
    

   Fill in the connection properties and copy the connection string to the clipboard.

   The Scrapfly API uses API Key authentication. The API key is passed as the key query parameter on every request.

   Using API Key Authentication

   Your Scrapfly API key is required to create a connection. To obtain your API key:

   1. Log into your Scrapfly account at scrapfly.io.
   2. Navigate to Dashboard and select API Keys.
   3. Copy your API key (begins with scp-live- for production or scp-test- for the test environment).

   After obtaining your API key, set the following connection properties:

   • AuthScheme: Set this to APIKey.
   • APIKey: Set this to your Scrapfly API key.

   Example connection string:

   Profile=C:\profiles\Scrapfly.apip;AuthScheme=APIKey;ProfileSettings='APIKey=your_api_key';
   

   👁 Using the built-in connection string designer to generate a JDBC URL (Salesforce is shown.)
   
3. Set URL to the connection string, e.g.,

   jdbc:api:Profile=C:\profiles\Scrapfly.apip;AuthScheme=APIKey;ProfileSettings='APIKey=your_api_key';

4. Click "Apply" and "OK" to save the connection string 👁 A configured Data Source (Salesforce is shown).
   

At this point, you will see the data source in the Data Explorer.

Execute SQL Queries Against Scrapfly

To browse through the Scrapfly entities (available as tables) accessible through the JDBC Driver, expand the Data Source.

To execute queries, right click on any table and select "New" -> "Query Console."

In the Console, write the SQL query you wish to execute. For example:

SELECT , FROM Account WHERE = ''

Download a free, 30-day trial of the CData API Driver for JDBC and start working with your live Scrapfly data in DataGrip. Reach out to our Support Team if you have any questions.