VOOZH about

URL: https://www.cdata.com/kb/tech/paylocity-cloud-databricks.rst

⇱ Connect and Query Live Paylocity Data in Databricks with CData Connect AI

Connect and Query Live Paylocity Data in Databricks with CData Connect AI

👁 Mohsin Turki
Mohsin Turki
Technical Marketing Engineer
Use CData Connect AI to integrate live Paylocity data into Databricks and enable direct, live querying and analysis without replication.

Databricks is a leading AI cloud-native platform that unifies data engineering, machine learning, and analytics at scale. Its powerful data lakehouse architecture combines the performance of data warehouses with the flexibility of data lakes. Integrating Databricks with CData Connect AI gives organizations live, real-time access to Paylocity data without the need for complex ETL pipelines or data duplication—streamlining operations and reducing time-to-insights.

In this article, we'll walk through how to configure a secure, live connection from Databricks to Paylocity using CData Connect AI. Once configured, you'll be able to access Paylocity data directly from Databricks notebooks using standard SQL—enabling unified, real-time analytics across your data ecosystem.

Overview

Here is an overview of the simple steps:

  1. Step 1 — Connect and Configure: In CData Connect AI, create a connection to your Paylocity source, configure user permissions, and generate a Personal Access Token (PAT).
  2. Step 2 — Query from Databricks: Install the CData JDBC driver in Databricks, configure your notebook with the connection details, and run SQL queries to access live Paylocity data.

Prerequisites

Before you begin, make sure you have the following:

  1. An active Paylocity account.
  2. A CData Connect AI account. You can log in or sign up for a free trial here.
  3. A Databricks account. Sign up or log in here.

Step 1: Connect and Configure a Paylocity Connection in CData Connect AI

1.1 Add a Connection to Paylocity

CData Connect AI uses a straightforward, point-and-click interface to connect to available data sources.

  1. Log into Connect AI, click Sources on the left, and then click Add Connection in the top-right.
    2. 👁 Adding a Connection in CData Connect AI
  2. Select "Paylocity" from the Add Connection panel.
    3. 👁 Selecting a data source
  3. Enter the necessary authentication properties to connect to Paylocity.

    Set the following to establish a connection to Paylocity:

    • RSAPublicKey: Set this to the RSA Key associated with your Paylocity, if the RSA Encryption is enabled in the Paylocity account.

      This property is required for executing Insert and Update statements, and it is not required if the feature is disabled.

    • UseSandbox: Set to true if you are using sandbox account.
    • CustomFieldsCategory: Set this to the Customfields category. This is required when IncludeCustomFields is set to true. The default value for this property is PayrollAndHR.
    • Key: The AES symmetric key(base 64 encoded) encrypted with the Paylocity Public Key. It is the key used to encrypt the content.

      Paylocity will decrypt the AES key using RSA decryption.
      It is an optional property if the IV value not provided, The driver will generate a key internally.

    • IV: The AES IV (base 64 encoded) used when encrypting the content. It is an optional property if the Key value not provided, The driver will generate an IV internally.

    Connect Using OAuth Authentication

    You must use OAuth to authenticate with Paylocity. OAuth requires the authenticating user to interact with Paylocity using the browser. For more information, refer to the OAuth section in the Help documentation.

    The Pay Entry API

    The Pay Entry API is completely separate from the rest of the Paylocity API. It uses a separate Client ID and Secret, and must be explicitly requested from Paylocity for access to be granted for an account. The Pay Entry API allows you to automatically submit payroll information for individual employees, and little else. Due to the extremely limited nature of what is offered by the Pay Entry API, we have elected not to give it a separate schema, but it may be enabled via the UsePayEntryAPI connection property.

    Please be aware that when setting UsePayEntryAPI to true, you may only use the CreatePayEntryImportBatch & MergePayEntryImportBatchgtable stored procedures, the InputTimeEntry table, and the OAuth stored procedures. Attempts to use other features of the product will result in an error. You must also store your OAuthAccessToken separately, which often means setting a different OAuthSettingsLocation when using this connection property. 👁 Configuring a connection (Salesforce is shown)

  4. Click Save & Test in the top-right.
  5. Navigate to the Permissions tab on the Paylocity Connection page and update the user-based permissions based on your preferences. 👁 Updating permissions

1.2 Generate a Personal Access Token (PAT)

When connecting to Connect AI through the REST API, the OData API, or the Virtual SQL Server, a Personal Access Token (PAT) is used to authenticate the connection to Connect AI. PAT functions as an alternative to your login credentials for secure, token-based authentication. It is a best practice to create a separate PAT for each service to maintain granularity of access.

  1. Click on the Gear icon () at the top right of the Connect AI app to open the settings page.
  2. On the Settings page, go to the Access Tokens section and click Create PAT.
  3. Give the PAT a name and click Create. 👁 Creating a new PAT
  4. Note: The personal access token is only visible at creation, so be sure to copy it and store it securely for future use.

Step 2: Connect and Query Paylocity Data in Databricks

Follow these steps to establish a connection from Databricks to Paylocity. You'll install the CData JDBC Driver for Connect AI, add the JAR file to your cluster, configure your notebooks, and run SQL queries to access live Paylocity data data.

2.1 Install the CData JDBC Driver for Connect AI

  1. In CData Connect AI, click the Integrations page on the left. Search for JDBC or Databricks, click Download, and select the installer for your operating system.
  2. Once downloaded, run the installer and follow the instructions:
    • For Windows: Run the setup file and follow the installation wizard.
    • For Mac/Linux: Unpack the archive and move the folder to /opt or /Applications. Make sure you have execute permissions.
  3. After installation, locate the JAR file in the installation directory:
    • Windows: 
      C:\Program Files\CData\CData JDBC Driver for Connect AI\lib\cdata.jdbc.connect.jar
    • Mac/Linux: 
      /Applications/CData/CData JDBC Driver for Connect AI/lib/cdata.jdbc.connect.jar

2.2 Install the JAR File on Databricks

  1. Log in to Databricks. In the navigation pane, click Compute on the left. Start or create a compute cluster. 👁 Launching a compute cluster in Databricks
  2. Click on the running cluster, go to the Libraries tab, and click Install New at the top right. 👁 Accessing the Libraries tab in Databricks
  3. In the Install Library dialog, select DBFS, and drag and drop the cdata.jdbc.connect.jar file. Click Install. 👁 Uploading the JDBC driver JAR to DBFS

2.3 Query Paylocity Data in a Databricks Notebook

Notebook Script 1 — Define JDBC Connection:

  1. Paste the following script into the notebook cell:
driver = "cdata.jdbc.connect.ConnectDriver"
url = "jdbc:connect:AuthScheme=Basic;User=your_username;Password=your_pat;URL=https://cloud.cdata.com/api/;DefaultCatalog=Your_Connection_Name;"
  1. Replace:
    • your_username - With your CData Connect AI username
    • your_pat - With your CData Connect AI Personal Access Token (PAT)
    • Your_Connection_Name - With the name of your Connect AI data source, from the Sources page
  2. Run the script.

Notebook Script 2 — Load DataFrame from Paylocity data:

  1. Add a new cell for this second script. From the menu on the right side of your notebook, click Add cell below.
  2. Paste the following script into the new cell:
remote_table = spark.read.format("jdbc") \
 .option("driver", "cdata.jdbc.connect.ConnectDriver") \
 .option("url", "jdbc:connect:AuthScheme=Basic;User=your_username;Password=your_pat;URL=https://cloud.cdata.com/api/;DefaultCatalog=Your_Connection_Name;") \
 .option("dbtable", "YOUR_SCHEMA.YOUR_TABLE") \
 .load()
  1. Replace:
    • your_username - With your CData Connect AI username
    • your_pat - With your CData Connect AI Personal Access Token (PAT)
    • Your_Connection_Name - With the name of your Connect AI data source, from the Sources page
    • YOUR_SCHEMA.YOUR_TABLE - With your schema and table, for example, Paylocity.Employee
  2. Run the script.

Notebook Script 3 — Preview Columns:

  1. Similarly, add a new cell for this third script.
  2. Paste the following script into the new cell:
display(remote_table.select("ColumnName1", "ColumnName2"))
  1. Replace ColumnName1 and ColumnName2 with the actual columns from your Paylocity structure (e.g. FirstName, LastName, etc.).
  2. Run the script.
👁 Previewing Paylocity data data in Databricks notebook

You can now explore, join, and analyze live Paylocity data directly within Databricks notebooks—without needing to know the complexities of the back-end API and without replicating Paylocity data.

Try CData Connect AI Free for 14 Days

Ready to simplify real-time access to Paylocity data? Start your free 14-day trial of CData Connect AI today and experience seamless, live connectivity from Databricks to Paylocity.

Low code, zero infrastructure, zero replication — just seamless, secure access to your most critical data and insights.