Connect to Spark Data in JRuby

JRuby is a high-performance, stable, fully threaded Java implementation of the Ruby programming language. The CData JDBC Driver for Spark makes it easy to integrate connectivity to live Spark data in JRuby. This article shows how to create a simple JRuby app that connects to Spark data, executes a query, and displays the results.

Configure a JDBC Connection to Spark Data

Before creating the app, note the installation location for the JAR file for the JDBC Driver (typically C:\Program Files\CData\CData JDBC Driver for Spark\lib).

JRuby natively supports JDBC, so you can easily connect to Spark and execute SQL queries. Initialize the JDBC connection with the getConnection function of the java.sql.DriverManager class.

Set the Server, Database, User, and Password connection properties to connect to SparkSQL.

Built-in Connection String Designer

For assistance in constructing the JDBC URL, use the connection string designer built into the Spark JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

java -jar cdata.jdbc.sparksql.jar

Fill in the connection properties and copy the connection string to the clipboard.

Below is a typical JDBC connection string for Spark:

jdbc:sparksql:Server=127.0.0.1;

Create a JRuby App with Connectivity to Spark Data

Create a new Ruby file (for example: SparkSQLSelect.rb) and open it in a text editor. Copy the following code into your file:

require 'java'
require 'rubygems'
require 'C:/Program Files/CData/CData JDBC Driver for Spark 2018/lib/cdata.jdbc.sparksql.jar'

url = "jdbc:sparksql:Server=127.0.0.1;"
conn = java.sql.DriverManager.getConnection(url)
stmt = conn.createStatement
rs = stmt.executeQuery("SELECT City, Balance FROM Customers")
while (rs.next) do
 puts rs.getString(1) + ' ' + rs.getString(2)
end

With the file completed, you are ready to display your Spark data with JRuby. To do so, simply run your file from the command line:

jruby -S SparkSQLSelect.rb

Writing SQL-92 queries to Spark allows you to quickly and easily incorporate Spark data into your own JRuby applications. Download a free trial today!