Integrate RabbitMQ Data in Your Informatica Cloud Instance

Informatica Cloud allows you to perform extract, transform, and load (ETL) tasks in the cloud. With the Cloud Secure Agent and the CData API Driver for JDBC, you get live access to RabbitMQ data, directly within Informatica Cloud. In this article, we will walk through downloading and registering the Cloud Secure Agent, connecting to RabbitMQ through the JDBC Driver and generating a mapping that can be used in any Informatica Cloud process.

Informatica Cloud Secure Agent

To work with the RabbitMQ data through the JDBC Driver, install the Cloud Secure Agent.

1. Navigate to the Administrator page in Informatica Cloud 👁 Image
   
2. Select the Runtime Environments tab
3. Click "Download Secure Agent" 👁 Image
   
4. Make note of the Install Token 👁 Image
   
5. Run the installer on the client machine and register the Cloud Secure Agent with your username and install token 👁 Image
   

NOTE: It may take some time for all of the Cloud Secure Agent services to get up and running.

Connecting to the RabbitMQ JDBC Driver

With the Cloud Secure Agent installed and running, you are ready to connect to RabbitMQ through the JDBC Driver.

Adding the JDBC Driver to the Secure Agent Machine

1. Navigate to the following directory on the Secure Agent machine:

   %Secure Agent installation directory%/ext/connectors/thirdparty/

2. Create a folder and add the driver JAR file (cdata.jdbc.api.jar) based on the type of mapping that you want to configure.

   For mappings, create the following folder and add the driver JAR file:

   informatica.jdbc_v2/common

   For mappings in advanced mode, also create the following folder and add the driver JAR file:

   informatica.jdbc_v2/spark

3. Restart the Secure Agent.

Connecting to RabbitMQ in Informatica Cloud

After installing the driver JAR file, you are ready to configure your connection to RabbitMQ in Informatica Cloud. Start by clicking the Connections tab and clicking New Connection. Fill in the following properties for the connection:

• Connection Name: Name your connection (i.e.: CData RabbitMQ Connection)
• Type: Select "JDBC_V2"
• Runtime Environment: Select the runtime environment where you installed the Secure Agent
• JDBC Driver Class Name: The name of the JDBC driver class: cdata.jdbc.api.APIDriver
• JDBC Connection URL: Set this to the JDBC URL for RabbitMQ. Your URL will look similar to the following:
  
  

  jdbc:api:Profile=C:\profiles\\RabbitMQ.apip;AuthScheme=Basic;URL=http://localhost:15672;User=guest;Password=guest;

  About RabbitMQ Management HTTP API

  RabbitMQ is an open-source message broker that supports multiple messaging protocols. The RabbitMQ Management HTTP API provides HTTP-based access to management and monitoring data for a RabbitMQ server. The API exposes information about virtual hosts, exchanges, queues, bindings, connections, channels, consumers, users, permissions, policies, and cluster-wide statistics.

  The Management plugin must be enabled on the RabbitMQ server for the HTTP API to be available. By default, the management interface listens on port 15672.

  Using Basic Authentication

  RabbitMQ Management HTTP API uses HTTP Basic authentication. You must supply the username and password of a RabbitMQ management user.

  To enable access to the management API:

  1. Ensure the RabbitMQ Management plugin is enabled on your server (rabbitmq-plugins enable rabbitmq_management).
  2. Use an existing management user or create one with the appropriate management tag (management, policymaker, monitoring, or administrator).
  3. Note the full base URL of your RabbitMQ Management HTTP API (e.g., http://localhost:15672).

  After configuring your RabbitMQ server, set the following connection properties to connect:

  • AuthScheme: Set this to Basic.
  • URL: Set this to the base URL of your RabbitMQ Management HTTP API (e.g., http://localhost:15672).
  • User: Set this to your RabbitMQ management username (e.g., guest).
  • Password: Set this to your RabbitMQ management password.

  Example connection string:

  Profile=C:\profiles\RabbitMQ.apip;AuthScheme=Basic;URL=http://localhost:15672;User=guest;Password=guest;
  

  Available Tables

  The RabbitMQ profile provides access to the following tables:

  • Overview - Cluster-wide statistics and information about the RabbitMQ node
  • Nodes - Information about individual nodes in the RabbitMQ cluster
  • NodeMemory - Detailed memory usage breakdown for a specific cluster node
  • Connections - List of all open AMQP connections to the broker
  • Channels - List of all open AMQP channels across all connections
  • Consumers - List of all consumers registered across all queues
  • Exchanges - List of exchanges declared across all virtual hosts
  • Queues - List of queues declared across all virtual hosts
  • Bindings - List of all bindings between exchanges and queues
  • VirtualHosts - List of virtual hosts configured on the broker
  • VhostPermissions - User permissions within a specific virtual host
  • Users - List of all RabbitMQ users
  • Permissions - Permission records for all users across all virtual hosts
  • TopicPermissions - Topic-level permission records for all users
  • Policies - List of policies applied to queues and exchanges in virtual hosts
  • OperatorPolicies - List of operator policies applied to queues in virtual hosts
  • Parameters - List of component parameters (e.g., federation, shovel) per virtual host
  • GlobalParameters - List of global parameters that apply across all virtual hosts
  • VhostLimits - Resource limits configured for specific virtual hosts
  • UserLimits - Resource limits configured for specific users
  • FeatureFlags - List of feature flags and their enabled/disabled state on the node
  • DeprecatedFeatures - List of deprecated features and their usage state
  • AuthAttempts - Authentication attempt statistics for the node
  • ClusterName - The name of the RabbitMQ cluster
  • WhoAmI - Information about the currently authenticated management user
  • ExchangeBindingsSource - Bindings for which a specific exchange is the source
  • ExchangeBindingsDestination - Bindings for which a specific exchange is the destination
  • QueueBindings - Bindings for a specific queue within a virtual host

  Built-In Connection String Designer

  For assistance in constructing the JDBC URL, use the connection string designer built into the RabbitMQ JDBC Driver. Either double-click the .jar file or execute the .jar file from the command-line.

   java -jar cdata.jdbc.api.jar
   

  Fill in the connection properties and copy the connection string to the clipboard.

  👁 Using the built-in connection string designer to generate a JDBC URL (Google Sheets is shown.)
  
• Username: Set this to the username for RabbitMQ
• Password: Set this to the password for RabbitMQ

👁 Configuring the Connection (Google Sheets is shown.

Create a Mapping for RabbitMQ Data

With the connection to RabbitMQ configured, we can now access RabbitMQ data in any Informatica process. The steps below walk through creating a mapping for RabbitMQ to another data target.

1. Navigate to the Data Integration page 👁 Image
   
2. Click New.. and select Mapping from the Mappings tab 👁 Image
   
3. Click the Source Object and in the Source tab, select the Connection and set the Source Type 👁 Selecting the Source Connection and Source Type
   
4. Click "Select" to choose the table to map 👁 Selecting the Source Object
   
5. In the Fields tab, select the fields from the RabbitMQ table to map 👁 Selecting Source Fields to map
   
6. Click the Target object and configure the Target source, table and fields. In the Field Mapping tab, map the source fields to the target fields. 👁 Selecting the Target Field Mappings
   

With the mapping configured, you are ready to start integrating live RabbitMQ data with any of the supported connections in Informatica Cloud. Download a free, 30-day trial of the CData API Driver for JDBC and start working with your live RabbitMQ data in Informatica Cloud today.