Build Data Flows from CSV to SQL Server using SSIS

Using SQL Server as a backup for critical business data provides an essential safety net against loss. Backing up data to SQL Server enables business users to more easily connect that data with features like reporting, analytics, and more.

This example demonstrates how to use the CData SSIS Tasks for CSV inside of a SQL Server SSIS workflow to transfer CSV data into a Microsoft SQL Server database.

Add the Components

To get started, add a new CSV source and SQL Server ADO.NET destination to a new data flow task.

Create a New Connection Manager

Follow the steps below to save CSV connection properties in a connection manager.

1. In the Connection Manager window, right-click and then click New Connection. The Add SSIS Connection Manager dialog is displayed.
2. In the Connection Manager type menu, select CSV. The CData CSV Connection Manager is displayed.
3. Configure connection properties.

   Connecting to Local or Cloud-Stored (Box, Google Drive, Amazon S3, SharePoint) CSV Files

   CData Drivers let you work with CSV files stored locally and stored in cloud storage services like Box, Amazon S3, Google Drive, or SharePoint, right where they are.

   Setting connection properties for local files

   Set the URI property to local folder path.

   Setting connection properties for files stored in Amazon S3

   To connect to CSV file(s) within Amazon S3, set the URI property to the URI of the Bucket and Folder where the intended CSV files exist. In addition, at least set these properties:

   • AWSAccessKey: AWS Access Key (username)
   • AWSSecretKey: AWS Secret Key

   Setting connection properties for files stored in Box

   To connect to CSV file(s) within Box, set the URI property to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect to Box.

   Dropbox

   To connect to CSV file(s) within Dropbox, set the URI proprerty to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect to Dropbox. Either User Account or Service Account can be used to authenticate.

   SharePoint Online (SOAP)

   To connect to CSV file(s) within SharePoint with SOAP Schema, set the URI proprerty to the URI of the document library that includes the intended CSV file. Set User, Password, and StorageBaseURL.

   SharePoint Online REST

   To connect to CSV file(s) within SharePoint with REST Schema, set the URI proprerty to the URI of the document library that includes the intended CSV file. StorageBaseURL is optional. If not set, the driver will use the root drive. OAuth is used to authenticate.

   Google Drive

   To connect to CSV file(s) within Google Drive, set the URI property to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect and set InitiateOAuth to GETANDREFRESH.

   👁 Configuring a connection (Salesforce is shown).
   

Configure the CSV Source

Follow the steps below to specify the query to be used to extract CSV data.

1. Double-click the CSV source to open the source component editor.
2. In the Connection Manager menu, select the connection manager previously created.
3. Specify the query to use for the data extraction. For example:

   SELECT City, SUM(TotalDue) FROM Customer GROUP BY City
   

   👁 The SQL query to retrieve records. (Salesforce is shown.)
   
4. Close the CSV Source control and connect it to the ADO.NET Destination.

Configure the SQL Server Destination

Follow the steps below to specify the SQL server table to load the CSV data into.

1. Open the ADO.NET Destination and add a New Connection. Enter your server and database information here.
2. In the Data access mode menu, select "table or view".
3. In the Table Or View menu, select the table or view to populate.
4. Configure any properties you wish to on the Mappings screen. 👁 The mappings from the SSIS source component to SQL Server. (Salesforce is shown.)
   

Run the Project

You can now run the project. After the SSIS Task has finished executing, your database will be populated with CSV data.