Automate CSV Integration Tasks from PowerShell

The CData Cmdlets for CSV are standard PowerShell cmdlets that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time access to CSV.

PowerShell Cmdlets or ADO.NET Provider?

The Cmdlets are not only a PowerShell interface to CSV, but also an SQL interface; this tutorial shows how to use both to retrieve CSV data. We also show examples of the ADO.NET equivalent, which is possible with the CData ADO.NET Provider for CSV. To access CSV data from other .NET applications, like LINQPad, use the CData ADO.NET Provider for CSV.

Once you have acquired the necessary connection properties, accessing CSV data in PowerShell can be enabled in three steps.

Connecting to Local or Cloud-Stored (Box, Google Drive, Amazon S3, SharePoint) CSV Files

CData Drivers let you work with CSV files stored locally and stored in cloud storage services like Box, Amazon S3, Google Drive, or SharePoint, right where they are.

Setting connection properties for local files

Set the URI property to local folder path.

Setting connection properties for files stored in Amazon S3

To connect to CSV file(s) within Amazon S3, set the URI property to the URI of the Bucket and Folder where the intended CSV files exist. In addition, at least set these properties:

• AWSAccessKey: AWS Access Key (username)
• AWSSecretKey: AWS Secret Key

Setting connection properties for files stored in Box

To connect to CSV file(s) within Box, set the URI property to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect to Box.

Dropbox

To connect to CSV file(s) within Dropbox, set the URI proprerty to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect to Dropbox. Either User Account or Service Account can be used to authenticate.

SharePoint Online (SOAP)

To connect to CSV file(s) within SharePoint with SOAP Schema, set the URI proprerty to the URI of the document library that includes the intended CSV file. Set User, Password, and StorageBaseURL.

SharePoint Online REST

To connect to CSV file(s) within SharePoint with REST Schema, set the URI proprerty to the URI of the document library that includes the intended CSV file. StorageBaseURL is optional. If not set, the driver will use the root drive. OAuth is used to authenticate.

Google Drive

To connect to CSV file(s) within Google Drive, set the URI property to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect and set InitiateOAuth to GETANDREFRESH.

PowerShell

1. Install the module:

   Install-Module CSVCmdlets

2. Connect:

   $csv = Connect-CSV -URI "$URI"
   

3. Search for and retrieve data:

   $firstname = "Bob"
   $customer = Select-CSV -Connection $csv -Table "Customer" -Where "FirstName = `'$FirstName`'"
   $customer
   

   You can also use the Invoke-CSV cmdlet to execute SQL commands:

   $customer = Invoke-CSV -Connection $csv -Query 'SELECT * FROM Customer WHERE FirstName = @FirstName' -Params @{'@FirstName'='Bob'}
   

ADO.NET

1. Load the provider's assembly:

   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for CSV\lib\System.Data.CData.CSV.dll")
    

2. Connect to CSV:

    
   $conn= New-Object System.Data.CData.CSV.CSVConnection("URI=/PATH/TO/MyCSVFilesFolder;")
   $conn.Open()
   

3. Instantiate the CSVDataAdapter, execute an SQL query, and output the results:

   $sql="SELECT City, TotalDue from Customer"
   
   $da= New-Object System.Data.CData.CSV.CSVDataAdapter($sql, $conn)
   $dt= New-Object System.Data.DataTable
   $da.Fill($dt)
   
   $dt.Rows | foreach {
   	Write-Host $_.city $_.totaldue
   }
    

CodeProject