Automate Elasticsearch Integration Tasks from PowerShell

The CData Cmdlets for Elasticsearch are standard PowerShell cmdlets that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time and bidirectional access to Elasticsearch.

Getting Started

---

PowerShell Cmdlets or ADO.NET Provider?

The Cmdlets are not only a PowerShell interface to Elasticsearch, but also an SQL interface; this tutorial shows how to use both to create, retrieve, update, and delete Elasticsearch data. We also show examples of the ADO.NET equivalent, which is possible with the CData ADO.NET Provider for Elasticsearch. To access Elasticsearch data from other .NET applications, like LINQPad, use the CData ADO.NET Provider for Elasticsearch.

Once you have acquired the necessary connection properties, accessing Elasticsearch data in PowerShell can be enabled in three steps.

Set the Server and Port connection properties to connect. To authenticate, set the User and Password properties, PKI (public key infrastructure) properties, or both. To use PKI, set the SSLClientCert, SSLClientCertType, SSLClientCertSubject, and SSLClientCertPassword properties.

The data provider uses X-Pack Security for TLS/SSL and authentication. To connect over TLS/SSL, prefix the Server value with 'https://'. Note: TLS/SSL and client authentication must be enabled on X-Pack to use PKI.

Once the data provider is connected, X-Pack will then perform user authentication and grant role permissions based on the realms you have configured.

PowerShell

1. Install the module:

   Install-Module ElasticsearchCmdlets

2. Connect:

   $elasticsearch = Connect-Elasticsearch -Server "$Server" -Port "$Port" -User "$User" -Password "$Password"
   

3. Search for and retrieve data:

   $shipcity = "New York"
   $orders = Select-Elasticsearch -Connection $elasticsearch -Table "Orders" -Where "ShipCity = `'$ShipCity`'"
   $orders
   

   You can also use the Invoke-Elasticsearch cmdlet to execute SQL commands:

   $orders = Invoke-Elasticsearch -Connection $elasticsearch -Query 'SELECT * FROM Orders WHERE ShipCity = @ShipCity' -Params @{'@ShipCity'='New York'}
   

ADO.NET

1. Load the provider's assembly:

   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for Elasticsearch\lib\System.Data.CData.Elasticsearch.dll")
    

2. Connect to Elasticsearch:

    
   $conn= New-Object System.Data.CData.Elasticsearch.ElasticsearchConnection("Server=127.0.0.1;Port=9200;User=admin;Password=123456;")
   $conn.Open()
   

3. Instantiate the ElasticsearchDataAdapter, execute an SQL query, and output the results:

   $sql="SELECT OrderName, Freight from Orders"
   
   $da= New-Object System.Data.CData.Elasticsearch.ElasticsearchDataAdapter($sql, $conn)
   $dt= New-Object System.Data.DataTable
   $da.Fill($dt)
   
   $dt.Rows | foreach {
   	Write-Host $_.ordername $_.freight
   }
    

Update Elasticsearch Data

PowerShell

Update-Elasticsearch -Connection $Elasticsearch -Columns @('OrderName','Freight') -Values @('MyOrderName', 'MyFreight') -Table Orders -Id "MyId"

ADO.NET

$cmd = New-Object System.Data.CData.Elasticsearch.ElasticsearchCommand("UPDATE Orders SET ShipCity='New York' WHERE Id = @myId", $conn)
$cmd.Parameters.Add((New-Object System.Data.CData.Elasticsearch.ElasticsearchParameter("@myId","10456255-0015501366")))
$cmd.ExecuteNonQuery()

Insert Elasticsearch Data

PowerShell

Add-Elasticsearch -Connection $Elasticsearch -Table Orders -Columns @("OrderName", "Freight") -Values @("MyOrderName", "MyFreight") 

ADO.NET

$cmd = New-Object System.Data.CData.Elasticsearch.ElasticsearchCommand("INSERT INTO Orders (ShipCity) VALUES (@myShipCity)", $conn)
$cmd.Parameters.Add((New-Object System.Data.CData.Elasticsearch.ElasticsearchParameter("@myShipCity","New York")))
$cmd.ExecuteNonQuery()

Delete Elasticsearch Data

PowerShell

Remove-Elasticsearch -Connection $Elasticsearch -Table "Orders" -Id "MyId"

ADO.NET

$cmd = New-Object System.Data.CData.Elasticsearch.ElasticsearchCommand("DELETE FROM Orders WHERE Id=@myId", $conn)
$cmd.Parameters.Add((New-Object System.Data.CData.Elasticsearch.ElasticsearchParameter("@myId","001d000000YBRseAAH")))
$cmd.ExecuteNonQuery()

CodeProject