LINQ to Kafka Data

This article illustrates using LINQ to access tables within the Kafka via the CData ADO.NET Data Provider for Kafka. To achieve this, we will use LINQ to Entity Framework, which facilitates the generation of connections and can be seamlessly employed with any CData ADO.NET Data Providers to access data through LINQ.

See the help documentation for a guide to setting up an EF 6 project to use the provider.

1. In a new project in Visual Studio, right-click on the project and choose to add a new item. Add an ADO.NET Entity Data Model.
2. Choose EF Designer from Database and click Next.
3. Add a new Data Connection, and change your data source type to "CData Kafka Data Source".

4. Enter your data source connection information.

   Set BootstrapServers and the Topic properties to specify the address of your Apache Kafka server, as well as the topic you would like to interact with.

   Authorization Mechanisms

   • SASL Plain: The User and Password properties should be specified. AuthScheme should be set to 'Plain'.
   • SASL SSL: The User and Password properties should be specified. AuthScheme should be set to 'Scram'. UseSSL should be set to true.
   • SSL: The SSLCert and SSLCertPassword properties should be specified. UseSSL should be set to true.
   • Kerberos: The User and Password properties should be specified. AuthScheme should be set to 'Kerberos'.

   You may be required to trust the server certificate. In such cases, specify the TrustStorePath and the TrustStorePassword if necessary.

   Below is a typical connection string:

   User=admin;Password=pass;BootStrapServers=https://localhost:9091;Topic=MyTopic;

👁 Required connection properties for the data source. (QuickBooks is shown.)

5. If saving your entity connection to App.Config, set an entity name. In this example we are setting ApacheKafkaEntities as our entity connection in App.Config.
6. Enter a model name and select any tables or views you would like to include in the model.

Using the entity you created, you can now perform select , update, delete, and insert commands. For example:

ApacheKafkaEntities context = new ApacheKafkaEntities();

var sampletable_1Query = from sampletable_1 in context.SampleTable_1
 select sampletable_1;

foreach (var result in sampletable_1Query) {
 Console.WriteLine("{0} {1} ", result.Id, result.Id);
}

See "LINQ and Entity Framework" chapter in the help documentation for example queries of the supported LINQ.