Spring Boot Kafka Producer Example

Spring Boot provides seamless integration with Apache Kafka to build scalable event-driven applications. Using Spring for Apache Kafka, developers can easily produce and publish messages to Kafka topics with minimal configuration.

• Enables real-time message publishing between distributed systems.
• Simplifies Kafka integration using Spring Boot auto-configuration.

Prerequisite: Apache Kafka

Step-by-Step Implementation

Follow the steps below to create and run a Kafka Producer using Spring Boot.

Step 1: Create a Spring Boot Project

Go to Spring Initializr and create a new Spring Boot project.

Project Configuration:

• Project: Maven
• Language: Java
• Spring Boot Version: Latest stable version
• Group: com.amiya
• Artifact: Apache-kafka-producer-demo
• Packaging: Jar
• Java Version: 17 or higher

Add Dependencies:

Select the following dependencies:

• Spring Web
• Spring for Apache Kafka

Download the project, extract it, and open it in your IDE.

Step 2: Configure Kafka Properties

Open the application.properties file located in src/main/resources and add the following configuration.

server.port=8081
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=test-group
spring.kafka.consumer.auto-offset-reset=earliest

These properties configure the Kafka server connection and application port.

Step 3: Create Kafka Producer Controller

Create a controller class DemoController.java to send messages to a Kafka topic.

DemoController.java

Explanation:

• KafkaTemplate is used to send messages to Kafka topics.
• The send() method publishes the message to the specified topic.
• The REST endpoint /publish/{message} allows sending messages through a browser or API call.

Step 4: Start Apache Kafka Services

Before running the Spring Boot application, ensure that Kafka and Zookeeper servers are running.

Start Zookeeper Server

Run the following command:

C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

Zookeeper manages Kafka broker coordination.

Start Kafka Server

Run the following command:

C:\kafka>.\bin\windows\kafka-server-start.bat .\config\server.properties

This command starts the Kafka broker which handles message publishing and consumption.

Step 5: Listen to Kafka Messages

To monitor the messages published to the topic, run the following command:

C:\kafka>.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic NewTopic --from-beginning

This command listens to messages from the NewTopic topic.

Step 6: Run the Spring Boot Application

Run the main Spring Boot class:

SpringBootKafkaProducerApplication.java

You can start the application by:

• Right-click Project -> Run As -> Spring Boot App
• The application will start on:

http://localhost:8081

Step 7: Publish Messages to Kafka

Open a browser or API client and send a request:

http://localhost:8081/publish/GeeksforGeeks

As we have passed "GeeksforGeeks" here you can see we got "Published Successfully" in return. And in real-time you can see the message has been published on the server also. The streaming of the message is in real-time. 

Similarly, if we have passed "Hello World" here you can see we got "Published Successfully" in return. And in real-time you can see the message has been published on the server also.