Spring Boot | How to publish JSON messages on Apache Kafka

Apache Kafka is a distributed messaging system used for building real-time data pipelines and streaming applications. Spring Boot provides seamless integration with Kafka, allowing applications to publish structured messages such as JSON objects.

• Enables sending structured JSON data between distributed services.
• Simplifies Kafka integration using Spring Boot and Spring for Apache Kafka.

Step-by-Step Implementation

Follow the steps below to publish JSON messages to Apache Kafka using Spring Boot.

Step 1: Create a Spring Boot Project

Go to Spring Initializr and create a new Spring Boot project.

Project Configuration:

• Project: Maven
• Language: Java
• Spring Boot Version: Latest stable version
• Group: com.gfg
• Artifact: springboot-kafka-json
• Packaging: Jar
• Java Version: 17 or higher

Add Dependencies:

Select the following dependencies:

• Spring Web
• Spring for Apache Kafka

Download the project, extract it, and open it in your IDE.

Step 2: Create a Model Class

Create a Student model class that represents the JSON message we want to send to Kafka.

Student.java

Step 3: Create Kafka Configuration Class

Create a configuration class StudentConfig.java to configure Kafka producer settings and enable JSON serialization.

StudentConfig.java

Step 4: Create Controller Class

Create a REST controller that sends JSON messages to a Kafka topic.

UserResource.java

Step 5: Start Kafka and Zookeeper

Before running the application, ensure Kafka and Zookeeper servers are running.

Start Zookeeper

C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

Start Kafka Server

C:\kafka>.\bin\windows\kafka-server-start.bat .\config\server.properties

Step 6: Create Kafka Topic

Create a topic named StudentExample.

Windows:

.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic StudentExample

Mac / Linux:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic StudentExample

Step 7: Listen to Kafka Messages

To monitor messages published to Kafka, run the following command.

Windows:

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic StudentExample --from-beginning

Mac / Linux:

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic StudentExample --from-beginning

Step 8: Run the Application and Test API

Run the Spring Boot application and open the following URL in a browser:

http://localhost:8080/gfg/publish/1/John/Doe

Output:👁 Image

Checking the message in real time: 👁 Image