VOOZH about

URL: https://www.geeksforgeeks.org/java/spring-boot-consume-message-through-kafka-save-into-elasticsearch-and-plot-into-grafana/

⇱ Spring Boot - Consume Message Through Kafka, Save into ElasticSearch, and Plot into Grafana - GeeksforGeeks

  • Courses
  • Tutorials
  • Interview Prep

Spring Boot - Consume Message Through Kafka, Save into ElasticSearch, and Plot into Grafana

Last Updated : 26 Mar, 2026

In modern applications, processing real-time data efficiently is key. This tutorial shows how to integrate Spring Boot with Kafka for message consumption, Elasticsearch for storing and indexing data, and Grafana for real-time visualization.

  • Consume messages from Kafka using Spring Boot.
  • Store and index consumed messages in Elasticsearch.
  • Visualize data in Grafana dashboards.

Why do we mention the version?

Version compatibility is of paramount importance in the case of Elastic and Spring Boot. If your elastic-search version doesn't match with the spring boot version or vice versa, then you face problems in configuring both. Below is the list of version compatibility : 

Spring Data Elasticsearch

Elastic-Search

Spring framework

Spring Boot

4.4.X

7.17.3

5.3.X

2.7.X

4.3.X

7.15.2

5.3.X

2.6.X

4.2.X

7.12.0

5.3.X

2.5.X

4.1.X

7.9.3

5.3.2

2.4.X

4.0.X

7.6.2

5.2.12

2.3.X

3.2.X

6.8.12

5.2.12

2.2.X

3.1.X

6.2.2

5.1.19

2.1.X

3.0.X

5.5.0

5.0.13

2.0.X

2.1.X

2.4.0

4.3.25

1.5.X

Steps to Implement Kafka Producer, Consumer with Elasticsearch and Grafana

Step 1: Download Required Tools

Download the required software:

  • Elastic-Search
  • Apache Kafka

Download and extract from your system.

pom.xml – Contains required dependencies.

application.properties

server.port=1234

Step 2: Create Kafka Producer Application

Create a spring boot project named Producer.ProducerApplication.java -> Main class to run the application.

ProducerApplication.class

Step 3: Create Configuration class.

In this class, we have provided the configuration of Kafka. KafkaProducerConfig.java -> Contains Kafka configuration.

KafkaProducerConfig.java

Step 4: Create User class

Model class where we store the user information.

User.java

Step 5: Create Service class

Create Service class to write the logic. Service class uses kafka template to send the data to the consumer.
Uses KafkaTemplate to send data to Kafka. 

KafkaService.java

Step 6: Create the controller class.

Create the Controller class. REST controller for sending API requests.

ProducerController.java

Create Kafka Consumer and Configure the ElastisSearch Application

Create another spring boot application named ElasticConsumer.

Step 1. Create Application class.

Create Application class ->Main class to run the consumer application.

ComsumerApplication.java

Step 2. Create Config class.

This class Contains Kafka consumer configuration.

KafkaConsumerConfig.java

Step 3: Create User class

Model class used for storing data in Elasticsearch.

Important annotations:

  • @Document : Defines the Elasticsearch index
  • @Id : Unique identifier for each document
  • @Field :Specifies field type inside the document

Step 4. Create Repository class

Repository interface used to perform operations with Elasticsearch.

KafkaUserRepository.java

Step 5. Create Service class

Service class that consumes Kafka messages and stores them in Elasticsearch.

  • This application receives messages from Kafka and saves them to Elasticsearch.

KafkaUserService.java

Step 6: Run Elasticsearch

Open Command Prompt and navigate to the Elasticsearch bin folder.

Run the following command:

elasticsearch.bat

After starting, open the browser and check:

http://localhost:9200

If Elasticsearch is running successfully, it will display cluster information in JSON format.

  • Elasticsearch must run before starting the consumer application.

Run Producer and ElasticConsumer Spring Application

Send JSON data using postman. Here data is sent by the Producer app to ElasticConsumer. And in the ElasticConsumer console data would be printed and saved into ElasticSearchDB in the form of JSON.

Producer App APIs:

  • Send single object -> http://localhost:1234/producer
  • Send list of objects -> http://localhost:1234/producerlist
👁 Image

ElasticConsumer app APIs

  • Fetch all records from elastic db -> localhost:8080/getElasticUserFromKafka

Grafana Dashboard

Grafana dashboard is running on http://localhots:3000. Watch the configuration video below.

Output

Output Video 1:

Output Video 2:

Some ElasticSearch APIs

  • To show the records of index -> http://localhost:9200/<index_name>/_search
  • To Delete index -> http://localhost:9200/<index_name>
  • List all indices -> http://localhost:9200/_cat/indices
  • Show schema of index -> http://localhost:9200/<index_name>
Comment
Article Tags: