Listing Kafka Topics

Kafka topics are the categories that organize communications. Each topic has a unique name across the Kafka cluster. Messages are sent and viewed from certain topics. In other words, producers add data to topics, while consumers read data from them. Kafka topics support multiple subscribers.

What is Kafka topic?

Kafka Topics are Virtual Groups or Logs that store messages and events in logical order, allowing users to easily transmit and receive data between Kafka Servers. When a Producer transmits messages or events to a given Kafka Topic, the topics add the messages one after the other, resulting in a log file. Furthermore, producers can push messages into the tail of these newly formed logs, whereas consumers can pull messages from a specified Kafka topic.

Where are Kafka topics stored?

Kafka topics are stored in the cluster. Specifically, each Kafka partition associated with a topic is an ordered, immutable sequence of records that is constantly added to. These partitions are spread throughout the Kafka cluster's servers, with each server allocated a specific number of partitions.

Partitions for Kafka topics are actually saved as log files on the Kafka brokers' filesystems. Each log file only carries data for one partition.

How do you create a Kafka topic?

• If you are using Kafka v2.2+ then use the Kafka hostname and port, such as localhost:9095.
• For a previous version of Kafka, use the Zookeeper URL and port, such as localhost:2181.
• Provide the following required parameters: topic name, number of partitions, and replication factor.
• Use the kafka-topics.sh command with the --create option.

Implementation of Listing Kafka Topics

Step 1: Navigate to Kafka Installation Directory

Locate the installation directory for Kafka by opening a terminal or command prompt and going there. For instance:

cd /path/to/kafka

Step 2: Using Command-Line Tool

Open your terminal of your PC and locate your Kafka installation’s bin directory then the below command, replacing <broker-address> with the address of one of your Kafka brokers.

$ ./kafka-topics.sh --list --bootstrap-server <broker-address>

Step 3: Create the bootstrap server for a listing

The single-instance Kafka cluster uses the 9095 port, thus we set "localhost:9092" as the bootstrap server. Simply said, bootstrap servers function as Kafka brokers.

$ ./bin/kafka-topics.sh --list

OUTPUT

Step 4: Execute Topic Details

Once you have a list of themes, we can look into the specifics of one of them. To accomplish this, we can use the "-describe -topic <topic name>" options:

$ ./bin/kafka-topics.sh --bootstrap-server=localhost:9095 --describe --topic users.registrations

OUTPUT