Kafdrop | Tools And Frameworks

Introduction

Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. It offers a convenient way to monitor the health of your Kafka cluster and analyze messages in real-time. This tutorial will guide you through the installation, configuration, and usage of Kafdrop.

Prerequisites

Before you start, ensure you have the following:

A running Kafka cluster
Java 8 or higher installed on your machine
Docker (optional, for containerized deployment)

Installation

Using Docker

The easiest way to get Kafdrop up and running is by using Docker. Use the following command to pull the Kafdrop Docker image:

docker pull obsidiandynamics/kafdrop

Then, run the Docker container with the command:

docker run -d -p 9000:9000 -e KAFKA_BROKERCONNECT=your_kafka_broker:9092 obsidiandynamics/kafdrop

Replace your_kafka_broker:9092 with your Kafka broker's address.

Running from JAR file

You can also run Kafdrop directly from a JAR file. First, download the latest JAR file from the Kafdrop Releases Page. Then, run the JAR file with the following command:

java -jar kafdrop-.jar --kafka.brokerConnect=your_kafka_broker:9092

Replace <version> with the version of the JAR file you downloaded and your_kafka_broker:9092 with your Kafka broker's address.

Configuration

Kafdrop can be configured using environment variables or command-line arguments. Here are some of the most common configuration options:

KAFKA_BROKERCONNECT: Comma-separated list of Kafka brokers (e.g., broker1:9092,broker2:9092)
SERVER_PORT: Port for the Kafdrop web UI (default is 9000)
SPRING_PROFILES_ACTIVE: Spring profiles to activate (e.g., prod)

For example, to run Kafdrop with a different port, you can use:

docker run -d -p 8080:9000 -e KAFKA_BROKERCONNECT=your_kafka_broker:9092 -e SERVER_PORT=8080 obsidiandynamics/kafdrop

Using Kafdrop

Once Kafdrop is running, you can access the web UI by navigating to http://localhost:9000 (or the port you configured) in your web browser.

Viewing Topics

The Topics view provides a list of all topics in your Kafka cluster. You can see the number of partitions, replication factor, and current offsets for each topic.

Example:
Topic Name: example-topic
Partitions: 3
Replication Factor: 2
Current Offsets: 0, 1, 2

Browsing Consumer Groups

The Consumer Groups view lists all consumer groups and their current offsets. This helps you monitor consumer lag and identify any issues with message consumption.

Example:
Consumer Group: example-group
Topic: example-topic
Partition: 0
Current Offset: 10
Lag: 2

Viewing Messages

You can view messages for a specific topic and partition. This is useful for debugging and analyzing message content.

http://localhost:9000/topic/example-topic/partition/0

Example Message:
Offset: 0
Key: null
Value: {"message": "Hello, World!"}
Timestamp: 2023-10-01T12:00:00Z

Conclusion

Kafdrop provides a powerful and convenient way to monitor and troubleshoot your Kafka cluster. By following this tutorial, you should be able to install, configure, and use Kafdrop effectively. For more advanced configurations and features, refer to the Kafdrop GitHub Repository.

Kafdrop Tutorial