Skip to main content

Working with Kafka from the Command Line

Hands-On Lab

 

Photo of Will Boyd

Will Boyd

DevOps Team Lead in Content

Length

00:30:00

Difficulty

Intermediate

In this lesson, we will have the opportunity to work with a real Kafka cluster using the command line tools provided through Kafka. Ultimately, Kafka makes it easy to build applications that interact with the Kafka cluster, but it also provides some command line tools. We can use these to create ad-hoc producers and consumers, plus perform a variety of administrative tasks that require you to interact with the cluster.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Working with Kafka from the Command Line

Introduction

In this hands-on lab, we will create a topic to store inventory data using the command-line tools provided through Kafka.

Solution

Begin by logging in to the lab servers using the credentials provided on the hands-on lab page:

  ssh cloud_user@PUBLIC_IP_ADDRESS

Create a Kafka Topic for the Inventory Purchase Data

  1. Create the topic using the kafka-topics command:

    kafka-topics --create --bootstrap-server localhost:9092 --replication-factor 3 --partitions 6 --topic inventory_purchases

    Note: After the creation command is ran, we may encounter a warning indicating that we should simply avoid creating another topic with a '.' called inventory.purchases.

Test the Setup by Publishing and Consuming Some Data

  1. Start a command line producer:

    kafka-console-producer --broker-list localhost:9092 --topic inventory_purchases
  2. Develop a few lines of data that can be used for testing purposes. Since we are working with merely test data, a specific format is not required. It could look like this:

    product: apples, quantity: 5
    product: lemons, quantity: 7
  3. Once the test images are published, we can exit the producer.

  4. Start up a command line consumer:

    kafka-console-consumer --bootstrap-server localhost:9092 --topic inventory_purchases --from-beginning

    Note: The --from-beginning flag is used because we want to target our test messages.

  5. We should see the test messages that were published earlier:

    product: apples, quantity: 5
    product: lemons, quantity: 7

Conclusion

Congratulations - you've completed this hands-on lab!