Skip to main content

Writing Tests for a Kafka Producer

Hands-On Lab

 

Photo of Will Boyd

Will Boyd

DevOps Team Lead in Content

Length

00:45:00

Difficulty

Intermediate

Unit testing is an important part of developing software with good practices, and this even applies to your custom Kafka producer code. Luckily, Kafka offers test fixtures that can help you easily write such tests for your Kafka producers. In this lab, we will work with Kafka's producer test fixtures by building a few unit tests for some existing producer code.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Writing Tests for a Kafka Producer

Introduction

Unit testing is an important part of developing software with good practices, and this even applies to your custom Kafka producer code. Luckily, Kafka offers test fixtures that can help you easily write such tests for your Kafka producers. In this lab, we will work with Kafka's producer test fixtures by building a few unit tests for some existing producer code.

Solution

Log in to the lab server using the credentials provided on the hands-on lab page:

ssh cloud_user@PUBLIC_IP_ADDRESS

Clone the Starter Project from GitHub and Perform a Test Run

  1. Clone the starter project from GitHub (first changing to the home directory if you aren't already in it):

    cd ~/
    git clone https://github.com/linuxacademy/content-ccdak-producer-tests-lab.git
  2. Change to the new content-ccdak-producer-tests-lab directory:

    cd content-ccdak-producer-tests-lab
  3. Perform a test run to make sure the code is able to compile and run:

    ./gradlew test

    The code should compile, but the tests should fail since they are not implemented yet.

Implement the Unit Tests for the MemberSignupsProducer

  1. Edit the test class for MemberSignupsProducer:

    vi src/test/java/com/linuxacademy/ccdak/producer/MemberSignupsProducerTest.java
  2. Implement the testHandleMemberSignup_sent_data test:

    @Test
    public void testHandleMemberSignup_sent_data() {
        // Perform a simple test to verify that the producer sends the correct data to the correct topic when handleMemberSignup is called.
        // Verify that the published record has the memberId as the key and the uppercased name as the value.
        // Verify that the records is sent to the member_signups topic.
        memberSignupsProducer.handleMemberSignup(1, "Summers, Buffy");
    
        mockProducer.completeNext();
    
        List<ProducerRecord<Integer, String>> records = mockProducer.history();
        Assert.assertEquals(1, records.size());
        ProducerRecord<Integer, String> record = records.get(0);
        Assert.assertEquals(Integer.valueOf(1), record.key());
        Assert.assertEquals("SUMMERS, BUFFY", record.value());
        Assert.assertEquals("member_signups", record.topic());    
    }
  3. Implement the testHandleMemberSignup_partitioning test:

    @Test
    public void testHandleMemberSignup_partitioning() {
        // Verify that records with a value starting with A-M are assigned to partition 0, and that others are assigned to partition 1.
        // You can send two records in this test, one with a value that begins with A-M and the other that begins with N-Z.
        memberSignupsProducer.handleMemberSignup(1, "M");
        memberSignupsProducer.handleMemberSignup(1, "N");
    
        mockProducer.completeNext();
        mockProducer.completeNext();
    
        List<ProducerRecord<Integer, String>> records = mockProducer.history();
        Assert.assertEquals(2, records.size());
        ProducerRecord<Integer, String> record1 = records.get(0);
        Assert.assertEquals(Integer.valueOf(0), record1.partition());
        ProducerRecord<Integer, String> record2 = records.get(1);
        Assert.assertEquals(Integer.valueOf(1), record2.partition());
    }
  4. Implement the testHandleMemberSignup_output test:

    @Test
    public void testHandleMemberSignup_output() {
        // Verify that the producer logs the record data to System.out.
        // A text fixture called systemOutContent has already been set up in this class to capture System.out data.
        memberSignupsProducer.handleMemberSignup(1, "Summers, Buffy");
    
        mockProducer.completeNext();
    
        Assert.assertEquals("key=1, value=SUMMERS, BUFFYn", systemOutContent.toString());
    }
  5. Implement the testHandleMemberSignup_error test:

    @Test
    public void testHandleMemberSignup_error() {
        // Verify that the producer logs the error message to System.err if an error occurs when seding a record.
        // A text fixture called systemErrContent has already been set up in this class to capture System.err data.
        memberSignupsProducer.handleMemberSignup(1, "Summers, Buffy");
    
        mockProducer.errorNext(new RuntimeException("test error"));
    
        Assert.assertEquals("test errorn", systemErrContent.toString());
    }
  6. Save and exit the file.

  7. Run your tests and make sure they pass:

    ./gradlew test

Conclusion

Congratulations on successfully completing this hands-on lab!