Skip to main content

Configuring DynamoDB Streams

Hands-On Lab

 

Photo of

Training Architect

Length

01:00:00

Difficulty

Intermediate

DynamoDB is one of the premiere hosted features of the AWS ecosystem, but it isn't capable of everything. Sometimes, the data needs to be moved to another destination, such as a relational database, in order to perform necessary analytics on the data because joins and other complex operations are not possible. In this hands-on lab, we will configure a DynamodDB stream to output DynamoDB data to a Lambda function that will then insert the data into a MySQL database on EC2.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Configuring DynamoDB Streams

Introduction

In this hands-on lab, we will configure a DynamodDB stream to output DynamoDB data to a Lambda function that will then insert the data into a MySQL database on EC2.

Solution

Log in to the live AWS environment using the credentials provided. Make sure you're in the N. Virginia (us-east-1) region throughout the lab.

Then, open a new browser tab and log in to the Jupyter notebook using the provided credentials. Once you're logged in, open the dynamodbStreams.ipynb notebook.

Create a DynamoDB Table

  1. In the dynamodbStreams Jupyter notebook, run the command listed in the first cell.
  2. In the second cell, add the #endpoint and #lambdaRole values provided on the lab page.
  3. Once they're replaced, uncomment them by deleting the # before endpoint and lambdaRole.
  4. Run the code in that cell.
  5. Run the code in the next two cells to establish a connection to the server and create the database.
  6. Run the next cell to ensure the database was created.
  7. Run the next three cells to connect to the database and create the table.
  8. Run the next two cells to add a sample record and ensure the database is set up properly.
  9. Run the next cell to create the DynamoDB table. It might take a few minutes — wait until the * in the brackets turns into a number.
  10. Once it's finished creating, navigate to DynamoDB in the AWS console.
  11. Click Tables in the left-hand menu to see our listed table.

Create a Lambda Function

  1. In the Jupyter notebook, run the next cell to create the Lambda function.

  2. Back in the AWS console, navigate to Lambda.

  3. Click our listed ddbStream function.

  4. In the Configuration section, select DynamoDB from the Designer list.

  5. In the Configure triggers section, set the following values:

    • DynamoDB table: Movies
    • Batch size: 1
    • Starting position: Latest
    • Enable trigger: Check
  6. Click Add.

  7. Click Save.

  8. Back in the Jupyter notebook, run the next two cells to add test items.

  9. In the AWS console, navigate back to DynamoDB, and select our listed table.

  10. Click the Items tab to see the items we've added.

  11. In the Jupyter notebook, run the next cell to ensure the data streamed.

  12. Run the next cell to add 100 records to the DynamoDB table.

  13. Run the next cell to do a specific query in the relational database.

Conclusion

Congratulations on completing this hands-on lab!