Skip to main content

Amazon DynamoDB Deep Dive

Course

Intro Video

Photo of Mark Richman

Mark Richman

AWS Training Architect II in Content

Length

15:21:32

Difficulty

Intermediate

Videos

54

Hands-on Labs

11

Course Details

Welcome to the Linux Academy Amazon DynamoDB Deep Dive course. This course will help you master DynamoDB!

In this course, you will learn the basics of DynamoDB, and how it differs from traditional relational database management systems. A real-world scenario project will help guide you through each of the concepts presented.

This course is intended for all skill levels, even if you're totally new to database development.

By the end of this course, you'll have the necessary skills and knowledge to effectively build scalable, high performance applications using DynamoDB.

Thank you for taking the course...let's get started!

Syllabus

Course Introduction

Getting Started

Course Introduction

00:01:44

Lesson Description:

Welcome to this Linux Academy course - Amazon DynamoDB Deep Dive

About the Training Architect

00:00:58

Lesson Description:

Get to know a little bit more about me, the instructor for this course!

Course Features and Tools

00:03:30

Lesson Description:

Let's talk about features and tools that you will find helpful during the course.

Databases 101

Databases 101

Database Management Systems

00:11:03

Lesson Description:

In this lesson, we get a bit of a history lesson on databases and introduce some foundational information on relational databases, as well as some key terminology.

Relational DBs and SQL

00:11:43

Lesson Description:

In this video, we discuss the ACID properties of relational databases, the pros and cons of scaling databases using both replication and sharding, and Brewer's CAP theorem. Further Reading Principles of Transaction-Oriented Database Recovery Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services

NoSQL Database Models

00:11:54

Lesson Description:

In this lesson, we get an introduction to NoSQL databases, how they differ from SQL databases, and what problems they solve. We explore the various NoSQL engine types and cover the BASE properties of NoSQL databases. Finally, we discuss how scaling NoSQL databases works. Further Reading Base: An Acid Alternative by Dan Pritchett, Ebay An Unorthodox Approach to Database Design: The Coming of the Shard by Todd Hoff

DynamoDB and Scenario Introduction

DynamoDB and Scenario Introduction

DynamoDB 10,000 FT

00:06:49

Lesson Description:

In this video, we cover a brief history of DynamoDB and where it sits in the AWS product set. Further Reading The DynamoDB Paper

Introducing Pinehead Records

00:05:25

Lesson Description:

In this lesson, we review the sample application, "Pinehead Records," used throughout the course. We also review the migration plan, which evolves the application from MySQL to DynamoDB and beyond.

Scenario Discussion and Architecture Thoughts

00:09:14

Lesson Description:

In this lesson, we walk through the high-level application architecture, discussing some of its limitations. We also review some common objections to adopting DynamoDB. Course Repository: https://github.com/linuxacademy/content-dynamodb-deepdive

DynamoDB Fundamentals

Tables and Item Architecture

DynamoDB Architecture

00:07:34

Lesson Description:

In this video, we discuss DynamoDB architecture at the highest level. We cover how partitions work and define some basic concepts, such as tables and keys. Finally, we discuss the various capacity modes in DynamoDB.

DynamoDB Items

00:04:00

Lesson Description:

In this video, we continue our discussion of DynamoDB architecture, focusing on items and attributes. We cover item limits and the role of primary keys, as well as the various data types supported by DynamoDB.

Accessing DynamoDB

Using the AWS Management Console

00:13:31

Lesson Description:

In this video, we take a tour of the AWS Management Console features for DynamoDB.

Using the AWS CLI

00:10:09

Lesson Description:

This video introduces the AWS Command Line Interface (CLI). You can use the AWS CLI for various ad hoc operations, like creating and managing tables in DynamoDB. You can also use it to embed DynamoDB operations in your DevOps tooling. In this lesson, we use the Linux Academy Cloud Sandbox environment within the Cloud Playground. If you'd like to install the AWS CLI in your own environment, here are the instructions: https://docs.aws.amazon.com/en_pv/cli/latest/userguide/cli-chap-install.html Commands Used in This Lesson

aws dynamodb create-table 
    --table-name Music 
    --key-schema AttributeName=Artist,KeyType=HASH 
                 AttributeName=SongTitle,KeyType=RANGE     
    --attribute-definitions 
        AttributeName=Artist,AttributeType=S 
        AttributeName=SongTitle,AttributeType=S     
    --provisioned-throughput 
        ReadCapacityUnits=5,WriteCapacityUnits=5

aws dynamodb describe-table --table-name Music

aws dynamodb put-item 
    --table-name Music 
    --item '{
        "Artist": {"S": "Dream Theater"},
        "AlbumTitle": {"S": "Images and Words"},        
        "SongTitle": {"S": "Under a Glass Moon"} }'

aws dynamodb scan --table-name Music

SDK Options

00:07:36

Lesson Description:

This lesson covers the various AWS SDK options available for interacting with DynamoDB. Object persistence interface example: https://github.com/linuxacademy/content-dynamodb-deepdive/tree/master/4.2.3-SDK_Options/sdkoptions

DynamoDB Local

00:07:42

Lesson Description:

DynamoDB Local is a downloadable version of DynamoDB that enables developers to develop and test applications using a version of DynamoDB running in your own development environment. Download DynamoDB Local using one of the links on this page: https://docs.aws.amazon.com/en_pv/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Creating a DynamoDB Table

00:45:00

Monitoring and Metrics

Console Metrics and CloudWatch

00:05:29

Lesson Description:

CloudWatch is a network monitoring service that can be used to track various metrics. CloudWatch can help us answer various questions about our DynamoDB performance, such as: How can I determine how much of my provisioned throughput is being used?How can I determine which requests exceed the provisioned throughput limits of a table?How can I determine if any system errors occurred? DynamoDB Metrics and Dimensions: https://docs.aws.amazon.com/en_pv/amazondynamodb/latest/developerguide/metrics-dimensions.html

Alerts and Alarms

00:05:02

Lesson Description:

In this lesson, we cover CloudWatch alarms, which watch a single metric over a time period you specify and perform one or more actions based on the value of the metric relative to a given threshold over a number of time periods.

Errors and Codes

00:06:55

Lesson Description:

This video explains how the various AWS SDKs can automatically retry failed API calls using an exponential backoff algorithm. Exponential backoff example: https://github.com/linuxacademy/content-dynamodb-deepdive/blob/master/4.3-Monitoring/backoff.py Boto3 retry handler: https://github.com/boto/botocore/blob/c632931a6cc5eab4113c976d430bcb9c059f829f/botocore/retryhandler.py DynamoDB retry defaults: https://github.com/boto/botocore/blob/develop/botocore/data/_retry.json#L128 AWS Service Health Dashboard: https://status.aws.amazon.com/

Working with Items

Partitions, Partition and Sort Keys

00:05:47

Lesson Description:

This video covers how partition and sort keys are used to uniquely write and read items. The Consistent Hashing algorithm is also explained. Consistent Hashing: https://en.wikipedia.org/wiki/Consistent_hashing

Performance Units: RCU/WCU

00:05:45

Lesson Description:

This lesson covers Read Capacity Units (RCU) and Write Capacity Units (WCU) in detail.

Consistency Model (Strongly vs. Eventual)

00:02:23

Lesson Description:

This video covers the implementation details of DynamoDB's consistency model, including storage node availability and replication. Paxos: https://en.wikipedia.org/wiki/Paxos_(computer_science)

Scans and Queries

00:07:36

Lesson Description:

This lesson covers Scans and Queries in more detail, including their strengths and limitations. Scan Filtering results Sometimes you might need to write an expression containing an attribute name that conflicts with a DynamoDB reserved word. In this example, we replace name with an expression attribute name such as #n.

aws dynamodb scan --table-name artist 
    --filter-expression "#n = :name" 
    --expression-attribute-values '{":name":{"S":"Dream Theater"}}' 
    --expression-attribute-names '{"#n":"name"}'
Result:
{
    "Items": [
        {
            "id": {
                "N": "154555"
            },
            "name": {
                "S": "Dream Theater"
            }
        }
    ],
    "Count": 1,
    "ScannedCount": 973696,
    "ConsumedCapacity": null
}
Strongly Consistent Read
aws dynamodb scan --table-name artist 
    --filter-expression "#n = :name" 
    --expression-attribute-values '{":name":{"S":"Dream Theater"}}' 
    --expression-attribute-names '{"#n":"name"}' 
    --consistent-read
Limit the number of items returned
aws dynamodb scan --table-name track --limit 3
Result:
{
    "Items": [
        {
            "album_id": {
                "N": "1855173"
            },
            "length": {
                "N": "123000"
            },
            "id": {
                "N": "21662086"
            },
            "name": {
                "S": "Please Please Me"
            },
            "number": {
                "S": "A"
            }
        },
        {
            "album_id": {
                "N": "245766"
            },
            "length": {
                "N": "280000"
            },
            "id": {
                "N": "3424622"
            },
            "name": {
                "S": "Die rosarote Brille"
            },
            "number": {
                "N": "1"
            }
        },
        {
            "album_id": {
                "N": "1710199"
            },
            "length": {
                "N": "300000"
            },
            "id": {
                "N": "19937970"
            },
            "name": {
                "S": "Sofre"
            },
            "number": {
                "S": "B3"
            }
        }
    ],
    "Count": 3,
    "ScannedCount": 3,
    "LastEvaluatedKey": {
        "id": {
            "N": "19937970"
        }
    }
}
Pagination The LastEvaluatedKey in the response indicates that not all of the items have been retrieved. The AWS CLI then issues another Scan request to DynamoDB. This request and response pattern continues, until the final response.
aws dynamodb scan --table-name artist --page-size 10 --debug
Query Find items based on primary key values
aws dynamodb query 
    --table-name artist 
    --key-condition-expression "id = :id" 
    --expression-attribute-values  '{":id":{"N":"154555"}}'
Result:
{
    "Items": [
        {
            "id": {
                "N": "154555"
            },
            "name": {
                "S": "Dream Theater"
            }
        }
    ],
    "Count": 1,
    "ScannedCount": 1,
    "ConsumedCapacity": null
}

PutItem

00:02:32

Lesson Description:

In this video, we will discuss the various PutItem operations: PutItem, UpdateItem, and DeleteItem, along with an example of a Write Conflict.

Batch Operations

00:03:51

Lesson Description:

DynamoDB provides the ability to run multiple read and write operations as a single batch, improving performance with large-scale operations. In this lesson, we will cover the BatchGetItem and BatchWriteItem API calls.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Using a DynamoDB Table Basic IO

01:30:00

Table Performance

Provisioned vs. On-Demand Capacity Modes

00:05:21

Lesson Description:

In this video, we explain the different use cases for provisioned capacity vs. on-demand modes, as well as pricing scenarios for both.

Auto Scaling

00:07:18

Lesson Description:

DynamoDB auto scaling uses the AWS Application Auto Scaling service to dynamically adjust provisioned throughput capacity on your behalf, in response to actual traffic patterns. In this lesson, we cover how auto scaling works, how to configure it, and observe it in action. Code for This Lesson https://github.com/linuxacademy/content-dynamodb-deepdive/blob/master/4.5.2-Auto-Scaling/run.py

Data Model - Version 1

Version 1 Data Model Overview

00:07:13

Lesson Description:

In this video, we review what we refer to as our "Version 1" data model. Here, we have performed a naïve migration from MySQL to DynamoDB. To perform this migration on your own, follow these steps on an Amazon Linux 2 instance:

sudo yum install python3

pip3 install --user boto3 pymysql


# Set region and optionally set access keys (if not using an IAM role on the EC2 instance)
aws configure

curl https://raw.githubusercontent.com/linuxacademy/content-dynamodb-deepdive/master/labs/migration_v2.py | python3 /dev/stdin -s 1 -o csv -f s3://dynamodblabs/artist.csv,s3://dynamodblabs/album.csv,s3://dynamodblabs/track.csv
If you run this on a t3.medium instance, this should take around eight minutes to complete.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Implementing a Simple DynamoDB Application

01:00:00

Intermediate DynamoDB

Table Indexes

Indexes: Part 1

00:04:22

Lesson Description:

In this lesson, we cover global secondary indexes (GSIs) and local secondary indexes (LSIs), including how they can help improve performance and reduce costs. Creating the album_indexed Table Clone the GitHub repo for this course: https://github.com/linuxacademy/content-dynamodb-deepdive

cd content-dynamodb-deepdive/5.1.1-Indexes-Part1
gunzip album_indexed.csv.gz
create_table.sh
load.py

Indexes: Part 2

00:08:49

Lesson Description:

In this lesson, we dive deeper into indexes, demonstrating index queries and projection expressions. We also cover sparse indexes. Code Examples https://github.com/linuxacademy/content-dynamodb-deepdive/tree/master/5.1.2-Indexes-Part2

Importing Tables Using AWS Database Migration Service

00:10:22

Lesson Description:

AWS Database Migration Service (DMS) helps you migrate and replicate your data from many different types of sources into AWS. In this lesson, we cover how to import a legacy database running on MySQL to DynamoDB.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Working with DynamoDB Local and Global Indexes

00:30:00

Backup and Recovery

On-Demand vs. Continuous (PITR) Backup

00:03:11

Lesson Description:

In this lesson, we cover on-demand vs. continuous, or point-in-time recovery, backups.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Implementing and Working with Backups and Restores in DynamoDB

00:30:00

Advanced Performance and Scaling Considerations

Offloading Large Attribute Values to S3

00:03:27

Lesson Description:

In this video, you will learn how to improve your scan and query performance, and reduce your costs, by offloading your large binary objects to S3.

Hot and Cold Partition Imbalance

00:04:00

Lesson Description:

In this lesson, we will learn how partitions are initially allocated upon table creation, what a hot partition is, and how DynamoDB uses resharding to improve performance. We will also cover Adapdive Capacity, which is how DynamoDB can continues to serve imbalanced traffic indefinitely, even if the workload remains unbalanced.

Data Consistency and Management

Conditional and Update Expressions

00:05:28

Lesson Description:

In this video, we will review what a write conflict is, and how conditional update expressions can help avoid them. We will also cover atomic updates, and how conditional updates impact capacity consumption. Code examples used in this lesson: https://github.com/linuxacademy/content-dynamodb-deepdive/tree/master/5.4.1-Conditional_Updates

Transactions

00:06:17

Lesson Description:

DynamoDB transactions provide developers atomicity, consistency, isolation, and durability (ACID) across one or more tables within a single AWS account and region. You can use transactions when building applications that require coordinated inserts, deletes, or updates to multiple items as part of a single logical business operation.

Time to Live (TTL)

00:04:08

Lesson Description:

Time to Live (TTL) allows you to you define when table items expire so that they can be automatically deleted. Code example used in this lesson https://github.com/linuxacademy/content-dynamodb-deepdive/blob/master/5.4.3-TTL/ttl.py Epoch & Unix Timestamp Conversion Tools https://www.epochconverter.com/

Data Resilience, Security, and Encryption

Global Tables

00:05:46

Lesson Description:

Global tables provide a fully managed solution for deploying a multiregion, multi-master database, without having to build and maintain your own replication solution.

Encryption

00:10:38

Lesson Description:

This lesson covers the various encryption options available for DynamoDB, including at-rest, in-transit, and the DynamoDB Encryption Client. DynamoDB Encryption Client for Python https://github.com/aws/aws-dynamodb-encryption-python/ Awsume https://awsu.me/

DynamoDB VPC Endpoints

00:03:26

Lesson Description:

A VPC endpoint for DynamoDB enables EC2 instances to use their private IP addresses to access DynamoDB with no exposure to the public internet.

Database Model v2

Data Model v2 Review

00:06:01

Lesson Description:

In this lesson, we review the changes to our DynamoDB data model, explaining the justification for those changes. We see a demo of just how much better this model performs over the "v1" model, as well as highlight some areas for improvement.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Managing DynamoDB Data Migration with DMS

01:00:00

Advanced DynamoDB

Streams and Triggers

Streams

00:05:50

Lesson Description:

DynamoDB Streams captures a time-ordered sequence of item-level modifications in a DynamoDB table and durably stores the information for up to 24 hours. In this lesson, we cover the basic architecture of DynamoDB Streams and use cases for its application.

Triggers (Lambda Invoke)

00:14:24

Lesson Description:

In this lesson, we create a Lambda function to consume events from a DynamoDB stream. This technique can be used to send order confirmation emails for each new order entered into DynamoDB. Emails are sent using Simple Email Service (SES).

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Processing DynamoDB Streams Using Lambda

01:00:00

Advanced Index Usage

Selective Write Sharding

00:09:32

Lesson Description:

One way to better distribute writes across a partition key space in DynamoDB is to expand the space. For write-heavy items, avoid hot keys by applying a random suffix to partition keys.

Aggregation with Streams

00:07:24

Lesson Description:

In this lesson, you'll learn how to use the data coming from DynamoDB streams to aggregate our voting data in realtime and use Lambda to scale up without any operational overhead.

Implementing High-Performance Architectures

Static Data Dumps (Streams to Static File)

00:11:15

Lesson Description:

In this lesson, we work with DynamoDB Streams and Lambda to create an ad hoc reporting solution. This architecture will generate a real-time report for top-selling orders, storing a static file in an S3 bucket, which is configured for website hosting. Setting Up a Static Website on S3 https://docs.aws.amazon.com/en_pv/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html Source Code for This Lesson https://github.com/linuxacademy/content-dynamodb-deepdive/tree/master/6.3.1-Static-Data-Dumps

DynamoDB Accelerator (DAX) Architecture

00:11:42

Lesson Description:

Amazon DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement — from milliseconds to microseconds — even at millions of requests per second. In this lesson, we learn the architecture of DAX and dive into some code to perform an A/B test, demonstrating a real-world performance boost.

SQS Write Buffer

00:13:08

Lesson Description:

This lesson introduces the use of an SQS queue to buffer writes to DynamoDB and reduce provisioned write throughput. Source Code for This Lesson https://github.com/linuxacademy/content-dynamodb-deepdive/tree/master/6.3.3-SQS-Write-Buffer Create a DynamoDB Table

aws dynamodb create-table --table-name orders 
  --attribute-definitions AttributeName=id,AttributeType=S 
                          AttributeName=timestamp,AttributeType=N 
  --key-schema AttributeName=id,KeyType=HASH 
               AttributeName=timestamp,KeyType=RANGE 
  --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1
Note: We deliberately set RCU/WCU=1 to induce throttling. If your orders table already exists, you must change the provisioned throughput to match the above. Create SQS Queue
aws sqs create-queue --queue-name orders
Create IAM Execution Role for Lambda Use the lambda_execution_role.json to create an IAM policy: Lambda-SQSWriteBuffer-Policy.Edit the policy to reflect your AWS account ID.Attach this policy to a new IAM role: Lambda-SQSWriteBuffer-Role. Create Lambda Function Create a new Python 3.7 function named SQSWriteBuffer with the execution role Lambda-SQSWriteBuffer-Role. Set the environment variable QUEUE_NAME to orders.Set the environment variable DYNAMODB_TABLE to orders.Ensure the function timeout is less than the queue's visibility timeout (e.g., set function timeout = 15 sec)Add a trigger for SQS specifying the orders queue.Wait up to 1 min for the trigger to create in the Enabled state. Sending Messages to SQS Run the provided script orders.py to send messages to SQS Example: Send a message containing a random order to the orders queue every 0.1 second (10 messages per second):
cd content-dynamodb-deepdive/6.3.3-SQS-Write-Buffer
pipenv install
pipenv shell
./orders.py -q orders -i 0.1`
Press Ctrl+C to quit. Review Review the CloudWatch logs for the Lambda function. You may see messages that read: ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. This indicates your messages are being throttled. After some time, observe the orders have been inserted into the orders table. Also observe the SQS queue is empty.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Using Elasticache to Improve DynamoDB Performance

00:45:00

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Implementing DAX on an Existing DynamoDB Solution

00:30:00

Advanced Security

Federated and Fine-Grained Access

00:11:11

Lesson Description:

With fine-grained access control (FGAC), requests to a DynamoDB table can be restricted to specific items or attributes. Requests can also be authenticated and authorized directly by DynamoDB.

Auditing Admin Access Using CloudTrail

00:11:03

Lesson Description:

CloudTrail is very important in the security realm because it records every API call executed on our resources. We can also create trails that allow us to store logs longer than 90 days and use them to trigger automation events. It is a best security practice to make sure CloudTrail logging is always enabled. This video gives information about what CloudTrail does and how to configure it for use with DynamoDB.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Recovering and Auditing Access to DynamoDB

01:00:00

Database Model v3

Database Model v3 Review

00:04:29

Lesson Description:

In this lesson, we review the various iterations of our migration from a relational database to DynamoDB. In "version 3" of our data model, we explore its various features and benefits, as well as its tradeoffs.

Hands-on Labs are real live environments that put you in a real scenario to practice what you have learned without any other extra charge or account to manage.

Implementing Fine-Grained Access Control for DynamoDB

00:30:00

Integration

Amazon EMR

00:08:35

Lesson Description:

In this lesson, we see how DynamoDB integrates with Amazon EMR. EMR lets you create tables using Apache Hive, which can store data natively in an EMR cluster or can be mapped to data stored externally such as in DynamoDB tables and S3 buckets.

Amazon Elasticsearch Service

00:15:51

Lesson Description:

In this lesson, we learn how to integrate DynamoDB with Elasticsearch, enabling you to perform full-text queries on your data. Elasticsearch is a popular open-source search and analytics engine designed to simplify real-time search and big data analytics. Elasticsearch integration is easy to set up using a simple Lambda function and DynamoDB Streams. GitHub Repository for This Lesson https://github.com/linuxacademy/content-dynamodb-deepdive/tree/master/6.6.2-Elasticsearch Create Elasticsearch Domain Deployment type: Development and testing is sufficient. Elasticsearch domain name can be anything you like; this is not an internet domain name. Select all other defaults for instance type, number of instances, etc. Network Configuration Choose VPC Access.Choose a subnet.Enable a security group with ports 22/tcp and 443/tcp open inbound.Set domain access policy to Do not require signing request with IAM credential. Domain initialization takes about 10 minutes. You cannot load data or run queries against your domain until the initialization is complete. The domain status will change to Active as soon as your domain is ready to use. Create the DynamoDB Table Table: pinehead_records_s3PK: type (String)SK: id (Number)On-demand capacity mode Enable streams for "New and old images". Create IAM Execution Role for Lambda Function Role name: Lambda-ddb2es-Role Use policy in lambda-role.json with Lambda as the trusted entity. Package Up the Lambda Function

cd content-dynamodb-deepdive/6.6.2-Elasticsearch/ddb2es
pip3 install requests -t .
pip3 install requests_aws4auth -t .
zip -r ../ddb2es.zip *
Create Lambda Function Name: ddb2esRuntime: Python 3.7Function code: Upload the ddb2es.zip file created in the previous step.Execution role: Use the role Lambda-ddb2es-Role created in the previous step. Configure basic settings for 1024 MB memory and 30 sec timeout. Configure the Lambda function for your VPC. Select all subnets. Ensure you select a security group with outbound access to 0.0.0.0/0 for all ports. Configure the DynamoDB Trigger Table: pinehead_records_s3Batch size: 1000 (this is the maximum)Starting position: Trim horizon Trim horizon will start reading at the oldest record in the shard. Import Data into DynamoDB This will create the v3 data model in your account (the -c or --clean flag will delete the existing table, if specified): Make sure you use have at least 10 GB free memory for the bootstrap script to run without errors. This runs most quickly on an EC2 instance in the same region as the target DynamoDB table.
aws configure set default.region us-east-1
sudo yum install python3 -y
pip3 install --user boto3
curl https://raw.githubusercontent.com/linuxacademy/content-dynamodb-deepdive/master/labs/bootstrap/tablebootstrap.py | python3 /dev/stdin -s 3 -f s3://dynamodblabs/artist.csv,s3://dynamodblabs/album.csv,s3://dynamodblabs/track.csv
Query Elasticsearch You may create an SSH tunnel to simulate a local install of Elasticsearch:
ssh -i ~/.ssh/your-key.pem ec2-user@your-ec2-instance-public-ip -N -L 9200:vpc-your-amazon-es-domain.region.es.amazonaws.com:443
Navigate to https://localhost:9200/_plugin/kibana/ in your web browser. You might need to acknowledge a security exception. Create Index Pattern Define an index pattern of * will match the lambda-index index. Select Next Step. Select I don't want to use the Time Filter. Select Create index pattern. Visualize The Visualize / Create screen will show you the Count metric, indicating the number of items in the index. This value should continue to grow until the Lambda function has processed the entire DynamoDB stream. You can change this to Unique Count on the _id field. Discover Search using KQL syntax. For example, a partial match on track name containing the string Intro:
type.S:track and name_title.S:Intro

What's Next?

What's Next?

What's Next?

00:01:59

Lesson Description:

Thanks for taking this course! Please don't forget to press the thumbs up button on all the lessons you liked! In this lesson, we look at some other courses you may enjoy next. Related Courses on Linux Academy AWS Certified SysOps Administrator - Associate (SOA-C01) AWS Certified Big Data - Specialty Automating AWS with Lambda, Python, and Boto3

Get Recognized!

00:00:46

Lesson Description:

You have completed the course. Get recognized for your achievement!

Take this course and learn a new skill today.

Transform your learning with our all access plan.

Start 7-Day Free Trial