Skip to main content

Common S3 Errors when Developing with AWS S3

Hands-On Lab

 

Photo of

Training Architect

Length

00:30:00

Difficulty

Beginner

In this live AWS environment, you will use Boto3 and Python to trigger and debug some common AWS S3 errors. In this process, you will create S3 buckets and S3 objects as well as see how you can use the AWS SDK for Python (Boto3) to interact with AWS S3. This experience will allow you to more effectively develop with the S3 service because you will be able to quickly identify the causes of different errors and you will get experience using an AWS SDK to interact with the S3 service.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Common S3 Errors when Developing with AWS S3

Introduction

In this live AWS environment, you will use Boto3 and Python to trigger and debug some common AWS S3 errors. In this process, you will create S3 buckets and S3 objects as well as see how you can use the AWS SDK for Python (Boto3) to interact with AWS S3. This experience will allow you to more effectively develop with the S3 service because you will be able to quickly identify the causes of different errors and you will get experience using an AWS SDK to interact with the S3 service.

Getting Started

Let's get started by logging in to the AWS console. The necessary links and credentials are provided on the hands-on lab page under the Credentials section.

  1. Click on the Open AWS Console button to open the AWS console login page.
  2. Copy and paste the username and password listed under AWS ACCOUNT in the Credentials section of the hands-on lab page. The Account ID or alias field should be filled in for you already.
  3. Now that we are logged in to the AWS console, we need to connect to our server's public IP address using SSH.

    • Open a Terminal window on your local machine

    • Connect to your server by entering the following command, replacing <PUBLIC IP> with the IP provided under Public IP address of public instance in the Credentials section of the hands-on lab page:

      ssh cloud_user@<PUBLIC IP>
    • At the prompt "Are you sure you want to continue connecting (yes/no)?", enter yes

    • Use the password for the cloud_user provided under Cloud Server and Public Instance in the Credentials section of the hands-on lab page.

Now that we are logged in to the AWS console and connected to our server using SSH, we are ready to get started!

Debugging S3 Errors with the AWS SDK for Python (Boto3)

In our Terminal window, let's use the ls command to show the files that we will be working with. The only file in this directory is s3_error_examples.py. We can show the contents of this file by using the command cat s3_error_examples.py. Inside this file, we have a function that will try and return an object from S3, as well as several steps which will show examples of various errors that are commonly encountered. The # No such key step requires that we have an existing S3 bucket created, so let's go ahead and create that bucket.

Create an S3 Bucket

In our Terminal window, let's create an S3 bucket by executing the aws s3 mb s3://the-la-cda-s3-bucket command. We can verify that this bucket has been created successfully by running the command aws s3 ls, which will show the buckets that currently exist in the account. We can also navigate to the S3 section in our AWS Console to display this information.

Now that we have an S3 bucket, let's create a file with some text in it to use for testing. The command echo 'Hello, AWS!' > ourtest.txt creates a file named ourtest.txt and places the text Hello, AWS! inside of this file. With the S3 bucket and test file created, we are now ready to set up the steps in our s3_error_examples.py file. Open the file in a text editor with the command sudo nano s3_error_examples.py. Since we are using sudo, we will need to provide the password for the cloud_user user. This password is provided in the Credentials section of the hands-on lab page.

With our s3_error_examples.py file open in a text editor, let's put the name of the S3 bucket we created on the existing_bucket line. Replace <REPLACE WITH YOUR BUCKET NAME> with the name of our bucket, the-la-cda-s3-bucket. We can now save and close this file (for the nano text editor, use Ctrl+X, type Y, and press Enter). We can verify our change saved correctly by showing the contents of the file with the cat s3_error_examples.py command.

Our bucket is created, our test file is in place, and our bucket name has been added to our Python script. Let's start running some code!

Begin by starting a new Python shell with the python3 command. The first thing we should run is the function that is defined in our s3_error_examples.py script and the commands listed under the # Access denied error section. Copy and paste this into our Python window:

import boto3

s3 = boto3.resource('s3')

def try_to_get_object(bucket, key):
    try:
        obj = s3.Object(bucket, key)
        return obj.get()['Body'].read().decode('utf-8')
    except Exception as e:
        print('Error:')
        print(e)

After pasting in our function, press Enter twice and then paste in the commands listed under # Access denied error:

no_access_bucket = 'www.fernandomc.com'
no_access_key = 'index.html'
try_to_get_object(no_access_bucket, no_access_key)

When we run this section of our code, it will produce the message "An error occurred (AcessDenied) when calling the GetObject operation: Access Denied."

We can continue on to the next example error by copy and pasting the next set of code into our Python window:

exisiting_bucket = 'the-la-cda-s3-bucket'
fake_key = 'something.something.not.real'
try_to_get_object(exisiting_bucket, fake_key)

Executing this section of code will produce the message "An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist."

We can continue running the other sets of code to produce their respective error examples. Once we have seen all of the error examples, exit the Python session by executing the exit() command.

Add a File to Your S3 Bucket

Remember the ourtest.txt file we created after we made our S3 bucket? Let's copy that file from our server to the S3 bucket by running the command aws s3 cp ./ourtest.txt s3://the-la-cda-s3-bucket/ourtest.txt. Navigating to the Overview page of our the-la-cda-s3-bucket bucket in the Amazon Console will show the ourtest.txt file has been uploaded successfully.

No such key Error

We have seen what the error looks like. Now, let's learn how to resolve it. Start by modifying our s3_error_examples.py file again to include the new file that was uploaded. Open the file in a text editor, providing the cloud_user password when prompted from using sudo:

sudo nano s3_error_examples.py

We need to replace the line:

fake_key = 'something.something.not.real'

With the name of our test file:

fake_key 'ourtest.txt'

Save and exit the file (if you are using nano, use Ctrl+X, type Y, and press Enter). Display the contents of the file for use by executing cat s3_error_examples.py.

Start a new Python shell by executing python3 and copy and paste in the following code from our s3_error_examples.py script:

import boto3

s3 = boto3.resource('s3')

def try_to_get_object(bucket, key):
    try:
        obj = s3.Object(bucket, key)
        return obj.get()['Body'].read().decode('utf-8')
    except Exception as e:
        print('Error:')
        print(e)

After pasting in our function, press Enter twice and then paste in the commands listed under # No such key error:

exisiting_bucket = 'the-la-cda-s3-bucket'
fake_key = 'ourtest.txt'
try_to_get_object(exisiting_bucket, fake_key)

Since we replaced the fake_key line with a file that exists, our function is now able to return a result instead of the No such key error. The text of our file is displayed.

Conclusion

In this live AWS environment, you used Boto3 and Python to trigger and debug some common AWS S3 errors. In this process, you created S3 buckets and S3 objects as well as saw how you can use the AWS SDK for Python (Boto3) to interact with AWS S3. This experience has allowed you to more effectively develop with the S3 service because you are able to quickly identify the causes of different errors and you now have experience using an AWS SDK to interact with the S3 service. Great job!

The Amazon Error Responses page is very helpful when troubleshooting any errors that we generally encounter. You can find that page here: https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html. It's a good idea to study the list of codes to familiarize yourself with what type of errors exist and how to resolve them.