Configuring AWS Backup and Archiving Solutions in S3 with Lifecycle Policies and Versioning

Hands-On Lab

 

Photo of Wayde Gilchrist

Wayde Gilchrist

Training Architect

Length

00:30:00

Difficulty

Intermediate

In this live AWS environment, you will learn how to implement S3 archival and backup solutions using S3 versioning and lifecycle policies. This knowledge will allow you to create cost-efficient methods of backing up different systems when using S3. By the end of the environment, you will have configured multiple lifecycle policies, implemented object versioning and should feel confident implementing policies that may be more customized to the needs of your own backup and archival processes. The files for this environment can be found here - https://github.com/linuxacademy/csa-a-2018/tree/master/s3-backups-archiving

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Configuring AWS Backup and Archiving Solutions in S3 with Lifecycle Policies and Versioning

Introduction

In this live AWS environment, you will learn how to implement S3 archival and backup solutions using S3 versioning and lifecycle policies. This knowledge will allow you to create cost-efficient methods of backing up different systems when using S3. By the end of the lab, you will have configured multiple lifecycle policies, implemented object versioning, and should feel confident implementing policies that may be more customized to the needs of your own back up and archival processes.

The files for this environment can be found here: https://github.com/linuxacademy/csa-a-2018/tree/master/s3-backups-archiving.

Getting Started

Let's get started by logging in to the AWS Console. The necessary links and credentials are provided on the hands-on lab page under the Credentials section.

  • Click on the Open AWS Console button to open the AWS console login page.
  • Copy and paste the username and password listed under AWS ACCOUNT in the Credentials section of the hands-on lab page. The Account ID or alias field should be filled in for you already.

Now that we are logged in to the AWS Console, let's open up the S3 service by clicking the Services dropdown in the top-left of the screen and clicking on S3 under the Storage section.

The files we will use later in the lab are available on GitHub. Let's go ahead and download these files by navigating to https://github.com/linuxacademy/csa-a-2018/tree/master/s3-backups-archiving in a new tab in our browser, click on the green Clone or download button, and click Download ZIP. Next, extract the ZIP file so that we have the files ready to be used later in the lab. This operation is different for each operating system. If you are using a Mac, you can double-click on the file to extract it. If you are using Windows, you can right-click the file and choose Extract.

S3 Object Versioning

Create an S3 Bucket

Now that we have everything we need to get started, let's switch back to the S3 tab to begin creating an S3 bucket. Click on the blue Create bucket button. The bucket name will need to be something unique. For this lab, the instructor used the name medical-app-bucket-2018-asgdvb123. The name of your bucket isn't important for this lab, just be sure use a unique name. Once we have a unique name chosen for our bucket, click Next. We won't be changing any of the other settings for our bucket, so click Next two more times, and then click Create bucket.

Create Folders in an S3 Bucket

After our bucket is created, click on the bucket name to bring us to the Overview tab. From here, let's create a folder prefix inside of our bucket by clicking Create folder. We'll call our folder prefix images and use the default settings for encryption. Click Save to create the folder prefix.

We will need to create two more folder prefixes. Repeat this process to create the call-logs and medical folder prefixes.

Upload Files to an S3 Bucket

Now that our three folders have been created, we can begin uploading files to them. Let's start by clicking on the call-logs folder and then clicking Upload. On the Select files tab, click Add files to select which files to upload. Navigate to the extracted csa-a-2018-master folder, open the s3-backups-archiving folder, select the two call log files (log-2018-06-12-01.m4a and log-2018-06-12-02.m4a), and then click Open. The two call log files are now displayed in the Select files tab. The other settings can be left at their default, so click Next three more times, and then click Upload.

Click on the name of our bucket at the top of the screen to take us back to the list of folders. Click on the medical folder and repeat this process to upload the mr-1 and mr-2 files. Click on the name of our bucket at the top of the screen again to take us back to the list of folders. Click on the images folder. Click Upload and upload the mountain.png file.

Enable Object Versioning on an S3 Bucket

Now that we have some files added, let's enable versioning and see how it affects our files. Object versioning needs to be enabled at the bucket level, so let's click the name of our bucket again to take us back to the list of folders. Click on the Properties tab and click on Versioning to expand the Versioning section. In this section, click the button next to Enable versioning and then click Save.

Now we can see what happens when we upload existing objects. Navigate back to the images folder by clicking on the Overview tab and then clicking on images. We will need to rename our new file to use the same name as the file we already have uploaded. Click Upload and Add files, delete the mountain.png file, and then rename mountain2.png to mountain.png. Now, upload the mountain.png file.

We still see only the mountain.png file in our images folder. If we click on Show next to Versions, we will be able to see the version history for our mountain.png file. If we wanted to download the original file, we can click on the original file in the list and then click Download in the details window for that image.

Now that we have seen what object versioning looks like in an S3 bucket, let's take a look at lifecycle policies to help manage the number of versions that are retained.

S3 Lifecycle Policies

Click on the name of our bucket at the top of the screen to take us back to our list of folders. Click on the Management tab and be sure that Lifecycle is selected. From here, click on Add lifecycle rule to begin creating our lifecycle policy.

Call Logs Lifecycle Policy

We will be creating our first lifecycle policy for our call logs, so let's name our rule Call Logs. We also want to limit our lifecycle policy to our call logs folder, so let's type call-logs in the Add filter to limit scope to prefix/tags field. Click on prefix call-logs in the list to add it to our filter. Click Next to take us to the Transitions tab. In this tab, check the boxes for Current version and Previous versions.

Let's add a transition for the current versions of our objects by clicking the Add transition button next to For current versions of objects. In the Object creation field, select the Transition to One Zone-IA after option and set the Days after creation field to 30. Add another transition for our previous objects by clicking the Add transition button next to For previous versions of objects. In the Object creation field, select the Transition to One Zone-IA after option and set the Days after creation field to 30. Now that we have our transitions added, click Next to take us to the Expiration tab.

In the Expiration tab, click the boxes for Current version and Previous version. Set Expire current version of object to 365 and Permanently delete previous versions to 365. These settings will ensure an object is marked as a previous version one year after the object is created and then permanently deleted after another year. This means it will take two full years before an object is permanently deleted. We also want to enable Clean up incomplete multipart uploads and set that to 7 days to ensure any failed uploads are cleaned up after 7 days. Click Next to take us to the Review tab. Feel free to review the configuration and then click Save to continue. Note: You may have to click the refresh button in the top-right of this section to see the new lifecycle policy.

Medical Records Lifecycle Policy

Our second lifecycle policy will be for our medical folder. Click Add lifecycle rule to begin adding this rule. We can name this rule Medical Records, type medical in the Add filter to limit scope to prefix/tags field, and select prefix medical. Click Next to take us to the Transitions tab.

In the Transitions tab, check the boxes for Current version and Previous versions again. Click Add transition next to For current versions of objects, set the Object creation field to Transition to Standard-IA after, and set Days after creation to 90. We also want to transition our objects to Amazon Glacier, so let's add another transition with an Object creation of Transition to Amazon Glacier after and Days after creation of 365.

We will do the same thing for our previous versions. Add another transition for our previous objects by clicking the Add transition button next to For previous versions of objects. In the Object creation field, select the Transition to Standard-IA after option and set the Days after creation field to 90. We also want to transition our objects to Amazon Glacier, so let's add another transition with an Object creation of Transition to Amazon Glacier after and Days after creation of 365. Click Next to take us to the Expiration tab.

In this case, we want to keep all of our medical records, so we will not be configuring any expiration in the Expiration tab. Click Next to take us to the Review tab. After reviewing our configuration, click Save. Refresh the page by clicking the refresh button at the top-right of the list of lifecycle policies to see the newly created Medical Records lifecycle policy.

Conclusion

In this live AWS environment, you have learned how to implement S3 archival and backup solutions using S3 versioning and lifecycle policies. This knowledge will allow you to create cost-efficient methods of backing up different systems when using S3. Over the course of this lab, you have configured multiple lifecycle policies, implemented object versioning, and should feel confident implementing policies that may be more customized to the needs of your own back up and archival processes. Great job!