Skip to main content

AWS EC2 Custom Logging with CloudWatch

Hands-On Lab

 

Photo of Craig Arcuri

Craig Arcuri

AWS Training Architect II in Content

Length

00:30:00

Difficulty

Intermediate

Welcome to this hands-on AWS Learning Activity in which you will learn how to send logging data from EC2 to Amazon CloudWatch. When deploying instances it is often desirable to have logging information for the instance. This can be accomplished by installing the CloudWatch Logs Agent on the EC2 instance. The logging data can then be streamed to CloudWatch for analysis and even sent further downstream to other resources such as Elastic Map Reduce or Kinesis for in-depth analysis. The key to all of this is to install the CloudWatch Logs Agent on the EC2 instance, turn on the log service, and configure CloudWatch to receive these messages.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

AWS EC2 Custom Logging with CloudWatch

Introduction

In this hands-on lab, we'll learn how to send logging data from EC2 to Amazon CloudWatch.

Log in to the AWS environment using the cloud_user credentials provided on the lab instructions page.

Once you're logged in to the AWS account, make sure you are using us-east-1 (N. Virginia) as the selected region.

Custom EC2 Logging

Create an EC2 instance

  1. From the AWS Management Console dashboard, navigate to EC2.
  2. Click Launch Instance.
  3. Select Amazon Linux AMI (not Amazon Linux 2 AMI).
  4. Select t2.micro, and click Next: Configure Instance Details.
  5. On the Configure Instance Details page:
    • Set Auto-assign Public IP to Enable.
    • In the IAM role dropdown, select the role listed (there should just be one, which includes the phrase InstanceLoggingProfile).
    • Leave the other defaults.
  6. Click Review and Launch.
  7. Click Launch.
  8. Select Create a new key pair from the dropdown, and type "EC2Logging" for Key pair name. Click Download Key Pair.
  9. Click Launch Instances.
  10. Click View Instances.

Log In to the Instance Via SSH

  1. Open your terminal application.

  2. Switch to your downloads directory.

    cd downloads
  3. From your downloads directory, change permissions on the key pair:

    chmod 400 EC2Logging.pem
  4. Go back to the AWS Console in your browser. Once our new instance is in a running state, select it by clicking the checkbox on the left.

  5. Click Connect.

  6. Copy the command under Example in the popup menu, and paste it in to your command line. It will look something like this (make sure you replace the <PUBLIC_IP_ADDRESS> parameter:

    ssh -i "EC2Logging.pem" ec2-user@<PUBLIC_IP_ADDRESS>.compute-1.amazonaws.com
  7. Type yes at the prompt.

  8. Update the instance.

    sudo yum update -y
  9. Install the awslogs package.

    sudo yum install -y awslogs
  10. Change to the awslogs directory.

    cd /etc/awslogs
  11. List the contents to see the configuration files.

    ls

    >Note: The /etc/awslogs/awscli.conf file contains credentials and region information, so if we needed to change anything for those items, we'd do it there. The /etc/awslogs/awslogs.conf file contains the settings for CloudWatch logging, so from that file, we could determine which logs to watch, add or remove log streams, and change default settings like time zone. But we won't change any of these items in this lab.

  12. Turn on the awslogs service.

    sudo service awslogs start
  13. Check for errors.

    less /var/log/awslogs.log
  14. Exit by entering:

    q
  15. Ensure the service starts after reboots.

    sudo chkconfig awslogs on

Verify CloudWatch Logs Sent from EC2

  1. Head back to the AWS Management Console dashboard, and navigate to the CloudWatch service.
  2. Click Logs in the sidebar.
  3. On the Log Groups page, click /var/log/messages.
  4. Under Log Streams, click the listed instance ID to view the log files.

Conclusion

Congratulations on completing this lab!