AWS CloudWatch - Reporting on RAM, Disk Usage & More

Amazon Web Services reports some good metrics on the console by default, like CPU, but it’s missing some key metrics like memory usage or disk space; these are important to monitor to ensure instance uptime and health.

In this post we’ll look at how we can use CloudWatch to monitor these extended metrics, allowing you to build reports and even send alerts to Slack based on set thresholds and defined metric upper bounds.

It’s a good idea keeping everything in the same place, so we can leave CPU and all the other default metrics as they are, but in addition append the extra ones we want, like how much disk space we have, or how much memory is being used.

The monitoring scripts are authored by Amazon themselves, but aren’t included unless you set them up yourself, which isn’t always obvious. The scripts are available for a variety of different operating systems that could be running on your instances, however we will focus on Linux-based systems in this post. Amazon's own documentation on this topic, while comprehensive, is hard to find; hopefully this post will help you with your own instance monitoring.

Installing prerequisites on the instance

The monitoring scripts used in this post require some additional Perl libraries to be installed on your Linux instance image. Some operating systems already have certain modules installed by default; run the commands below for a general guide.

For Ubuntu run

sudo apt-get update
sudo apt-get install libwww-perl libdatetime-perl

For RHEL based systems run

sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA

For SUSE Linux Enterprise run

sudo zypper install perl-Switch perl-DateTime "perl(LWP::Protocol::https)"

Please note, If you don’t have the unzip package installed on your server, you will also need this in order to unzip the contents of the monitoring scripts. On pretty much every Linux system, this is simply called ‘unzip’.

For Ubuntu run

sudo apt-get update
sudo apt-get install unzip

For RHEL

sudo yum install zip unzip

For SUSE

sudo zypper install unzip

Setting up a IAM user on AWS

We’ll need to create authenticated access to Cloud Watch and EC2 Instance tags.

You can use the policy below as an inline policy on a user with programmatic access to the cloud through the form of an Secret Key / Access Key.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1491836185126",
            "Action": [
                "cloudwatch:PutMetricData"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Sid": "Stmt1491836200583",
            "Action": [
                "cloudwatch:GetMetricStatistics"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Sid": "Stmt1491836212902",
            "Action": [
                "cloudwatch:ListMetrics"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Sid": "Stmt1491836285195",
            "Action": [
                "ec2:DescribeTags"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Alternatively, you can associate a role with your instance; however, it will require access to the rights below:

• cloudwatch:PutMetricData
• cloudwatch:GetMetricStatistics
• cloudwatch:ListMetrics
• ec2:DescribeTags

Installing the monitoring scripts on the instance

Run the command below to fetch a zip folder of the monitoring scripts from Amazon.

curl http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O

Unzip the contents and remove the zip folder that was just downloaded.

unzip CloudWatchMonitoringScripts-1.2.1.zip
rm CloudWatchMonitoringScripts-1.2.1.zip

Move the folder to the desired location, we’ll move this to the below for the purposes of the post.

mv aws-scripts-mon .aws-scripts-mon

Navigate to the new folder

cd .aws-scripts-mon

Copy the AWS creds template to a new file

cp awscreds.template credentials.conf

Add the content below to the file

AWSAccessKeyId=[YOUR ACCESS KEY ID]
AWSSecretKey=[YOUR SECRET KEY]

To create a new cron run the following command

crontab -e

Create a new cron job with the following content replacing [USER] with your linux username.

*/5 * * * * ~/.aws-scripts-mon/mon-put-instance-data.pl --aws-credential-file="/home/[USER]/.aws-scripts-mon/credentials.conf" --mem-util --mem-used --disk-space-util --disk-path=/ --from-cron

This cron job will run every 5 mins, sending memory utilisation, the amount of memory being currently used, and the disk space utilisation metrics to Cloudwatch.

Doing something with the data

1. Log into your AWS console, navigating to Cloudwatch and ensuring the correct region is selected.
2. Choose browse metrics.
3. You should now see Linux System under custom namespaces
4. Choose instance ID, and you will see a list of the new memory and disk metrics against your Instance ID.
5. Clicking on a metric will graph the data. If you’ve just set this up, there won’t be much data to graph.

Now that data from instances is being sent to Cloudwatch, there are a number of things we can do with it. For instance, we can create responses to defined threshold, which could include posting messages to a Slack channel or modifying a cloud scaling group on an instance. Setting up some of the aforementioned items are quite labour-intensive, but there are a lot of useful things you can extract from the extra data.

In addition to just RAM and Disk metrics, the script can measure a few more items. For more information on the script you can follow Amazon’s documentation