Cloudwatch | Monitoring And Logging | Shell Scripting Tutorial

Using AWS CloudWatch with Shell Scripts

Introduction to AWS CloudWatch

AWS CloudWatch is a monitoring and management service built for developers, system operators, site reliability engineers (SRE), and IT managers. It provides data and actionable insights to monitor applications, respond to system-wide performance changes, optimize resource utilization, and get a unified view of operational health. Integrating CloudWatch with shell scripts can help automate the monitoring process.

Setting Up AWS CLI

Before integrating CloudWatch with shell scripts, you need to set up the AWS Command Line Interface (CLI). Follow these steps to install and configure the AWS CLI:


#!/bin/bash

# Update the package list and install AWS CLI
sudo apt-get update
sudo apt-get install awscli -y

# Configure AWS CLI
aws configure

During configuration, you will be prompted to enter your AWS Access Key ID, Secret Access Key, region, and output format. Make sure to have your AWS credentials ready.

Monitoring EC2 Instances

You can use shell scripts to monitor your EC2 instances using CloudWatch. Here is an example script to get CPU utilization of an EC2 instance:


#!/bin/bash

# Define variables
INSTANCE_ID="your_instance_id"
NAMESPACE="AWS/EC2"
METRIC_NAME="CPUUtilization"
START_TIME=$(date -u -d '1 hour ago' +"%Y-%m-%dT%H:%M:%SZ")
END_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
PERIOD=3600

# Get CPU utilization from CloudWatch
aws cloudwatch get-metric-statistics --namespace $NAMESPACE --metric-name $METRIC_NAME --dimensions Name=InstanceId,Value=$INSTANCE_ID --start-time $START_TIME --end-time $END_TIME --period $PERIOD --statistics Maximum

Replace your_instance_id with the ID of your EC2 instance. This script fetches the maximum CPU utilization in the past hour.

Creating Alarms

CloudWatch Alarms allow you to monitor CloudWatch metrics and automatically perform actions when a threshold is breached. Here is a script to create a CPU utilization alarm for an EC2 instance:


#!/bin/bash

# Define variables
ALARM_NAME="HighCPUUtilization"
INSTANCE_ID="your_instance_id"
NAMESPACE="AWS/EC2"
METRIC_NAME="CPUUtilization"
THRESHOLD=80.0
COMPARISON_OPERATOR="GreaterThanOrEqualToThreshold"
EVALUATION_PERIODS=1
ALARM_ACTIONS="arn:aws:sns:your_region:your_account_id:your_sns_topic"

# Create CloudWatch alarm
aws cloudwatch put-metric-alarm --alarm-name $ALARM_NAME --metric-name $METRIC_NAME --namespace $NAMESPACE --statistic Average --period 300 --threshold $THRESHOLD --comparison-operator $COMPARISON_OPERATOR --evaluation-periods $EVALUATION_PERIODS --dimensions Name=InstanceId,Value=$INSTANCE_ID --alarm-actions $ALARM_ACTIONS

Replace the placeholders with your specific values. This script creates an alarm that triggers when the average CPU utilization of the specified EC2 instance exceeds 80%.

Logging to CloudWatch

CloudWatch Logs allows you to monitor, store, and access log files from Amazon EC2 instances, AWS CloudTrail, and other sources. Here is a script to configure CloudWatch Logs for your application logs:


#!/bin/bash

# Define variables
LOG_GROUP_NAME="your_log_group"
LOG_STREAM_NAME="your_log_stream"

# Create a log group
aws logs create-log-group --log-group-name $LOG_GROUP_NAME

# Create a log stream
aws logs create-log-stream --log-group-name $LOG_GROUP_NAME --log-stream-name $LOG_STREAM_NAME

# Push a log event
TIMESTAMP=$(date +%s000)
MESSAGE="This is a test log message"
aws logs put-log-events --log-group-name $LOG_GROUP_NAME --log-stream-name $LOG_STREAM_NAME --log-events timestamp=$TIMESTAMP,message="$MESSAGE"

Replace the placeholders with your specific values. This script creates a log group and a log stream, then pushes a test log message to CloudWatch Logs.

Automating CloudWatch Setup with Shell Scripts

Here is a complete shell script to automate the setup of CloudWatch monitoring and logging:


#!/bin/bash

# Install AWS CLI
sudo apt-get update
sudo apt-get install awscli -y
aws configure

# Define variables
INSTANCE_ID="your_instance_id"
NAMESPACE="AWS/EC2"
METRIC_NAME="CPUUtilization"
START_TIME=$(date -u -d '1 hour ago' +"%Y-%m-%dT%H:%M:%SZ")
END_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
PERIOD=3600
ALARM_NAME="HighCPUUtilization"
THRESHOLD=80.0
COMPARISON_OPERATOR="GreaterThanOrEqualToThreshold"
EVALUATION_PERIODS=1
ALARM_ACTIONS="arn:aws:sns:your_region:your_account_id:your_sns_topic"
LOG_GROUP_NAME="your_log_group"
LOG_STREAM_NAME="your_log_stream"

# Get CPU utilization from CloudWatch
aws cloudwatch get-metric-statistics --namespace $NAMESPACE --metric-name $METRIC_NAME --dimensions Name=InstanceId,Value=$INSTANCE_ID --start-time $START_TIME --end-time $END_TIME --period $PERIOD --statistics Maximum

# Create CloudWatch alarm
aws cloudwatch put-metric-alarm --alarm-name $ALARM_NAME --metric-name $METRIC_NAME --namespace $NAMESPACE --statistic Average --period 300 --threshold $THRESHOLD --comparison-operator $COMPARISON_OPERATOR --evaluation-periods $EVALUATION_PERIODS --dimensions Name=InstanceId,Value=$INSTANCE_ID --alarm-actions $ALARM_ACTIONS

# Create a log group
aws logs create-log-group --log-group-name $LOG_GROUP_NAME

# Create a log stream
aws logs create-log-stream --log-group-name $LOG_GROUP_NAME --log-stream-name $LOG_STREAM_NAME

# Push a log event
TIMESTAMP=$(date +%s000)
MESSAGE="This is a test log message"
aws logs put-log-events --log-group-name $LOG_GROUP_NAME --log-stream-name $LOG_STREAM_NAME --log-events timestamp=$TIMESTAMP,message="$MESSAGE"

Save this script as setup_cloudwatch.sh and make it executable:


chmod +x setup_cloudwatch.sh
./setup_cloudwatch.sh