AWS Resource Usage Tracking integrated with Cron Job

In this article, we will explore how to track AWS resource usage in our project and set up automated reports that update at specified intervals using a Cron job.

What is CronJob?

The word “Cron” means ‘time”, that is time-based-scheduling jobs. The particular task will be executed at a specified time interval. This can be achieved using the “Crontab” Linux command.

Crontab:

The Crontab Linux command goes on to check the time on the device and when a particular time arrives, it performs pre-defined tasks automatically. And this crontab contains lots of commands to be executed as a cronjob.

Syntax:

MIN HOUR DOM MON DOW CMD

→ MIN (Minutes 0-59)

→ HOUR (Hour 0-23)

→ DOM (Day of Month 1-31)

→ MON (Month 1-12)

→ DOW (Day of Week 0-7)

CMD → Command that want to execute.

For ex:

I want to display “HELLO WORLD!” in every minute.

Here ‘greetings.sh’ will be executed every single minute.

Inside the ‘greetings.sh’, it has echo" HELLO WORLD!” . '>>' means stores that output in the specified file.

When we print the contents of the ‘output’ file, it displays “HELLO WORLD” as it was added every single minute.

Script Part :

In this project, we will check the usage of AWS S3, AWS EC2, AWS Lambda functions, and AWS IAM Users

1. List s3 buckets :

   aws s3 ls » resource_tracker

   The list of s3 buckets will be stored in the resource_tracker file.

2. List ec2 instance :

   aws ec2 describe-instances

   This will give a huge JSON data. But we can get a particular value we want using the ‘jq’ command line tool. Using ‘jq’ we extract the specified field from the JSON data type.

   aws ec2 describe-instances | jq ‘ .Reservations[].Instances[].InstanceId’ » resource_tracker

   Here we extract the ‘instance id’ only. That is inside the Instance[] list, that was inside the Reservations[] list. Using the ‘.’ operator we can achieve this.

3. List AWS Lambda:

   aws lambda list-functions » resource_tracker

4. List IAM Users:

   aws iam list-users » resource_tracker

Understanding of chmod command :

The chmod command is used to set file or directory permissions. Permissions control who can read, write, and execute files and directories.

+ means adds the permission, - means removes the permission.

Numeric notation :

4 → Read(r)

2 → Write(w)

1 → Execute(x)

EX:

1. Making a file executable for everyone

   chmod +x filename

2. Making a full permission to the user and read/execute permission to the group and others.

   chmod 755 filename

   7 (4+2+1): User has read, write and execute permissions.

   5 (4+0+1): The group have read and execute permissions.

   5(4+0+1): Others have read and execute permissions.

Integrating this script with Cronjob:

#Making a file Executable:

chmod +x /home/ubuntu/aws_resource_tracker.sh

#Open crontab with edit option:

crontab -e

#Reporting AWS Resource usage on every day at 6:00 pm

0 18 * /home/ubuntu/aws_resource_tracker.sh

Explanation :

min → 0, hour → 18, day_of_month → * (for all days in the month), month → \ (for all months), day_of_week → \ (for all days in the week)

Resource_tracker file contains the report :

We actually done our AWS resource usage tracking and that will automatically updated in the resource_tracker file at 6:00 pm.

Reference :

Project link: https://github.com/VasukiElsa/aws_resource_tracker.git

AWS CLI Docs: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/index.html

HAPPY DEVOPS JOURNEY!