Shell scripting is a powerful tool for automating tasks and managing resources in the cloud. In this post, we'll explore how to use shell scripting to list AWS resource usage, including EC2 instances, S3 buckets, Lambda functions, IAM users, etc.

When shell script want to talk to AWS either we can call using API or using aws cli. If we do with API call we have to follow many steps to avoid that we use aws cli. If you where using python then you can connect using Boto3

Prerequisites

AWS CLI installed and configured with your credentials
Bash shell or any compatible shell

Getting Started

Create a new file for your script, e.g., aws_resource_list.sh.

Copy past the code from https://gist.github.com/sharmilasp/4a827ec20e94a49fdbbe5f1fb72a14ca
Add the shebang line at the beginning of the file: #!/bin/bash.

Include metadata about the script, such as author, version, and purpose

 # Author: Sharmila/Devops Team
 # Version: v0.0.1
 # Purpose: List AWS resource usage

Listing AWS Resources

Check if the required number of arguments are passed
```
 if [ $# -ne 2 ]; then
     echo "Usage: ./aws_resource_list.sh <aws_region> <aws_service>"
     echo "Example:./aws_resource_list.sh us-east-1 ec2"
     exit 1
 fi
```
if [ $# -ne 2]
1. Here $# stores the total number of arguments
2. Check total number of arguments not equal to 2 that is user is passing two argument or not if not it will echo the output

Assign the argument to variables and convert the service to lowercase

 aws_region=$1
 aws_service=$(echo "$2" | tr '[:upper:]' '[:lower:]')

Check if the AWS CLI is installed
```
 if ! command -v aws &> /dev/null; then
     echo "AWS CLI is not installed. Please install the AWS CLI and try again."
     exit 1
 fi
```
if ! command -v aws &> /dev/null
1. you're getting a stderr response when no aws cli is found so we move the output to > /dev/null which discard stderr message from an output
2. & -->execute the command asynchronously in a subshell it will not wait for the first command to finish immediately run the second command in shell
3. > /dev/null --> To suppress that return message in stderr

which mainly used to discard standard output and standard error from an output.

Check if the AWS CLI is configured
```
 if [ ! -d ~/.aws ]; then
     echo "AWS CLI is not configured. Please configure the AWS CLI and try again."
     exit 1
 fi
```
[ ! -d ~/.aws ] verify if particular directory exist

-d -->represent directory

~ (tilde) is a quick way of specifying your home directory

~/.aws --> check in your home directory .aws file present or not if not print the output

List the resources based on the service

 case $aws_service in
     ec2)
         echo "Listing EC2 Instances in $2"
         aws ec2 describe-instances --region $aws_region
         ;;
     rds)
         echo "Listing RDS Instances in $aws_region"
         aws rds describe-db-instances --region $aws_region
         ;;
     s3)
         echo "Listing S3 Buckets in $aws_region"
         aws s3api list-buckets --region $aws_region
         ;;
     cloudfront)
         echo "Listing CloudFront Distributions in $aws_region"
          aws cloudfront list-distributions --region $aws_region
         ;;
     vpc)
         echo "Listing VPCs in $aws_region"
         aws ec2 describe-vpcs --region $aws_region
         ;;
     iam)
         echo "Listing IAM Users in $aws_region"
         aws iam list-users --region $aws_region
         ;;
     route5)
         echo "Listing Route53 Hosted Zones in $aws_region"
         aws route53 list-hosted-zones --region $aws_region
         ;;
     cloudwatch)
         echo "Listing CloudWatch Alarms in $aws_region"
         aws cloudwatch describe-alarms --region $aws_region
         ;;
     cloudformation)
         echo "Listing CloudFormation Stacks in $aws_region"
         aws cloudformation describe-stacks --region $aws_region
         ;;
     lambda) echo "Listing Lambda Functions in $aws_region"
         aws lambda list-functions --region $aws_region
         ;;
     sns)
         echo "Listing SNS Topics in $aws_region"
         aws sns list-topics --region $aws_region
         ;;
     sqs)
         echo "Listing SQS Queues in $aws_region"
         aws sqs list-queues --region $aws_region
         ;;
     dynamodb)
         echo "Listing DynamoDB Tables in $aws_region"
         aws dynamodb list-tables --region $aws_region
         ;;
     ebs)
         echo "Listing EBS Volumes in $aws_region"
         aws ec2 describe-volumes --region $aws_region
         ;;
     *)
         echo "Invalid service. Please enter a valid service."
         exit 1
         ;;
 esac

there is no point to list all the aws services as part of organization you can select the services used by organization

For listing we can either go by if else or by switch condition here we are going by switch statement then only the complexity is reduced when we use else if concept then we have to check one by one service to avoid unnecessary search we use switch statement where as it will directly jump to the condition

Executing the Script

Make the script executable:

chmod 771 aws_resource_list.sh

For security best practice read write execute permission for owner(me) and group(team), only execute permission for other user it might change based on your organization needs.

Run the script:

./aws_resource_list.sh us-east-1 ec2

Run the script followed by ./file_name.sh, region and service

./aws_resource_list.sh <region> <service>

Before installing aws cli you run the script you will get the error response as follows

Install aws cli -->create or get an access secret key in AWS Security credentials--> use aws configure command in CLI

List all the s3 buckets that are present

If there is no resource found show as null resource

If use the service which is not maintained by organization show as invalid service

Listing active resource of service this can be used by manager any one who interest in understanding active resources on AWS Account

aws_resource_list.sh

When you execute this script followed by two argument region and services, then they can able to list active running resources in particular service it can be ec2, s3, rds, lambda etc. any service mentioned in script on particular region. This is important because cost-optimization is very important for Organizations

Key Features

Output Redirection: The script can redirect its output to a file, allowing you to keep a record of your resource usage. This is particularly useful for reporting purposes.
Cron Job Integration: By integrating the script with a cron job, you can automate the execution at scheduled intervals, ensuring that you always have up-to-date information without manual intervention.
User-Friendly Comments: The script includes comments that explain each section, making it easier for others (or yourself in the future) to understand the purpose and functionality of the code.

Conclusion

Finally, you can use this script to view key performance indication and monitoring for your active services. This helps you keep an eye on how everything is performing and spot any issues early.

This scripts help you quickly gather and understand important information about your cloud resources, making your job easier and your management more efficient.

Feel free to try these out, customize and extend this script to suit your specific needs. Also, let me know if you have any questions or need help with anything else!

Shell Scripting for Listing AWS Resource Usage

Table of contents