Executive Summary

This document outlines a solution for AWS cost optimization through automated monitoring and reporting. By implementing this solution, organizations can gain regular insights into their AWS spending patterns, identify cost spikes, and take appropriate actions to optimize cloud expenses.

Key Points

• Measurement First Approach: You cannot optimize what you cannot measure
• Daily Cost Tracking: Automated daily reports provide consistent visibility
• Service-based Analysis: Breakdown of costs by AWS service helps pinpoint expensive resources
• Spike Detection: Comparing daily costs helps identify unusual spending patterns
• Automated Slack Notifications: Reduces manual monitoring effort and ensures timely awareness

Tools and Technologies

---

Prerequisites

Before diving into this project, here are some skills and tools you should be familiar with:

• Terraform installed on your machine.
• A GitHub account.
• A GitHub personal access token with the necessary permissions to create repositories.

Clone repository for terraform code

💡 Note: Replace resource names and variables as per your requirement in terraform code

• For EC2 VM: change in terraform.tfvars(i.e keyname-MYLABKEY`*)

Implementation Steps

Setting Up the Infrastructure

I have created a Terraform code to set up the entire infrastructure, including the installation of required tools and EC2 automatically created.

1. Setup Environment

• Create a virtual machine (or use existing server)
• Update system packages
• Install NodeJS using NVM
• Install AWS CLI

2. Install and Configure AWS Cost CLA

• Install AWS Cost CLA using npm
• Configure AWS credentials using aws configure
• Test basic cost reporting with command aws-cost

💡 Note: ⇒ Part of creating EC2 virtual machine it will installed all the packages and AWS CLI set up automatically.

The script has been updated with several improvements to ensure the aws-cost command is available to the ubuntu user when you SSH in:

Added a symbolic link to make aws-cost available in the system path (/usr/local/bin/aws-cost)
Properly set up both .bashrc and .profile to include NVM configuration
Created a robust run-aws-cost.sh script in the ubuntu home directory
Added a system-wide aws-cost-run script in /usr/local/bin
With these changes, after the EC2 instance is provisioned, you should be able to:

• SSH into the instance and run aws-cost directly (thanks to the symbolic link)
  • If that doesn't work, you can run source ~/.bashrc && aws-cost Or simply use the helper script: ~/run-aws-cost.sh
  • As a last resort, use the system-wide script: aws-cost-run

3. Create Slack Integration

• Create a dedicated Slack channel for cost reporting
• Create a new Slack application
• Configure the following permissions:
  • chat:write
  • chat:write.public
  • files:write (for file uploads)
• Install the app to your workspace and save the Bot token

• Copy the Slack channel ID

• Login on https://slack.com/intl/en-au/

• For creating a apps https://api.slack.com/apps

Invite the Bot in Slack Channel /invite @AWS Cost Notifier and click on send

4. Create Python Script for Report Uploads

• Install Python and the Slack SDK
• Create a script (upload_cost_report.py) that:
  • Reads cost report data
  • Uploads it as a file to the Slack channel
  • Confirms successful upload

• . Install Python & pip (if not already)

   sudo apt update
   sudo apt install -y python3 python3-pip python3-venv
  

• Create a Virtual Environment (Optional but Recommended)

  python3 -m venv venv
  source venv/bin/activate
  

  

• Install Slack SDK:

   pip install slack-sdk
  

  

• Create Python Script (upload_cost_report.py): from slack_sdk

  ```py title="upload_cost_report.py" linenums="1" hl_lines="2-4""" from slack_sdk import WebClient from slack_sdk.errors import SlackApiError slack_token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxx" client = WebClient(token=slack_token)

  try:

  Use files_upload_v2 (latest method)

  response = client.files_upload_v2( channel="xxxxxxxxxxxxxxxxxxxxxxx", initial_comment="AWS Cost Report", file="cost-report.txt" ) print("File uploaded successfully:", response)

  except SlackApiError as e: print(f"Error uploading file: {e.response['error']}") ```

• Run the AWS Cost CLI + Python Script:

5. Generate cost report

run the following command- aws-cost --text > cost-report.txt

6. Upload to Slack

run the following command- python3 upload_cost_report.py

5. Set Up Automation

• Generate cost report files using AWS Cost CLA
• Execute the Python script to upload the report
• Configure cron job to run this process daily

Challenges

1. Limited Message Display: Slack has message size limitations that prevent displaying the complete service breakdown directly in messages

   • Solution: Upload reports as files instead

2. Permissions Configuration: Setting up proper permissions for both AWS and Slack requires careful configuration

   • Solution: Follow step-by-step process for configuring OAuth scopes and AWS credentials

3. Manual Analysis Required: While the system automates reporting, analysis still requires human interpretation

   • Solution: Establish baselines and patterns to make anomaly detection easier

4. Authentication Security: Using root AWS credentials is not recommended for production

   • Solution: Create dedicated IAM users with read-only cost explorer permissions

Benefits

1. Proactive Cost Management: Daily visibility into costs prevents end-of-month billing surprises
2. Quick Issue Detection: Identifying cost spikes within 24 hours allows for rapid response
3. Service-Level Insights: Understanding which services contribute most to costs enables targeted optimization
4. Automated Workflow: Reduces time spent manually checking cost dashboards
5. Centralized Communication: Cost reports delivered to Slack integrate with existing team communication channels

Conclusion

AWS cost optimization requires consistent monitoring and analysis of spending patterns. This solution provides an automated approach to cost reporting that enables teams to quickly identify unusual spending patterns or resource utilization. By implementing daily cost reports delivered directly to Slack, organizations can foster a cost-conscious culture and respond rapidly to unexpected expenses.

The combination of AWS Cost CLA, Slack integration, and scheduled reporting creates a seamless system that reduces the manual effort required for cost monitoring while increasing visibility across the organization. This approach transforms cost optimization from a reactive, month-end activity into a proactive, daily practice that can significantly reduce cloud spending over time.

Ref Link:

• YouTube Link
• Download NodeJs
• Repo Link
• My Repo