Automating AWS IAM Key Rotation Securely โ€” Zero Downtime, JWT Auth, Secrets Manager

Managing IAM credentials securely at scale is a critical challenge in modern DevOps workflows. Hardcoded or stale AWS access keys can be a huge security risk.

So, I built a complete automated key rotation system using:

  • JWT-secured FastAPI

  • Zero-downtime rotation logic

  • AWS Secrets Manager

  • CloudFormation deployment

  • API Gateway + IP Whitelisting

  • CloudWatch monitoring

And the best part? Itโ€™s reusable, scalable, and production-ready.


The Problem

Organizations often use programmatic IAM credentials for automation (CI/CD tools, scripts, pipelines). But:

  • These access keys often never get rotated.

  • Manual key rotation is slow and error-prone.

  • Hardcoded keys can leak or expire without notice.

We needed a system that would:

  1. Rotate keys automatically

  2. Provide fresh keys to approved users

  3. Secure API access (JWT + API key)

  4. Monitor everything centrally


The Solution: Secure IAM Key Rotation as a Service

๐Ÿ’ก Key Highlights

FeatureDetails
๐Ÿ”„ Auto RotationRotates keys via Lambda every 10 mins
๐Ÿ” API SecurityJWT (2 min token) + IP whitelist + API key
๐Ÿ“ฆ Infrastructure-as-CodeCloudFormation (key_rotation_simple.yaml)
๐Ÿ’พ Secret StorageAWS Secrets Manager
๐Ÿ“Š MonitoringCloudWatch logs
๐Ÿš€ Zero DowntimeNew key is created before old one is deleted

Stack Overview

  • Python 3.9

  • FastAPI

  • AWS Lambda

  • CloudFormation

  • Secrets Manager

  • Boto3

  • API Gateway

  • CloudWatch


How It Works โ€” Behind the Scenes

Key Rotation Logic

The core Lambda function:

  • Lists IAM users

  • Identifies existing access keys

  • Creates a new one

  • Marks old one as inactive

  • Deletes old key after confirmation

๐Ÿ‘‰ This ensures zero disruption to services using the credentials.


Deployment Guide

If youโ€™d like to deploy this project in your own AWS environment, follow these steps:


1. Clone the Repository

git clone https://github.com/yourusername/aws-key-rotation.git
cd aws-key-rotation/aws_key_rotation_task

2. Set Up a Python Virtual Environment

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install Dependencies

pip install -r requirements.txt

4. Configure AWS CLI

Make sure you have AWS CLI installed and configured with appropriate permissions:

aws configure

You will be prompted to enter:

  • AWS Access Key ID

  • AWS Secret Access Key

  • Default Region (e.g. us-east-1)

  • Output format (can leave blank or use json)


5. Deploy with the Python Script

Now run the deployment script to set everything up:

python deploy.py --admin-email your@email.com --sender-email same@email.com

What it does step-by-step:

  1. Optional Cleanup:

    • Deletes old CloudFormation stack

    • Deletes secrets

  2. IAM User Discovery:

    • Lists all existing IAM users
  3. Secrets Creation:

    • Generates API key + endpoint secret

    • Stores them securely in Secrets Manager

  4. Stack Deployment:

    • Uses key_rotation_simple.yaml to deploy:

      • Lambda function

      • IAM role

      • API Gateway

      • Logging + Security configs

  5. Prints Credentials:

    • Shows your API key & endpoint

    • Shows how to use the secure curl commands


๐Ÿงพ Output Example:

API Key: iam-key-rotation-us-east-1-52688...
API Endpoint: https://.../prod/active-key

# Generate Token
curl -X POST -H "x-api-key: ..." "https://.../generate-token"

# Use token to fetch credentials
curl -H "x-api-key: ..." -H "Authorization: Bearer YOUR_TOKEN" "https://.../active-key?username=test-user"

The CloudFormation Template โ€œkey_rotation_simple.yamlโ€

This CloudFormation template is the heart of the project. It defines all AWS resources โ€” infrastructure as code โ€” needed to automate IAM key rotation and expose a secure API.

Letโ€™s walk through it step-by-step:


1. Lambda Function: RotateKeyFunction

This function:

  • Runs the IAM key rotation logic

  • Is invoked every 10 minutes (for testing phase. Update it as per your use case ) via an event trigger

  • Has access to Secrets Manager and IAM APIs

What it does on each run:

  • Lists IAM users

  • Creates a new access key for each

  • Deactivates old keys

  • Stores the new key securely in Secrets Manager


2. IAM Role for Lambda Execution

A dedicated IAM role (LambdaExecutionRole) is created that:

Grants least-privilege access to:

  • iam:*AccessKey* and iam:ListUsers

  • secretsmanager:GetSecretValue, PutSecretValue

  • logs:* (for writing CloudWatch logs)

This ensures the Lambda can rotate keys and log securely โ€” nothing more, nothing less.


3. API Gateway: Secure Public Endpoints

Two RESTful endpoints are deployed via API Gateway:

EndpointPurpose
/generate-tokenIssues a short-lived JWT
/active-keyFetches current IAM credentials for a given user

Secured using:

  • API Keys

  • JWT Bearer token

  • Optional: IP Whitelisting via API Gateway config or Lambda logic


4. Secrets in AWS Secrets Manager

Secrets Manager is used to store:

  • The API Key

  • The JWT secret key for signing tokens

  • Active IAM user credentials (as needed)

Using AWS-native secret storage avoids hardcoding credentials and ensures secure retrieval during runtime.


5. CloudWatch Logs

Logging is automatically enabled for:

  • Lambda execution

  • API Gateway access logs

  • Rotation success/failure events

These logs help with debugging, auditing, and compliance.


Bonus: Logical Resource Names

  • KeyRotationAPI: Main API Gateway resource

  • RotateKeyFunction: Core logic handler (Lambda)

  • LambdaExecutionRole: Secure IAM role for Lambda

  • RotationTrigger: Timer that invokes Lambda every 10 minutes


Optional: Testing Scripts

These scripts are included but not part of the production setup:

ScriptPurpose
trigger_rotation.pyManual rotation trigger
test_rotation.pyTest IAM rotation logic

Security at Every Layer

  • JWT auth (2-minute expiry)

  • Secrets stored in Secrets Manager

  • API secured by:

    • IP Whitelist

    • API Key

    • Authorization Bearer Token

  • IAM permissions tightly scoped


Monitoring & Logs

Everything is automatically logged to AWS CloudWatch:

  • Rotation success/failures

  • API access logs

  • Security issues


๐Ÿ’ก Final Thoughts

This system gives you:

Peace of mind
Zero-touch key rotation
No manual mistakes
Compliance-friendly security
Reusable infra for any team


Interested?

Drop a comment or DM if you want to:

  • Try the system in your org

  • Get help adapting it to other workflows (e.g. rotating RDS creds)

  • Extend it with email/SMS alerts

Letโ€™s make cloud security better โ€” together. โ˜๏ธ

0
Subscribe to my newsletter

Read articles from Krishank Kaushik directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Krishank Kaushik
Krishank Kaushik