FeedbackHub Under Fire: Real-World AWS Auto Scaling, Alarms, and SNS Alerts in Action

🚀 Introduction

Recently, we put our FeedbackHub platform through a rigorous real-world scenario—testing its ability to handle intense traffic spikes. By leveraging AWS ECS Auto Scaling, CloudWatch Alarms, and SNS notifications, we validated our infrastructure's resilience and responsiveness.

In this article, we'll dive into:

• The stress test using 50,000 requests.

• How CloudWatch Alarms reacted to the CPU spike.

• ECS service scaling in real-time.

• Instant SNS notifications that kept us informed.

---

🧪 Load Test: Pushing the Limits

We simulated intense traffic using the load-testing tool hey:

hey -n 50000 -c 200 http://feedbackhub-production-alb-454348641.ap-south-1.elb.amazonaws.com

As expected, CPU usage soared to 74.51%, exceeding our test threshold set at 10% to observe the system's reaction.

---

📊 CloudWatch Alarms in Action

Our configured CloudWatch alarm, feedbackhub-service-cpu-high, instantly detected the spike:

• Threshold: Set at 10% for test purposes

• Alarm Status: Quickly transitioned to ALARM state and returned to OK once traffic reduced.

Alarm state verification via CLI:

aws cloudwatch describe-alarms --alarm-names feedbackhub-service-cpu-high

---

📬 SNS Email Notifications: Keeping Us Alerted

SNS promptly sent email notifications to our inbox (deepakaryan1988@gmail.com):

• High CPU Alert: Immediate email upon alarm trigger.

• CPU Normalized Alert: Notification once the system returned to safe levels.

These alerts meant we didn’t have to constantly check dashboards.

---

⚙️ ECS Scaling Events: Real-Time Adaptation

Our ECS service (feedbackhub-service on feedbackhub-production-cluster) automatically scaled up and down based on real-time metrics:

• Initially, the service had 1 running task.

• As CPU usage persisted, it scaled progressively: 1 → 3 → 4 → 5 tasks.

• When load eased, tasks scaled back down smoothly to 1 task.

Scaling events validation via CLI:

aws ecs describe-services
aws application-autoscaling describe-scaling-activities

---

🧠 AI Integration: Smart Log Summaries with AWS Bedrock

We integrated AWS Bedrock and Lambda to intelligently summarize ECS logs, simplifying troubleshooting:

• Summaries stored securely in S3 bucket: feedbackhub-production-lambda-summaries

• Provided quick, clear insights into system events during the scaling test.

---

🔗 Important Resources

• GitHub: FeedbackHub on AWS

• LinkedIn: Deepak Kumar

---

📌 Key Takeaways

• ECS Auto Scaling flawlessly handled traffic spikes.

• CloudWatch Alarms and SNS alerts ensured immediate visibility.

• AI log summarization greatly streamlined our monitoring workflow.

---

💬 Question for Readers: Have you stress-tested your ECS deployments? How effectively did your auto-scaling strategy handle real-world spikes?