AI Engineering: Deploy Streamlit application to AWS ECS & Automated CICD with Github actions

Hello Techies👋! I’m Samiksha, Hope you all are doing amazing stuff. I’m back with another blog about the Best MLOps practices for an AI Engineering. Today I’m going to cover the Deployment Stratergy for your AI Chatbot Application i.e. This Article covers Step by step process on how to deploy your Chatbot Streamlit application as a Container service on AWS ECS.

Please NOTE: This article is beginner with AWS Cloud + Docker image creation knowledge and for the professionals who wants to deploy their application on to the AWS ECS Cluster.

What you’ll get after Practically following this Article: You’'ll be able to deploy any of your application on to the AWS ECS cluster, Create Service for your task to launch as a Managed Service for long-running and production-grade setup.

If you are total beginner and wanted to learn these technologies, Follow teckbakers GenerativeAI series, where such topics are covered and going to cover soon.. Here: https://teckbakers.hashnode.dev/series/generative-ai

Pre-requisites:

1. AWS Account/CLI setup.

2. Docker installation/Docker Desktop: to test the images on local before deployment.

3. Github account.

BONUS: At the end We will automate the whole process from Container built from local to push the image to Container Registry → Create the ECS cluster → Create the task definitions → Create ECS managed service Through Github Actions..

Superb, Let’s Get into the journey to Load your container on to the ECS BOAT⛵️ & Let’s Ship your Chatbot Application.

Before going to the practical application setup, As I’m going to cover How I developed and deployed the streamlit application for “Advance Hybrid RAG AI consultant Chatbot“ using ECS. Please find the Code here: https://github.com/kolhesamiksha/Hybrid-Search-RAG.

Let’s First understand the Basic terminologies!!

What is ECS?

Amazon Elastic Container Service (ECS) is a fully managed container orchestration service from Amazon Web Services (AWS) that simplifies the deployment, management, and scaling of containerized applications at any scale. ECS orchestrates Docker containers, allowing developers to run, stop, and manage them without having to build and operate their own container management infrastructure. It offers a choice between a serverless compute plane with AWS Fargate or hosting containers on your own Amazon EC2 instances, and integrates deeply with other AWS services for security, scaling, and load balancing.

Key Features and Benefits

• Fully Managed Orchestration:

  ECS handles the complexities of managing your container cluster, including scheduling containers, scaling resources, and monitoring container health.

• Serverless or EC2 Options:

  • AWS Fargate: This is a serverless option that allows you to run containers without managing the underlying EC2 instances.

  • EC2 Launch Type: You can also run containers on a cluster of Amazon EC2 instances that you manage, giving you more control over the underlying infrastructure.

• Deep AWS Integration:

  ECS integrates with other AWS services like Elastic Load Balancing, Auto Scaling, Amazon VPC, and AWS IAM to build scalable and secure applications.

• Simplified Deployment:

  You can define and run applications using the AWS Management Console, AWS CLI, or API, making it easy to deploy and manage your containers.

• Advanced Security:

  It provides advanced security features for building and managing your containerized applications.

• Scalability:

  ECS allows you to easily scale your applications by scheduling tasks based on your resource needs and availability requirements.

• Monitoring:

  It works with Amazon CloudWatch to monitor the health and utilization of containers and clusters, allowing you to set up alarms for scaling and other events.

Why & When to use ECS if we have AWS EKS and AWS Lambda Functions to Automate?

Please NOTE: If you are new to Kubernetes concepts and wanted to learn it practically: checkout my blog on this concepts here: https://teckbakers.hashnode.dev/decoding-the-mlops-a-step-by-step-guide-to-setup-kubeflow-on-aws-using-microk8s && For In-detailed Devops Practices follow Devops & Cloud series of Teckbakers!!

AWS Elastic Container Service (ECS) remains a relevant choice for container orchestration despite the availability of AWS Elastic Kubernetes Service (EKS) and AWS Lambda Functions, due to specific use cases and operational preferences.

Reasons to use ECS:

• Simplicity and AWS-Native Integration:

  ECS offers a simpler, more opinionated approach to container orchestration, tightly integrated with other AWS services like Amazon EC2, Amazon S3, and Amazon CloudWatch. This can be beneficial for teams primarily operating within the AWS ecosystem and seeking a less complex management overhead than Kubernetes.

• Cost Efficiency for Specific Workloads:

  While EKS has a cluster management fee, ECS can be more cost-effective for certain workloads, especially when leveraging AWS Fargate for serverless container execution, or utilizing EC2 Spot Instances for cost optimization.

• Reduced Operational Overhead:

  For teams with limited Kubernetes expertise or those seeking to minimize infrastructure management, ECS (especially with Fargate) reduces the operational burden of managing the underlying container orchestration layer. AWS handles much of the heavy lifting.

• Batch Processing and Long-Running Tasks:

  ECS is well-suited for long-running services, batch processing jobs, and applications requiring persistent storage or complex networking configurations that might not be ideal for the ephemeral, event-driven nature of Lambda functions.

• Specific Use Cases for Containerization:

  When migrating legacy applications to containers, building microservices, or running specific Docker-based workloads without the need for the extensive features and ecosystem of Kubernetes, ECS provides a straightforward and efficient solution.

In essence, while EKS offers the power and flexibility of Kubernetes, and Lambda excels in event-driven, serverless computing, ECS provides a balanced solution for containerized applications within the AWS environment, prioritizing simplicity, cost-effectiveness, and seamless integration for specific use cases.

Now Let’s Delve into the Article Practically!!

1. Login to your AWS account on aws.amazon.com. then you can see the below screen.

• Checkout this Repository: https://github.com/kolhesamiksha/Hybrid-Search-RAG.where Dockerfile, CICD manifest file and the streamlit code to package is present which i will be reffering further.

• Containerize your application First:

  • Create a Dockerfile for your application, defining how to build its Docker image.

  • Build the Docker image using docker build.

  • Test the Docker image locally to ensure it functions as expected.

• Now Before pushing the Docker Image to AWS ECR, let’s first create the registry to store the Container Image first to AWS ECR.

• 1. Navigate to the AWS console first & search ECR. there click on create registry as Private Registry & then click on create Repository.

• 

  2. Provide the name of your repository: IMP - This repository name will be used to tag the docker image cuz, It’s mandatory to tag the image with same name as that of ECR repository name.

• 

• Now Push the Docker image to Amazon ECR (Elastic Container Registry):

  • Authenticate your Docker client with ECR.

  • Tag your local Docker image with the ECR repository URI.

  • Push the tagged image to your ECR repository.

After executing above you can see the Image with it’s tag: latest, got pushed to the ECR repository below:

Now We have Successfully pushed the Image to the ECR Registry, let’s first create the ECS cluster.

• In the Navigation bar search for ECS then you can see below Console for ECS service.

• AWS ECS Cluster:

  An ECS cluster is a logical grouping of resources where your containerized applications run. It serves as the foundation for your ECS services and tasks. Clusters can be provisioned with different types of infrastructure capacity:

  • Amazon EC2 instances:

    You manage and scale a fleet of EC2 instances that act as container instances within your cluster.

  • AWS Fargate:

    This is a serverless compute engine for containers, where AWS manages the underlying infrastructure, and you only pay for the resources consumed by your tasks.

• What is AWS FARGATE?

• using AWS's serverless compute engine, Fargate, with the Amazon Elastic Container Service (ECS) for running containerized applications without managing underlying servers or clusters of EC2 instances. You package your application in containers, define its resource needs (CPU, memory), and Fargate automatically provisions and scales the necessary compute infrastructure, allowing you to focus on application development rather than server management.

• Create an Amazon ECS Cluster:

  • Navigate to the Amazon ECS console.

  • Choose "Clusters" and then "Create Cluster."

  • Select your desired infrastructure:

    • AWS Fargate (serverless): AWS manages the underlying EC2 instances.

    • EC2 instances: You manage the EC2 instances that host your containers.

  • Configure the cluster name and other settings.

Now Before Creating the Service to manage our application, We have to create the task definitions first to link to the service, Because service manages the workloads don’t create the tasks.

• AWS Task Definition:

  An ECS service is designed to maintain a desired number of running tasks and to manage their deployment and scaling. The service needs to know what tasks to run, and this "what" is defined by the task definition. The task definition acts as a blueprint for your application, specifying details like:

• Container images: Which Docker images to use for your application's containers.

• CPU and memory: The resources required by your containers.

• Port mappings: How network traffic is directed to your containers.

• Environment variables: Configuration settings for your application.

• Volumes: How data is stored and accessed by your containers.

When you create an ECS service, you must link it to a specific task definition. This ensures that the service knows exactly how to launch and manage the containers that make up your application.

Hence Let’s first create the task definition:

1. Running Tasks:

• Definition:

  A task in ECS is an instantiation of a task definition. A task definition acts as a blueprint, specifying the container image(s) to use, CPU and memory allocation, port mappings, logging configuration, and other settings for your application.

• Use Case:

  Running a task directly is suitable for short-lived, one-off processes, batch jobs, or scheduled tasks, similar to how you might use a cron job. You explicitly launch a task, and it runs until completion or an error occurs.

2. Creating Services:

• Definition:

  An ECS service is used to maintain a desired number of running instances of a task definition within an ECS cluster. Services ensure the availability and scaling of your applications.

• Use Case:

  Services are ideal for long-running applications like web servers, APIs, or microservices that require continuous operation and potentially load balancing. They manage deployments, handle task failures by restarting new ones, and can integrate with load balancers for traffic distribution.

• Types:

  Services can be configured as "replica" (maintaining a specific desired count of tasks) or "daemon" (running one task on each container instance).

In summary, while you can directly run individual tasks in ECS for specific, short-term needs, services provide a robust and automated way to manage and scale long-running containerized applications, ensuring their availability and desired state.

1. Run Task: Let’s first run the task and see the working of the application then create the service.

The above task failed 😳 to run because we didn’t configure the IAM permissions to communicate in between two AWS services.. Hence Now Let’s setup the necessary IAM permissions. & create the customized task definition.

The task_definition.json is further we will automate through Github Actions. If you want to access the json file for this chatbot application. checkout here: https://github.com/kolhesamiksha/Hybrid-Search-RAG/blob/main/task-definition.json

follow this tutorial to setup the ECS task execution policy for your container: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html

Now after configuring the above 2 things, We can see the task running successfullylllyly🥳

• Create an Amazon ECS Service:

  • Navigate to your ECS cluster and choose "Services" then "Create."

  • Select the task definition you created.

  • Specify the desired number of tasks (instances of your application).

  • Configure networking (VPC, subnets, security groups).

  • Optionally, integrate with an Application Load Balancer (ALB) for traffic distribution and health checks.

  • Choose a deployment type (e.g., rolling update, blue/green with AWS CodeDeploy).

Follow this article by AWS to navigate creating service: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-service-console-v2.html

You can see below, My service “hybrid_rag_service“ for the task is configured and it’s running, You can also see the logs of the deployment below.

• Monitor and manage your application:

  • Use Amazon CloudWatch for logging and monitoring application performance.

  • Utilize AWS CodeDeploy for automated deployments and rollbacks, especially for blue/green deployments.

  • Scale your service based on demand by adjusting the desired task count.

Now, We have setup ECR registry, ECS cluster, ECS task definition & ECS service.. Let’s do follow the Above process in an Automated way Using Github Actions.

Automation using Github Actions:

NOTE: The task_definition.json must be present at the same level where action manifest file is present. hence, in the actions this will be get captured.

Now, let’s setup the credentials in the github secrets.

Hurrrayyyy🥳, After running the github actions it has successfully pushed run the code → built and push the Image to the ECR → from task_definition.json connected the ECR to ECS task and then further service.. And that’s how Everytime, new image will be get created and pushed to the service when we run the github actions.

Yeahhhh, You can see My Chatbot applcation is running on cloud, with self managed services configured by our ECS cluster we setup!!!!!!!!!!

You are now all set to deploy your chatbot application to AWS ECS cluster!

Please feel free to contribute to this article in comments, share your insights and experience in optimizing docker images for your Large scale application. This will help everyone to learn from each others experience!!.

till then, Stay tuned and follow our newsletter to get daily updates & Built Project End to end!! Connect with me on linkedin, github, kaggle.

Let's Learn and grow together:) Stay Healthy stay Happy✨. Happy Learning!!