Day 70 of 90 Days of DevOps Challenge: Getting Hands-On with AWS EC2

After understanding the fundamentals of networking protocols, such as DHCP and DNS, in yesterday’s session, I took the next big step in my cloud journey by launching compute power on the cloud using AWS EC2.

Today was all about diving deep into Amazon EC2 (Elastic Compute Cloud), understanding its architecture, learning how to create an instance step-by-step, exploring instance types, and grasping essential concepts like key pairs and security groups.

In this post, I’ll Walk you through everything I learned, especially the most useful information for anyone starting with AWS or aiming to master the compute layer of cloud infrastructure.

What is Amazon EC2?

Amazon EC2 (Elastic Compute Cloud) is a core AWS service that allows you to launch virtual servers called instances in the cloud. These instances run your applications and workloads just like a traditional server, but without the need to buy or manage hardware.

Why EC2 matters in DevOps:

• Run CI/CD pipelines

• Host web apps, APIs, or microservices

• Set up Dev/Test environments

• Automate provisioning with tools like Ansible or Terraform

EC2 Instance Architecture:

1. AMI (Amazon Machine Image)

• A pre-configured image that contains the OS and software stack.

• You choose an AMI while launching an instance (e.g., Ubuntu, Amazon Linux, Windows).

• You can also create custom AMIs for repeatable deployments.

2. Instance Types

EC2 provides various instance types based on CPU, memory, and network performance.

Examples: t2.micro (Free Tier, general purpose, m5.large (balanced workloads), c5.xlarge (compute optimized)

3. Storage:

1. EBS (Elastic Block Store)

• Persistent storage volumes attached to EC2.

• You can use it to store application files, databases, or logs.

• Volumes can be backed up via snapshots.

2. Instance Store

• Temporary storage tied to the life of the instance.

• Lost when the instance stops or terminates (not recommended for critical data).

4. Networking

• Every EC2 instance runs inside a VPC (Virtual Private Cloud) and a subnet.

• It receives:

  • A private IP for internal communication

  • An optional public IP for external access (if in a public subnet)

• Elastic IPs can be assigned for static public IP access.

5. Security Groups

• Virtual firewalls that control inbound and outbound traffic to EC2 instances.

• You must allow specific ports (like 22 for SSH, 80/443 for web) to connect.

Here’s a list of important port numbers that every DevOps engineer should know for working with servers, containers, firewalls, cloud networking, and troubleshooting:

Authentication & Remote Access

• 22 – SSH (Secure Shell): For remote access to Linux servers

• 3389 – RDP (Remote Desktop Protocol): Remote access to Windows servers

Web Traffic

• 80 – HTTP: Standard web traffic

• 443 – HTTPS: Secure web traffic (SSL/TLS)

File Transfer

• 20/21 – FTP: File Transfer Protocol (insecure)

• 22 – SFTP/SCP: Secure file transfer (uses SSH)

Database Ports

• 3306 – MySQL / MariaDB

• 5432 – PostgreSQL

• 1433 – Microsoft SQL Server

• 1521 – Oracle DB

• 27017 – MongoDB

• 6379 – Redis

Web Servers & Reverse Proxies

• 8080 – Alternative HTTP (often used by Tomcat, Jenkins)

• 8443 – Alternative HTTPS

• 8000–9000 – Common for local development servers

Email Services

• 25 – SMTP (Sending emails)

• 110 – POP3 (Receiving)

• 143 – IMAP

• 465 / 587 – SMTP (with SSL/TLS)

DevOps Tools

• 8080 – Jenkins default UI

• 9000 – SonarQube default port

• 3000 – Grafana / React apps

• 5601 – Kibana (ELK Stack)

• 9200 – Elasticsearch

• 9090 – Prometheus

• 9093 – Alertmanager

Container & Orchestration

• 2375 / 2376 – Docker API (unencrypted/encrypted)

• 6443 – Kubernetes API Server

• 10250–10255 – Kubelet & cluster communication

• 30000–32767 – Kubernetes NodePort services

6. Key Pairs

• Used to SSH securely into Linux-based EC2 instances.

• A key pair consists of a public key (stored on AWS) and a private key (downloaded to your machine).

Types of EC2 Instances

1. General Purpose Instances

Balanced compute, memory, and networking. Ideal for applications that use these resources in equal proportion.

Popular Families:

• T Series (e.g., t2, t3, t4g):

  • Burstable performance: CPU credits are earned when idle and spent under load.

  • Use Case: Web servers, dev/test environments, microservices.

  • Free Tier includes t2.micro and t3.micro.

• M Series (e.g., m5, m6g):

  • Steady performance with balanced resources.

  • Use Case: Application servers, small/medium databases, backend services.

2. Compute Optimized Instances

High-performance processors with a better CPU-to-memory ratio.

Popular Families:

• C Series (e.g., c5, c6g):

  • Optimized for compute-bound applications.

  • Use Case: Gaming servers, batch processing, ad serving, scientific modeling, HPC (high-performance computing).

3. Memory Optimized Instances

Designed to deliver fast performance for memory-intensive workloads.

Popular Families:

• R Series (e.g., r5, r6g):

  • More memory per vCPU.

  • Use Case: In-memory caches (Redis), real-time big data processing, high-performance databases.

• X Series (e.g., x1, x1e):

  • High memory capacities, up to several TB.

  • Use Case: SAP HANA, large-scale enterprise databases, high-performance computing.

• Z Series (e.g., z1d):

  • High memory with high CPU clock speeds.

  • Use Case: EDA (Electronic Design Automation), financial modeling, latency-sensitive apps.

4. Storage Optimized Instances

High disk throughput and IOPS (input/output operations per second).

Popular Families:

• I Series (e.g., i3, i4i):

  • NVMe SSD storage with high IOPS.

  • Use Case: NoSQL databases like Cassandra, low-latency workloads, transactional systems.

• D Series (e.g., d2, d3):

  • Dense HDD storage.

  • Use Case: Data warehousing, log processing, Hadoop clusters.

• H Series (e.g., h1):

  • High disk throughput with larger HDDs.

  • Use Case: MapReduce, distributed file systems, Big Data processing.

5. Accelerated Computing Instances

Powered by GPUs or FPGAs for hardware acceleration.

Popular Families:

• P Series (e.g., p3, p4):

  • NVIDIA GPUs for ML, AI, deep learning.

  • Use Case: Model training, large-scale ML inference, research computing.

• G Series (e.g., g4dn, g5):

  • Graphics-intensive instances.

  • Use Case: Game streaming, 3D rendering, remote desktops.

• F Series (e.g., f1):

  • Field Programmable Gate Arrays (FPGAs) for custom hardware logic.

  • Use Case: Cryptography, genomics, algorithm acceleration.

Instance Size and Pricing

Each family (like m5) comes in multiple sizes, such as:

• m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, etc.

This lets you scale horizontally or vertically based on workload and budget.

Launching an EC2 Instance

1. Choose AMI

   • Select an OS (e.g., Amazon Linux, Ubuntu, Windows).

2. Select Instance Type

   • Pick hardware config: CPU, memory (e.g., t2.micro for free tier).

3. Configure Instance Details

   • Set VPC, subnet, auto-assign public IP, IAM role.

   • Add user data scripts for automation.

4. Add Storage (EBS)

   • Default SSD volume (gp2/gp3), add more if needed.

5. Add Tags

   • Add key-value tags (e.g., Name=Web-Server) for resource tracking.

6. Configure Security Group

   • Add inbound rules (SSH - 22, HTTP - 80, HTTPS - 443).

   • Best practice: Restrict SSH to your IP.

7. Create/Select Key Pair (.pem file)

   • Required for SSH.

   • Download and save securely. you cannot recover it later.

8. Launch Instance

   • Wait for status checks → Connect via SSH.

   • Ready to install apps, deploy code, or monitor via CloudWatch.

Final Thoughts

Mastering EC2 is the first real hands-on experience in working with cloud computing infrastructure. It bridges your networking and Linux knowledge with actual, scalable virtual servers. Every DevOps workflow from deployment to monitoring starts with launching and managing EC2 instances.

Next, I’ll explore Elastic Load Balancers (ELB), Auto Scaling Groups (ASG), and Launch Templates to make applications highly available and resilient.

Let’s keep building cloud-native muscle! Stay tuned for Day 71!