Project-10: Scalable and modular infrastructure setup for cloud environments using Terraform

Problem Overview

In typical infrastructure deployments, environments like dev, QA, and production might have different requirements (e.g., dev doesn’t need a load balancer or Route53). Managing these differences with a single Terraform codebase can lead to manual changes, which is inefficient. By breaking the code into modules, you can dynamically include/exclude components based on environment requirements, making the infrastructure easier to manage.

Solution

We break the infrastructure into the following modules:

• Network: VPC, subnets, routing

• Compute: EC2 instances (public and private)

• Security Groups (SG): For securing VPC resources

• NAT: NAT gateway for private instance internet access

• ELB: Elastic Load Balancers (optional)

• IAM: Identity and Access Management

Folder Structure

You can find the complete codebase for this project on GitHub: https://github.com/PakeezaPakeeza/Terraform-Modules-Project.git

Step-by-Step Setup

Step 1: Create the Network Module

The network module is the foundation of the infrastructure. It includes resources like the VPC, subnets, and routing configurations.

Files in /modules/network:

• vpc.tf: Defines the VPC and Internet Gateway.

• public_subnets.tf: Configures public subnets.

• private_subnets.tf: Configures private subnets.

• routing.tf: Creates routing tables for public and private subnets.

• variables.tf: Input variables required for the module.

• outputs.tf: Exports important values like VPC ID and subnet IDs.

• locals.tf: Sets local values for naming conventions or environment-based configuration.

Import the Network Module in Development

In /development/infra.tf, reference the network module:

module "dev_vpc_1" {
  source  = "../modules/network"
  vpc_cidr = var.vpc_cidr
  # Add other necessary variables
  ...
}

Deploy the Network Module

cd development
terraform init
terraform fmt
terraform validate
terraform apply

---

Step 2: Configure for Production

After testing in development, replicate the infrastructure for the production environment.

Steps:

1. Copy Files: Duplicate the setup from /development to /production.

2. Update Variables:

   • Ensure var.environment is set to "production".

Deploy for Production

cd production
terraform init
terraform fmt
terraform apply

---

Step 3: Add the Security Groups Module

The security groups module defines firewall rules for your infrastructure.

Files in /modules/sg:

• sg.tf: Contains security group configurations.

• variables.tf: Defines input variables like allowed ports and CIDR ranges.

• outputs.tf: Exports security group IDs for other modules to use.

Import the Security Groups Module in Development

Add the security groups module in /development/infra.tf:

module "dev_sg_1" {
  source = "../modules/sg"
  vpc_id = module.dev_vpc_1.vpc_id
  # Add other necessary variables
  ...
}

Deploy the Security Groups Module

cd development
terraform get
terraform apply

Replicate for Production:

Copy the security groups module configuration to /production, updating variables for production settings.

---

Step 4: Add the EC2 (Compute) Module

The compute module handles the creation of public and private EC2 instances.

Files in /modules/compute:

• private_ec2.tf: Provisions private EC2 instances.

• public_ec2.tf: Provisions public EC2 instances.

• variables.tf: Defines instance-related variables (e.g., AMI, instance type).

• outputs.tf: Exports EC2 instance IDs or other metadata.

Deploy EC2 in Development

Reference the compute module in /development/ec2.tf:

module "dev_compute_1" {
  source  = "../modules/compute"
  vpc_id  = module.dev_vpc_1.vpc_id
  sg_id   = module.dev_sg_1.sg_id
  # Add other necessary variables
  ...
}

Replicate for Production:

Follow the same process in /production, updating variables as required.

---

Step 5: Add the ELB Module

The ELB module provisions the Elastic Load Balancer, its listener, and associated configurations.

Files in /modules/elb:

• elb.tf: Configures the ELB resource.

• listener.tf: Configures ELB listeners.

• variables.tf: Defines input variables like subnets and target group configurations.

• outputs.tf: Exports ELB-related outputs like listener ARNs or DNS names.

Deploy ELB in Development

Reference the ELB module in /development/infra.tf:

module "dev_elb_1" {
  source = "../modules/elb"
  vpc_id = module.dev_vpc_1.vpc_id
  subnets = module.dev_vpc_1.public_subnets
  sg_id = module.dev_sg_1.sg_id
  # Add other necessary variables
  ...
}

Replicate for Production:

Follow the same process in /production, ensuring proper adjustments for production configurations.

---

Step 6: Add the NAT Gateway Module

The NAT gateway module allows private instances to access the Internet.

Files in /modules/nat:

• natgw.tf: Configures the NAT gateway.

• variables.tf: Input variables like subnet IDs.

• outputs.tf: Exports NAT gateway IDs.

Deploy NAT Gateway in Development

Add the NAT module to /development/infra.tf:

module "dev_natgw_1" {
  source           = "../modules/nat"
  vpc_id           = module.dev_vpc_1.vpc_id
  public_subnets_id = module.dev_vpc_1.public_subnets_id
  # Add other variables
  ...
}

Deploy for Production

Repeat the same process in /production, ensuring all variables are correctly configured.

Lets Test each of the environment one by one:

• Development Env:

    cd /development
     terraform init
     terraform plan
     terraform apply
     terraform state list
     terraform destroy
  

  Ec2-instances:

  

  Load Balancer:

  

  Network Infra created:

  

  Production Env:

    cd /production
     terraform init
     terraform plan
     terraform apply
     terraform state list
     terraform destroy
  

  Load Balancer:

  

  Ec2 Instances:

  

  Network Infra Created: