Optimizing Infrastructure Scalability Using Terraform's COUNT Argument

Joel ThompsonJoel Thompson
6 min read

Introduction

Managing infrastructure at scale can be challenging, especially when deploying multiple similar resources. Terraform's count argument simplifies this process by allowing you to create multiple instances of a resource or module using an incrementing index. This is particularly useful when deploying identical or nearly identical resources, such as EC2 instances across multiple subnets.

In this tutorial, you'll:

  1. Provision a VPC, load balancer, and EC2 instances on AWS

  2. Use the count argument to dynamically deploy multiple EC2 instances in each private subnet

  3. Refactor Terraform configurations to improve scalability and maintainability

Prerequisites

Before starting, ensure you have the following:

  1. Latest version of Terraform installed

  2. An AWS Account with IAM permissions to create VPCs, EC2 instances, and load balancers

  3. HCP Terraform (formerly Terraform Cloud) account

  4. VS Code (or any preferred code editor)

  5. AWS CLI configured (optional, but recommended for testing)

Command to configure AWS credentials:
export AWS_ACCESS_KEY_ID="your_access_key"
export AWS_SECRET_ACCESS_KEY="your_secret_key"


Step 1: Clone the Starter Repository

Start by cloning the example repository:

git clone https://github.com/hashicorp-education/learn-terraform-count
cd learn-terraform-count

This repository contains a basic Terraform configuration for:

  1. A VPC with public and private subnets

  2. An Application Load Balancer (ALB)

  3. EC2 instances in private subnets

Step 2: Initialize and Apply the Configuration

  1. Set your HCP Terraform organization name (replace <YOUR_ORG>):
export TF_CLOUD_ORGANIZATION=<YOUR_ORG>
  1. Initialize Terraform:
terraform init

  1. Apply the configuration:

At this point, the configuration deploys only one EC2 instance per private subnet, which isn't scalable.

Step 3: Refactor the Configuration for Scalability

Problem with the Current Setup

The initial configuration hardcodes EC2 instances (app_a, app_b), making it inflexible. If you increase private_subnets_per_vpc, new instances won't be created automatically.

Solution: Use count for Dynamic Scaling

BEFORE: Original Configuration

variables.tf (Before)

variable "instances_per_subnet" {
  description = "Number of EC2 instances per private subnet"
  type        = number
  default     = 2  # Creates 2 instances per subnet
}

main.tf (Before)

resource "aws_instance" "app" {
  count = var.instances_per_subnet * length(module.vpc.private_subnets)
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.instance_type
  subnet_id     = module.vpc.private_subnets[count.index % length(module.vpc.private_subnets)]
  vpc_security_group_ids = [module.app_security_group.security_group_id]

  tags = {
    Name        = "${var.project_name}-instance-${count.index}"
    Environment = var.environment
    Project     = var.project_name
    Terraform   = "true"
  }
}

resource "aws_elb_attachment" "this" {
  count    = length(aws_instance.app)
  elb      = module.elb_http.this_elb_id
  instance = aws_instance.app[count.index].id
}

AFTER: Enhanced Configuration

variables.tf (After)

# ==============
# CORE SETTINGS
# ==============
variable "instances_per_subnet" {
  description = "Number of identical EC2 instances to deploy per private subnet (recommend 2+ for HA)"
  type        = number
  default     = 2
  validation {
    condition     = var.instances_per_subnet >= 1
    error_message = "Must deploy at least 1 instance per subnet for availability"
  }
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.micro"
}

# ==============
# RESOURCE TAGS
# ==============
variable "project_name" {
  description = "Project identifier for resource tagging (e.g. 'client-webapp')"
  type        = string
}

variable "environment" {
  description = "Deployment tier (dev/stage/prod)"
  type        = string
  default     = "dev"
}

variable "additional_tags" {
  description = "Custom tags to apply to all resources"
  type        = map(string)
  default     = {}
}

main.tf (After)

# ===================
# VPC CONFIGURATION
# ===================
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.0"

  name = "${var.project_name}-vpc"
  cidr = "10.0.0.0/16"
  azs  = ["us-east-1a", "us-east-1b"]

  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]
  enable_nat_gateway = true
}

# ===================
# SECURITY GROUP
# ===================
module "app_security_group" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "~> 4.0"

  name   = "${var.project_name}-sg"
  vpc_id = module.vpc.vpc_id

  ingress_with_cidr_blocks = [
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }
  ]
}

# ===================
# AMI DATA SOURCE
# ===================
data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

# ===================
# COMPUTE RESOURCES
# ===================
resource "aws_instance" "app" {
  count         = var.instances_per_subnet * length(module.vpc.private_subnets)
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.instance_type
  subnet_id     = module.vpc.private_subnets[count.index % length(module.vpc.private_subnets)]
  vpc_security_group_ids = [module.app_security_group.security_group_id]

  tags = {
    Name        = "${var.project_name}-instance-${count.index}"
    Environment = var.environment
  }
}

# ===================
# LOAD BALANCER
# ===================
resource "aws_lb" "app" {
  name               = "${var.project_name}-alb"
  load_balancer_type = "application"
  subnets            = module.vpc.public_subnets
  security_groups    = [module.app_security_group.security_group_id]
}

Key Changes:

  1. Declare a new variable in variables.tf:

  2. Refactor the EC2 resource block in main.tf:

    • Remove hardcoded instances (app_a, app_b)

    • Replace with a single aws_instance block using count:

How This Works:

  • count.index assigns a unique index (0, 1, 2, ...) to each instance

  • Modulo (%) ensures instances are distributed evenly across subnets

Update the Load Balancer Configuration:

Modify the aws_lb_target_group_attachment to dynamically attach all instances:

resource "aws_lb_target_group_attachment" "app" {
  count            = length(aws_instance.app)
  target_group_arn = aws_lb_target_group.app.arn
  target_id        = aws_instance.app[count.index].id
}

For your Terraform configuration using count, here are the corresponding "after" files when you change the instance count, along with whether they need manual updates:

1. Outputs.tf

Before:

output "instance_ids" {
  value = aws_instance.app[*].id
}

After (Enhanced):

output "instance_summary" {
  description = "Detailed instance distribution report"
  value = {
    total_instances  = length(aws_instance.app)
    instances_per_az = var.instances_per_subnet
    az_distribution = {
      for idx, instance in aws_instance.app :
      instance.id => element(module.vpc.azs, idx % length(module.vpc.private_subnets))
    }
  }
}

output "load_balancer_dns" {
  description = "DNS name of the application load balancer"
  value       = aws_lb.app.dns_name
}


Step 4: Test the Scalable Configuration

  1. Run terraform plan to preview changes

  2. Apply the updated configuration:

terraform apply

Now, Terraform will:

  1. Deploy 2 instances per private subnet (default)

  2. Distribute them evenly across subnets

  3. Automatically attach all instances to the load balancer

Step 5: Adjust Scaling Dynamically

Want more instances per subnet? Just update the variable:

variable "instances_per_subnet" {
  default = 3  # Now deploys 3 instances per subnet!
}

Re-run terraform apply, and Terraform will adjust accordingly.

Best Practices When Using count

  1. Use for identical resources - If instances need different configurations, consider for_each

  2. Avoid hardcoding indexes - Use count.index dynamically (e.g., subnet distribution)

  3. Combine with length() - Makes the configuration adapt to list/map changes

Conclusion

By using Terraform's count argument, you can:

  1. Eliminate repetitive code (no more app_a, app_b, etc.)

  2. Scale infrastructure dynamically (just update instances_per_subnet)

  3. Improve maintainability (fewer lines of code, easier updates)

Next Steps

  1. Experiment with for_each for non-identical resources

  2. Explore auto-scaling groups (ASGs) for even more flexibility

Additional Considerations

count vs. for_each

While count uses numeric indexes for identical resources, for_each creates resources from a map or set, making it better for:

  1. Unique resource configurations

  2. Preventing destroy/recreate when order changes

  3. Explicit resource naming via map keys

Limitations of count

  1. All instances must be identical in configuration

  2. Removing an item from the middle forces re-creation of subsequent resources

  3. Less readable than named resources (e.g., instance["web"] vs instance[2])

Auto Scaling Groups (ASGs) Alternative

For true dynamic scaling:

  1. ASGs automatically adjust instance count based on metrics/load

  2. Better integrated with load balancers (no manual target attachments)

  3. Supports instance replacements and rolling updates

Use count for simple static scaling, ASGs for dynamic workloads.

๐Ÿš€ Happy Terraforming!

0
Subscribe to my newsletter

Read articles from Joel Thompson directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Joel Thompson
Joel Thompson