Terraform

Arindam BaidyaArindam Baidya
50 min read

Table of contents

❓ What is IaC (Infrastructure as Code)?

Infrastructure as Code (IaC) is the practice of managing and provisioning IT infrastructure using code, instead of manual processes or interactive configuration tools.

With IaC, you define your infrastructure (servers, databases, networks, etc.) in machine-readable configuration files. This allows you to:

πŸ”§ Automate setup and configuration
πŸ” Repeat the same deployment consistently
πŸ› οΈ Version control infrastructure using Git
βš™οΈ Reduce errors caused by manual steps
πŸš€ Scale easily across environments (dev, staging, prod)


πŸ” Example Tools that Use IaC:

  • Terraform (multi-platform, cloud-agnostic) - (Provisioning Tool)

  • AWS CloudFormation (for AWS only)

  • Ansible, Chef, Puppet (focus on configuration management)

CategoryPurposeToolsDescription
Provisioning ToolsCreate and manage infrastructure resources (VMs, networks, etc.)Terraform, AWS CloudFormation, PulumiDefine and provision infrastructure using declarative or imperative code
Configuration ManagementInstall software and configure systems after provisioningAnsible, Chef, Puppet, SaltStackEnsure systems are configured consistently (e.g., install NGINX, apply security settings)
Orchestration ToolsCoordinate multiple tasks across machines or systemsKubernetes, Docker SwarmManage containerized applications and automate deployments
Image Building ToolsCreate machine or container images with pre-installed configurationsPacker, DockerfileBuild reusable machine images (e.g., AMIs, Docker images)
Secret Management ToolsSecurely manage and access sensitive informationVault, AWS Secrets Manager, Azure Key VaultManage credentials, API keys, and secrets for secure access
Policy as Code ToolsDefine and enforce security and compliance policiesOPA (Open Policy Agent), SentinelEnsure infrastructure follows organizational rules and compliance standards

🌐 Introduction to Terraform

Terraform is a powerful, open-source Infrastructure as Code (IaC) tool developed by HashiCorp. It allows you to provision, manage, and destroy infrastructure across a wide range of environmentsβ€”whether on public clouds like AWS, Azure, and GCP, or private/on-premises platforms such as vSphere.

Terraform uses a declarative language called HCL (HashiCorp Configuration Language), where you define the desired state of your infrastructure in simple configuration files (with .tf extension). Terraform then automatically figures out the steps needed to reach that state from the current oneβ€”handling all the provisioning logic for you.

It works in three key phases:

  1. terraform init – Initializes the project and configures providers (API connectors for each platform).

  2. terraform plan – Generates a detailed execution plan showing what changes will be made.

  3. terraform apply – Applies the necessary changes to reach the desired infrastructure state.

Terraform is resource-based, meaning it manages everything (VMs, databases, networks, etc.) as individual resources, taking care of their entire lifecycleβ€”from creation and configuration to decommissioning.

What is resource?

Resource is a object that Terraform manages. It could be a file on the local host or a virtual machine, services like s3 bucket, IAM users


HashiCorp Configuration Langauage (HCL)

Syntax:

<block> <params> {
    key1 = value1
    key2 = value2
}

Examples:

Provisioning an AWS EC2 instance

For creating an AWS S3 bucket

Workflows

  1. Write the configuration file

  2. Run terraform init command

  3. terraform plan for review the execution plan

  4. terraform apply to apply the changes

terraform show to show the resource we have created

Update and destroy resources in Terraform

terraform destroy to delete the infrastructure completly.

  • When we don’t want to produce output by the terraform plan and terraform apply commands printed on the screen.
resource "local_sensitive_file" "games" {
  filename     = "/root/favorite-games"
  content  = "FIFA 21"
}

🧩 Terraform Providers and the terraform init Command

When you run the terraform init command inside a directory that contains your Terraform configuration files, Terraform performs several important tasks:

πŸ”§ What terraform init Does:

TaskDescription
πŸ“₯ Downloads ProvidersIdentifies the providers used in the configuration and downloads the required plugins for them.
βš™οΈ Installs PluginsInstalls these provider plugins locally in a .terraform directory.
πŸ”„ Prepares BackendInitializes the backend configuration for storing state, if defined.

πŸ› οΈ Plugin-Based Architecture

Terraform is built with a plugin-based architecture, meaning it can integrate with hundreds of infrastructure platforms by using external plugins (providers). This makes it highly modular and extensible.


🌐 Terraform Registry

All major Terraform providers are:

πŸ“Š Three Tiers of Terraform Providers

TierDescriptionExamplesMaintainerSupport Level
Tier 1Official providers maintained and supported directly by HashiCorp.AWS, Azure, Google Cloud, KubernetesHashiCorpHigh (regular updates, docs, support)
Tier 2Providers maintained by partners or vendors, with some HashiCorp involvement.Datadog, Cloudflare, VMware vSpherePartner/Vendor + HCModerate (semi-official support)
Tier 3Community or third-party maintained providers, often for niche platforms.GitHub Actions, Netlify, UptimeRobotCommunityLow (may be outdated, limited support)

When we run terraform init it shows the version of the plugin that is being installed.


Configuration Directory

we can create configuration as many as we want in a single configuration ddirectory.

The common practive is to have one single configuration file with all the resource blocks required to provision the infrastructure. A single configuration file can have as many number of configuration clocks that we need.

Common naming convention used for such configuration file is to call it the main.tf.

Other configuration file can be created within the directory are variables.tf, outputs.tf, provider.tf.


Multiple providers

Terraform supports the use of multiple providers within the same configuration. to ilustrate this let’s use of another provider called random. This provider allows us to create random resources, such as random ID, a random integer, a random password, etc. Let us create a resources called random pet. This resource type will generate a random pet when applied.

resource "local_file" "pet" {
    filename = "/root/pets.txt"
    content = "We love pets!"
}

resource "random_pet" "my-pet" {
    prefix = "Mrs"
    separator = "."
    length = "1"
}

Here, for random resource we have used three arguments

FieldDescription
prefixAdds a custom prefix to the name (e.g., Mrs).
seperatorSpecifies what character separates the prefix and name (typo β†’ should be separator).
lengthNumber of words (pet name components) to generate (1 = single word).

πŸ” What terraform init Will Do:

ProviderStatus
localβœ… Already installed earlier β†’ will be reused
randomπŸ”„ Not yet installed β†’ Terraform will download it

πŸ› οΈ Terminal Output Example (Expected):

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/local from the dependency lock file
- Finding latest version of hashicorp/random...
- Installing hashicorp/random v3.5.1...
- Installed hashicorp/random v3.5.1 (signed by HashiCorp)

Terraform has been successfully initialized!

Input variables

resource "local_file" "pet" {
    filename = "/root/pets.txt"
    content = "We love pets!"
}

resource "random_pet" "my-pet" {
    prefix = "Mrs"
    separator = "."
    length = "1"
}

This way of configuration file are hard-coded and not reusable, which resist the rule of using IAC. To improve our technique, we can use another file for varible like variables.tf, from which our configuation file will collect the defined variables.

Variable Blocks

Variable block in Terraform accepts three parameters.

  1. defaults (Optional)

  2. type (optional)

  3. description (optional)

type arguments are optional, but when used, it enforces the type of variable being used. If it is not specified in the varibale block, it is set to the type Any by default.

Other types of variable types rather string, number, bool

List β†’ Can have duplicate variables

Map

Set β†’ Similar to a List but can’t have duplicates values

Objects β†’ Combinned of all (Complex data structure)

Tuples β†’ Similar to List but here we can use elements of different variable types

Using variables in Terraform (Multiple ways)

  1. When default parameter in variable block are empty.

It will allows user to enter each variable in interactive mode.

  1. Command line flags (-var β€œvariable_name” = β€œvalue”)

  1. Using environment variables with TF_VAR_<name_of_the_declared_variable>="value"

  1. Variable definition file (When need to use lots of variables) - Custom variable

Variable Definition Precedence

When we use multiple ways to take value it take value by maintaining some rule

Resource Attributes Reference

We will learn how to link to resources by making use of resource attributes.

Initial State

After making resource attributes (referencing) - reference expression

Resource Dependencies

Here we are explicitly mentioning the local_file resource is depends on random_pet resource. And, so random_pet resource will first created, and during deletion, it will deleted at the last. Here, we are not declaring what resource should created first, but Terraform will do that for us.

Here, we are using depends_on to create explicit dependency.

Output Variables

Here, in output block, mandatory argument for value is the reference expression.

terraform apply can be used to see the output in the terminal. Once the resource has been created we can run terraform output command to print the value of the output variables. This command will print all the output variabels defined in the current configuration directory. we can also use the output command to specificlly print the value of an existing output variable like below:

πŸ“„ Terraform State (terraform.tfstate)

πŸ”Ή What is Terraform State?

Terraform uses a state file (terraform.tfstate) to track the current state of the infrastructure it manages. The content of this file is in json format.

Terraform uses state file to map the resource configuration to the real world infrastructure.


🧠 Why is it important?

  • Keeps a record of all deployed resources

  • Helps Terraform determine what needs to be added, changed, or destroyed

  • Enables incremental changes without affecting existing infrastructure


πŸ“Œ Key Points:

FeatureDescription
terraform.tfstateCreated automatically after executing terraform apply at least once
LocationStored locally by default, but can be stored remotely (e.g., S3, GCS)
SyncingMust always be in sync with actual infrastructure state
Sensitive InfoMay contain secrets (use remote encrypted storage and restrict access)

πŸ“„ Purpose of Terraform State

Terraform needs to keep track of what it has created so it can manage your infrastructure correctly. That’s why it uses a state file called terraform.tfstate.


βœ… Why is Terraform State Important?

  1. Records What Exists
    The state file stores details about all resources Terraform manages β€” like IDs, IP addresses, and names. This helps Terraform remember what it deployed.

  2. Detects Changes
    When you run terraform plan, Terraform compares the current state (in terraform.tfstate) with your configuration code to figure out what has changed.

  3. Prevents Re-Creation
    Without state, Terraform wouldn’t know what resources already exist β€” and would try to recreate everything each time.

  4. Supports Collaboration
    When using remote state (e.g., AWS S3), teams can safely share and work on infrastructure without conflicts.

  5. Stores Metadata
    It keeps track of dependencies and links between resources to apply changes in the correct order.

πŸ“Œ Terraform State Considerations

Managing Terraform state properly is crucial for safe and predictable infrastructure changes. Here are key considerations:


1. 🧠 State File Is Critical

  • The terraform.tfstate file contains the full record of Terraform-managed infrastructure.

  • If it's lost or corrupted, Terraform cannot track or manage your resources properly.


2. πŸ” Sensitive Information

  • State files may contain sensitive data like passwords, API keys, or IP addresses.

  • Always encrypt state files and restrict access (especially in teams).


3. ☁️ Use Remote State for Teams

  • For team environments, use remote backends (e.g., AWS S3, Azure Blob, GCS) to:

    • Centralize the state file

    • Prevent accidental overwrites

    • Enable locking (e.g., DynamoDB for AWS)


4. ♻️ Do Not Manually Edit

  • Avoid editing the state file directly unless absolutely necessary (and with backups).

  • Instead, use commands like terraform state mv, rm, or import.


5. πŸ”’ Use State Locking

  • Prevents multiple users or pipelines from modifying the same state file at once.

  • Most remote backends (e.g., S3 + DynamoDB) support state locking.


6. πŸ“ Store in Version Control?

  • ❌ Do NOT commit terraform.tfstate or .terraform/ directories to version control.

  • βœ… You can track .tfstate.backup or version-controlled *.tf files β€” but never the actual state file itself in Git.


7. πŸ“€ State Can Be Split

  • For large projects, you can split infrastructure into multiple state files (using workspaces or modules) to improve manageability.

Terraform Commands

After making the configuration file:

terraform validate to check syntax in configuration file used is correct. And the error (if exist) with hint to fix it.

terraform fmt scans the configuration files in current working directory and formats the code into a canonical format.

terraform show prints out the current state of the infrastructure as seen by terraform.

terraform show -json prints the content in json format.

terraform providers list of all providers used in configuration directory.

terraform providers mirror /root/terraform/new_local_file to copy provider plugins needed for the current configuration to another directory.

terraform output to print all the output in the configuration directory.

terraform output <pet-name> output of a specific variable.

terraform apply -refresh-only used to sync terraform with the real-world infrastructure. For example, if there are any changes made to a resource created by terraform outside its control such as manual update, the terraform refresh command will pick it up and update the state file. The reconcilation is useful to determine what action to take during the next apply. This command will not modify any infrastructure resource but it will modify the state file.

terraform graph used to create a visual representation of the dependencies in a terraform configuration or an execution plan. The graph generated in a format called dot.

πŸ†š Mutable vs Immutable Infrastructure in IaC

AspectMutable InfrastructureImmutable Infrastructure
πŸ” DefinitionYou update the existing infrastructure in placeYou replace existing infrastructure with new ones
βš™οΈ Change MethodModify (upgrade, patch) existing servers/resourcesDestroy and recreate with updated configuration
πŸ› οΈ Common ExampleSSH into a server and run updatesReplace an old AMI with a new one during deployment
πŸ”„ State BehaviorInfrastructure state is changed in-placeInfrastructure is discarded and rebuilt
⚠️ RisksDrift, inconsistency, configuration rotMore reliable, predictable, clean state
πŸš€ Use CaseQuick patches or dev environmentsProduction deployments, container-based apps

πŸ“˜ Example

πŸ”§ Mutable:

You update an EC2 instance in place:

resource "aws_instance" "web" {
  ami           = "ami-123"
  instance_type = "t2.micro"
  user_data     = "apt update && apt install nginx"
}

Later, you just change the user_data script and run terraform apply. The instance stays the same, but its config changes β€” mutable behavior.


🧱 Immutable:

You change the AMI to a new, pre-configured one:

resource "aws_instance" "web" {
  ami           = "ami-456" # new AMI with pre-installed nginx
  instance_type = "t2.micro"
}

Terraform will destroy the old instance and create a new one β€” immutable behavior.


βœ… What Does Terraform Use?

Terraform supports both, but it naturally leans toward immutable infrastructure.

🧠 Why? Because:

  • Resources are declared declaratively

  • Changing a property often leads to recreating the resource (e.g., changing AMI, volume size)

  • This ensures a clean, predictable state

πŸ” Terraform Lifecycle Rules

In Terraform, the lifecycle block inside a resource lets you customize how Terraform manages resource creation, update, and deletion. This helps handle complex infrastructure scenarios more safely and predictably.


🎯 Purpose:

To control the behavior of Terraform when a resource is:

  • Re-created

  • Changed

  • Deleted


πŸ”‘ Three Primary Lifecycle Meta-Arguments

Meta-ArgumentDescription
create_before_destroyEnsures a new resource is created before destroying the old one. Useful to avoid downtime.
prevent_destroyPrevents a resource from being accidentally destroyed. Terraform will throw an error if a destroy is attempted.
ignore_changesTells Terraform to ignore specific attributes even if they change outside Terraform (e.g., manually or by automation).

πŸ“˜ Examples

1️⃣ create_before_destroy

resource "aws_instance" "example" {
  ami           = "ami-123"
  instance_type = "t2.micro"

  lifecycle {
    create_before_destroy = true
  }
}

βœ… Ensures a new EC2 instance is created before destroying the old one β€” useful for zero-downtime deployments.


2️⃣ prevent_destroy

resource "aws_s3_bucket" "important" {
  bucket = "my-critical-logs"

  lifecycle {
    prevent_destroy = true
  }
}

βœ… Prevents accidental deletion of important S3 buckets β€” Terraform will error out if a destroy is attempted.


3️⃣ ignore_changes

resource "aws_instance" "web" {
  ami           = "ami-abc"
  instance_type = "t2.micro"

  lifecycle {
    ignore_changes = [ami]
  }
}

βœ… If someone changes the AMI manually in the cloud, Terraform won’t try to revert it during future applies.

πŸ“˜ Terraform Data Sources

πŸ” What is a Data Source?

A data source in Terraform allows you to fetch information from external sources or existing infrastructure without creating or modifying them.

βœ… Think of data sources as read-only lookups.


🧠 Why Use Data Sources?

  • To reference existing infrastructure (e.g., an existing AWS AMI, VPC, or S3 bucket).

  • To fetch dynamic values that are managed outside of Terraform.

  • To use outputs from one module in another without duplication.


πŸ“¦ Example:

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }
}

resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t2.micro"
}

🧾 Here, we use a data source to get the latest Ubuntu AMI, and then use that AMI to launch an EC2 instance.


πŸ” Resource vs Data Source: Key Differences

AspectResourceData Source
PurposeCreates, updates, or deletes infrastructureReads existing infrastructure or external data
BehaviorManages lifecycle of infrastructureRead-only access
Exampleresource "aws_instance"data "aws_ami"
State FileTracked in Terraform stateReferenced but not created or modified
Use CaseDeploying EC2, S3, VMs, databases, etc.Fetching AMI IDs, VPC info, secrets, etc.
Can be destroyed?βœ… Yes❌ No – only used for reading

πŸ”’ What is count in Terraform?

The count meta-argument in Terraform allows you to create multiple instances of a resource using a single configuration block.

It’s like a loop that helps with scaling resources easily.


βœ… 1. Using count with a fixed number

πŸ”§ main.tf

resource "local_file" "notes" {
  count    = 3
  filename = "file_${count.index}.txt"
  content  = "This is file number ${count.index}"
}

This creates 3 files:

  • file_0.txt

  • file_1.txt

  • file_2.txt


βœ… 2. Using count with a variable

We’ll now control the number of resources using an input variable.


πŸ“ File: variables.tf

variable "file_count" {
  description = "How many files to create"
  type        = number
  default     = 2
}

πŸ“ File: main.tf

resource "local_file" "notes" {
  count    = var.file_count
  filename = "note_${count.index}.txt"
  content  = "This is note ${count.index}"
}

When you change the file_count value (e.g., via terraform.tfvars or CLI), it changes how many files get created.


βœ… 3. Using count = length(var.list)

When you have a list of values, you can dynamically control the count using length().


πŸ“ File: variables.tf

variable "file_names" {
  description = "List of filenames"
  type        = list(string)
  default     = ["math", "science", "history"]
}

πŸ“ File: main.tf

resource "local_file" "subjects" {
  count    = length(var.file_names)
  filename = "${var.file_names[count.index]}.txt"
  content  = "This file is about ${var.file_names[count.index]}"
}

Terraform creates:

  • math.txt

  • science.txt

  • history.txt

πŸ” Terraform for_each Meta-Argument

πŸ” What is for_each?

for_each is a Terraform meta-argument used to create multiple instances of a resource or module by looping over a set or map.

Unlike count, for_each gives you more control and clarity, especially with named items.


βœ… Use Case 1: Using for_each with a set (directly from variables.tf)


πŸ“ variables.tf

variable "file_names" {
  description = "Set of file names"
  type        = set(string)
  default     = ["alpha", "beta", "gamma"]
}

πŸ“ main.tf

resource "local_file" "my_files" {
  for_each = var.file_names
  filename = "${each.key}.txt"
  content  = "This is file named ${each.key}"
}

βœ… Result:

Terraform will create:

  • alpha.txt

  • beta.txt

  • gamma.txt


βœ… Use Case 2: Using for_each with a list, but convert it to a set

for_each cannot be used directly with a list β€” it must be a set or map. So we convert the list to a set using toset().


πŸ“ variables.tf

variable "topics_list" {
  description = "List of topics"
  type        = list(string)
  default     = ["devops", "cloud", "terraform"]
}

πŸ“ main.tf

resource "local_file" "topic_files" {
  for_each = toset(var.topics_list)
  filename = "${each.key}.md"
  content  = "This topic is about ${each.key}"
}

βœ… Result:

Terraform will create:


🧠 Summary: for_each vs count

Featurefor_eachcount
Input Typesset or mapnumber or expression
Resource Accesseach.key, each.value (if map)count.index
UniquenessNamed instancesIndexed instances
Flexibilityβœ… Better for identifying unique items🚫 Less flexible with maps/sets

πŸ“Œ Terraform Version Constraints

πŸ” What Are Version Constraints?

Version constraints in Terraform allow you to control which version of a provider (like hashicorp/local) Terraform installs. This avoids breaking changes from new major versions and ensures your infrastructure stays stable.


βœ… Without Version Constraint

πŸ“ main.tf

resource "local_file" "pet" {
  filename = "/root/pet.txt"
  content  = "We love pets!"
}

πŸ“¦ When you run:

$ terraform init

You’ll see:

The following providers do not have any version constraints in configuration, so the latest version was installed.
To prevent automatic upgrades to new major versions, we recommend adding version constraints.

βœ… Best Practice: Always define a version constraint to ensure consistent behavior.


βœ… Adding Version Constraints

πŸ“ main.tf

terraform {
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = "1.4.0"
    }
  }
}

resource "local_file" "pet" {
  filename = "/root/pet.txt"
  content  = "We love pets!"
}

πŸ“¦ Output after terraform init:

Installing hashicorp/local v1.4.0...
Terraform has been successfully initialized!

🎯 Different Version Constraints

Constraint TypeExampleMeaning
Exact version"1.4.0"Only install version 1.4.0
Greater than"> 1.1.0"Any version newer than 1.1.0
Less than"< 1.4.0"Any version older than 1.4.0
Not equal to"!= 2.0.0"Exclude version 2.0.0
Range with exclusion"> 1.2.0, < 2.0.0, != 1.4.0"Between 1.2.0 and 2.0.0, except 1.4.0
Pessimistic (~>) version"~> 1.2"Compatible patch versions, e.g., >= 1.2.0, < 2.0.0
Pessimistic with patch"~> 1.2.0"Patch updates only, e.g., >= 1.2.0, < 1.3.0

βœ… Example: Using ~> (Pessimistic Constraint)

πŸ“ main.tf

terraform {
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = "~> 1.2.0"
    }
  }
}

resource "local_file" "pet" {
  filename = "/root/pet.txt"
  content  = "We love pets!"
}

πŸ“¦ Output after terraform init:

Installing hashicorp/local v1.2.2...
Terraform has been successfully initialized!

ℹ️ Installed version is within the allowed patch range (>= 1.2.0 and < 1.3.0).


πŸ“˜ Getting Started with AWS

πŸ”₯ Why AWS?

AWS (Amazon Web Services) is a leader in cloud infrastructure services, recognized by Gartner for over 10 years.

πŸ“Œ Source: AWS Named a Cloud Leader - Gartner MQ

βœ… AWS Core Services:

CategoryExamples
ComputeEC2 (Elastic Compute Cloud)
StorageS3, EBS
DatabasesDynamoDB, RDS
AnalyticsAthena, Redshift
Machine LearningSageMaker
IoTIoT Core
NetworkingVPC, Route 53
  • US: Ohio, Oregon, N. California, GovCloud (West/East)

  • Europe: London, Frankfurt, Ireland, Paris, Milan

  • Asia Pacific: Mumbai, Tokyo, Hong Kong, Sydney

  • Others: SΓ£o Paulo, Beijing, Canada (Central)


πŸ› οΈ AWS with Terraform

Terraform allows Infrastructure as Code (IaC), letting you automate AWS resources like:

AWS ServiceTerraform Resource Example
EC2aws_instance
S3aws_s3_bucket
DynamoDBaws_dynamodb_table
VPCaws_vpc
Route 53aws_route53_zone
EBSaws_ebs_volume

πŸ‘€ AWS IAM (Identity and Access Management)

Concepts:

  • Root User: Default AWS admin (avoid daily use)

  • IAM Users: Individual accounts for people/applications

  • IAM Groups: Group permissions (e.g., Developer Group)

  • Policies: JSON-based permissions (e.g., AdministratorAccess, AmazonEC2FullAccess)

IAM Policy Example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*"
    }
  ]
}

🧩 Policy Name Examples:

  • AdministratorAccess

  • Billing

  • AmazonS3FullAccess

  • AmazonEC2FullAccess


πŸ” Programmatic Access

How to Configure AWS CLI:

$ aws configure
AWS Access Key ID: <YOUR_ACCESS_KEY>
AWS Secret Access Key: <YOUR_SECRET_KEY>
Default region name: us-west-2
Default output format: json

Config files:

~/.aws/config
~/.aws/credentials

Useful CLI Commands:

aws iam create-user --user-name lucy
aws s3api create-bucket --bucket my-bucket --region us-east-1
aws ec2 describe-instances

πŸ‘‰ CLI Reference: https://docs.aws.amazon.com/cli/latest/reference


πŸ“¦ Installing AWS CLI

Linux/Mac:

$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install
$ aws --version

Windows:

Download from: https://awscli.amazonaws.com/AWSCLIV2.msi


🌐 Creating IAM Users on AWS with Terraform


πŸ”° Introduction

IAM (Identity and Access Management) in AWS allows you to manage access to AWS services and resources securely. With Terraform, you can automate IAM user creation, assign policies, and tag users consistently across environments.


🧱 Terraform Block Structure

resource "aws_iam_user" "admin-user" {
  name = "lucy"
  tags = {
    Description = "Technical Team Leader"
  }
}
BlockDescription
aws_iam_userTerraform resource type
admin-userLocal name within your configuration
nameIAM user name in AWS (e.g., lucy)
tagsOptional metadata for the user

βœ… Full main.tf Example with Provider

provider "aws" {
  region     = "us-west-2"
}

resource "aws_iam_user" "admin-user" {
  name = "lucy"
  tags = {
    Description = "Technical Team Leader"
  }
}

πŸ”‘ Providing AWS Credentials (3 Methods)

  1. Inline in provider block (Not recommended for production):
provider "aws" {
  region     = "us-west-2"
  access_key = "AKIAI44QH8DHBEXAMPLE"
  secret_key = "je7MtGbClwBF/2tk/h3yCo8nvbEXAMPLEKEY"
}
  1. Using environment variables (Recommended):
$ export AWS_ACCESS_KEY_ID=AKIAI44QH8DHBEXAMPLE
$ export AWS_SECRET_ACCESS_KEY=je7MtGbClwBF/2tk/h3yCo8nvbEXAMPLEKEY
$ export AWS_REGION=us-west-2
  1. Using ~/.aws/credentials and ~/.aws/config:
# ~/.aws/credentials
[default]
aws_access_key_id = AKIAI44QH8DHBEXAMPLE
aws_secret_access_key = je7MtGbClwBF/2tk/h3yCo8nvbEXAMPLEKEY

# ~/.aws/config
[default]
region = us-west-2
output = json

βš™οΈ Terraform Commands

  1. Initialize the configuration

     terraform init
    
  2. Review the execution plan

     terraform plan
    
  3. Apply the configuration

     terraform apply
    

πŸ’‘ Example Output After apply

aws_iam_user.admin-user: Creating...
aws_iam_user.admin-user: Creation complete after 1s [id=lucy]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

πŸ” Creating IAM Policies with Terraform


βœ… Objective

In this guide, you’ll learn how to:

  • Create an IAM user (lucy)

  • Define a custom IAM policy (AdminUsers)

  • Attach the policy to the user using aws_iam_user_policy_attachment

  • Use Heredoc syntax (<<EOF) to define inline JSON policy


🧱 Complete main.tf Example

hclCopyEditprovider "aws" {
  region = "us-west-2"
}

# 1. Create an IAM User
resource "aws_iam_user" "admin-user" {
  name = "lucy"
  tags = {
    Description = "Technical Team Leader"
  }
}

# 2. Define an IAM Policy using Heredoc Syntax
resource "aws_iam_policy" "adminUser" {
  name = "AdminUsers"
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*"
    }
  ]
}
EOF
}

# 3. Attach the Policy to the User
resource "aws_iam_user_policy_attachment" "lucy-admin-access" {
  user       = aws_iam_user.admin-user.name
  policy_arn = aws_iam_policy.adminUser.arn
}

πŸ“œ Policy Explained

This policy grants Administrator Access by:

  • Allowing all actions ("Action": "*")

  • On all resources ("Resource": "*")

  • Which is equivalent to the AdministratorAccess managed policy.


πŸ”§ Terraform Commands

  1. Initialize Terraform
terraform init
  1. Preview the Plan
terraform plan
  1. Apply the Configuration
terraform apply

βœ… Sample Output (Simplified)

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

aws_iam_user.admin-user: Created [id=lucy]
aws_iam_policy.adminUser: Created [id=arn:aws:iam::123456789012:policy/AdminUsers]
aws_iam_user_policy_attachment.lucy-admin-access: Created [id=lucy-xyz123]

πŸ“ Option: External Policy File

If you have a policy JSON file like admin-policy.json:

Contents:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*"
    }
  ]
}

Updated Terraform:

resource "aws_iam_policy" "adminUser" {
  name   = "AdminUsers"
  policy = file("admin-policy.json")
}

🧠 Pro Tip: Use Version Constraints

Add a version block to prevent provider upgrades that might break your setup:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.6.0"
    }
  }
}

πŸͺ£ Getting Started with AWS S3 (Simple Storage Service)


πŸ“Œ What is Amazon S3?

Amazon S3 (Simple Storage Service) is a scalable object storage service used to store and retrieve any amount of data at any time from anywhere on the web.


πŸ“‚ S3 Storage Structure

S3 organizes data in a flat structure using:

ComponentDescription
BucketTop-level container for storing objects
ObjectFile stored in a bucket (e.g. .jpg, .mp4)
KeyUnique identifier for each object (like a filepath)
ValueThe actual file/data (object content)
MetadataData about the object (size, owner, timestamp, etc.)

πŸ“ Example Bucket: all-pets (us-west-1)


πŸ” Permissions and Access Control

There are two major ways to control access:

1. ACL (Access Control List)

  • Assigned to individual objects

  • Example: dog.jpg β†’ Only Lucy can read

2. Bucket Policy

JSON-based permission rules at the bucket level.

Example: read-objects.json – allow Lucy to GetObject from bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::all-pets/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::123456123457:user/Lucy"
        ]
      }
    }
  ]
}

🧠 Important Notes

FeatureDetails
Bucket NameMust be globally unique and DNS-compliant
Max Object SizeUp to 5 TB
URL Formathttps://<bucket-name>.<region>.amazonaws.com/<key>
OwnerUser who created the bucket/object
Last ModifiedTracks last time an object was updated

🧾 Creating and Managing AWS S3 Buckets with Terraform


πŸͺ£ 1. Create an S3 Bucket

To provision an S3 bucket using Terraform:

resource "aws_s3_bucket" "finance" {
  bucket = "finanace-21092020"  # Must be globally unique
  tags = {
    Description = "Finance and Payroll"
  }
}

βœ… After running terraform apply, the output:

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Bucket ID: finanace-21092020

πŸ“„ 2. Upload a File to the Bucket

To upload a document (e.g. finance-2020.doc) to the S3 bucket:

resource "aws_s3_bucket_object" "finance-2020" {
  bucket  = aws_s3_bucket.finance.id
  key     = "finance-2020.doc"
  content = file("/root/finance/finance-2020.doc")
}

πŸ’‘ The key represents the object's name (like a file path), and content is loaded from your local filesystem.


πŸ‘₯ 3. Attach IAM Group & Define Permissions

IAM Group (Data Source)

data "aws_iam_group" "finance-data" {
  group_name = "finance-analysts"
}

πŸ” 4. Set Bucket Policy to Allow Group Access

Create Bucket Policy with <<EOF Heredoc syntax:

resource "aws_s3_bucket_policy" "finance-policy" {
  bucket = aws_s3_bucket.finance.id
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "arn:aws:s3:::${aws_s3_bucket.finance.id}/*",
      "Principal": {
        "AWS": [
          "${data.aws_iam_group.finance-data.arn}"
        ]
      }
    }
  ]
}
EOF
}

🧠 Tip: <<EOF is heredoc syntax to write multiline JSON inside HCL.


πŸ§ͺ 5. Apply Everything

Run:

terraform apply

You will see Terraform output showing resources like:

βœ… All created successfully!


🧾 Introduction to Amazon DynamoDB


πŸ”Ή What is DynamoDB?

Amazon DynamoDB is a:

  • πŸ”Έ Fully managed, serverless, NoSQL database service by AWS.

  • πŸ”Έ Designed for high performance with single-digit millisecond latency.

  • πŸ”Έ Provides horizontal scaling and multi-region replication out-of-the-box.


βœ… Key Features

FeatureDescription
Fully ManagedNo need to manage servers or infrastructure
Highly ScalableHandles millions of requests per second seamlessly
Low LatencySingle-digit millisecond read/write
NoSQLFlexible schema, great for unstructured/semi-structured data
Global TablesReplicate data across multiple AWS regions
Built-in SecurityEncryption at rest, IAM access controls

πŸ—‚ Sample Table: cars

Let's consider a DynamoDB table storing car inventory:

πŸ“Œ Sample Items

{
  "Manufacturer": "Toyota",
  "Make": "Corolla",
  "Year": 2004,
  "VIN": "4Y1SL65848Z411439"
}
{
  "Manufacturer": "Honda",
  "Make": "Civic",
  "Year": 2017,
  "VIN": "DY1SL65848Z411432"
}
{
  "Manufacturer": "Dodge",
  "Make": "Journey",
  "Year": 2014,
  "VIN": "SD1SL65848Z411443"
}
{
  "Manufacturer": "Ford",
  "Make": "F150",
  "Year": 2020,
  "VIN": "DH1SL65848Z41100"
}

πŸ”‘ DynamoDB Table Design

AttributeDescription
ManufacturerPartition Key (Primary Identifier)
ModelSort Key (Optional)
YearNumeric attribute
VINUnique Identifier (Secondary Index or attribute)

⚠️ Note: DynamoDB requires a Primary Key, which can be:

  • A Partition Key (e.g., Manufacturer), or

  • A combination of Partition Key + Sort Key (e.g., Manufacturer + Model)


🌐 Use Cases

  • Real-time inventory tracking (e.g., car dealers)

  • Session state storage

  • IoT device data logging

  • Leaderboards and user profiles for games

  • Event logging and analytics


πŸ” Example Access Pattern

Want to find all cars by Honda:

aws dynamodb query \
  --table-name cars \
  --key-condition-expression "Manufacturer = :m" \
  --expression-attribute-values '{":m":{"S":"Honda"}}'

πŸ› οΈ DynamoDB with Terraform


βœ… Step 1: Create a DynamoDB Table

To create a table named cars with a primary key (VIN) using Terraform:

resource "aws_dynamodb_table" "cars" {
  name         = "cars"
  hash_key     = "VIN"
  billing_mode = "PAY_PER_REQUEST"  # On-demand pricing

  attribute {
    name = "VIN"
    type = "S"  # "S" = String
  }
}

πŸ“Œ PAY_PER_REQUEST means you're charged per read/write β€” no need to define capacity units.


βœ… Step 2: Add an Item to the Table

Use aws_dynamodb_table_item to insert a car record:

resource "aws_dynamodb_table_item" "car_items" {
  table_name = aws_dynamodb_table.cars.name
  hash_key   = aws_dynamodb_table.cars.hash_key

  item = <<EOF
{
  "Manufacturer": {"S": "Toyota"},
  "Model": {"S": "Corolla"},
  "Year": {"N": "2004"},
  "VIN": {"S": "4Y1SL65848Z411439"}
}
EOF
}

πŸ“ Use jsonencode() or Heredoc syntax (<<EOF ... EOF) to structure your JSON item.


πŸ” Sample Multiple Items

You can repeat the aws_dynamodb_table_item resource block for each item, or modularize for dynamic item insertion (advanced):

{
  "Manufacturer": {"S": "Honda"},
  "Model": {"S": "Civic"},
  "Year": {"N": "2017"},
  "VIN": {"S": "DY1SL65848Z411432"}
}

πŸ§ͺ Terraform Workflow

  1. βœ… Initialize:

     terraform init
    
  2. πŸ” Preview plan:

     terraform plan
    
  3. πŸš€ Apply resources:

     terraform apply
    

    Confirm with yes.


βœ… Output Example

aws_dynamodb_table.cars: Creation complete [id=cars]
aws_dynamodb_table_item.car-items: Creation complete [id=VIN=4Y1SL65848Z411439]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

πŸ“Œ Notes

  • Use "S" for string, "N" for number, "BOOL" for boolean.

  • Each item block must be valid JSON and match the defined schema.

  • For production, use IAM roles and policies for permission control.

  • You can define additional attributes like range_key, TTL, or tags.


πŸ“¦ Terraform Remote State & Best Practices


πŸ“˜ What is Terraform State?

  • terraform.tfstate stores the current state of infrastructure.

  • Maps real-world resources to your Terraform configuration.

  • Tracks metadata (e.g., resource IDs, IPs, volumes).

  • Enables performance optimization by caching state data.


⚠️ Why You Should NOT Store State Files in Version Control

❌ Never commit terraform.tfstate to Git or any VCS.

Reasons:

  • It can contain sensitive information (access keys, IPs, passwords).

  • Leads to merge conflicts when used by teams.

  • Insecure and not designed for collaborative access.

  • Risk of accidental changes and security breaches.

βœ… Use remote backends (like AWS S3, Terraform Cloud) for secure, shared state management.


βœ… Remote State: Real-World Usage

Remote Backends Examples:

  • AWS S3 (with optional DynamoDB for state locking)

  • Terraform Cloud

  • Google Cloud Storage

  • HashiCorp Consul


πŸ” State Locking

When using remote backends, Terraform can lock the state file to prevent concurrent writes.

Example Error:

Error: Error locking state: Error acquiring the state lock: resource temporarily unavailable

πŸ”’ Why Locking Matters:

  • Prevents simultaneous state updates from multiple users.

  • Ensures infrastructure consistency.

  • Uses DynamoDB (in AWS) or similar to track active locks.


πŸ§ͺ Example: Remote State with AWS S3

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "env/dev/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-locks"   # For state locking
  }
}

πŸ›  Benefits of Remote State

FeatureLocal StateRemote State
CollaborationβŒβœ…
SecurityβŒβœ… (Encrypted)
State LockingβŒβœ…
Backup & RecoveryβŒβœ…
Automation FriendlyβŒβœ… (CI/CD Ready)

πŸ“ Common Files

  • main.tf β†’ Infrastructure definitions

  • terraform.tfstate β†’ Auto-generated state file (⚠️ don’t track it!)

  • terraform.tfvars β†’ Variable values

  • .terraform/ β†’ Terraform cache folder


βœ… Final Best Practices

  • βœ”οΈ Use remote state for collaboration and automation.

  • ❌ Never push terraform.tfstate to Git.

  • βœ”οΈ Enable state locking to avoid conflicts.

  • βœ”οΈ Use backend encryption for added security.


πŸ“¦ Remote State with S3 Backend (Best Practice)


βœ… Why Use Remote Backend?

  • Store terraform.tfstate securely in a central S3 bucket.

  • Enable team collaboration without state file conflicts.

  • Support automatic state locking with DynamoDB.

  • Avoid checking .tfstate into version control.


🧾 File Structure

$ ls
main.tf          # Resource definitions
terraform.tf     # Backend config (S3 + DynamoDB)

πŸ“ main.tf – Resource Definition

resource "local_file" "pet" {
  filename = "/root/pets.txt"
  content  = "We love pets!"
}

βš™οΈ terraform.tf – Remote Backend Configuration

⚠️ Do not put this block inside main.tf. Keep it separate in terraform.tf.

terraform {
  backend "s3" {
    bucket         = "kodekloud-terraform-state-bucket01"
    key            = "finance/terraform.tfstate"
    region         = "us-west-1"
    dynamodb_table = "state-locking"  # Enables state locking
  }
}

πŸ” State Locking with DynamoDB

  • Prevents simultaneous modifications.

  • Table: state-locking

  • Required when working with remote teams.


▢️ Commands Workflow

  1. Initialize backend

     $ terraform init
    
    • Prompts to copy existing local state β†’ Enter yes.
  2. Remove local state

     $ rm -rf terraform.tfstate
    
  3. Apply configuration

     $ terraform apply
    
    • You’ll see messages like:

        Acquiring state lock...
        Releasing state lock...
      

πŸ›‘ Warning: Never Track State Files in Version Control

❌ Do not commit terraform.tfstate, .terraform/, or .terraform.lock.hcl.

Use .gitignore:

terraform.tfstate
.terraform/
terraform.tfstate.backup

🧠 Summary

FeatureLocal StateRemote State (S3)
Team CollaborationβŒβœ…
State LockingβŒβœ… (DynamoDB)
SecurityβŒβœ… (S3 encryption)
Performance & BackupβŒβœ… (Versioning support)

πŸ“¦ Terraform State Management

Terraform maintains infrastructure state in a .tfstate file. This file tracks the mapping between your Terraform configurations and real-world resources.


πŸ”§ Common State Subcommands

CommandPurpose
state listLists all resources tracked in the state
state showShows detailed attributes of a specific resource
state mvRenames or moves a resource in the state
state pullRetrieves the raw state data
state rmRemoves a resource from the state

πŸ“œ Examples

πŸ“‹ 1. terraform state list

Lists all resources stored in the current state:

$ terraform state list
aws_dynamodb_table.cars
aws_s3_bucket.finance-2020922

πŸ” 2. terraform state show [resource]

Shows detailed info about a resource from the state:

$ terraform state show aws_s3_bucket.finance-2020922

πŸ“Œ Output snippet:

bucket = "finance-2020922"
region = "us-west-1"
tags = {
  "Description" = "Bucket to store Finance and Payroll Information"
}

πŸ” 3. terraform state mv

Moves or renames a resource inside the state (without recreating):

$ terraform state mv aws_dynamodb_table.state-locking aws_dynamodb_table.state-locking-db

βœ… Output:

Successfully moved 1 object(s).

πŸ“€ 4. terraform state pull

Fetches the entire raw state in JSON format:

$ terraform state pull | jq '.resources[] | select(.name=="state-locking-db") | .instances[].attributes.hash_key'

βœ… Output:

"LockID"

❌ 5. terraform state rm

Removes a resource from the state (but not from AWS):

$ terraform state rm aws_s3_bucket.finance-2020922

⚠️ This will orphan the real-world resource unless you terraform import it again.


πŸ›‘ Important: Do NOT Edit .tfstate Manually

Always use the Terraform CLI to safely view or modify state. Editing the state file directly can corrupt it and cause resource drift.


πŸš€ Introduction to AWS EC2 (Elastic Compute Cloud)

AWS EC2 provides resizable compute capacity in the cloud. It allows you to launch virtual machines (instances) with various configurations of CPU, memory, storage, and networking.


πŸ–ΌοΈ Amazon Machine Images (AMIs)

An AMI is a pre-configured template for your EC2 instance including the OS and software.

OS / PlatformAMI ID (Example)
Amazon Linux 2ami-0c2f25c1f66a1ff4d
RHEL 8 (Web Server)ami-04312317b9c8c4b51
Ubuntu 20.04 (MySQL)ami-0edab43b6fa892279
Windows Server (ASP.NET)Windows Server 2019 AMI

πŸ’‘ Instance Types

Different types for different use cases:

πŸ“Š General Purpose (T2 Series)

InstancevCPUMemory (GB)
t2.nano10.5
t2.micro11
t2.small12
t2.medium24
t2.large28
t2.xlarge416
t2.2xlarge832

πŸ”— More EC2 Instance Types


πŸ’Ύ EBS Volume Types (Storage)

Elastic Block Store (EBS) is block-level storage attached to EC2 instances.

NameTypeDescription
io1SSDBusiness-critical apps
io2SSDLatency-sensitive transactions
gp2SSDGeneral purpose
st1HDDLow-cost, frequently accessed
sc1HDDLowest cost, infrequent access

πŸ”— More on EBS Volumes


πŸ” Access Methods

  • Linux/Ubuntu/RHEL: SSH using Key Pair (PEM file)

  • Windows: Connect via RDP using username/password


πŸ”§ User Data (Linux Example)

Use user data for automation during instance launch:

#!/bin/bash
sudo apt update
sudo apt install nginx -y
systemctl enable nginx
systemctl start nginx

This script installs and starts Nginx on an Ubuntu web server.


βš™οΈ Example Use Cases

OSWorkload Type
UbuntuMySQL Database
RHELWeb Server (Apache/Nginx)
WindowsASP.NET Core Application

πŸ“ Summary

  • EC2 = Virtual Machine in the cloud

  • AMI = Prebuilt OS image

  • Instance Type = Choose based on workload

  • EBS Volume = Persistent storage

  • User Data = Automate instance configuration


πŸš€ Provisioning AWS EC2 Web Server with Terraform

πŸ“¦ Purpose

To launch an Ubuntu 20.04 EC2 instance with:

  • NGINX pre-installed

  • SSH access enabled

  • SSH key pair for authentication

  • Security Group allowing inbound SSH

  • Output the public IP for remote login.


πŸ“ Terraform Project Structure

project/
β”œβ”€β”€ main.tf
β”œβ”€β”€ provider.tf
β”œβ”€β”€ output.tf
└── /root/.ssh/web.pub (your SSH public key)

πŸ”§ provider.tf

provider "aws" {
  region = "us-west-1"
}

πŸ”§ main.tf

resource "aws_key_pair" "web" {
  public_key = file("/root/.ssh/web.pub")
}

resource "aws_security_group" "ssh-access" {
  name        = "ssh-access"
  description = "Allow SSH access from the Internet"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "webserver" {
  ami           = "ami-0edab43b6fa892279" # Ubuntu 20.04 LTS
  instance_type = "t2.micro"
  key_name      = aws_key_pair.web.id

  vpc_security_group_ids = [aws_security_group.ssh-access.id]

  user_data = <<-EOF    #USER-DATA WILL RUN ONLY FOR THE FIRST TIME
              #!/bin/bash
              sudo apt update
              sudo apt install nginx -y
              systemctl enable nginx
              systemctl start nginx
              EOF

  tags = {
    Name        = "webserver"
    Description = "An NGINX WebServer on Ubuntu"
  }
}

πŸ“€ output.tf

output "publicip" {
  value = aws_instance.webserver.public_ip
}

βœ… Terraform Commands

$ terraform init         # Initialize provider plugins
$ terraform plan         # Review execution plan
$ terraform apply        # Provision the infrastructure

🧠 SSH Access

After successful apply:

$ ssh -i /root/.ssh/web ubuntu@<public_ip>

🧩 Replace <public_ip> with the output you got from Terraform.


🟒 Validate NGINX

On the EC2 instance:

$ systemctl status nginx

You should see active (running) status.


πŸš€ Terraform Provisioners

Provisioners in Terraform allow you to execute scripts or commands either on the local machine (where Terraform is running) or on the remote resource (like an EC2 instance).


πŸ”Ή Types of Provisioners

TypePurpose
remote-execRun commands on the remote resource via SSH or WinRM
local-execRun commands on the local machine (e.g., for automation/logging)

πŸ”§ Example: AWS EC2 with remote-exec

πŸ“ main.tf

resource "aws_key_pair" "web" {
  public_key = file("/root/.ssh/web.pub")
}

resource "aws_security_group" "ssh-access" {
  name        = "ssh-access"
  description = "Allow SSH access from the Internet"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "webserver" {
  ami           = "ami-0edab43b6fa892279"
  instance_type = "t2.micro"
  key_name      = aws_key_pair.web.id
  vpc_security_group_ids = [aws_security_group.ssh-access.id]

  provisioner "remote-exec" {
    inline = [
      "sudo apt update",
      "sudo apt install nginx -y",
      "sudo systemctl enable nginx",
      "sudo systemctl start nginx"
    ]

    connection {
      type        = "ssh"
      user        = "ubuntu"
      private_key = file("/root/.ssh/web")
      host        = self.public_ip
    }
  }

  tags = {
    Name = "webserver"
  }
}

βœ… What it does: Once the EC2 instance is created, Terraform connects to it using SSH and installs NGINX.


πŸ”§ Example: AWS EC2 with local-exec

resource "aws_instance" "webserver" {
  ami           = "ami-0edab43b6fa892279"
  instance_type = "t2.micro"
  key_name      = aws_key_pair.web.id
  vpc_security_group_ids = [aws_security_group.ssh-access.id]

  provisioner "local-exec" {
    command = "echo ${self.public_ip} >> /tmp/ips.txt"
  }

  tags = {
    Name = "webserver"
  }
}

βœ… What it does: After the EC2 instance is created, the instance's public IP is written to a local file (/tmp/ips.txt).


⚠️ Best Practices for Provisioners

  • πŸ›‘ Avoid relying on provisioners in production. Use cloud-init, user_data, or configuration management tools like Ansible or Chef.

  • βœ… Use provisioners for quick testing, POCs, or initial configuration.

  • πŸ›‘οΈ Always ensure SSH key access and network rules (Security Groups) are configured before using remote-exec.


βš™οΈ Terraform Provisioner Behavior

Provisioners allow you to run scripts or commands during the resource lifecycle β€” at creation or during destruction.


🟒 1. Creation-Time Provisioner

These run immediately after the resource is created.

πŸ“„ main.tf

resource "aws_instance" "webserver" {
  ami           = "ami-0edab43b6fa892279"
  instance_type = "t2.micro"

  provisioner "local-exec" {
    command = "echo Instance ${self.public_ip} Created! > /tmp/instance_state.txt"
  }
}

βœ… Output

$ cat /tmp/instance_state.txt
Instance 3.96.136.157 Created!

πŸ”΄ 2. Destroy-Time Provisioner

These run just before the resource is destroyed by using when = destroy.

πŸ“„ main.tf

resource "aws_instance" "webserver" {
  ami           = "ami-0edab43b6fa892279"
  instance_type = "t2.micro"

  provisioner "local-exec" {
    command = "echo Instance ${self.public_ip} Created! > /tmp/instance_state.txt"
  }

  provisioner "local-exec" {
    when    = destroy
    command = "echo Instance ${self.public_ip} Destroyed! > /tmp/instance_state.txt"
  }
}

βœ… Output after terraform destroy

$ cat /tmp/instance_state.txt
Instance 3.96.136.157 Destroyed!

⚠️ 3. Provisioner Failure Behavior

Use on_failure to control what Terraform should do when a provisioner fails:

OptionBehavior
fail❌ Default – stops execution and fails the Terraform run
continueβœ… Logs the error but continues execution

πŸ“„ Example with fail

provisioner "local-exec" {
  command    = "echo ${self.public_ip} > /temp/pub_ip.txt" # invalid path
  on_failure = "fail"
}

β›” This will stop the apply due to an invalid path (/temp instead of /tmp).

πŸ“„ Example with continue

provisioner "local-exec" {
  command    = "echo ${self.public_ip} > /temp/pub_ip.txt" # still invalid
  on_failure = "continue"
}

βœ… Execution continues even if the provisioner command fails.


βœ… Terraform Provisioners: Key Considerations

Provisioners allow custom scripts/commands to run after resource creation or before destruction. Use them wisely and sparingly.


🧭 Provisioner Types

TypeRuns OnUse Case Example
local-execLocal machineNotify via Slack, copy data to a local file, etc.
remote-execRemote resource (e.g. EC2)Install packages, configure services (e.g. NGINX)

πŸ› οΈ Example: remote-exec

resource "aws_instance" "webserver" {
  ami           = "ami-0edab43b6fa892279"
  instance_type = "t2.micro"
  tags = {
    Name        = "webserver"
    Description = "An NGINX WebServer on Ubuntu"
  }

  provisioner "remote-exec" {
    inline = ["echo $(hostname -i) >> /tmp/ips.txt"]
  }

  connection {
    type        = "ssh"
    user        = "ubuntu"
    private_key = file("/root/.ssh/web")
    host        = self.public_ip
  }
}

πŸ” Remote-Exec Requirements

βœ… Ensure the following for successful remote-exec:

  • πŸ”“ Security Group allows SSH (port 22) or WinRM (port 5985/5986)

  • πŸ—οΈ SSH Key Pair is available and injected

  • 🌐 EC2 instance has public IP (or private IP with VPN/Direct Connect)

  • πŸ“‘ Hostname resolution or direct IP accessible


❗ Considerations

  • πŸ“¦ Provisioners are not idempotent β€” if a provisioner fails or succeeds partially, re-running may produce unintended results.

  • ❌ Provisioners do not show up in terraform plan β€” they are part of the apply phase only.

  • πŸ”„ Avoid using provisioners for routine configurations. Prefer user_data, AMIs, or config management tools.


πŸ†š Provisioner vs Cloud-Native Bootstrapping

Use CaseRecommended Method
Simple package installuser_data, custom_data, metadata_startup_script
Complex software/config setupPre-built AMI or Configuration Management (Ansible, Chef)
One-time notification/cleanuplocal-exec or remote-exec
Launching scripts post-deployUse remote-exec (with caution)

πŸ“‹ Provider Specific Metadata Options

Cloud ProviderResource TypeMetadata / Script Field
AWSaws_instanceuser_data
Azureazurerm_virtual_machinecustom_data
GCPgoogle_compute_instancemetadata_startup_script
VMware vSpherevsphere_virtual_machineuser_data

🎯 Alternative to Provisioners: Custom AMI

Instead of scripting NGINX installation each time, build a custom AMI using tools like Packer:

nginx-build.json
{
  "builders": [{ ... }],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "sudo apt update",
        "sudo apt install -y nginx"
      ]
    }
  ]
}

➑️ Use this AMI in Terraform:

ami = "ami-XYZ"  # Your custom NGINX AMI

🏁 Final Notes

  • πŸ“Œ Use provisioners only when absolutely necessary

  • πŸ›‘οΈ For robust and secure infrastructure, prefer immutable infrastructure principles

  • πŸ“‚ Use user_data for most bootstrapping tasks in cloud VMs


🧨 Terraform taint Command

πŸ” What is taint?

terraform taint marks a resource for forced recreation during the next terraform apply.

πŸ” Useful when the resource is still working but misconfigured or corrupted, and you want Terraform to destroy and recreate it.


βœ… Syntax

terraform taint <resource_name>

πŸ“Œ Example:

terraform taint aws_instance.webserver

πŸ“€ Result:

  • Terraform marks the resource as "tainted"

  • The next terraform apply will:

    • destroy the current resource

    • then recreate it


🧹 To Reverse It

terraform untaint <resource_name>

πŸ“Œ Example:

terraform untaint aws_instance.webserver

πŸ“€ Result:

  • Terraform removes the taint flag

  • No changes will be made in next apply (if config is unchanged)


πŸ““ Terraform Plan Output (after taint)

# aws_instance.webserver is tainted, so must be replaced
-/+ resource "aws_instance" "webserver" {

This means Terraform will destroy the current instance and then create a new one.


πŸ› οΈ Terraform Provisioner Failure Example

provisioner "local-exec" {
  command = "echo ${aws_instance.webserver.public_ip} > /temp/pub_ip.txt"
}

❌ This causes an error on Windows:

Error: The system cannot find the path specified.

βœ… Fix:
Use a valid directory path, such as:

command = "echo ${aws_instance.webserver.public_ip} > C:\\temp\\pub_ip.txt"

🐞 Terraform Debugging

πŸ” Enable Debug Logs

export TF_LOG=TRACE

πŸ“Œ Other log levels: TRACE > DEBUG > INFO > WARN > ERROR

TRACE provides the most detailed information.


πŸ“ Log to File

export TF_LOG_PATH=/tmp/terraform.log

βœ… View logs:

head -10 /tmp/terraform.log

🧼 Turn off logging:

unset TF_LOG
unset TF_LOG_PATH

βœ… Terraform Import

Terraform allows importing existing resources (e.g., EC2, S3, Route53) so they can be managed via Terraform configuration and state.


🎯 Objective: Import an existing EC2 instance into Terraform


πŸͺœ Step-by-Step Guide


βœ… Step 1: Use data block to read existing resource (optional)

You used a data block like this:

data "aws_instance" "newserver" {
  instance_id = "i-026e13be10d5326f7"
}

output "newserver" {
  value = data.aws_instance.newserver.public_ip
}

Then:

$ terraform apply

πŸ“¦ Output:

Apply complete!
Outputs:
newserver = 15.223.1.176

βœ”οΈ This is just to view an existing resource, but not manage it.


βœ… Step 2: Attempt to import the resource (Fails without config)

You ran:

$ terraform import aws_instance.webserver-2 i-026e13be10d5326f7

πŸ›‘ Error:

Error: resource address "aws_instance.webserver-2" does not exist in the configuration.
Before importing this resource, please create its configuration in the root module.

βœ… Step 3: Add the matching resource block to main.tf

You created:

resource "aws_instance" "webserver-2" {
  # (resource arguments)
}

Just a basic skeleton β€” doesn't need full values yet.


βœ… Step 4: Run terraform import again

$ terraform import aws_instance.webserver-2 i-026e13be10d5326f7

πŸ“¦ Output:

aws_instance.webserver-2: Importing from ID "i-026e13be10d5326f7"...  
Import successful!

βœ… Step 5: Review in terraform.tfstate

After import, Terraform records the actual live configuration:

{
  "type": "aws_instance",
  "name": "webserver-2",
  "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
  "instances": [
    {
      "attributes": {
        "ami": "ami-0edab43b6fa892279",
        "instance_type": "t2.micro",
        "key_name": "ws",
        "tags": {
          "Name": "old-ec2"
        },
        "vpc_security_group_ids": ["sg-8064fdee"]
      }
    }
  ]
}

βœ… Step 6: Update main.tf with accurate values

You refined your resource block to match the imported state:

resource "aws_instance" "webserver-2" {
  ami                    = "ami-0edab43b6fa892279"
  instance_type          = "t2.micro"
  key_name               = "ws"
  vpc_security_group_ids = ["sg-8064fdee"]

  tags = {
    Name = "old-ec2"
  }
}

βœ… Step 7: Run terraform plan

$ terraform plan

βœ”οΈ Output:

No changes. Infrastructure is up-to-date.

βœ… This confirms that Terraform state and config are now in sync.


βœ… Terraform Modules


πŸ“¦ What is a Module?

A module is a container for multiple Terraform resources grouped together. It helps with:

  • Reusability

  • Maintainability

  • Cleaner main.tf

  • Easier collaboration and team management


πŸͺœ Section 1: Using a Local Module


🧱 1. Create Reusable Module – /aws-instance

πŸ“ Folder: /root/terraform-projects/aws-instance

main.tf

resource "aws_instance" "webserver" {
  ami           = var.ami
  instance_type = var.instance_type
  key_name      = var.key
}

variables.tf

variable "ami" {
  type        = string
  default     = "ami-0edab43b6fa892279"
  description = "AMI ID"
}

variable "instance_type" {
  type    = string
  default = "t2.micro"
}

variable "key" {
  type = string
}

πŸ”§ 2. Consume Module in Root Project – /development

πŸ“ Folder: /root/terraform-projects/development

main.tf

module "dev-webserver" {
  source        = "../aws-instance"
  ami           = "ami-0edab43b6fa892279"
  instance_type = "t2.micro"
  key           = "dev-key"
}

▢️ 3. Initialize & Apply

$ terraform init
$ terraform apply

βœ”οΈ Output: EC2 created from the module.


🧩 Section 2: Complex Modules with Reuse (Payroll App)


πŸ” Reusable Module – /modules/payroll-app

πŸ“ Folder: /root/terraform-projects/modules/payroll-app

Module Files:

βœ… Sample – app_server.tf

resource "aws_instance" "app_server" {
  ami           = var.ami
  instance_type = "t2.medium"
  tags = {
    Name = "${var.app_region}-app-server"
  }
  depends_on = [
    aws_dynamodb_table.payroll_db,
    aws_s3_bucket.payroll_data
  ]
}

βœ… Sample – s3_bucket.tf

resource "aws_s3_bucket" "payroll_data" {
  bucket = "${var.app_region}-${var.bucket}"
}

βœ… Sample – dynamodb_table.tf

resource "aws_dynamodb_table" "payroll_db" {
  name         = "user_data"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "EmployeeID"

  attribute {
    name = "EmployeeID"
    type = "N"
  }
}

βœ… Sample – variables.tf

variable "app_region" {
  type = string
}

variable "bucket" {
  type    = string
  default = "flexit-payroll-alpha-22001c"
}

variable "ami" {
  type = string
}

πŸ‡ΊπŸ‡Έ US Payroll App

πŸ“ /us-payroll-app

main.tf

module "us_payroll" {
  source      = "../modules/payroll-app"
  app_region  = "us-east-1"
  ami         = "ami-24e140119877avm"
}

provider.tf

provider "aws" {
  region = "us-east-1"
}

πŸ› οΈ Run:

$ terraform init
$ terraform apply

πŸ‡¬πŸ‡§ UK Payroll App

πŸ“ /uk-payroll-app

main.tf

module "uk_payroll" {
  source      = "../modules/payroll-app"
  app_region  = "eu-west-2"
  ami         = "ami-35e140119877avm"
}

provider.tf

provider "aws" {
  region = "eu-west-2"
}

πŸ› οΈ Run:

$ terraform init
$ terraform apply

βœ… Standardized config for multiple regions with shared logic.


🌐 Section 3: Using Modules from Terraform Registry

πŸ“ Example: SSH Security Group Module

main.tf

module "security-group_ssh" {
  source               = "terraform-aws-modules/security-group/aws//modules/ssh"
  version              = "3.16.0"
  vpc_id               = "vpc-7d8d215"
  ingress_cidr_blocks  = [ "10.10.0.0/16" ]
  name                 = "ssh-access"
}

πŸ‘‰ Get the module:

$ terraform get

βœ… Benefits of Modules

Before ModulesWith Modules
Duplicate .tf filesShared logic via module blocks
Hard to maintain infra per regionRegion-based folder (us/uk) w/ modules
High risk on changesLower risk with modular changes
Complex top-level main.tfSimpler root configs

🧠 Terraform Functions & Conditional Logic – Explained with Examples


πŸ“˜ 1. Why Functions?

Terraform functions allow you to transform, manipulate, or validate values inside configurations and templates. They increase reusability, reduce duplication, and improve logic-based deployments.


πŸ”’ 2. Numeric Functions

FunctionDescriptionExampleResult
max()Returns max numbermax(-1, 2, -10, 200, -250)200
min()Returns min numbermin(-1, 2, -10, 200, -250)-250
ceil()Rounds up to nearest integerceil(10.1) / ceil(10.9)11
floor()Rounds downfloor(10.9)10
variable "num" {
  type        = set(number)
  default     = [250, 10, 11, 5]
  description = "A set of numbers"
}

πŸ”€ 3. String Functions

FunctionDescriptionExampleResult
split()Splits string to listsplit(",", "a,b,c")[a, b, c]
join()Joins list to stringjoin(",", ["a", "b", "c"])a,b,c
lower()Converts to lowercaselower("ABC")abc
upper()Converts to uppercaseupper("abc")ABC
title()Capitalizes each wordtitle("abc,def")Abc,Def
substr()Gets substringsubstr("ami-xyz,ABC", 0, 7)ami-xyz

🧺 4. Collection Functions

List Example

variable "ami" {
  type        = list(string)
  default     = ["ami-xyz", "AMI-ABC", "ami-efg"]
}
FunctionDescriptionExampleResult
length()Count itemslength(var.ami)3
index()Index of itemindex(var.ami, "AMI-ABC")1
element()Item at indexelement(var.ami, 2)ami-efg
contains()Check if item existscontains(var.ami, "AMI-ABC")true

πŸ—ΊοΈ 5. Map Functions

Map Example

variable "ami" {
  type = map(string)
  default = {
    "us-east-1"    = "ami-xyz",
    "ca-central-1" = "ami-efg",
    "ap-south-1"   = "ami-ABC"
  }
}
FunctionDescriptionExampleResult
keys()List all map keyskeys(var.ami)["ap-south-1",...]
values()List all map valuesvalues(var.ami)["ami-ABC",...]
lookup()Get value by key (w/ default)lookup(var.ami, "us-west-2", "ami-pqr")ami-pqr

πŸ” 6. Type Conversion

FunctionPurposeExampleResult
toset()Convert list to set (remove dupes)toset(["a", "a", "b"])["a", "b"]
tolist()Convert other types to listtolist(toset(["a", "b"]))["a", "b"]
tonumber()String to numbertonumber("5")5

πŸ”€ 7. Operators

Numeric, Equality, Comparison

> 1 + 2         // 3
> 8 == 8        // true
> 5 > 7         // false
> 4 < 5         // true

Logical

> 8 > 7 && 8 < 10       // true
> 8 > 10 || 8 < 10      // true
> ! true                // false

❓ 8. Conditional Expressions

Syntax:

condition ? true_val : false_val

πŸ” Example – Password Generator

resource "random_password" "password-generator" {
  length = var.length < 8 ? 8 : var.length
}
variable "length" {
  type        = number
  description = "The length of the password"
}

Run Example:

$ terraform apply -var=length=5 -auto-approve

βœ”οΈ Output: Will create password with length = 8


πŸ§ͺ 9. terraform console – Test Functions

$ terraform console

> length([1,2,3])
3

> split(",", "a,b,c")
["a", "b", "c"]

> var.length < 8 ? 8 : var.length
8

πŸ”§ Terraform Workspaces – Real World Usage

βœ… What are Terraform Workspaces?

Terraform Workspaces allow you to use the same configuration to manage multiple environments (like dev, staging, prod) while keeping independent state files.


πŸ“‚ Project Structure Overview

/root/terraform-projects/project/
β”œβ”€β”€ main.tf
β”œβ”€β”€ variables.tf
β”œβ”€β”€ terraform.tfstate.d/
β”‚   β”œβ”€β”€ ProjectA/
β”‚   β”‚   └── terraform.tfstate
β”‚   └── ProjectB/
β”‚       └── terraform.tfstate

πŸ—‚οΈ Why Use Workspaces?

  • βœ… Separate environments (e.g., ProjectA, ProjectB)

  • βœ… Prevent state file overwrites

  • βœ… Simplifies environment isolation without duplicating code


πŸ› οΈ Steps to Use Terraform Workspaces

  1. Initialize Terraform Project
$ terraform init
  1. Create and Switch to a Workspace
$ terraform workspace new ProjectA
$ terraform workspace new ProjectB
  1. Check Current Workspace
$ terraform workspace show
  1. Switch Workspace
$ terraform workspace select ProjectA
  1. List Workspaces
$ terraform workspace list

πŸ“¦ Dynamic Configuration with Workspaces

πŸ“ variables.tf

variable "ami" {
  type = map(string)
  default = {
    "ProjectA" = "ami-0edab43b6fa892279",
    "ProjectB" = "ami-0c2f25c1f66a1ff4d"
  }
}

variable "instance_type" {
  default = "t2.micro"
}

πŸ“ main.tf

resource "aws_instance" "project" {
  ami           = lookup(var.ami, terraform.workspace)
  instance_type = var.instance_type

  tags = {
    Name = terraform.workspace
  }
}

πŸ” Verify in Console

$ terraform console
> terraform.workspace
"ProjectA"
> lookup(var.ami, terraform.workspace)
"ami-0edab43b6fa892279"

βœ… Final Results

Each workspace manages its own state:

terraform.tfstate.d/
β”œβ”€β”€ ProjectA/ β†’ EC2 with AMI-A
└── ProjectB/ β†’ EC2 with AMI-B

πŸ“Œ Use Case

Perfect for managing multiple isolated deployments (per project, team, environment) without duplicating Terraform code.

0
Subscribe to my newsletter

Read articles from Arindam Baidya directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Arindam Baidya
Arindam Baidya