Storing Terraform State Files in S3

What is Terraform?

Terraform is an IaC tool developed by HashiCorp that enables developers and operators to manage and provision infrastructure as code. It allows you to define and automate your cloud, on-premises, or hybrid infrastructure in a declarative configuration language. With Terraform, you can create, modify, and version your infrastructure effortlessly, making it an indispensable tool for modern DevOps practices.

The Role of the State File

The state file in Terraform serves as a crucial record-keeper. It tracks the existing resources, their attributes, and the relationships between them. Terraform references this file to understand the current state of the infrastructure, enabling it to plan and execute modifications efficiently. The terraform state file, by default, is named terraform. tfstate and is held in the same directory where Terraform is run. It is created after running terraform apply . The actual content of this file is a JSON formatted mapping of the resources defined in the configuration and those that exist in your infrastructure.

Migrating Terraform to the Cloud: Why It Matters

Now that we've got the hang of Terraform and how it keeps track of our work locally, let's talk about moving our state file to the cloud. Why should we bother? Well, it's a big deal for a few reasons that make our teamwork smoother and our work safer. Here are some reasons:

1. Teamwork Made Easier:

Imagine you're building a Lego castle with your friends. You need to make sure everyone's building on the same blueprint, right? Moving Terraform to the cloud is like putting that blueprint where everyone can see it. With a cloud-based Terraform state file, everyone in your team can work together without stepping on each other's toes. It's like having a shared drawing board where everyone knows what's going on.

2. Safety First:

In the cloud, you can put your Terraform state file in a digital vault, guarded by high-tech locks. Only the right people (your team members) can access it. Plus, cloud providers have backup systems, so even if something bad happens, like your computer crashing, your work stays safe and sound.

3. Keeping Track of Changes:

Ever wanted to go back in time to see how your Lego castle looked a month ago? Cloud-based Terraform state files let you do something similar. They keep a record of all the changes you make. So, if you accidentally mess something up, you can easily go back to the version you liked. It's like having an 'undo' button for your Lego creations.

The State Lock Advantage

1. Preventing Conflicts:

Imagine two friends trying to build the same part of your digital Lego castle at the same time. That can create a mess! State locks act like traffic lights, ensuring only one person can work on a specific part at any given moment. This prevents conflicts, making sure changes don't clash and cause problems.

2. Data Integrity:

State locks also ensure the integrity of your Terraform state file. They guarantee that only one operation happens at a time, avoiding any data corruption. It's like having a supervisor ensuring everyone follows the building plan step by step, maintaining the stability of your digital creation.

Guide: Migrating Terraform State to AWS S3

1. Creating the S3 Bucket:

resource "aws_s3_bucket" "terraform-state" {
    bucket = "terraform-state-unique-bucket"

    # Prevent accidental deletion of the bucket
    lifecycle {
      prevent_destroy = true
    }  
}

This block creates an AWS S3 bucket named "terraform-state-unique-bucket". The lifecycle configuration ensures that the bucket is protected from accidental deletion.

1. Enabling Versioning:

resource "aws_s3_bucket_versioning" "terraform-state" {
    bucket = aws_s3_bucket.terraform-state.id

    versioning_configuration {
      status = "Enabled"
    } 
}

This block enables versioning for the S3 bucket, allowing multiple versions of the Terraform state file to be stored for tracking changes over time.

1. Configuring Server-Side Encryption:

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform-state" {
    bucket = aws_s3_bucket.terraform-state.id

    rule {
        apply_server_side_encryption_by_default {
          sse_algorithm = "AES256"
        }
    } 
}

This block configures server-side encryption for the S3 bucket, ensuring that data stored in the bucket is encrypted using the AES256 encryption algorithm.

1. Applying Public Access Restrictions:

resource "aws_s3_bucket_public_access_block" "terraform-state" {
    bucket                  = aws_s3_bucket.terraform-state.id
    block_public_acls       = true
    block_public_policy     = true
    ignore_public_acls      = true
    restrict_public_buckets = true  
}

This block restricts public access to the S3 bucket by blocking public ACLs, public policies, and ignoring existing public ACLs, enhancing the security of the stored Terraform state file.

---

Creating DynamoDB Table for State Locking Information

1. Creating a DynamoDB Table:

resource "aws_dynamodb_table" "terraform-state" {
    name            = "terraform-state"
    billing_mode    = "PAY_PER_REQUEST"
    hash_key        = "LockID"

    attribute {
      name = "LockID"
      type = "S"
    }
}

This block creates an AWS DynamoDB table named "terraform-state" with a hash key "LockID". DynamoDB is used for state locking, ensuring only one Terraform operation can execute at a time, preventing conflicts.

---

Configuring Terraform Backend for S3 Storage

Configuring Terraform Backend:

terraform {
  backend "s3" {
    bucket         = "terraform-state-unique-bucket"
    key            = "global/s3/terraform.tfstate"
    dynamodb_table = "terraform-state"
    region         = "us-west-2"
    encrypt        = true

    # Use the "terraform" AWS profile for credentials
    # profile = "terraform"
  }
}

This block configures Terraform to use S3 as the backend for storing the state file. It specifies the S3 bucket name, key (path within the bucket), DynamoDB table for state locking, region, and encryption settings. Uncomment the profile line if you're using a specific AWS profile for credentials. This setup ensures that Terraform state is stored securely in the specified S3 bucket, with state locking managed by DynamoDB, providing a robust and reliable infrastructure management environment.

Conclusion

Congratulations, you've successfully migrated your Terraform state to AWS S3. Now, you and your team can build, modify, and collaborate on your infrastructure projects with ease, all while ensuring the security and integrity of your state file. Happy cloud-based building!