How to Create a Complete Multi-Cloud Terraform Setup for Azure, AWS, and GCP

Terraform by HashiCorp is a powerful Infrastructure-as-Code (IaC) tool that allows you to define, provision, and manage infrastructure across multiple cloud platforms, including Azure, AWS, and Google Cloud Platform (GCP). In this guide, we will walk you through creating a complete Terraform setup for multi-cloud environments. We’ll cover the directory structure, core concepts, and best practices to streamline infrastructure management across Azure, AWS, and GCP.

Whether you’re managing a multi-cloud deployment or seeking flexibility, this guide will help you get started with a solid Terraform foundation.

Prerequisites
Before diving in, ensure you have the following:
- Basic knowledge of Terraform.
- Installed Terraform CLI (`v1.0` or later).
- Cloud provider accounts (Azure, AWS, GCP) with appropriate permissions.
- Installed CLI tools for each cloud provider:
 — Azure CLI
 — AWS CLI
 — Google Cloud SDK
- An IDE (e.g., Visual Studio Code) with the Terraform plugin.

Directory Structure and Arborescence

One of the best practices in Terraform is to keep your configuration files organized. For a multi-cloud setup, the structure can be a bit complex but ensures modularity and reusability.

Here’s a recommended directory structure for managing infrastructure across Azure, AWS, and GCP:

multi-cloud-terraform/
├── modules/
│   ├── azure/
│   │   ├── network/
│   │   │   ├── main.tf
│   │   │   ├── variables.tf
│   │   │   ├── outputs.tf
│   │   └── compute/
│   │       ├── main.tf
│   │       ├── variables.tf
│   │       ├── outputs.tf
│   ├── aws/
│   │   ├── network/
│   │   │   ├── main.tf
│   │   │   ├── variables.tf
│   │   │   ├── outputs.tf
│   │   └── compute/
│   │       ├── main.tf
│   │       ├── variables.tf
│   │       ├── outputs.tf
│   ├── gcp/
│   │   ├── network/
│   │   │   ├── main.tf
│   │   │   ├── variables.tf
│   │   │   ├── outputs.tf
│   │   └── compute/
│   │       ├── main.tf
│   │       ├── variables.tf
│   │       ├── outputs.tf
├── environments/
│   ├── dev/
│   │   ├── main.tf
│   │   ├── variables.tf
│   ├── prod/
│   │   ├── main.tf
│   │   ├── variables.tf
├── provider/
│   ├── azure.tf
│   ├── aws.tf
│   ├── gcp.tf
├── terraform.tfvars
└── backend.tf

Directory Breakdown:
- modules/: Contains reusable code for each provider (Azure, AWS, GCP). This modular approach allows you to define core components like networking and compute resources in one place and reuse them across multiple environments.
- environments/: Includes environment-specific configurations (e.g., development, production). This enables you to define different infrastructure states for each environment while reusing the modules.
- provider/: Contains configuration files for each cloud provider. These files will define how Terraform interacts with Azure, AWS, and GCP.
- terraform.tfvars: Stores variables that are common across the infrastructure, like region, instance sizes, and other cloud-specific details.
- backend.tf: Configures the backend storage for your Terraform state files (e.g., S3 for AWS, Azure Blob Storage, or Google Cloud Storage).

Setting Up Terraform for Azure

Step 1: Configure the Provider

The `provider/azure.tf` file configures Terraform to work with Azure. It includes details about authentication and the subscription ID.

provider "azurerm" {
  features {}

  subscription_id = var.azure_subscription_id
  client_id       = var.azure_client_id
  client_secret   = var.azure_client_secret
  tenant_id       = var.azure_tenant_id
}

Step 2: Create a Basic Network Module

Inside `modules/azure/network/main.tf`, define a virtual network (VNet) and subnets:

resource "azurerm_virtual_network" "main" {
  name                = var.vnet_name
  address_space       = var.address_space
  location            = var.location
  resource_group_name = var.resource_group_name
}

resource "azurerm_subnet" "subnet" {
  name                 = var.subnet_name
  resource_group_name  = var.resource_group_name
  virtual_network_name = azurerm_virtual_network.main.name
  address_prefixes     = var.subnet_prefixes
}

Step 3: Define Variables and Outputs

In `modules/azure/network/variables.tf`, define the necessary variables:

variable "vnet_name" {
  description = "Name of the Virtual Network"
  type        = string
}

variable "address_space" {
  description = "Address space for the VNet"
  type        = list(string)
}

variable "location" {
  description = "Azure region for the resources"
  type        = string
}

variable "resource_group_name" {
  description = "Name of the resource group"
  type        = string
}

In `outputs.tf`, output the necessary information:

output "vnet_id" {
  description = "ID of the virtual network"
  value       = azurerm_virtual_network.main.id
}

Step 4: Deploy Resources to Azure

Once the modules are defined, you can deploy the infrastructure by running the following commands in the `environments/dev/` directory:

terraform init
terraform plan
terraform apply

Setting Up Terraform for AWS

Step 1: Configure the Provider

In `provider/aws.tf`, define the AWS provider configuration:

provider "aws" {
  region     = var.aws_region
  access_key = var.aws_access_key
  secret_key = var.aws_secret_key
}

Step 2: Create a Basic Network Module

In `modules/aws/network/main.tf`, define a VPC, subnets, and Internet Gateway:

resource "aws_vpc" "main" {
  cidr_block = var.cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "subnet" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = var.subnet_cidr_block
  availability_zone       = var.availability_zone

  tags = {
    Name = var.subnet_name
  }
}

Step 3: Deploy Resources to AWS

To deploy the AWS resources, navigate to the environment directory (e.g., `environments/dev/`) and run:

terraform init
terraform plan
terraform apply

4. Setting Up Terraform for GCP

Step 1: Configure the Provider

In `provider/gcp.tf`, define the GCP provider configuration:

provider "google" {
  credentials = file(var.gcp_credentials_file)
  project     = var.gcp_project
  region      = var.gcp_region
}

Step 2: Create a Basic Network Module

In `modules/gcp/network/main.tf`, define a VPC and subnets:

resource "google_compute_network" "main" {
name = var.network_name
auto_create_subnetworks = false
}
resource "google_compute_subnetwork" "subnet" {
name = var.subnet_name
ip_cidr_range = var.cidr_block
network = google_compute_network.main.name
region = var.region
}

Step 3: Deploy Resources to GCP

To deploy the GCP resources, navigate to the environment directory and run:

terraform init
terraform plan
terraform apply

Backend Configuration for Terraform State Management

To avoid inconsistencies in your infrastructure state and enable collaboration, use remote state management. Below is an example of configuring remote backends for Azure, AWS, and GCP.

Example: AWS S3 Backend

In `backend.tf`, configure the S3 backend for storing Terraform state files:

terraform {
  backend "s3" {
    bucket = "your-terraform-state-bucket"
    key    = "global/s3/terraform.tfstate"
    region = "us-west-2"
  }
}

Example: Azure Blob Storage Backend

terraform {
  backend "azurerm" {
    storage_account_name = "yourstorageaccount"
    container_name       = "tfstate"
    key                  = "terraform.tfstate"
  }
}

Example: Google Cloud Storage Backend

terraform {
  backend "gcs" {
    bucket  = "your-terraform-state-bucket"
    prefix  = "terraform/state"
  }
}

Run `terraform init` again after configuring the backend to migrate your state to the remote backend.

Conclusion

Congratulations! You’ve successfully created a multi-cloud Terraform setup for Azure, AWS, and GCP. By structuring your Terraform configurations in a modular and environment-specific way, you ensure scalability and manageability of your infrastructure.

Terraform’s cloud-agnostic capabilities provide the flexibility to deploy and manage resources across multiple cloud platforms from a single codebase, making it a powerful tool in the modern multi-cloud environment.

Feel free to experiment with more complex configurations, integrate CI/CD pipelines, and expand your Terraform knowledge further. Happy coding!