Automating Infrastructure Deployment with AI Models in Amazon Bedrock, Terraform, AWS SageMaker, and AWS CodeBuild

In today's fast-paced world of technology, automating infrastructure deployment is crucial for maintaining efficiency and consistency. This blog post will guide you through the process of generating Terraform code using AI models available via Amazon Bedrock, executing it from AWS SageMaker, and setting up a CI/CD pipeline with AWS CodeBuild. Let's dive right in!

Here is what I have setup as my Environment Playground

Access to your Pay as you go, AWS Account.

Enable access to Claude 3.5 Sonnet LLM model from Anthropic within Amazon Bedrock.

Appropriate permissions to Amazon Bedrock and AWS Sagemaker for your IAM User / Group.

We wil be executing Jupyter Notebooks from within AWS Sagemaker.

1. Generating Terraform Code with AI

We'll start by using Amazon Bedrock to generate Terraform code for our infrastructure. Here's how you can do this in a SageMaker notebook:

import json
import boto3

# Install necessary packages
!pip install --upgrade boto3 awscli

# Initialize the Bedrock client
bedrock = boto3.client('bedrock-runtime')

# Your prompt
prompt = "Generate Terraform code for an AWS VPC with public and private subnets, an EC2 instance, and an RDS database"

# Prepare the request
request = {
    "prompt": f"Human: {prompt}\n\nAssistant:",
    "max_tokens_to_sample": 2000,
    "temperature": 0.7,
    "top_p": 0.9,
}

# Call the Bedrock API
response = bedrock.invoke_model(
    body=json.dumps(request),
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0"
)

# Parse the response
response_body = json.loads(response['body'].read())
generated_text = response_body['completion']

print(generated_text)

This code sends a prompt to the Cloude 3.5 Sonnet AI model inside the Amazon Bedrock and receives Terraform code as a response.

2. Executing Terraform from SageMaker

Let us now save the generated code and execute it directly from AWS SageMaker:

import os
import subprocess

# Save the Terraform code to a file
with open('main.tf', 'w') as f:
    f.write(generated_text)

print("Terraform code saved to main.tf")

# Install Terraform
!curl -O https://releases.hashicorp.com/terraform/1.5.7/terraform_1.5.7_linux_amd64.zip
!unzip terraform_1.5.7_linux_amd64.zip
!mv terraform /usr/local/bin/
!terraform version

# Initialize Terraform
init_result = subprocess.run(['terraform', 'init'], capture_output=True, text=True)
print("Terraform Init Output:")
print(init_result.stdout)
print(init_result.stderr)

# Auto-apply the Terraform configuration
apply_result = subprocess.run(['terraform', 'apply', '-auto-approve'], capture_output=True, text=True)
print("Terraform Apply Output:")
print(apply_result.stdout)
print(apply_result.stderr)

This script saves the generated code to a file, installs Terraform, initializes the working directory, and applies the terraform configuration.

3. Setting Up CI/CD with AWS CodeBuild

For continuous deployment, we can use AWS CodeBuild. Create a file named buildspec.yml in your GitHub repository with the following content:

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.9
    commands:
      - echo "Installing Terraform..."
      - curl -O https://releases.hashicorp.com/terraform/1.5.7/terraform_1.5.7_linux_amd64.zip
      - unzip terraform_1.5.7_linux_amd64.zip
      - mv terraform /usr/local/bin/
      - terraform version

  pre_build:
    commands:
      - echo "Initializing Terraform..."
      - terraform init

  build:
    commands:
      - echo "Planning Terraform changes..."
      - terraform plan -out=tfplan
      - echo "Applying Terraform changes..."
      - terraform apply -auto-approve tfplan

  post_build:
    commands:
      - echo "Terraform apply completed on `date`"

artifacts:
  files:
    - '**/*'

This buildspec file instructs CodeBuild to install Terraform, initialize the working directory, plan the changes, and apply them automatically.

Setting Up the CI/CD Pipeline

1. Create a new CodeBuild project in the AWS Console.

2. Connect it to your GitHub repository.

3. Configure the project to use the buildspec.yml file.

4. Ensure the CodeBuild service role has necessary permissions to create AWS resources.

5. Set up any required environment variables in the CodeBuild project settings.

Now, every time you push changes to your GitHub repository, CodeBuild will automatically execute your Terraform code, updating your infrastructure as needed.

Conclusion

By combining AI-generated Terraform code, SageMaker for testing, and CodeBuild for CI/CD, we've created a powerful, automated infrastructure deployment pipeline. This approach allows for rapid iteration and ensures your infrastructure stays in sync with your code.

Remember to always review all your AI-generated code, be cautious with the terraform auto-apply in production environments, and properly manage your Terraform state. Happy automating!

---