Day-16 | Infrastructure as Code

In today's session, we'll explore Infrastructure as Code (IaC), an essential concept in modern DevOps practices.

The Problem

Imagine you're a DevOps engineer working for an organization like Flipkart, which operates hundreds of applications—let’s say around 300. Each application requires deployment environments such as servers, storage, and databases. These compute resources (CPU, RAM, etc.) can be hosted in various ways:

1. Cloud platforms like AWS, Azure, or Google Cloud Platform (GCP).

2. On-premises physical servers in the organization's data centers.

3. Other platforms such as Oracle Cloud or DigitalOcean.

Scenario 1: Hosting on AWS

Let’s assume the entire infrastructure is hosted on AWS. To streamline operations, reduce manual efforts, and enhance productivity, you decide to automate the creation and management of resources using AWS CloudFormation Templates (CFTs). These templates enable you to define and provision infrastructure using code.

After writing hundreds of scripts and stabilizing them, your workflow is smooth. Developers can quickly request resources (e.g., 10 virtual machines), and automation takes care of provisioning.

Challenge 1: Moving to Azure

Suddenly, due to issues such as unsatisfactory support or high costs on AWS, your organization decides to migrate to Microsoft Azure. However, AWS CloudFormation Templates (CFTs) are specific to AWS and cannot be used on Azure.

Now, you need to rewrite all your scripts using Azure Resource Manager (ARM) templates. After significant effort, you successfully migrate the scripts and automate the process on Azure.

Challenge 2: Moving to On-Premises

After some time, the organization is dissatisfied with Azure due to similar reasons. This time, they decide to move to an on-premises infrastructure, deploying servers in their own data centers using OpenStack.

To automate resource provisioning on OpenStack, you need to use Heat Templates, a tool specific to OpenStack. Once again, you rewrite the scripts, spending valuable time and effort to adapt to this new platform.

Scenario 2: The Rise of Hybrid Cloud

Modern organizations are increasingly adopting the hybrid cloud model, where infrastructure is hosted across multiple platforms based on specific needs:

1. Example Use Case:

   • AWS is chosen for its superior storage services.

   • Azure is preferred for its DevOps tools and project management services.

In this scenario, Flipkart might host:

• Storage services on AWS.

• DevOps-related services on Azure.

The Complexity of Hybrid Cloud

As a DevOps engineer in a hybrid cloud environment, you need to:

• Learn AWS CFTs for automating tasks on AWS.

• Learn Azure Resource Manager for automation on Azure.

• Learn Heat Templates for on-premises infrastructure.

This approach creates a significant challenge: managing multiple cloud platforms requires mastering several tools and frameworks, leading to inefficiency and a steep learning curve.

The Solution: Terraform

Terraform, developed by HashiCorp, solves this problem by providing a unified tool for infrastructure automation across multiple cloud platforms. It simplifies your work in the following ways:

1. Single Tool for All Platforms
   Instead of learning AWS CFT, Azure ARM, and other tools, you only need to learn Terraform. It works with all major cloud providers (AWS, Azure, GCP, etc.) and even on-premise solutions.

2. Smooth Migration Between Cloud Providers
   If your organization decides to migrate from one cloud provider (e.g., AWS) to another (e.g., Azure), you only need to update minimal details in Terraform scripts, such as:

   • Provider information.

   • Modules specific to the new provider.

This eliminates the need to rewrite scripts from scratch.

How Terraform Works

Terraform automates resource creation by using a concept called API as Code. It eliminates the need to directly interact with cloud APIs by acting as a bridge between you and the APIs. Here’s how it works:

1. Modules and Simplified Syntax

   • Terraform provides pre-built modules for common tasks. For example, to create an EC2 instance in AWS, you can use the ec2 module.

   • Instead of writing complex API calls, you write a simple Terraform script in a declarative syntax resembling plain English.

2. API Abstraction

   • When you submit a Terraform script, Terraform processes your input.

   • It translates your script into API calls specific to the cloud provider (AWS, Azure, etc.).

   • Terraform handles the interaction with the provider’s API and executes the request to create, update, or delete resources.

3. Unified Tool

   • By learning Terraform, you avoid the need to master multiple tools like AWS CloudFormation or Azure Resource Manager.

Key Concepts: Infrastructure as Code (IaC) and API as Code

Infrastructure as Code (IaC)

• IaC is a practice where infrastructure is managed using code rather than manual processes.

• Examples:

  • AWS CloudFormation Templates.

  • Azure Resource Manager Templates.

• Terraform also falls under IaC but offers broader compatibility across providers.

API as Code

• Terraform goes a step further with API as Code, a concept where Terraform interacts with cloud provider APIs on your behalf.

• You focus on writing Terraform scripts, and Terraform handles the API communication in the background.

For example:

• You write a script to provision a virtual machine.

• Terraform talks to the cloud provider’s API, executes the necessary API calls, and provisions the VM.

Why Terraform Stands Out

• Simplifies Multi-Cloud Management:

  • Manage AWS, Azure, GCP, and more using a single tool.

• Reduces Complexity:

  • No need to write custom scripts for API calls or learn multiple tools.

• Reusable Modules:

  • Use Terraform modules for repeatable tasks, ensuring consistency and reducing errors.

• Smooth Workflow:

  • Migrate between providers with minimal changes, making Terraform ideal for hybrid or multi-cloud environments.