Creating a Webserver using Pulumi to Build and Manage Infrastructure

Pulumi is a tool that lets you create and manage your cloud infrastructure using any of the programming languages it supports. This way, you can leverage your existing coding skills and tools (such as your preferred IDE) and follow the best practices and workflows of software development. Pulumi works with multiple cloud providers and has the unique feature of using programming languages to define your infrastructure.

Instructions

1. In the AWS Management Console search bar, enter EC2, and click the EC2 result under Services:

2. To see available instances, click Instances in the left-hand menu:

The instances list page will open, and you will see an instance named cloudacademylabs:

If you don't see a running instance then the lab environment is still loading. Wait until the Instance state is Running.

3. Right-click the cloudacademylabs instance, and click Connect:

The Connect to your instance form will load.

4. In the form, ensure the EC2 Instance Connect tab is selected:

You will see the instance's Instance ID and Public IP address displayed.

5. In the User name textbox, enter ec2-user:

Note: Ensure there is no space after ec2-user or connect will fail.

6. To open a browser-based shell, click Connect:

If you see an error it's likely that the environment hasn't finished setting up. When you see the following button show that the lab's cloud environment has fully provisioned, you are ready to use the virtual machine.

1. To download and install Pulumi, in the Linux shell, enter the following commands:

curl -fsSL https://get.pulumi.com | sh

These commands do the following:

• This will install the pulumi CLI to ~/.pulumi/bin and add it to your path. When it can’t automatically add pulumi to your path, you will be prompted to add it manually.

There are several different methods for installing Pulumi, here you are using a manual installation as it is the most suitable for this lab. In a non-lab environment, the method you should choose will depend on factors such as the operating system you are using and your infrastructure as code workflow. See the Pulumi documentation on installing for details on the available installation methods.

2. To test that Pulumi has been installed, enter the following command:

pulumi --help

You will see information displayed about how to use the Pulumi command-line tool.

3. To configure Pulumi to use a local backend for state, enter the following command:

pulumi login --local

You will see a response similar to:

Infrastructure as code tools such as Cloudformation, Terraform, and Pulumi store state about the infrastructure they create and manage. The state they store has several purposes, primarily it's used to define the desired state of the infrastructure. Infrastructure as code tools compare the desired state with the actual state of the infrastructure and determine what actions are required to make the actual state match the desired state.

Storage of this state information is an important topic, as state information usually contains sensitive information. If an attacker obtained the state information of an existing system, they may be able to use it to compromise the system.

Pulumi has different options for managing state, notably the company that created Pulimi recommend using their service to host the state information. The benefits of using their service include handling authentication, encryption, checkpointing, and access control. This hosted backend state service differentiates Pulumi from Terraform and other cloud-provider agnostic Infrastructure as Code tools.

To learn more see the Pulumi documentation on state.

In this lab, you are storing state locally on the EC2 instance you connected to. Storing state locally is designed for development and testing purposes and is not recommended for a production environment.

4. To create a new directory and change to it, enter the following:

mkdir webserver
cd webserver

The Pulumi command-line tool requires you to be in an empty directory when creating a new project.

5. To create a new AWS typescript project, enter the following command:

pulumi new aws-typescript

A key feature of Pulumi is that it allows you to describe your infrastructure using your preferred programming language. Languages supported by Pulumi include Node.js, Python, .NET Core, and Go.

In comparison, Cloudformation supports JSON and YAML, and Terraform uses HCL, these are static and declarative file formats. Pulumi allows you to use a language that you may already be familiar with to describe your desired infrastructure. And you can use features of the language such as loops and conditionals, that aren't present in HCL, JSON, or YAML.

Using a programming language over a declarative file format is a somewhat controversial choice. Some people think that declarative is better because it is static and doesn't have common programming language features like loops and conditionals available. Nevertheless, being unable to easily loop is a common complaint when working with existing Infrastructure as Code tooling. There is a difference in philosophy between Pulumi and Infrastructure as Code tools, both believe that code should be the single source of truth to describe the infrastructure, but Pulumi promotes using a capable programming language over a declarative format.

In this lab, you are using AWS as the cloud provider, and TypeScript (a variant of JavaScript with static typing) as the language (NodeJS has been pre-installed on this instance).

6. Enter the following information as you are prompted:

• project name: webserver

• project description: Lab webserver

• stack name: Press enter to accept the default dev

• passphrase to protect config/secrets: Enter Secret123 (you will be prompted to enter this two times)

• aws:region: us-west-2 (Important: You won't be able to complete this lab in any other region)

Pulumi will create the project and install dependencies.

7. To configure Pulumi to use the IAM role associated with this Amazon EC2 instance, enter the following:

pulumi config set aws:skipMetadataApiCheck false

There are several methods Pulumi's AWS provider can use authenticate and use credentials. They are documented in the Configuration section of the documentation for the AWS Classic Pulumi package.

8. To save your passphrase and prevent Pulumi from asking you to re-enter it, enter the following:

export PULUMI_CONFIG_PASSPHRASE=Secret123

This command creates a Bash variable for your passphrase that Pulumi will look for before prompting for it. Without it, you would be asked to enter the passphrase each time you perform an operation using Pulumi.

In a non-lab environment, you should carefully consider the handling of passwords, keys, and passphrases that are used to access potentially sensitive information.

9. To see the IP address of the EC2 instance, enter the following:

curl https://checkip.amazonaws.com

You will see an IP address in response.

10. Open a new browser tab, and paste the following in the address bar, replacing <YOUR IP ADDRESS> with the IP address from the previous instruction:

http://<YOUR IP ADDRESS>:3000

You will see a web-based IDE (integrated development environment) load.

This IDE has been installed on the EC2 instance you connected to for convenient file editing and viewing. Alternatively, if you are familiar with and comfortable using command-line text editors such as Vim or Nano, they are available.

11. In the web IDE, in the left side menu, click the explorer icon:

The explorer side-menu will expand and you will see the files and directories in your home directory.

12. To view the Pulumi project files, in the explorer side-menu, click the webserver directory:

The files in this directory were created when you created a new Pulumi project earlier.

Tip: You can make the explorer wider by dragging its edge.

Here is an explanation of these files:

• node_modules: Pulumi has installed it's dependencies here

• .gitignore: Defines files to be ignored when using Git for source control (Git isn't used in this lab)

• index.ts: Defines the infrastructure you want to create, by default it will create an S3 bucket

• package-lock.json and package.json: Defines the dependences of this TypeScript project

• Pulumi.dev.yaml: This is the stack file, see the documentation on stacks for more information

• Pulumi.yaml: Project settings

• tsconfig.json: Used to configure static typing

Feel free to look at each one and familiarise yourself with them.

Leave this browser tab open, you will use it in the next lab step.

13. In the Linux command-line, to deploy this Pulumi stack, enter:

pulumi up

Pulumi will preview the update, and ask you if you want to perform the update operation.

14. Using your keyboard arrow keys, select yes:

You will see some output confirming the change was performed:

15. To verify that Pulumi created the S3 bucket, enter the following:

aws s3 ls

This command uses the AWS command-line tool to list buckets in this AWS account.

You will see a bucket listed beginning with my-bucket-.

Pulumi has created the bucket that is defined in the index.ts file.

1. In the Linux shell, to delete the bucket, enter the following:

pulumi destroy

Pulumi will ask you if you want to perform the destroy operation.

2. To see information about what will happen, select details.

Pulumi will show you that it will delete the bucket:

3. To perform the destroy operation, select yes.

You will see output confirming the deletion of the stack and the bucket.

4. To confirm that Pulumi has deleted the bucket, enter the following again:

aws s3 ls

You will not see any output, denoting that the bucket has been deleted.

5. Return to your browser tab with the web IDE open.

6. In the explorer side-menu, select index.ts:

You will see the code that created the S3 bucket in the previous lab step:

7. Using the web IDE, remove all lines apart from the import statements at the top.

The import statements are required to make Pulumi API calls.

The proceeding instructions will instruct you to add code to index.ts file, building up to the final web server infrastructure.

8. To define a security group for your webserver, in the index.ts file, add the following:

let group = new aws.ec2.SecurityGroup("web-secgrp", {
  ingress: [
    { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
  ],
});

If you have experience with security groups in AWS, the above should look familiar. This code is creating a security group called web-secgrp, and specifying an ingress rule to allow traffic on port eighty.

9. To assign the size of the instance you are going to create, add the following:

let size = aws.ec2.InstanceTypes.T2_Micro;

This line is assigning an instance type constant to a variable for later use.

10. To retrieve an AMI for the instance, add the following code:

let ami = aws
  .getAmi(
    {
      filters: [
        { name: "name", values: ["amzn2-ami-hvm-*-x86_64-gp2"] }
      ],
      owners: ["137112412989"], // Amazon
      mostRecent: true,
    },
    { async: true }
  )
  .then((result) => result.id);

This code is looking up an Amazon Machine Image (AMI) to use for the EC2 instance you are creating.

Notice the wildcard (*) in the value of the name filter, and use of the mostRecent boolean option. This means the call will retrieve the latest AMI whose name matches the filter.

For more information on this API call, you can visit the Pulumi documentation.

11. Add the following code to the index.ts file:

let html = `<!doctype html>
<html><body><div>
  <dl>
    <dt><b>Hostname</b></dt>
    <dd>$HOSTNAME</dd>
    <dt><b>Instance Id</b></dt>
    <dd>$INSTANCE_ID</dd>
    <dt><b>Public IP</b></dt>
    <dd>$PUBLIC_IP</dd>
    <dt><b>Security Group</b></dt>
    <dd>$SECURITY_GROUP</dd>
  </dl>
</div></body></html>`;

This code assigns a multi-line HTML template string containing Bash variables to a JavaScript variable called html.

12. Add the following multi-line assignment:

let userData = `#!/bin/bash
export HOSTNAME=$(curl http://169.254.169.254/latest/meta-data/hostname);
export PUBLIC_IP=$(curl http://169.254.169.254/latest/meta-data/public-ipv4);
export INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id);
export SECURITY_GROUP=$(curl http://169.254.169.254/latest/meta-data/security-groups);
echo "${html}" > index.html;
nohup python -m SimpleHTTPServer 80 &`;

This code is assigning a Bash user-data script to a JavaScript variable. A user data script is a script that is executed when an EC2 instance boots up.

There are three main parts to this user-data script, here's an explanation of them:

• Four Bash variable assignments, these are fetching data about the instance from the EC2 metadata API using curl

• An echo statement that echoes the HTML template from the previous instruction and redirects it to a file called index.html

• A statement that starts a web server using the Python SimpleHTTPServer module, this will serve the index file

The echo statement is where the Bash variables are substituted into the HTML template.

Notice the nohup and & elements in the last statement, these ensure the webserver will keep running once started.

This is a simple example of a webserver for demonstration purposes in this lab. In non-lab environments bootstrapping an application on an EC2 instance using user data often includes configuration management software, starting containers, or other techniques.

For more information on using the EC2 metadata instance API, visit the AWS documentation.

13. To define your web server, add the following code:

let server = new aws.ec2.Instance("web-server-www", {
  tags: { Name: "web-server-www" },
  instanceType: size,
  vpcSecurityGroupIds: [group.id],
  ami: ami,
  userData: userData,
});

This section of the code is creating a new EC2 instance JavaScript object and making use of the size, group, ami, and userData variables you previously defined.

14. To complete your webserver code definition, add the following lines:

exports.publicIp = server.publicIp;
exports.publicHostName = server.publicDns;

These are stack outputs, they are a way of making information about a stack easily accessible programmatically.

Visit the Pulumi documentation on stack outputs for more information about them.

Your index.ts file should look similar to:

15. To save your changes, in the web IDE, in the top-left, click File, and click Save:

16. To create your webserver, in the Linux shell, enter:

Copy code1
pulumi up -y

The -y option specifies that you don't want Pulumi to ask you for confirmation. Most Infrastructure as Code tools have similar options and they are useful when you want to automate creating or changing your infrastructure as a part of a DevOps pipeline.

You will see output similar to:

17. To retrieve a stack output, enter the following command:

Copy code1
pulumi stack output publicIp

In response, you will see the public IP address of the instance you created.

Stack outputs, sometimes called stack exports, are another common feature of Infrastructure as Code tools, and also useful for programmatic use in a DevOps pipeline.

14. Copy the IP address displayed from the previous instruction, open a new browser tab, and paste it in the address bar.

You will see a simple web page displaying details of the instance you created:

Please note that it will take two to three minutes for the instance to become ready and the user data to execute. If you don't see a web page, wait a couple of minutes, and refresh the browser tab.

Displayed on this page is the data that was fetched from the EC2 instance metadata service in the user-data script.

By allowing you to use a language you already know well Pulumi enables you to get started and develop Infrastructure as Code projects quickly and easily.