AWS CloudShell "Run From Web" Configuration Scripts

AWS CloudShell joins the ranks of hostless shells for operating in your cloud environment. Cloud shells are a huge help to training and enablement scenarios because they remove the pain of fussy configuration of a user-owned endpoint - which can have an endless variety of challenges installing utilities for something like managing a Kubernetes cluster. Additionally, once you get through the process of getting new utilities working, you often have to wonder what you broke that you don't know about yet.

On the flip side, AWS CloudShell cannot include every possible utility you may need or want to use. AWS CloudShell is currently based on Amazon Linux 2 and so the relevant commands apply.

Know Before You Install

Here are some tips to make note of when considerating doing installations and configurations within AWS CloudShell:

• AWS CloudShell is based on Amazon Linux, which means you must use yum. When you are reading installation documentation you want to look at the yum commands. The Amazon Linux yum repos are on by default.

• Since it is Amazon Linux, favor installing from Amazon Linux Extras if the software you are looking for is available there. These extras instructions also tell how to add the RHEL extras repo to Amazone Linux.

• Check if what you want is already installed on this list of AWS CloudShell Pre-installed Software.

• AWS CloudShell allows passwordless sudo by default - so using sudo to install things is easy.

• Making installations idempotent makes sure they are quick and error free. This essentially means checking if something is already present and only installing or configuring it if it is missing.

• AWS CloudShell does not persist configuration changes after you log off of the AWS account. They are also remembered on a per-account, per-region basis - so an installation into CloudShell while viewing us-east-1 is not retained if you switch to eu-central-1. This really means that at any given time you use CloudShell it will not be likely to have the configuration you need.

• For data storage (not installs and configuration) the $HOME folder can store up to 1GB of data per-region and is deleted after 120 days from the end of the last session.

• The $HOME directory will give a disk error at 1GB. A workaround is to use sudo in the /home directory for commands that will download more than 1GB. This content will not be retained, but if it is just temporary dependencies (like with complex terraform stacks) then at least you can get the work done.

• With regard to Kubernetes administration, the automatic kubeconfig setup using AWS CLI works great in AWS CloudShell. The code below includes config-kubectl.sh which do this configuration for you. It will enumerate the EKS clusters and prompt for one if you do not provide one.

Run From Web To The Rescue

For some years now I've been using what I call "Run From Web" code in both Bash and PowerShell to enable anyone to quickly use scripted solutions directly from git without a lot of fuss. This makes them ideal for helping with the fact that AWS CloudShell forgets configuration changes on logoff and only remembers them on a per-account, per-region basis in the first place.

The specific scripts also check if something is installed and don't install it if already present, making them lightning fast when doing incremental configuration.

Code Review

Below is the code block for installing terraform, notice:

• It self-documents how to run it from the web. (including hard lessons learned about not piping downloaded scripts directly into bash.)

• The if statement ensures we only do work if we need to. Also if you run this terraform configuration and then later run the add-all.sh script - which also includes terraform - the installs layer up without duplicate work or errors (idempotency).

• The final echo command tells us the installed version. It functions as both a test of whether an install worked as well as a visual confirmation that something is already installed when no install was required.

# Run this directly from the web with:
# curl -sSL https://gitlab.com/guided-explorations/aws/aws-cloudshell-configs/-/raw/main/add-terraform.sh -o $HOME/add-terraform.sh; chmod +x $HOME/add-terraform.sh; bash $HOME/add-terraform.sh

echo "AWS Cloudshell Configuration"

if [[ -z $(command -v terraform) ]]; then
  echo "adding terraform"
  sudo yum install -y yum-utils 
  sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo
  sudo yum -y install terraform
fi 
echo "TERRAFORM VERSION: $(terraform --version)"

Managing Terraform Templates and State

One of the main things I will be using AWS CloudShell for is running the Terraform templates of EKS Blueprints. These are likely to have more than 1GB of downloading dependencies, so operating in the $HOME directory of AWS CloudShell causes an out of disk space error. However, operating anywhere else causes the directory to be tossed as well as introduces some challenges with constantly use sudo to be able to change the disk.

The work around is to create a root directory and permission it for the cloud shell user and then user the -state switch of terraform to store the state in $HOME where it will persist for up to 120 days.

The below code repository includes prep-for-terraform.sh that will prepare the setup for you and shows commented terraform commands to go with a default setup.

Code For This Article

Here is the open source code: AWS CloudShell Run From Web Configuration Scripts

Mission Impossible Code Series Inclusion

• The solution is concise.

• The solution is idempotent (only takes steps necessary when things are missing).

• The solution runs directly from the web.