Setting Up Self-hosted Runners on AWS EC2 (Ubuntu)

Introduction

Self-hosted runners provide greater flexibility and customization for your GitHub Actions workflows. This guide will walk you through the process of setting up and managing your own runners on AWS EC2 using Ubuntu.

Why Use Self-hosted Runners on EC2?

Benefits of EC2-based Runners:

• Cost Control: Optimize instance types for your specific workloads

• Network Access: Direct access to your AWS resources and VPCs

• Customization: Install specific software and dependencies

• Scalability: Easily scale up or down based on workflow demands

• Persistence: Maintain state between workflow runs if needed

• Resource Control: Choose instance types with the CPU/RAM/storage you need

Prerequisites

Before you begin, make sure you have:

• An AWS account with permissions to create EC2 instances

• A GitHub repository where you want to use self-hosted runners

• Basic knowledge of AWS EC2 and SSH

• Admin access to the GitHub repository or organization

Step 1: Launch an EC2 Instance

1. Log in to AWS Console and Launch an Instance

1. Go to the AWS Management Console

2. Navigate to EC2 Dashboard

3. Click "Launch Instance"

2. Choose an Ubuntu AMI

1. Select "Ubuntu Server 22.04 LTS (HVM)"

2. This provides a stable, long-term supported environment for your runner

3. Select Instance Type

1. For basic workflows: t3.small (2 vCPU, 2 GB RAM)

2. For moderate workloads: t3.medium (2 vCPU, 4 GB RAM)

3. For resource-intensive tasks: m5.large or higher

4. Configure Instance Details

1. Network: Select your VPC

2. Subnet: Choose a subnet with internet access

3. Auto-assign Public IP: Enable

4. IAM role: Attach a role with necessary permissions if your workflows need AWS services

5. Add Storage

1. Root volume: At least 20 GB (more if you'll be building large projects)

2. Volume type: gp3 (general purpose SSD)

6. Configure Security Group

Create a security group with these rules:

• SSH (port 22): Restrict to your IP address

• HTTPS (port 443): Allow from anywhere (for GitHub communication)

• HTTP: Allow from anywhere

7. Launch Instance and Create/Select Key Pair

1. Launch the instance

2. Create or select an existing key pair

3. Download the key pair if creating new

4. Set proper permissions: chmod 400 your-key.pem

Step 2: Connect to Your EC2 Instance

ssh -i your-key.pem ubuntu@your-ec2-public-dns

Step 3: Prepare the EC2 Instance

1. Update System Packages

sudo apt update
sudo apt upgrade -y

2. Install Required Dependencies

# Install basic tools
sudo apt install -y curl wget git jq build-essential

# Install Docker (optional, if you need Docker for your workflows)
sudo apt install -y docker.io
sudo systemctl enable docker
sudo systemctl start docker
sudo usermod -aG docker ubuntu

3. Create a Dedicated User for the Runner (Optional but Recommended)

# Create a user
sudo adduser github-runner

# Add to necessary groups
sudo usermod -aG docker github-runner

# Switch to the new user
sudo su - github-runner

Step 4: Set Up the GitHub Runner

1. Get Runner Registration Token

For Repository-level Runner:

1. Go to your GitHub repository

2. Navigate to Settings → Actions → Runners

3. Click "New self-hosted runner"

4. Select Linux as a Runner Image

5. Select architecture as x64

6. Open a your EC2 instance terminal and run all command step by step as provide by GitHub

During configuration, you'll be asked to, just press enter, if you do not want to change anything:

1. Enter a runner group name (default)

2. Enter a runner name (default is the hostname)

3. Enter additional labels (e.g., ubuntu-ec2, production)

4. Enter a work folder (default is _work)

Step 5: Verify Runner Registration

1. Go back to your repository settings

2. Navigate to Settings → Actions → Runners

3. You should see your new runner listed as "Idle"

4. The status will change to "Active" when running a job

Step 6: Create a Hello World Workflow to Test Your Self-hosted Runner

Let's create a simple "Hello World" workflow to verify your self-hosted runner is working correctly.

Create a .github/workflows/hello-world-runner.yml file in your repository:

# ===================================================
# HELLO WORLD SELF-HOSTED RUNNER TEST - MODULE 3
# ===================================================
# LEARNING OBJECTIVES:
# - Understand how to target self-hosted runners
# - Learn how to verify runner functionality
# - See basic environment information retrieval
# - Experience the difference between GitHub-hosted and self-hosted runners
# ===================================================

name: Hello World Self-Hosted Runner

# LEARNING POINT: Multiple trigger types for flexibility
on:
  # Manual trigger from GitHub UI
  workflow_dispatch:

  # Automatic trigger on push to main branch
  push:
    branches: [ main ]

jobs:
  # LEARNING POINT: Simple job targeting self-hosted runner
  hello-world:
    # LEARNING POINT: This is how you specify a self-hosted runner
    # Instead of ubuntu-latest, windows-latest, etc.
    runs-on: self-hosted

    steps:
      # LEARNING POINT: Standard checkout action works on self-hosted runners too
      - name: Checkout code
        uses: actions/checkout@v4

      # LEARNING POINT: Basic hello world with runner information
      - name: Hello from self-hosted runner
        run: |
          echo "========================================"
          echo "👋 Hello, World! I'm a self-hosted runner!"
          echo "========================================"
          echo "🏷️ Runner name: ${{ runner.name }}"
          echo "💻 Runner OS: ${{ runner.os }}"
          echo "📂 Working directory: $(pwd)"
          echo "📦 Repository: ${{ github.repository }}"
          echo "🔄 Workflow: ${{ github.workflow }}"

      # LEARNING POINT: Simple system information for verification
      - name: Basic system info
        run: |
          echo "========================================"
          echo "📊 System Information"
          echo "========================================"
          echo "🖥️ Hostname: $(hostname)"
          echo "💾 Disk space:"
          df -h / | grep -v Filesystem

          # Show that we're running on the self-hosted machine
          echo "🌐 IP Address:"
          hostname -I || echo "IP address command not available"

# ===================================================
# LEARNING NOTES:
# ===================================================
# 1. Self-hosted runners are specified with "runs-on: self-hosted"
# 2. You can add labels to runners and target them with:
#    runs-on: [self-hosted, linux, production]
# 3. Self-hosted runners have access to their host environment
# 4. You can install custom software on self-hosted runners
# 5. Self-hosted runners can access internal networks
# ===================================================

How to Use This Test Workflow

1. Commit and push the workflow file to your repository

2. Go to the Actions tab in your GitHub repository

3. Select "Hello World Self-Hosted Runner" from the workflows list

4. Click "Run workflow" and select the branch to run on

5. Watch the workflow run on your self-hosted runner

What This Test Verifies

This simple workflow confirms:

• Your self-hosted runner is properly connected to GitHub

• The runner can check out code from your repository

• Basic system information about your EC2 instance

Learning Points

1. Self-hosted runners are specified with runs-on: self-hosted

2. You can add labels to runners and target them with: runs-on: [self-hosted, linux, production]

3. Self-hosted runners have access to their host environment

4. You can install custom software on self-hosted runners

5. Self-hosted runners can access internal networks

If you see output from this workflow, congratulations! Your self-hosted runner on EC2 is working correctly and ready to run your GitHub Actions workflows.

For Sample Output you can visit: Sample Output

EC2 Instance Maintenance

Regular Updates:

sudo apt update
sudo apt upgrade -y

Troubleshooting

Common issues:

• Incorrect permissions

• Network connectivity problems

• GitHub token expired

Jobs Not Being Assigned

Check:

1. Runner is online in GitHub UI

2. Job's runs-on label matches your runner's labels

3. Repository has access to the runner group

Network Connectivity Issues

Test GitHub connectivity:

curl -v https://github.com

Check outbound access:

# Test HTTPS connectivity
curl -v https://api.github.com

# Test network configuration
ip addr
route -n

Cost Optimization

1. Choose the Right Instance Type

Match the instance type to your workload:

• CPU-intensive: Compute-optimized instances (c5, c6g)

• Memory-intensive: Memory-optimized instances (r5, r6g)

• Balanced: General purpose instances (t3, t2, m5)

2. Use Spot Instances

For non-critical workflows, consider spot instances:

• Up to 90% cheaper than on-demand

• Configure fallback to GitHub-hosted runners if spot instances are terminated

Conclusion

You now have a self-hosted GitHub Actions runner running on an AWS EC2 Ubuntu instance. This setup gives you full control over your CI/CD environment while leveraging the flexibility of AWS infrastructure.

By using EC2 for your self-hosted runners, you can:

• Customize the environment to your exact needs

• Access AWS resources directly with low latency

• Control costs by choosing appropriate instance types

• Scale your CI/CD capacity as your needs grow

Additional Resources

• GitHub Actions Self-hosted Runners Documentation

• AWS EC2 User Guide

• GitHub Actions Runner Repository

• AWS Auto Scaling Documentation