A Lightweight, Serverless Self-Hosted GitHub Runners on AWS (Part 2)

Welcome to the second article in our series on building a lightweight, serverless, and cost-effective self-hosted GitHub Actions runner on AWS. In Part 1, we outlined the high-level architecture and the "why" behind this approach. Now, it's time to get our hands dirty and build the first crucial piece of the puzzle.

In this article, we'll walk through creating a GitHub App and configuring the webhook. This will serve as the event-driven trigger for our entire workflow, kicking things off the moment a new job is started in GitHub Actions.

---

GitHub Webhook: The Starting Point of the solution

The starting point for the entire serverless runner solution is the GitHub webhook. A webhook is an automated message sent from one system to another when a specific event occurs. For our purposes, when a job is started in a GitHub Actions workflow, GitHub will send a webhook event—a payload of JSON data—to a designated URL, it will be the AWS API Gateway and we'll discuss it in next article.

This GitHub webhook acts as the critical bridge connecting our GitHub organization, where the code and workflows reside, to the AWS environment, where the self-hosted runners will be provisioned. It's the event-driven trigger that enables us to react in real-time and build a truly on-demand system.

To configure this webhook, we could apply it directly to a repository or an entire organization. However, a more robust, secure, and scalable approach is to manage the webhook through a GitHub App. In this article, I'll walk through setting up the webhook via a new GitHub App.

Why a GitHub App?

Using a GitHub App to manage our webhook provides several key advantages over direct configuration:

• Enhanced Security: Apps have their own granular permissions, allowing us to grant them the minimum access required to function. We aren't relying on a user's personal access token as the best practice in an Enterprise environment.

• Centralized Management and Scalability: In GitHub Enterprise Cloud, the GitHub App can be created, managed in a centralise GitHub organization and installed on other enterprise-owned organization easily. It reduces the management overhead of creating and configuring the webhook each organization directly.

Step 1: Create the GitHub App

First, you'll need to be an owner of the GitHub organization where you want to create the App.

1. Navigate to Organization Settings: Go to your GitHub organization, click on Settings, then scroll down to the Developer settings section in the left-hand sidebar and click on GitHub Apps.

2. New GitHub App: Click the New GitHub App button.

3. Fill in App Details:

   • GitHub App name: Give it a descriptive name, like aws-severless-runner-app.

   • Homepage URL: This is required. You can use your company's website or a link to the GitHub organization, e.g., https://github.com/my-organization.

4. Configure the Webhook:

   • Webhook URL: This is the endpoint where GitHub will send the workflow_job events. For now, we don't have our API Gateway set up yet. Enter a placeholder gateway URL for now, https://my-serverless-runner.com/my-organization/placeholder-gateway-xxxxxxx

While you can use webhook.site for temporary Webhook event validation, it's not recommended due to data exposure risks.

* Webhook secret: This is critical for security. Generate a strong, random string and save it somewhere safe. We'll use this later in our Lambda function to verify that the webhook payloads are genuinely from GitHub. You should store this in AWS Secrets Manager. 5. Set Permissions: This is the most important step for security. We must follow the principle of least privilege. * Expand Repository permissions, find Actions and select Read-only from the dropdown. This is the only permission our app needs to receive the workflow_job event from repository where the GitHub Actions workflows are started. * Expand Organization permissions, find Self-hosted runners and select Read and write to view and managet self-hosted runners available to an organization. 6. Subscribe to Events: * Scroll down to the Subscribe to events section. * Select Workflow job. This tells GitHub to send a webhook event for all activities related to a workflow job (queued, in_progress, completed). 7. (Optional) Select This Enterprise: from Where can this GitHub App be installed?, if you want to install the App on other organizations in your GitHub Enterprise. 8. Create the App: Click Create GitHub App.

Step 2: Install the App in Your Organization

Creating the App doesn't automatically make it available. You need to install it.

1. From your new App's settings page, click on Install App in the left sidebar.

2. Click Install next to your organization's name.

3. You'll be asked which repositories the App should have access to. You can choose All repositories or Only select repositories. For now, you can select the specific repository where you plan to test your self-hosted runners.

4. Click Install.

Step 3: Trigger a Workflow and Inspect the Payload

Now for the fun part. Let's see our webhook in action.

1. Create a simple workflow file in the repository where you installed the App. Name it .github/workflows/test-runner.yml:

    name: Test Self-Hosted Runner
   
    on:
      push:
   
    jobs:
      build:
        runs-on: [self-hosted, my-runner]
        steps:
          - name: Checkout code
            uses: actions/checkout@v4
          - name: Simple command
            run: echo "Hello from a self-hosted runner!"
   

2. Commit and push this file. This will trigger the workflow. Since there's no runner with the labels self-hosted, my-runner available yet, the job will go into a queued state.

   

3. Go back to the GitHub Organization Settings, then Developer Settings. Check the Advanced settings of the newly created GitHub App. You should see some Recent Deliveries. This is our webhook events!

   

If you inspect the JSON payload, you'll see something like this (simplified for clarity):

{
  "action": "queued",
  "workflow_job": {
    "id": 123456789,
    "run_id": 987654321,
     "workflow_name": "Test Self-Hosted Runner",
     ...
    "status": "queued",
    "labels": [
      "self-hosted",
      "my-runner"
    ],
    ...
  },
  "repository": {
    "name": "repo-name",
    "full_name": "github-org-name/repo-name",
    ...
    "owner": {
        "login": "github-org-name",
        ...
     },
  },
  ...
}

If a placeholder Webhook URL was utilized for the GitHub App Webhook configuration, you may observe failed deliveries. The event JSON payload can still be examined by expanding the delivery details.

This payload is the lifeblood of our serverless solution. The action: "queued" tells us a new job is waiting. The workflow_job.labels array tells us exactly what kind of runner is needed. And the name: "repo-name", `login: "gitub-org-name" tells us the GitHub organization and the repository name where the workflow was started. This is the information our Lambda function will use to provision the correct Fargate task.

---

What's Next?

We've successfully set up the trigger for our automation. We have a GitHub App that securely notifies us whenever a specific type of job is requested.

In Part 3 of this series, we'll build the API Gateway endpoint. We'll replace our webhook placeholder URL with a real, secure API Gateway endpoint and write the initial Lambda code to receive and validate these webhook events.

Stay tuned!