Running local tools installed by Bazel

It's a common pattern that developers in your repo are expected to run some command-line tools as part of interacting with the code. For example, maybe they need to run terraform plan when working with Terraform files.

However, it's a non-stop hassle to get the right version of Terraform installed on everyone's machine. There's always someone asking in Slack "hey it seems like there's some unsupported syntax in this code can you help me" and after some debugging you realize they installed the tool three years ago and that version isn't supported by the current code.

Bazel is great for this! However, the pattern to set it up has always been fiddly. I recently had a client who needed a better answer, so here it is!

Fetching the tools

This post assumes the tools you need to run are statically-linked as an executable. If you're instead meant to gem install or pip install or npm install the tool, then this pattern is too simple to work.

I recently contributed to a nice utility ruleset for Bazel that gives you a JSON-format lockfile for the tools you want to fetch on their various platforms. Continuing with the terraform example, we could have a tools/multitool.lock.json file containing:

{
  "$schema": "https://raw.githubusercontent.com/theoremlp/rules_multitool/main/lockfile.schema.json",
  "terraform": {
    "binaries": [
      {
        "kind": "archive",
        "url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_darwin_arm64.zip",
        "sha256": "99c4d4feafb0183af2f7fbe07beeea6f83e5f5a29ae29fee3168b6810e37ff98",
        "os": "macos",
        "cpu": "arm64",
        "file": "terraform"
      },
      {
        "kind": "archive",
        "url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_darwin_amd64.zip",
        "sha256": "0eaf64e28f82e2defd06f7a6f3187d8cea03d5d9fcd2af54f549a6c32d6833f7",
        "os": "macos",
        "cpu": "x86_64",
        "file": "terraform"
      },
      {
        "kind": "archive",
        "url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_linux_amd64.zip",
        "sha256": "3ff056b5e8259003f67fd0f0ed7229499cfb0b41f3ff55cc184088589994f7a5",
        "os": "linux",
        "cpu": "x86_64",
        "file": "terraform"
      },
      {
        "kind": "archive",
        "url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_linux_arm64.zip",
        "sha256": "08631c385667dd28f03b3a3f77cb980393af4a2fcfc2236c148a678ad9150c8c",
        "os": "linux",
        "cpu": "arm64",
        "file": "terraform"
      }
    ]
  }
}

NB: I'd love to work on some tooling to generate this file for an arbitrary list of released binaries. Please fund us!

Now we just need to point Bazel at this JSON lockfile to get the tools installed. Paste the MODULE.bazel or WORKSPACE snippet from the latest release: https://github.com/theoremlp/rules_multitool/releases

Making it nice for developers

The desired end-user experience shouldn't involve Bazel and should be as simple as running the tool in your terminal:

$ ./tools/terraform -version
Terraform v1.7.5
on linux_amd64

First, as always, is to find a workaround for a Bazel footgun: bazel run cannot possibly behave correctly because it will change the working directory. (It's designed with the assumption that you'd only want to run programs you wrote yourself...) If the working directory is wrong, then we run ./tools/terraform plan it can't found the Terraform files: Error: No configuration files. Oops! Come visit https://github.com/bazelbuild/bazel/issues/3325 and give it a thumbs up.

You might see —-run_under suggested for this purpose, with a simple value to change the working directory like —-run_under=”cd $PWD &&”. However this is broken under Bazel as well, as it discards the analysis cache. Oops! Come visit https://github.com/bazelbuild/bazel/issues/10782 and ask for this to happen only when the value of run_under looks like a label.

To work around these issues, we're forced to use a mix of bazel build to fetch the tool into bazel-out, bazel cquery to know what path it's installed under, and bazel info to convert that relative path. Sigh.

Create a helper script in tools/_multirun_under_cwd.sh , make it executable and add this content:

#!/bin/sh
target="@multitool//tools/$(basename "$0")"
bazel 2>/dev/null build "$target" && exec $(bazel info execution_root)/$(bazel 2>/dev/null cquery --output=files "$target") "$@"

See an updated and more full-featured version of this script in our aspect init template repo: https://github.com/aspect-build/aspect-workflows-template/blob/main/%7B%7B%20.ProjectSnake%20%7D%7D/tools/_run_under_cwd.sh

This script assumes that the name of the program being run (basename "$0") is the name we gave in the JSON file earlier "terraform". So the final step is just a symlink:

cd tools; ln -s _multitool_run_under_cwd.sh terraform

Now the tool behaves the way users expect, running in whatever working directory they're in. Let's try under our /infrastructure folder:

/infrastructure$ ../tools/terraform init

Initializing the backend...
Initializing modules...

Nice! Now you can repeat this for other tools you need to run, just by adding an entry to the JSON file, and the corresponding symlink in the tools folder.

UPDATE May 2024: rules_multitool now has a dedicated cwd target: https://github.com/theoremlp/rules_multitool/pull/29 However the pattern above is still useful if you fetched your tools in some other way.