Dagger and Bazel

For those of you in the Bazel ecosystem, when you hear "Dagger", you probably think of the dependency injection framework for Java, Kotlin, and Android. And for good reason. Dependencies are frequently part of a build process. But there's another Dagger in the wild, a tool and platform for ephemerally building and testing multi-language projects. Sound familiar?

In this post, I look at what (the new) Dagger is, how it works, and how it compares to Bazel.

Dagger history and concepts

Dagger, started in 2022 by Solomon Hyke, the same creator as Docker, is part of the ecosystem of tools I call "code as infrastructure." In this model, you use your programming language of choice to define your infrastructure and test and build pipelines.

Under the hood, everything runs in containers, making Dagger pipelines portable and moveable between local runs or running on CI.

Dagger defines every task and workflow (including "infrastructure" setup) as a Dagger Function written in one of the currently available SDKs: Go, Typescript, and Python. Yes, this means you write functions to create Functions. It's a little confusing.

You can extend the core Dagger functionality with Daggerverse modules, which include a wide variety of use cases, from using helm charts to spinning up Kubernetes servers and various package managers.

At the center of everything is the Dagger Engine, which is open source and handles maintaining the connections between functions, caching, state, telemetry, and more.

There's also the Dagger Cloud, which currently provides a browser-based interface for tracing and debugging issues with Dagger Functions. This is Dagger's monetization strategy and is free for one user.

Example

This is the TypeScript example Daggerized application pipeline from the Dagger documentation. It builds two containers, runs tests, and then publishes one of the built containers.

import { dag, Container, Directory, object, func } from "@dagger.io/dagger"

@object()
class HelloDagger {
  /**
   * Publish the application container after building and testing it on-the-fly
   */
  @func()
  async publish(source: Directory): Promise<string> {
    await this.test(source)
    return await this.build(source).publish(
      "ttl.sh/myapp-" + Math.floor(Math.random() * 10000000),
    )
  }

  /**
   * Build the application container
   */
  @func()
  build(source: Directory): Container {
    const build = this.buildEnv(source)
      .withExec(["npm", "run", "build"])
      .directory("./dist")
    return dag
      .container()
      .from("nginx:1.25-alpine")
      .withDirectory("/usr/share/nginx/html", build)
      .withExposedPort(80)
  }

  /**
   * Return the result of running unit tests
   */
  @func()
  async test(source: Directory): Promise<string> {
    return this.buildEnv(source)
      .withExec(["npm", "run", "test:unit", "run"])
      .stdout()
  }

  /**
   * Build a ready-to-use development environment
   */
  @func()
  buildEnv(source: Directory): Container {
    const nodeCache = dag.cacheVolume("node")
    return dag
      .container()
      .from("node:21-slim")
      .withDirectory("/src", source)
      .withMountedCache("/root/.npm", nodeCache)
      .withWorkdir("/src")
      .withExec(["npm", "install"])
  }
}

You chain Functions together, meaning other Functions can call them. To run this example, you call the publish Function, passing a local code directory:

dagger call publish --source=.

The publish Function, in turn, calls test, which calls build, which calls buildEnv. You can pass variables between them, such as the code directory, and use any other aspect of the programming language you're using should you need them.

The Dagger-specific methods in each Function are fairly self-explanatory and often mirror their name and function in a Dockerfile. Again, they use chaining from the base dag client and any of its core types, such as a Container.

Once you run the pipeline and it is complete, besides anything retained for caching, the Dagger engine tears everything down. However, like with Docker, you can maintain state by mounting local volumes from the host system.

Bazel and/or Dagger

I initially set out to write this post looking at how to use Bazel and Dagger together. But I almost immediately came across this line in the documentation:

Before going any further, you should look for reasons not to adopt Dagger. Your project may not be a good fit for Dagger if: …

• You are happily using a monolithic toolchain, such as Gradle, Nix or Bazel, with no exception and no fragmentation within the team

There are also many discussions within the community on how the tools compare and contrast, often coming down to "stick with what works for you".

So, instead, what are the key differences and overlaps? In the general spirit of technology and developer tools, there are no hard and fast "best" answers. Often, it depends.

Native verses Containers

Dagger runs tasks in containers, which makes its pipelines portable. However, not everything can run in containers. They add their own overhead, such as container runtimes.

Bazel uses native environments to run tasks which are more performant, but can have inconsistencies between platforms and require accommodations for portability.

Configuration language

Bazel uses Starlark, which is Python-like but requires understanding some complex new concepts. It has widespread support for plugins and linters in IDEs.

Dagger uses Python, Go, or TypeScript with the relevant SDK, making it simpler to start. The world of "code as infrastructure" tools is interesting and includes something like Pulumi. I'm unsure if it has widespread adoption as a concept or if it's something that dev and DevOps teams actually want to mix together.

Language support

Support for most aspects of Bazel comes in the form of "rules". By default, Bazel has support for C, C++, Java, Objective-C, Proto Buffer, Python, and Shell. Through 3rd party community rules, Bazel can support many other languages natively.

While you can only write pipelines in one of the supported SDKs (or call the Dagger API directly), Dagger can build, test, and run whatever programming language you can run in containers.

Reproducibility

One of Bazel's main features is its reliable reproducibility no matter when or where you run a build or test.

Dagger relies on the reproducibility of containers for its own guarantees. As it has this reproducibility, it can add caching, helping speed up subsequent runs.

Community

Bazel originates in Google, but now has a large community of users, contributors, and a large ecosystem of rulesets and plugins around it to extend core functionality. Bazel is about nine years old. This is relatively new in the grand scheme of similar tools such as Nix or Gradle, but also perhaps considered "old" by some cutting-edge development teams.

Dagger is new and largely directed by one company. However, it is open source and has a healthy mix of contributors. The module ecosystem is reasonable, but it's always hard to predict how much maintainers will keep those up to date.

DevEx

Bazel treats most things as artifacts that depend on each other, which can require some rethinking for your processes but is also similar to Nix. Bazel often needs a reasonable amount of configuration files and rulesets to start.

While it uses common language patterns and containers (which are fairly established now), Dagger also requires some rethinking, especially if you're coming from "as code" tools. I also found that whilst the practice of chaining was conceptually straightforward, I found myself getting lost in it sometimes and passing lots of variables back and forth. Perhaps the biggest blocker is if you don't use Python, Go, or TypeScript, writing pipelines is more challenging. It's still possible to use it by calling the GraphQL API, but then you lose a lot of the advantages. And while containers are fairly flexible, there are still certain tasks that you can't or don't want to run in them.

Stick with what you know

Dagger and other "code as infrastructure" tools like it are interesting and tend to attract a lot of tech media attention. However, they are still new and often developed largely by one company funded by venture capital. This can mean the tool won't stick around, no matter how promising it is.

Echoing the statements I read in the Dagger community, but also those of any pragmatic developer tool community. There are always new shiny tools to try. They may offer benefits over what you use already, but if what you have works reliably, and you have staff who understand it, is it worth the effort? That's time you could spend on servicing and improving customer's needs.