Never Compile protoc Again

Background

Protocol Buffers needs a code generation step, turning your .proto files into “stub” code in your language of choice which provides marshaling/unmarshaling of proto messages (binary or JSON typically) into language-idiomatic data structures. If you use services, then you also want “stub” code to implement clients and/or servers that use gRPC.

This code generation needs a “compiler” for the protobuf language and the idiomatic compiler provided by the Protobuf team at Google is protoc. They provide binary releases of this tool on https://github.com/protocolbuffers/protobuf/releases and that’s where most protobuf users get it (maybe via a plugin for their Build system, such as https://github.com/google/protobuf-gradle-plugin)

However under Bazel, the common means of getting the tool is to compile it yourself from this cc_binary target. This is what Googlers expect, having lived in an extreme monorepo where everything is vendored, and security is paramount. See an interesting comment I got in a discussion:

Interesting. I wasn't aware there was a contingent of Bazel rules maintainers who are opposed to google3-style source-only builds.

Yes! I’m one of those maintainers. That’s because most of us don’t have perfect caching. Ideally you don’t notice protoc compilation other than “the first time”. In practice, I wait for protoc to build all the time, and it spams my CI log with gcc warnings that are irrelevant to me. And on some machines, the compilation fails. If you haven’t listened to the Aspect Insights podcast, this topic was all the way back at episode 1.

Avoiding it

The exciting news is that many Bazel projects can stop compiling protoc TODAY.

Aspect engineer Sahin worked with the Bazel team long ago to help introduce a new flag, allowing the protoc binary to be registered as a toolchain. This gives us the flexibility of registering something other than the cc_binary target. In fact, you’re free to register /bin/false as your protocol buffer compiler, if you want something very fast and very broken.

I then wrote https://github.com/aspect-build/toolchains_protoc which gives the convenience of downloading the official binary release, and registering that. See the docs on that repo, and especially the examples folder.

Unfortunately we haven’t made as much progress as we’d like, because there’s an ecosystem-wide change required. Everywhere that references the protobuf compiler needs to use the toolchain to resolve it. As of this writing, https://github.com/protocolbuffers/protobuf/pull/19679 is pending as a fix for the worst culprit.

Then there is a long, long tail of issues. For example, I was waiting 10 minutes to cut releases of our Bazel OSS rulesets, just because the stardoc documentation generator is a java_binary target with a hard-coded transitive dep on the cc_binary(name=”protoc”) target and the CD step wasn’t getting a cache hit for it. So I worked around that with another pre-build: https://github.com/alexeagle/stardoc-prebuilt and then applied a small patch to our rulesets to override the renderer attribute, for example: https://github.com/aspect-build/rules_js/pull/2156. It’s unfortunate to have to work around the problem, but in this case it’s worth it to fix the slowest step in our releases.

Making it official

We’re hoping to upstream our toolchains_protoc to the protobuf repository, so that the default Bazel experience will be fast. Of course, you can still choose to register the cc_binary target as your proto compiler toolchain, if you want to build it from source.