Internal developer portals aren't a silver bullet for platform engineering

A hard-to-ignore theme at KubeCon this year was how many organizations are grappling with the challenges of improving developer velocity and wrangling the increasing complexity of their tooling. To improve the situation, many organizations are now investing in, or exploring the idea of platform engineering as the solution to their developer velocity challenges.

In the absence of a prescriptive guide on how to build a platform, many organizations are turning to internal developer portals as their first investment. Whilst some organizations do achieve success, many are also struggling. The “average Backstage adoption rate is stuck at 10%” reported The New Stack. For inexperienced platform teams, internal developer portals risk becoming a hammer looking for a nail in a scramble to solve for developer velocity.

In this article, we explore the trend towards internal developer portals, and the reasons why many platform teams seem to be struggling with adoption. We then take a look at why our platform tools must use interfaces most relevant to our developers, if we are to avoid needing to resort to top-down mandates to spur platform tooling adoption. Drawing on my past experiences adopting Backstage, and stories from Gitpod, hopefully you’ll be able to avoid similar mistakes and instead achieve successful adoption for your platform.

Caption: "Build an ecosystem, not a wilderness" - slogan from the Backstage homepage.

Backstage first launched in 2020 by the engineering team at Spotify. The announcements caught the eye of the developer experience team I was working on at DAZN. What Spotify were articulating in their messaging felt so painfully similar to what we felt at the time, as we wrangled with trying to support in increasing the developer velocity of a scaling engineering organization, built on top of microservice principles and architecture.

Software sprawl was rife. Duplicated work was happening across many different libraries and packages across the organization. Developers complained of the amount of work and platform knowledge required to launch a new service. Reliability was a top concern, and during incidents, it was often hard to figure out which service belonged to which team, leading to longer than tolerable outages. Leadership could only look on, dismayed.

“Build an ecosystem not a wilderness” was the messaging of Backstage. Which resonated deeply with us, and soon we became early adopters. Since Backstage was announced the market for internal developer portals has exploded, and we see many other tools in the market like getport, cortex, opslevel offering various takes on the same underlying principles of having a centralized, extensible catalog “single-pane-of-glass” into your internal developer platform.

Caption: Five Backstage lessons in 5 minutes - Chris Westerhold, Thoughtworks

However, with the growth of Spotify Backstage, comes a growing emphasis on internal developer portals as a significant piece of the platform engineering puzzle. With many newly minted platform engineering functions risk making the cardinal sin of the platform, which is thinking “if we build it they will come”. As Chris Westerhold of Thoughtworks pointed out at his BackstageCon talk: “you can actually make your developer experience worse” (source)

It’s becoming a trope that every Backstage talk seems to start and end more-or-less identically with an opener like: “we had challenges with developer experience, we started our journey to platform engineering, then we discovered Backstage…”. before coming to a conclusion with some variation of: “internal developer portals aren’t a silver bullet”, or...

“If you build it they will come doesn’t work”.

This trend points to a repeating pattern.

Which makes me wonder: did these platform teams not watch the other adopter talks? Or, do these platform teams believe they are unique enough to not need to heed the warnings from others who have walked a similar path before them? And finally, what is it about internal developer portals that creates such a big challenge for adoption?

Caption: Spotify Engineering Culture.

Spotify’s Backstage feels to have a parallel to the Spotify model, a structure for engineering organizations that Spotify popularized. The spotify model was subsequently adopted by many organizations as the “silver bullet” to their software delivery challenges. However, as many organizations soon found out: what works for Spotify might not work for you.

Just as the human body can “reject” a foreign transplant, engineering organizations can reject “implanted” culture and tools, including developer portals.

I believe a big part of the challenge here is that platform teams make the mistake of jumping too early to the solution. Skipping over the—often difficult—step of building a deep, empathetic understanding for the problem they are solving. However, the realization of the overconfidence always comes too late in the process, typically when the developer portal is unveiled and the adoption numbers don’t grow as you might have hoped. If the agile revolution taught us anything, it’s that we must test our assumptions, and work in small iterations to de-risk.

Another challenge is that when the path of least resistance for new platform capabilities is the developer portal, portals become the go-to option for interfacing into the platform. Portals might be easy to implement, but that doesn’t mean they’re easy for developers to adopt. When adoption challenges arise, the common response is usually that the portal “needs more functionality”. However, what experience has taught me is that adding more functionality into a developer tool seldom solves adoption challenges—as the challenges are often more complex.

What I’ve learned from my time working on platform teams in the past, and now from working on product at Gitpod is how critical it is to

"Leave interface choice to the developer—and never break their workflow”

As articulated by Humanitec in: how to make an enterprise grade internal developer platform.

Because, if you’ve ever tried, you know just how difficult it is to change the habits of your users. And this is particularly true if your users are developers.

Caption: Mechanical Keyboard by Michelle Ding (Unsplash)

Developers are highly technical users, with incredibly specific behaviors that have been honed through their many years of their professional work. Developers are often tuned towards quite specific interface mental models when completing certain tasks.

When it comes to tooling adoption, what I’ve come to learn is:

The sane choice—often the only choice—is to: "meet developers where they are аt"

We need a deep understanding of our developers, and we need to understand their behavioral patterns. User researchers have understood the importance of observing users in their natural environment for some time to inform product decision making, researchers call this practice contextual inquiry. Which is the practice of observing users in their context, to more deeply understand how they perform certain tasks. Why? Because the best indicator of future behavior is past behavior. With platform tooling, we’re often trying to nudge developers towards best practices. To do this, we need to understand developers' current behaviors.

To underscore how critical it is to “meet developers where they are at”, let me give you a few examples from my time working at Gitpod. At the time of writing, Gitpod has over >1 million developers on the platform. In building out Gitpod, we’ve learned a lot about developer experience, and we’ve also certainly made our fair share of mistakes. But what we’ve also seen is that every time we move closer to meeting developers where they are at, we also see a spike in adoption, which is not merely a coincidence.

For context, Gitpod is a “Cloud Development Environment''. Which in simple terms means that when developers are using Gitpod, they don’t work on their own machines, but they work in the cloud. There is certainly a lot more to cloud development environments, in general, but that’s not the purpose of today's blog. All you need to know is that the primary interface to Gitpod is mostly an editor or IDE. And, as we’ve learned over the years, the editor is at the center of the developer workflow—a very sensitive part of the developer experience.

Caption: Header image from From Theia to OpenVSCode Server - A history of Cloud IDEs

The first time we saw one of those significant spikes in adoption was when Gitpod moved from the browser-based editor developed by Gitpod called Theia to VS Code. At the time, this move felt bold and risky. Theia was inspired by and looked incredibly similar to VS Code. But, what we learned in hindsight was how the developer experience simply wasn’t good enough for mass adoption. Developers wanted the full, undiluted VS Code experience they had come to rely on—and not some “diet coke” version. As I mentioned, the editor is one of the most sensitive aspects of the developer experience, and even subtle differences cause distraction and pull developers out of their flow state. Moving to VS Code marked one of the first major upticks in users on the platform, no doubt attributed to the familiarity and convenience of VS Code.

We then saw another similar spike in adoption when we announced our JetBrains integration. At the time we were astounded by just how many users were using VS Code with Gitpod, but were also secretly dying to use their more familiar JetBrains IDE. Again, possibly unsurprisingly, by giving developers literally their interface of preference, we again saw adoption spike. What I learned throughout both of these stories was just how specific developer requirements for interfaces can be. Admittedly, it can be difficult to comprehend this phenomena unless you’ve experienced it first hand, which is why I also believe that many platform teams are often overly optimistic about their adoption numbers when it comes to new developer tools.

I could share more examples, but by now I hope the point is fairly clear. Developers are power users, with unique mental models, behaviors and expectations for their tooling. If hoping developers change their workflows is tied to your platform's success, adoption is going to be a difficult, or even impossible task. Because, the more you diverge from meeting developers where they’re at, the more you’ll need to apply top-down pressure, mandates and send threatening emails to get your developers to use your platform tooling. Which goes against the mantra of most platform teams, who actively want their tools to be so good that developers actively want to use them.

Instead, if we’re looking for true adoption success, we should choose the right interface for the job, taking each tool on a case-by-case basis. Platform teams need to remember how many options we have: building command line interfaces, libraries, editor plugins, browser extensions, slack bots, native apps, pull request apps, continuous integration plugins and more to bring the interface of the platform closer to our developers to support with driving adoption. We need to avoid the temptation to put all of our platform tools into our portal, simply because we have one. That all being said, internal developer portals absolutely do have a very necessary role in our platform engineering stack, but we do need to be careful that portals don’t become a hammer looking for a nail.

If the original goal of platform engineering was to keep our developers in flow, then blindly baking every platform tool into a user interface doesn’t make sense. Instead, we should strive to meet developers where they are at and keep developers in flow.

---

A special thanks to the folks outside Gitpod who also helped by reviewing, proofreading or otherwise providing valuable inputs that lead to this blog: Roy Ronalds (Software Architect), Fernando Villalba (Platform Engineer, OpsLevel), Himanshu Mishra (Product, Harness), Boris Tane (Founder, Baselime).