Backstage: an Open-Source Developer Portal
A deepdive into what Backstage is, why it is gaining rapid popularity across large tech companies, and alternatives to it.
Over the last year, I have talked with software engineers at a variety of companies (Netflix, Grab, Wealthsimple, QuintoAndar, Wayfair). They all kept mentioning the same tool name: Backstage. All these companies were either planning, or in the process of adopting, Backstage as their developer portal.
After looking further, I observed that, although only released in 2020 in public, Backstage has seen surprisingly large adoption at larger tech companies. Other adopters include American Airlines, Booking.com, Brex, DAZN, Epic Games, Expedia, Glovo, HelloFresh, Monzo, PagerDuty, Splunk, Siemens, Trendyol, Twilio VMware, Wise, and hundreds of others.
I decided to look more into this topic. To do so, I initially contacted the most well-known Backstage SaaS provider, Roadie, for their insights, talked with an engineer from the team which created Backstage, and got in touch with Backstage adopters.
In this issue, we cover:
The need for a developer portal. Why do tech companies need a developer portal, and at what stage does this become necessary?
The history of Backstage. How did it start, and where is it today?
What is Backstage, and how does it work? An overview of the main parts: the software catalog, software templates, TechDocs, and other plugins.
Why was Backstage open sourced? Backstage could be considered a competitive advantage for Spotify. Why did they open source it?
Spotify’s version of Backstage. Spotify operates arguably the most advanced version of Backstage. What additional features have they built, and how do they use their developer portal?
Getting started with Backstage. How do you adopt the tool? A case study from RD Station and advice from Roadie.
Alternatives to Backstage. If you’re looking for a developer portal, what other alternatives do you have? A brief overview of Cortex, OpsLevel, Port, Clutch and Hygieia.
Further insights into developer portals. A follow-up to the original article.
This issue mentions several vendors related to developer portals. As per my ethics policy, I strive to provide an independent viewpoint, not taking any form of payment or another incentive to mention - or omit - any company, vendor, or topic. I disclose any conflicts of interest and affiliations, should I have them. I have no affiliations with any vendors mentioned in this article and no conflicts of interest.
1. The need for a developer portal
Let’s take the typical path of a tech startup. A technical founder builds a prototype of a web app, hosting the code on GitHub, running it on AWS. The project gets traction, and the founder gets investment and hires a team.
The team now builds a mobile app and starts to get more customers. They begin moving away from Lambdas to managing their own EC2 instances. The mobile team adds feature flags to easily roll back features, and all teams add monitoring, alerting, and error/crash reporting.
Fast forward to when the team is at 30 engineers, split between 5 Product teams, with the first Platform engineering team about to be born. The company is trying to get the timing of this team right.
How does the engineering team operate regarding the 'non-coding' work? They probably have 1 or more wiki pages that list:
The various services and codebases, and their owners
Steps to execute common operational tasks like adding more server capacity and debugging infrastructure issues
Links to SaaS providers they use for things like feature flags and observability, perhaps with some basic documentation on finding the relevant things
This setup still works for a small-enough team. However, as the engineering team grows, and the number of services, at one point:
Several of the new services won’t be added to this wiki page.
The documentation on the 'how to' tasks will become outdated.
The most productive engineers can recall what system to go to, to get something specific done.
A major challenge for a new joiner engineer becomes unpacking a lot of the tribal knowledge about developer tools and processes.
Engineering teams these days increasingly make use of the Cloud and various SaaS services for developer tasks. There’s an explosion of interfaces to learn, tools to use, and places to find service information. When a team has too many of these tools, and the company wants to cut down on the tribal knowledge they need to get around, as a developer: they will consider putting a developer portal in place.
This is where a developer portal like Backstage comes in. Spotify went through this type of same journey and faced the same challenges.
2. The history of Backstage
In 2014, Spotify was at around 600 employees, and over 100 engineers, but growing quickly. Even at that time, the engineering team, they observed issues with their developer experience. Martina Iglesias Fernández is currently the CTO of Roadie. At the time, she was a lead backend engineer at Spotify, and she described these challenges at the time:
A service sprawl. Spotify was spinning up new services weekly, or even more frequently.
Duplicated work. Two teams would be building a similar service.
Fragmentation. It was hard to tell what was deployed to production..
Lack of ownership. Some services were becoming “unowned” after the person who created them changed teams or left.
Backstage was born in response to these issues, with the project starting in 2014, called System Z. Here’s how Martina described the approach of the company:
“Backstage was conceived inside Spotify at a time when the company was experiencing rapid growth. New engineers were joining every week, and with them, the number of microservices was increasing.
Collectively, Spotify’s engineering team realized that it was becoming more difficult to know what we had running in production, and who was responsible for each component.
Eventually, one of the Platform teams decided to build a catalog of all microservices and to model systems and components in it. The catalog was called System-Z.
System-Z started as a place where Spotify teams could register microservices and their metadata. There would be a link to the code, the name of a team who owns it, and the name of the product owner. Eventually it grew to include information about its relation to other components, and groups of microservices began to be organized into cohesive systems.
As the usage of System-Z grew, we realized it was also the perfect place for us to put frontend UIs for developer infrastructure tools. So many tools that previously only had a CLI were extended to also have a UI frontend inside System-Z.
This huge growth of usage and features led to a rewrite of System Z in 2017, to what is known as Backstage today.”
Spotify initially built Backstage for their own use cases. The company saw good success by adopting System Z and extending it to what would become Backstage. They observed the following results by 2020:
A 55% decrease in onboarding time for their engineers. Onboarding time was measured by time until the 10th pull request.
280 engineering teams inside Spotify used Backstage.
2,000+ backend services, 300+ websites, 4,000+ data pipelines, and 200+ mobile features onboarded to Backstage.
The company realized that it was building something that similar companies could use, and company leaders decided to open source it in 2020. By the end of 2020, Backstage was accepted to the Cloud Native Computing Foundation.
As of today, Backstage is considered to be an incubating project, which is a stable project used successfully in production. More than 600 companies use Backstage to power their developer portals.
3. What is Backstage and how does it work?
Backstage is a platform for building developer portals. Its open-source version comes with a few components out of the box: