How Kubernetes is Built with Kat Cosgrove

The Pragmatic Engineer

0:00

-1:08:46

How Kubernetes is Built with Kat Cosgrove

Kat Cosgrove shares what Kubernetes actually does, how it’s structured and scaled, and why it’s become one of the most successful open-source projects in the world.

Gergely Orosz

May 14, 2025

Transcript

Stream the Latest Episode

Listen and watch now on YouTube, Spotify and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.

Brought to You By

•⁠ WorkOS — The modern identity platform for B2B SaaS.

•⁠ Modal⁠ — The cloud platform for building AI applications.

•⁠ Cortex⁠ — Your Portal to Engineering Excellence.

—

In This Episode

Kubernetes is the second-largest open-source project in the world. What does it actually do—and why is it so widely adopted?

In this episode of The Pragmatic Engineer, I’m joined by Kat Cosgrove, who has led several Kubernetes releases. Kat has been contributing to Kubernetes for several years, and originally got involved with the project through K3s (the lightweight Kubernetes distribution).

In our conversation, we discuss how Kubernetes is structured, how it scales, and how the project is managed to avoid contributor burnout.

We also go deep into:

An overview of what Kubernetes is used for
A breakdown of Kubernetes architecture: components, pods, and kubelets
Why Google built Borg, and how it formed the basis of Kubernetes
The benefits of large-scale open source projects—for companies, contributors, and the broader ecosystem
The size and complexity of Kubernetes—and how it’s managed
How the project protects contributors with anti-burnout policies
The size and structure of the release team
What KEPs are and how they shape Kubernetes features
Kat’s views on GenAI, and why Kubernetes blocks using AI, at least for documentation
Where Kat would like to see AI tools improve developer workflows
Getting started as a contributor to Kubernetes—and the career and networking benefits that come with it
And much more!

Takeaways

Some of the most interesting topics discussed in the conversation were these:

1. Kubernetes helps with managing large-scale backend applications. Google originally built an in-house tool to manage the tens of thousands (then hundreds of thousands and later millions) of machines: this internal tool is called Borg. The roots of Kubernetes come from Borg: but Google has since donated the project to the Cloud Native Computing Foundation (CNCF) – and today, Kubernetes is the second largest open source project, after Linux. We previously did a deepdive on How Linux is built, and touched on Google’s SRE roots in What is Reliability Engineering?

2. Kubernetes is a very well-structured and organized open source project. The structure of the project and all processes are well documented. The project has around 150-200 maintainers, has a few dozen SIGs (Special Interest Groups) and releases run on a 14-16 week cycle.

3. The “lead” and “shadow” concept is a clever one, utilized by Kubernetes. The Release Team within Kubernetes owns releases, and the Release Team has about 20-30 people participating in each release. About half of the participants in the Release Team are “shadows” who get to learn on the job how a release is done – and, hopefully, in a release or two, become leads themselves!

Unlike most open source projects where getting a spot on the release team is based on long tenure and impactful contributions: the Kubernetes team recruits “shadows” people via an application process. Even those with no prior Kubernetes contributions are invited to join and participate. To get notified of applications opening, subscribe to the relevant Kubernetes mailing lists. This is a very friendly policy – and another reminder that a project can come up with its own policies: no need to copy existing ones.

An interesting quote: why did Kubernetes “win”?

From the episode, starting at 29:26

Gergely: “Why did Kubernetes win? What did it do so well?”
Kat: “I think we caught on early because of hype. Because of the Google name brand behind us and the association with Docker, which was already very popular. Google was a familiar company, donating a project that relied on a tool many were already familiar with (Kubernetes / Borg). And that got us an initial hype cycle.
The continued popularity of Kubernetes is at least in part due to our truly exceptional documentation. If Kubernetes does something that you can touch as a user, as a cluster admin, we have documented it. Kubernetes uses something in the release cycle called a Kubernetes enhancement proposal, a KEP. This was inspired by Python's PEP (Python Enhancement Proposal).
One of the things we require for a KEP to be considered complete and thus, includable in a particular release is that it has a user-facing change at all – even if it's just a feature flag! – it must be documented, or we do not allow it in the release.
Today, as we're recording this, this is actually the docs freeze for Kubernetes version 1.33. So today, a lot of KEP owners will either merge documentation, or I will revert their PRs!