Cloud Development Environments
Software engineers at large tech companies are developing less locally, and more with remote dev environments. But what is the idea behind CDEs and how are companies building their own? A deep dive.
A quiet revolution is unfolding across Big Tech and at larger tech companies, about which I hear little external chatter: Cloud developer environments.
If you’re a software engineer who moves from a startup to a larger tech company like Uber, Slack, or Shopify, then something interesting happens. The codebase you work with gets much larger: there’s more lines of code, more static analysis rules, and more unit, integration and other automated tests.
And yet, when you start developing and make that first change, the build and the running of tests is fast. Indeed, your laptop’s CPU utilization doesn’t spike like usually happens when building a project; no fan whirs at high speed because your laptop is heating up under the load.
How can this be?
The reason is that many big companies use cloud-based developer environments, so the build doesn’t happen on your own machine, but on a much more powerful and remote one.
In today’s issue, we dive into the world of Cloud development environments (CDEs,) and how they’re used by a variety of businesses. We cover:
The idea behind CDEs. Large codebases come with slow build times: and CDEs can speed this up.
Upsides of CDEs. Faster builds; consistency and reproducibility; more collaboration; simpler security auditing and faster onboarding.
Pipedrive. The company had about 250 engineers and first moved to a local Docker Desktop-based environment. Then built their custom Devboxes which run on Kubernetes.
Uber. The ridesharing giant speeds up builds and startup times with a custom CDE called Devpods. Devpods run on machines with up to 48 cores and 96GB of RAM (!) and come with six different development environment “flavors.”
Slack. The company chose a simple but remarkably effective remote development environment approach based on AWS EC2 instances. 8 cores & 16GB RAM for backend; 16 cores and 32GB of RAM for frontend development.
The downsides of CDEs. Involved initial setup, a high cost to run and maintain, and the risk of vendor lcok-in when choosing a CDE vendor.
See also the follow-up articles:
Two months after publishing this article, Gartner added Cloud Development Environments to their 2023 Gartner Hype Cycle. Subscribe to The Pragmatic Engineer to keep a pulse on what is happening in the software engineering world — sometimes hearing about trends months before they are discussed by more mainstream publications.
1. The idea behind CDEs
A normal development workflow is something like this:
Set up the local development environment
Check out the codebase
Make changes
Build the code
Run tests locally, or deploy the code locally
Debug
All these steps happen on a local developer machine, traditionally. But the idea behind cloud development is to move the CPU and memory-intensive parts of the development workflow, like building the code, running tests, and deploying, to a machine in the cloud which is more powerful and faster.
Large codebases come with slow build and test times, which is why CDEs make sense as an alternative. There are plenty of examples of companies which opted to introduce cloud development environments, and one feature they all share is their ever-growing codebases. Building the code from scratch can take more than 10 minutes on a powerful developer machine, while running automated tests for the codebase may take even longer.
It makes sense to consider remote environments for large codebases: CDEs introduce a lot of additional complexity as they move build steps and other automation steps to a Cloud machine. Also, CDEs are more expensive to run and operate because the additional Cloud infrastructure usage costs money.
2. Upsides of CDEs
CDEs come with several benefits which offset costs in certain environments. These are: