Code Freezes: Part 1

At the end of last year, code deployments were frozen at many tech companies. Why was this done, what are the benefits, and what are the tradeoffs of this approach?

Happy New Year and welcome to the first issue of 2023! I hope you had a restful time during the holiday period.

If you’re a software engineer at a tech company, there’s a good chance the final two weeks of December were more peaceful than usual, not only because of colleagues going on holiday, but because of deployment freezes. These are often referred to as “code freezes,” short for “code deployment freeze.” In this article, I’ll use the term “code freezes” to mean both “code deployment freeze” and “deployment freeze.”

Today, we cover:

1. Big Tech and code freeze approaches. How do Meta, Amazon, Microsoft, Uber, Apple, Google and other larger tech companies go about deploy freezes during the holidays?

2. Code freezes at other companies. Details from Spotify, Twitter, N26, Podia, Bold.org, T-mobile and several startups.

3. Code freeze upsides. Fewer outages, people disconnecting and other reasons these freezes are helpful.

4. Downsides of code freezes. A rush to get changes in, risky refactorings, merge issues, and other problems the freeze introduces.

5. Companies which don’t do code freezes. A major e-commerce platform and a large SaaS provider don’t do code freezes. What did the teams there observe, as a result of this?

This is Part 1 of a three-part series. In Part 2, we covered this topic in more depth, touching on:

• Software product categories and code freezes. Web-based products, desktop apps, on-prem software, and embedded products tend to have very different deployment cycles. What are products where code freezes matter, and what are ones where they do not?

• Mandating a code deployment freeze. Informal and more formal approaches to do so, and questions to ask before deciding on the strategy.

• Code freeze trends across industries. What are trends within banking, fintech, e-commerce, wellness apps, and other groups?

• Code freeze alternatives. Code chills, close slush, and business as usual.

• “Wave and bake” instead of code freezes: Ocado. The grocery retailer platform has no code freeze and uses a unique approach with its isolated, single-tenant environments.

• Do you need to implement a code freeze? Things that play into this decision, from the maturity of deployment tooling to wanting to create space for staff to recharge for a few weeks.

1. Big Tech and code freeze approaches

It’s common for most of Big Tech to have two week-long code freezes during a year:

1. Thanksgiving week. The week of Thanksgiving is when Black Friday takes place. For most e-commerce businesses, this is when the single biggest daily revenues of the year are generated. Online retail companies or businesses whose customers are in this sector want to ensure maximum reliability of systems during this period. Additionally, many tech workers take at least a few days off during this week, and several take the whole week. A code freeze helps both with reliability and allows for more tech workers to take a vacation.

2. Christmas to New Year. In the US and Europe, it’s common for the majority of people working in tech to take off one or two weeks. It’s understood there will be fewer colleagues available to fix any breakages, so most companies ensure no such breakages happen by putting a code freeze in place. A code freeze also helps those on holiday to properly unplug, confident that no potentially troublesome new changes are making their way to production.

Meta puts a code freeze in place from November, which looks roughly like this:

Code freeze periods at Meta during November and December. For two months, yellow and red periods alternate, indicating when shipping to production needs to be done more carefully than usual.

The reason for the stricter cadence at the end of the year is to avoid issues ahead of and during the holiday periods of Black Friday, Christmas and New Year.

This practice operates for two reasons:

1. Minimize revenue lost due to outages, especially for advertising products.

2. Enables people to take time off to rest during Thanksgiving and at the end of the year.

Both the “yellow” and the “red” periods are enforced company-wide. “Red” periods are company-wide code freezes, with exceptions for some teams and scenarios. Teams working on internal tools and tests can usually push code changes, and outages – naturally – can also be mitigated by them. Teams working closely with revenue generating functions like Ads tend to be a lot stricter with code freezes, while smaller teams more distant from revenue-generating tools tend to have more leeway in pushing changes.

Amazon has a calendar of “Restricted” and “Blocked” days for deployments, formerly called “gray” and “black” days. These days are usually around Prime Day, Black Friday, Christmas, Amazon’s annual “Re-Invent” event and some other major calendar dates.

Amazon’s Restricted (gray) and Blocked (black) days. The above is an illustration of potential Restricted and Black days in the month of November. Blocked days are often communicated at short notice, often due to major operational events. Teams may have freezes that run beyond these Restricted and Blocked days.

On Restricted days, all deployment pipelines have automatic deployment blockers in place. These blockers may be limited to specific regions, or only for AWS deployments.

Blocked days are usually announced at short notice during big, operational events such as an outage, or unplanned events when temporarily suspending deployment pipelines is the safest option. Deploying changes during Restricted days requires Director-level approval. Deploying during Blocked days needs VP-level approval.

On top of Restricted and Blocked days, teams define their own code deployment freezes. For most codebases, a freeze lasts for 1-2 weeks around the time of Black Friday, and 1-2 weeks around Christmas and the New Year.

Microsoft does not have a company-wide policy; organizations decide how they approach deployments. 

For example, within the team responsible for the search engine Bing, there are two deployment freezes:

• One week spanning Thanksgiving through to Cyber Monday

• Two weeks spanning Christmas to New Year

Bing operates heavily based on experiments. During this time, experiments that were already running can continue, but new ones cannot begin. 

Other organizations are more hands-off in their approach. For example, GitHub does not mandate code freezes centrally: it leaves these decisions up to teams. Most engineers are on vacation at the end of the year, and the pace is generally slower. A current software engineer at GitHub put it that GitHub has an effective code freeze, mainly thanks to barely anyone being around, without any of this having been mandated.

Uber historically put freezes in place during the week before Thanksgiving, and from mid-December until early January. This was because