Organizing and Running Successful Hackathons
Hackathons are fun for engineers, beneficial for businesses, and a good way to shake things up. This article suggests approaches for running successful hackathons – and whether you should hold one.
👋 Hi, this is Gergely with a subscriber-only issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. To get articles like this in your inbox, every week, subscribe:
In September last year, I was asked to be a judge at a hackathon organized by my brother’s startup, Craft Docs; a 50-person Series B company. The theme of the hackathon was – unsurprisingly – experimenting with Artificial Intelligence (AI,) and was the company’s biggest-ever hackathon.
I’ve done plenty of hackathons; some memorable, some underwhelming. This event was one of the best I’ve seen: with around 80% of staff taking part, including designers, product managers, and customer support representatives within engineering teams. The winning team consisted of some customer support folks and engineers, who built tooling which improved the customer support team’s workflow by solving some persistent pain points.
How did this startup pull off such a well-run hackathon, and how can other companies do so? I turned to the organizers of successful hackathons to learn which approaches worked for them. Today, we cover:
Craft Docs’ AI hackathon. From idea to execution; how most of the company got involved, and the outcome.
Vercel’s annual hackathon. A few weeks after their annual Next.js conference, the company does a week-long hackathon; an ongoing source of product innovation.
“Explorer” hackathons at Cabify. Eight years ago, the startup did its first hackathon with about 30 people in the tech team. This year, almost 250 people participated, and Cabify keeps iterating on the format every year.
Other internal hackathons. How Booking.com, Sherpa, and Firefly Health, ran their recent ones.
Twilio’s enterprise hackathons. Following its 2016 IPO, Twilio started to use company hackathons as an efficient sales tool.
Voiceflow’s external hackathon. The AI agent-building platform allocated $10,000 to get feedback from users on newly launched features.
Do hackathons live up to expectations? Hackathons may cause confusion that outweighs the benefits, and some companies don’t miss them. A mental model to help decide if running a hackathon is worth it in your workplace.
1. AI hackathon at Craft Docs
Marin Dimitrov is director of engineering at Craft Docs (we were also colleagues at Uber,) and he organized this hackathon. Turns out, it was not as carefully planned as I’d assumed – and such an event isn’t a regular thing at the company. Marin says:
“The 2023 Craft hackathon materialized almost by chance and it was not something that was carefully planned ahead of time. I was aware that Craft organized a single hackathon in 2021 – which was before I joined the company. I was interested in how that event went, and if hackathons are something people are excited about. I raised this topic for a quick discussion with the Eng Leadership Team (ELT) at the end of August. It turned out everyone was quite positive about the last one. The discussion quickly turned from: “how we liked this last one,” to: “we should do a hackathon!”
On the spot, these themes were suggested:
- Dev tooling improvements
- Building Smart Link integrations
- Experimenting with AIThe hackathon immediately got buy-in from the executive team, and in the same week we agreed to provide all support needed for the hackathon. This meant re-aligning release plans so that teams had two full, uninterrupted days of hacking, and providing prizes totalling €10,000 for three prize-winning teams.
All this moved at light speed: from ideation on Monday, to commitment by the executive team on Wednesday. This speed was radically different (faster!) than my previous experience of hackathons at other companies, where weeks of “what-if and what-about” discussions happened before any action.”
Goals
Here’s the goals Marin and the organizing committee set for the hackathon – or, as they called it, the “Craftathon.”
Exciting and engaging: participants should have reason to be more motivated about these two days than “just another working day”
Build new relationships. The goal was for new relationships to form, preferably cross-functional ones, with people working together who hadn’t done so before.
Inclusive for non-engineering. It’s common to see hackathons where developers participate and everyone else else sits it out. For this hackathon, it was an explicit goal to include folks working in design, product, Marketing, customer support – and even the people team!
Create learning opportunities. The two days of “hacking” should help people try out and learn new technologies and approaches.
Preparation
The Craftathon was announced 4 weeks ahead of time. The goal was to let teams self-organize, but some lightweight guidance sped up the process:
1. A pitch template. A document of guidance on how individuals can pitch ideas and attract teammates to form a squad around an idea. The pitch had to outline important details like the problem statement, proposed solution, resources needed (e.g. datasets, third-party API access, budgets,) as well as the vision for the prototype and how it could be productised by Craft). Here’s the pitch template – created in Craft, of course!
2. Forming hackathon squads. As a constraint, every idea had to have at least 2 enthusiasts working on it, and nobody was to work alone. Without this constraint, the relationship-building goal would have been hard to meet. Thanks to this constraint, some ideas with only one supporter were merged into a bigger project.
3. The leadership team encouraged people to take part. Within each function, leaders created space and offered motivation for people to sign up for the event. It was voluntary, but if someone couldn’t commit due to work commitments, then leadership tried to find ways to free up their time.
4. Encourage non-engineers to pitch ideas. The problem with aiming for inclusive teams is that to get a realistic shot at winning one of the monetary prizes, the winning team had to have engineers to create a working prototype. But where would this leave teams with only one engineer, or none?
To make the event more inclusive, one of the three monetary prizes were reserved for the squad that came up with the best idea, design and presentation – even if the idea did not have a working prototype! This was deliberately done to encourage non-engineers to pitch ideas. And it worked! Recruiters and customer support got involved in surprisingly high numbers, pitched ideas, and attracted engineers to many of their ideas! In the end, every hackathon squad had an engineer and a working prototype!
Running the Craftathon
The event took 2.5 days: two days of building, and half a day of demos and evaluation.
The building days were protected as uninterrupted time for all participants. Release schedules, ETAs, and plans were adjusted in advance for all teams and functions. The company also provided breakfast, lunch and dinner for everyone in the office for the duration of the Craftathon, so that the ideation, discussions and excitement were not disrupted.
Everyone co-located for the hackathon: this was because on Day 3, a new office opening party was scheduled. The whole team was already in the Budapest offices, which created intense, efficient in-person brainstorming & collaboration.
Enabling teams to move faster took some additional preparation and extra work. A couple of engineers volunteered to do additional work in advance, so that they enabled the Craftathon squads with things like:
Dataset exports / imports
API access keys setup
Getting access to new services (e.g. to OpenAI APIs; the team got an early preview access to AWS Bedrock that some teams utilized.)
Demos and evaluation
The final demos and evaluation was the “goal” of the hackathon. Naturally, this required some organization:
An evaluation committee: this was 3 Craft leadership team members from product, marketing, and engineering, and an external guest (which was me – Gergely!)
A demo & Q&A slot: this was set as 12 minutes each. With 9 teams, this took 2 hours.
Evaluation criteria shared upfront, focusing on 5 areas
A 1-page evaluation summary: each team received this afterwards from the committee, providing detailed feedback on their idea, the completeness and quality of the prototype, as well as the demo & presentation quality. The feedback was open to view for the whole company and it included the video of the presentation (for those to view who might have missed it,) a highlights section, feedback on the idea, and on the presentation.
Outcome and next steps
Here’s how Marin summarizes his experience of the hackathon:
“The 2023 Craftathon was the best hackathon I've seen in my (long) career. This is true for:
Engagement: almost 80% of Craft employees participated in 9 hackathon squads
Inclusivity: every single function was represented: engineering, product, design, marketing, customer support, people & recruitment
The quality of the prototypes: these were solving real customer problems, and covered end-to-end workflows from data ingestion and backend processing, to integrations with the web or the mobile Craft apps.
“Mini production readiness”. Some squads even did a mini "production readiness" assessment, outlining what would be the operational costs like cloud infrastructure, third party API access, etc, if their prototype was integrated into Craft, and worked on actual production volumes for all our 1M+ customers.
It was amazing to see the excitement, buzz and energy in the office during the 2.5 days of the Craftathon. Everyone is asking when the next Craftathon will take place.”
My observations
As a guest and invited judge at this hackathon, what stood out to me was the cross-functional engagement. Team after team, marketers, recruiters, and product folks, demoed what they’d built together with engineers. Talking to the teams, many had worked together for the first time, and it felt like this event doubled as an intense team building exercise.
Even though the theme of the hackathon was AI, and it’s pretty easy to do creative things with these tools, I was still surprised just how much “practical creativity” was on display. There was a team building a tool to “score” a document and offer improvements, another team built a better search with the tool while discovering clever ways to use ChatGPT to give better search results than the “simple approaches,” and there was even a team that created “Craft Radio:” a podcast version of daily team updates, read by the AI-synthesized voice of team members!
Interestingly, having several real prizes seemed to create a bit of competitive spirit where teams pushed hard to make their ideas more production-ready. In most hackathons, there are teams that you can tell are there to have fun; but here, all teams had good ideas and executed on them surprisingly well in the two days. I was especially impressed by how integrating GenAI features didn’t seem to be a problem for most teams.
One thing that I was left wondering: was the hackathon so successful thanks to all the bottom-up experimentation which generative AI opens up? My suspicion is that this was a big driver. On the other hand, I can’t deny the hackathon was smartly organized, and took advantage of an inflow of energy with the whole company coming together, which usually only happens once a year. For more on this event, the Craft team published their own summary of this hackathon.
2. Vercel’s annual hackathon
Vercel is the leading frontend cloud platform, and creator of the popular Next.js frontend framework, with around 400-500 members of staff. VP of Engineering, Lindsey Simon, shared a few details about how Vercel runs its annual, internal hackathon.
Timing-wise, it takes place two to three weeks after the annual, flagship Next.js conference, usually in November or December, and lasts a week. Here’s the briefing participants are given, as shared by Lindsey:
“There are no restrictions on direction or goal. However, please do not work on something that is on your team’s current work stream - that’s what non-hackathon time is for (unless it’s something innovative that is not planned for the project, meaning it wouldn’t happen unless the hackathon made it happen.) The point is to be as innovative as possible, to create something interesting that you can demo and get feedback on.
Focus on something that can be "shippable" – not a proof-of-concept, and not a side/pet project.
Work on your own, or form a team. If you don’t have an idea, find someone who does. If you don’t have a skill you need, find someone who might. If you’re done with your project, find someone who’s not. This is a complete license to fail, so try something that feels big!”
Hackathons are sources of innovation at Vercel, Lindsey told me, due to a focus on building something shippable. Successful projects become new features which go to production. Here are a couple that were born out of a hackathon:
The "PowerPicker": An UI tweak for a “smarter” navigation bar. It’s a tweak that makes the “parent” menu item hold focus even after navigating to the “child” item.
In the image above, “Analytics” parent menu is visually selected, despite the mouse being above the “implementation-demo” child item. Before this change, “Analytics” would have been de-selected. It’s a seemingly simple UI tweak that is both useful and complicated to build.
Vercel Cron: Schedule tasks and automate workflows in seconds. Use cases include triggering updates to and from third-party APIs, monitoring service health, running critical jobs like data backups, and more. Launched in February 2023, after a fall 2022 hackathon.
CMS Editing: Old pages built with Vercel’s content management system (CMS) functionality could only be edited within the CMS itself. A hackathon project prototyped what it would be like to edit in-place, instead. This was how the Visual Editing feature was born at the 2022 hackathon, and launched in 2023.
Layout Shift tool within the Vercel Toolbar: A layout shift is when a visible element changes position between two render frames, which makes the webpage seem “jerky” and disorienting for users. This hackathon project built a tool to automatically detect shifts and report them to customers. This feature was built during an October 2023 hackathon, and shipped just a month later.
A Date Formatting Library: A platform called little-date was started at an October 2023 hackathon, and published in February 2024. From the project’s documentation:
“When displaying date-ranges in the UI, they are often too long and hard to read. This library tries to solve that problem.
Examples dates ✨
Jan 1 - 12
Jan 3 - Apr 20
January 2023
Q1 2023
Wasn't that easy to read? “
AI Help First on Vercel’s help page. Before submitting a ticket to Vercel’s customer support (CS) team, an AI service trained on the Vercel and Next.js documentation answers the question (see recent Klarna blog post.) The results so far show that 20% of requests are successfully answered by the AI. Additionally, if the answer is insufficient the AI automatically prefills the ticket submission form by analyzing the user’s described question/problem; saving time and better routing the customer to get help from the CS team.
“Friday Demo Days” are intended to keep “hacking” culture alive: Every Friday, anyone can get feedback on something they are building. The idea is to give people space to innovate outside of annual hackathons, and to infuse some of the hackathon spirit into daily work.