Are reports of StackOverflow’s fall greatly exaggerated?
A blog post suggests traffic is down 50% at Stack Overflow, due to ChatGPT gaining popularity. I checked in with the company for more details. Plus: a senior engineer's months-long job search story.
👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover two out of five topics from today’s subscriber-only The Pulse issue. To get full issues twice a week, subscribe:
Before we start: plenty of people who subscribe to the newsletter utilize a learning and development budget their company provides. In what is neat: The Pragmatic Engineer is now the first-ever newsletter listed on Learnerbly. Learnerbly is an L&D platform hundreds of tech companies use, as it makes administering these budgets much simpler — and it’s also a lot easier for people to request books, courses, or newsletters. If your company already has Learnerbly, you can now request the full newsletter with just one click.
Today, we cover two topics:
Are reports of StackOverflow’s fall greatly exaggerated? A blog post suggests traffic is down 50% at Stack Overflow, due to ChatGPT gaining popularity. I reached out to Stack Overflow for more details: the company admitted a drop, but it’s only 14% as per data shared with me. Exclusive.
A senior engineer/EM job search story. Davidson Fellipe, a software engineer with 15 years’ experience, based in New York, was recently let go. After 350 applications and 85 first-round interviews in 4 months, he secured 3 offers, and has now started his new job. He shares first-hand learnings about navigating the jobs market. Exclusive.
Are reports of StackOverflow’s fall greatly exaggerated?
A week and a half ago, machine learning engineer Ayhan Fuat Çelik wrote an interesting analysis titled The fall of Stack Overflow. Stack Overflow shares website data with some of its most active members with reputations higher than 25,000. I’d like to call out how neat this approach is and how it contributes to transparency, even when the data isn’t flattering.
Ayhan visualized this data and observed a definite fall in all metrics: page views, visits, questions asked, votes. Visualized:
The post stated Stack Overflow has lost about 50% of its traffic. However, the traffic data turned out to not account for a Google Analytics change. Allowing for this, the drop would be 35%. Still, the most worrying part of the statistics is not traffic, but the drop in questions asked and upvotes.
I pinged engineers at Stack Overflow to get their thoughts about what’s happening. What they said is that they are not seeing so dramatic a drop, internally, and that data shared with the most active contributors is inaccurate. I also reached out via official channels to Stack Overflow, and here’s what the company told me (the company later published a blog post with some of the below data included):
5%: the company wrote “overall, we're seeing an average of ~5% less traffic compared to 2022.“
14%: the sharp decrease in traffic in April 2023. The company said: “we can likely attribute this to developers trying GPT-4 after it was released in March.”
14%: this is by how much search engine traffic is down, year-on-year.
A predictable rise and fall, as with any sudden change. When global lockdowns started in 2020, Stack Overflow saw a spike and then a decrease in cloud migration questions and security-related ones. I sense the company is not surprised that AI had an impact on traffic and the types of questions.
Q&A activity is definitely down: the company is aware of this metric taking a dive, and said they’re actively working to address it.
While the pandemic and its associated lockdowns didn’t pose a threat to Stack Overflow’s business model, coding assistants like Copilot, Cody, ChatGPT and the other alternatives are different. I find myself reaching less for Google and StackOverflow as a first step when building a new project, and instead using an assistant to get started. More complicated problems – and when I can’t get unstuck with the GenAI helper – are reasons why I head for the browser and open up new tabs, which tend to lead to StackOverflow.
Still, GenAI tools are only as good as their training data. And StackOverflow has the world’s richest programming Q&A training data. So the obvious question is will StackOverflow be content with being scraped for this data – as they have been, until now – or will the company turn it into AI products?
Two weeks ago, CEO Prashanth Chandrasekar announced OverflowAI, a set of AI-related products StackOverflow is working on. The company also announced an upcoming improvement: better search, and also 4 AI tools coming to Stack Overflow for Teams.
Stack Overflow for Teams could be the product which benefits most from the GenAI wave. Until now, our attention has been on the “original” StackOverflow public site we all know and use – especially when fixing an annoying bug. However, the increasingly important revenue-maker for StackOverflow is StackOverflow for Teams, a “private” StackOverflow instance.
When it comes to GenAI, Stack Overflow for Teams is getting a lot more love. Here are features the company is building:
A new Slack integration to give answers to technical questions, using the Stack Overflow for Teams knowledge base
Stack Overflow for Visual Studio Code: a pair programmer to bring Stack Overflow content right into the IDE
Enterprise knowledge ingestion: ingesting content from Confluence, Google Drive, Github, and ServiceNow
Improved search: similar to the makeover of the public StackOverflow product
Stack Overflow will struggle to compete with AI tools scraping tools which ingest all public StackOverflow questions and answers. However, it can compete by offering tools for companies to utilize their own internal knowledge bases, and by ensuring that internal company data doesn’t leave the premises.
Over time, I won’t be surprised if Stack Overflow’s main business becomes Stack Overflow for Teams, and the public Stack Overflow serves as a lead magnet for Stack Overflow for Teams, while generating some – although increasingly less important – income via ads.
Could we see the fall of public Q&A sites as AI tools rise? A striking statistic is just how much the volume of questions asked has dropped. It’s not as if people have fewer questions, it’s just that developers are typing these questions into AI tools, instead.
For public Q&A sites like StackOverflow to thrive they’ll surely need to be where these questions are asked, which will be integrations in Slack that the company is working on, and programming AI agents, which is another thing Stack Overflow is building.
But the use case that will surely thrive is private, company-only knowledge bases. I would be surprised if Stack Overflow can do much about developers turning to AI coding tools first, before visiting it, but integrating private Stack Overflow for Teams into these chatbots seems like a smart strategy for being where customers ask questions.
The one thing left to answer is how will StackOverflow incentivize people to ask more answers? Looking at this broadly, when AI tools can answer most questions, who will write the answers for complex questions which AI tools don’t have answers for? And what will be their incentive to do so?
A senior engineer/EM job search story
Davidson Fellipe is a lead software engineer based in New York, with more than 15 years of experience. In April this year he was unfortunately impacted by job cuts at the company where he worked, and found himself on the job market. A few months later, he signed an offer to start a new company, and shared brief details of his job search on LinkedIn:
I was interested in learning more about how Davidson went about his job search, and his experience of being in the jobs market, so I reached out and he kindly shared more context and advice for navigating the current market.
Davidson shared that he eventually received one engineering manager offer and two individual contributor offers. He accepted the senior engineer offer.
How long was your job search?
‘My search lasted for about three months, until I received the first offer.
‘Starting in April 2023, I applied to a few positions, mostly targeting engineering manager roles. During that time, I dedicated a significant effort to applying using referrals whenever possible. The period from April to mid-May was challenging: I found myself in hiring freezes and canceled processes. For some applications, it took 2-3 weeks before I had a call with the hiring manager. At this time, more than half of my applications were rejected during the resume review, even with referrals.
‘If you attempt to apply everywhere, you can get a lot more frustrated. Instead of applying broadly, I focused on engineering manager positions for frontend, product engineering, and developer experience (front-end.) Over time, I gradually opened myself up to individual contributor roles, but only for frontend positions (senior, lead, staff, and founding engineer roles.)’
Which tools did you use to stay organized and to apply for so many positions?
‘To keep track of all my applications, I created a spreadsheet with columns like:
Company name
Position applied for
Application date
Status
Last stage
‘Other tools that helped:
Teal application tracker and Simplify for applications. Simplify helped me a lot to automatically fill applications in the most popular applicant tracking systems (ATS.) Another useful option was for me to have an "Identity" ready in 1Password, with the most common information requested by the ATS.
Notion: for every company I talked to, I took notes on Notion to record what we talked about.
Crafting tailored resumes: I dedicated time to crafting two, different one-page resumes, highlighting accomplishments, and showing my past experiences. I did one as a manager, and the other as an engineer: with the goal to improve my chances of converting interviews. For instance, my experience as a founding engineer at a startup helped secure interviews with some startups.’
How helpful did you find referrals?
‘People in my network offered referrals, and I also asked people I knew when I saw positions close to my profile at their company.
‘I also had cases when I saw a position I was interested in, but didn't know anyone at the company. In some cases, I added someone from the company via LinkedIn, and asked for a referral. I got a few interviews with this strategy!
‘I helped others impacted by layoffs, and the other way around. I shared opportunities or redirected recruiters to others I knew were let go, and people did the same for me. We were all experiencing the same challenges, after all!
‘I worked with external recruiters as well. These recruiters – which work with multiple companies – tend to have a number of opportunities that may match your skills and interests. Working with external recruiters helped in getting more interviews, especially with startups.’
How did you find the interview processes?
‘There are far more candidates for fewer positions, so the bar to go to the final round is higher.
‘Talking about startups, they are very creative with interviews! I’ve completed both technical quizzes and even cognitive aptitude tests. These types of interviews were usually an extra round before the coding interview or take-home assessment. Having at least 4 interviews or screening steps was pretty common.’
How did you manage your time and energy?
‘Job searching took a toll on my time and energy. I took a few weeks of vacation in my home country, then gradually started to increase the pace of interviews. My daily high-level plan was this:
Morning: walk, study, review answers and tailor my resume.
Afternoon: interviews, review job descriptions and take-home assessments that were timeboxed.
Evening: take-home assessments that were not timeboxed; fire off applications, and in the end: recharge!’
What are interesting observations about the hiring process, and what advice would you share with job seekers?
‘What was surprising was that most hiring processes took up to 3 weeks from the conversation with the recruiter to the conversation with the hiring manager!
‘One thing that helped: New York City’s Salary Transparency Law was very helpful for not applying for jobs whose salary was below my salary expectations.
‘Advice for others: Don't underestimate the power of your LinkedIn intro and LinkedIn headline! And don't be afraid to apply if you see “500 applicants” for a role.”
This is Gergely again. Thanks a lot Davidson, for sharing the reality of your job search, and congrats on securing offers and accepting one of them. You can follow Davidson on LinkedIn or Twitter. He announced this week which company he will join.
It’s nice to hear about the positive impact of salary transparency regulations, and a good reminder that companies which publish compensation bands in their job adverts will likely attract more qualified candidates, as Davidson passed on places whose compensation was not a fit, which saved both him and the company time.
These were two out of the five topics covered in this week’s The Pulse. The full edition additionally covers:
What kind of migration is causing a payout outage at Booking.com? Small business hosts on the travel booking platform are waiting more than a month to be paid. Booking.com says a systems migration is the reason for the delay. I talked with engineers at the company and discovered an SAP migration is to blame. Exclusive.
Amazon gets stricter about enforcing return to the office (RTO.) The online retail giant sent a warning email to employees “not meeting the expectation of joining colleagues in the office at least three days a week.” I talked with engineers about the response to this ominous, unfriendly email. Exclusive.
Zoom to end remote work: an RTO turning point? Remote work tool Zoom is having most staff return to the office for two days a week. It’s a symbolic turning point which may signal how many companies will operate in a similarly hybrid way, going forward. Analysis.