The productivity impact of AI coding tools
The productivity impact of AI coding tools
How much of a productivity boost do GitHub Copilot and ChatGPT give software engineers? Results from a survey with over 170 respondents, and a look into what to expect from AI coding tools.
A month ago, I asked developers and others working with AI coding tools like GitHub Copilot and OpenAI’s ChatGPT to share their experiences, so far. More than 170 respondents have provided personal insights and in this issue, we dig into this expert feedback, and I offer my thoughts, too.
Today, we cover:
The survey. An overview of questions and the profiles of respondents.
Comparable productivity gains. The gains which GitHub Copilot and ChatGPT offer are enticing, but not totally unprecedented within tech. Experienced engineers with decades in the business share examples of previous comparable productivity improvements.
GitHub Copilot. Its most common use cases, where the biggest gains are to be found, and when this tool isn’t so useful.
ChatGPT. Most common use cases and when to proceed with caution.
Copilot vs ChatGPT. How the tools compare. When is one better than the other?
The present and future of AI coding tools. Common observations and interesting predictions from survey respondents.
Copilot and ChatGPT alternatives. There are a growing number of AI tools; a list of other popular, promising options.
“Are AI coding tools going to take my job?” A very common question and source of concern for some engineers.
1. The survey
In the survey, I asked these questions:
What AI coding tool have you been using recently?
How long have you been using the tool for?
How have you been using the tool?
How would you summarize your view of the tool in one sentence?
Does this tool make you more efficient?
What areas have you seen the most productivity gains?
What is a tool that has comparable productivity gains?
What are areas where this tool is not that helpful?
What types of work is this tool the most helpful for?
Any other comments?
In total, 175 people responded with answers which I’ve analyzed. Of this number, 134 say they’ve been using GitHub Copilot, 39 say they use ChatGPT, and 2 responses mention other tools – including someone who uses Raycast AI together with Copilot: