The Pragmatic Engineer

Share this post

The productivity impact of AI coding tools

newsletter.pragmaticengineer.com

The productivity impact of AI coding tools

How much of a productivity boost do GitHub Copilot and ChatGPT give software engineers? Results from a survey with over 170 respondents, and a look into what to expect from AI coding tools.

Gergely Orosz
Apr 18, 2023
∙ Paid
131
10
Share
Share this post

The productivity impact of AI coding tools

newsletter.pragmaticengineer.com

A month ago, I asked developers and others working with AI coding tools like GitHub Copilot and OpenAI’s ChatGPT to share their experiences, so far. More than 170 respondents have provided personal insights and in this issue, we dig into this expert feedback, and I offer my thoughts, too.

Today, we cover:

  1. The survey. An overview of questions and the profiles of respondents.

  2. Comparable productivity gains. The gains which GitHub Copilot and ChatGPT offer are enticing, but not totally unprecedented within tech. Experienced engineers with decades in the business share examples of previous comparable productivity improvements.

  3. GitHub Copilot. Its most common use cases, where the biggest gains are to be found, and when this tool isn’t so useful.

  4. ChatGPT. Most common use cases and when to proceed with caution.

  5. Copilot vs ChatGPT. How the tools compare. When is one better than the other?

  6. The present and future of AI coding tools. Common observations and interesting predictions from survey respondents.  

  7. Copilot and ChatGPT alternatives. There are a growing number of AI tools; a list of other popular, promising options.

  8. “Are AI coding tools going to take my job?” A very common question and source of concern for some engineers.

1. The survey

In the survey, I asked these questions:

  • What AI coding tool have you been using recently?

  • How long have you been using the tool for?

  • How have you been using the tool?

  • How would you summarize your view of the tool in one sentence?

  • Does this tool make you more efficient?

  • What areas have you seen the most productivity gains?

  • What is a tool that has comparable productivity gains?

  • What are areas where this tool is not that helpful?

  • What types of work is this tool the most helpful for?

  • Any other comments?

In total, 175 people responded with answers which I’ve analyzed. Of this number, 134 say they’ve been using GitHub Copilot, 39 say they use ChatGPT, and 2 responses mention other tools – including someone who uses Raycast AI together with Copilot:

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2023 Gergely Orosz
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing