Simon is one of the best-known software engineers experimenting with LLMs to boost his own productivity: he’s been doing this for more than three years, blogging about it in the open.
We know that LLMs don't really understand words, they just do stochastic calculations on tokens which are representations of words and phrases, so essentially it's a math calculator - it knows the rules of the English language without actually knowing what language even is. So I assume it "learns" the rules of a programming language the same way. I can imagine how it learns the rules and patterns of non-mathematical constructs like class definitions, object inheritance, type hinting, basic data structures, method signatures - but since it is so terrible at classical math and logic, how on earth can it "understand" the workings of loops, mutable variables, complex data structures, algorithms, date-time calculations, and any real math that comprises typical application programming? Does it choke on incrementing values or a loop with increment because of its inability to count?
I have heard LLMs adapted pretty well for maths use cases (several teams are working in this area) for formal reasoning. But also, your point on LLMs being a poor choice for maths calculations is true, here is a hilarious example of wasting tons of compute to try a single enough floating point calculation:
> It takes a ton of effort to learn how to use these tools efficiently. As Simon puts it: “You have to put in so much effort to learn, to explore and experiment and learn how to use it. And there's no guidance.”
I'm slightly surprised that there don't seem to be more training courses popping up that promise to teach people these skills.
That seems like it should be a reliable way to make money from time invested in learning how to use LLMs as coding assistants.
The odd thing about software engineers embracing LLMs for writing code is that, sure, anything that can save you time and effort is cool and gets my vote, BUT, you can't ignore the fact that these machines are designed to replace you. We can call it an "augment", "assistant" or "weird intern" all we like, but it's inventors have a goal of making us the assitant and it the master. Are we training our own replacement?
Jim: the stated goal of the language COBOL was to be able to replace expensive programmers with business people who could all write programs to do what they want. That was around 1960. Turns out that introducing COBOL meant it created... COBOL engineers.
Looking back through the history of computing, there have always been (and always will be) pushes to "replace" expensive devs so people can create programs without them. The latest such push a few years ago was "no code" approaches (press buttons to build apps). But UI-defined programs, things like Microsoft Access, to some extent Excel etc have all done this.
In an interesting change, all these things did not reduce demand for developers (people who can write machine interpretable code to translate business expectations). Though you can argue tools have replaced entire disciplines like webmasters in the 90s.
It's hard to predict how things evolve: and there are many lofty promises made by vendors. This episode was about what works, today, and how.
I Hope you're right Gergely, but I personally don't think generativeAI can be drawn parallel to any previous technological innovation, it changes the game too much, so I do think there is a real risk that it could extract far more than it creates in terms of jobs. In the short term it seems likely that some companies will reduce developer headcount and some senior devs will remain to assist the machine and act as QA ~ not the same job but at least they'll still have one I suppose.
That's already what we are doing, cajoling the machine so that it outputs what we want. :-)
But for now the code is the interface. If tomorrow GenAI will be the interface then it won't change much, instead of writing the code we'll configure and fine tune the AI.
What's funny is that for the LLMs to work you need to feed them code. They can't generate the code without having as input a HUGE mass of code. Basically, you need to know how to code, and feed it to them, so that they can code for you. And even if the models are already trained, they are static, and still need to be constantly updated with new code, which is very expensive to do. As said in the podcast, they are good with very few widespread languages, but as soon as you start doing something that is not on their list, they are lost and need to be retrained with original content.
Also, if a cost reduction happens then it will allow for more production. Some small companies, and startups, will be able to create their products without spending millions which will actually increase the demand for developers.
Many different perspectives, for sure, it will be interesting to see what comes next. As of today, I tried many tools, and most of them are useful for only 10% of my tasks, and as soon as they hallucinate I actually waste more time than writing the code by myself.
Yes, AI code generation really is very limited, despite the hype. I noticed openAI has a new feature where you can port code from one language to another - but in many cases that makes no sense. Eg you can't port a SPA built in React to Python without losing all of the front-end interactivity because of course Python is a server language. In terms of how these tools will affect the future of software development my 2 main concerns are: 1. It will be an excuse to replace lots of developers even though it is not actually up to the task (there will be massive fallout down the road) 2. The job of software development will become a low-paid, meaningless task of just babysitting the AI - tidying up its mistakes. Compare this to what is happening to writers, who are being turned into mere AI editors: https://www.bbc.com/future/article/20240612-the-people-making-ai-sound-more-human
Solid first episode! I strong agree with the advice to experiment with different language models to build some intuition for the general strengths and weakness, notably hallucinations, of LLMs.
To this end I'd recommend playing with https://lmarena.ai/ as means of experimenting w/ different LLMs. Each query is answered by two different, randomly chosen LLMs and we the user rate which answer is better. The models are initially anonymous and their identity is only revealed after voting. It is fascinating to regularly see small, largely unknown models match performance of frontier LLMs, although we can also see them fail spectacularly on other queries.
I noticed that you didn't mention Cursor! It's the best IDE for AI codegen, we've reccommended it to all our readers.
I just subbed to your Substack, good stuff. I went ahead and recommended you to our readers, who I teach about AI code generation. I hope it helps more people find your stuff.
We know that LLMs don't really understand words, they just do stochastic calculations on tokens which are representations of words and phrases, so essentially it's a math calculator - it knows the rules of the English language without actually knowing what language even is. So I assume it "learns" the rules of a programming language the same way. I can imagine how it learns the rules and patterns of non-mathematical constructs like class definitions, object inheritance, type hinting, basic data structures, method signatures - but since it is so terrible at classical math and logic, how on earth can it "understand" the workings of loops, mutable variables, complex data structures, algorithms, date-time calculations, and any real math that comprises typical application programming? Does it choke on incrementing values or a loop with increment because of its inability to count?
You probably saw this, but a recap of how LLMs work, from the ChatGPT team: https://blog.pragmaticengineer.com/how-does-chatgpt-work
I have heard LLMs adapted pretty well for maths use cases (several teams are working in this area) for formal reasoning. But also, your point on LLMs being a poor choice for maths calculations is true, here is a hilarious example of wasting tons of compute to try a single enough floating point calculation:
https://www.linkedin.com/posts/luiscaires_dont-expect-gen-ai-on-its-own-to-be-a-technology-activity-7243540643971534848-98CF/?utm_source=share&utm_medium=member_desktop
Clearly, the weak areas of LLMs need to be complemented with more sensible approaches!
One of the most useful ways we found to use it in our company is to built an internal chatbot that knows all our documentation:
https://medium.com/taranis-ag/building-an-internal-chatbot-that-knows-all-your-docs-a-step-by-step-guide-d910dfb26041
Super useful, and saves tons of time!
This is awesome. Too many AI tools are advertised as better than they are for clickbait. I appreciate the honest recommendations here
This was exactly what I needed to hear today.
https://newsletter.pragmaticengineer.com/p/ai-tools-for-software-engineers-simon-willison?utm_source=publication-search
It a such great article
> It takes a ton of effort to learn how to use these tools efficiently. As Simon puts it: “You have to put in so much effort to learn, to explore and experiment and learn how to use it. And there's no guidance.”
I'm slightly surprised that there don't seem to be more training courses popping up that promise to teach people these skills.
That seems like it should be a reliable way to make money from time invested in learning how to use LLMs as coding assistants.
The odd thing about software engineers embracing LLMs for writing code is that, sure, anything that can save you time and effort is cool and gets my vote, BUT, you can't ignore the fact that these machines are designed to replace you. We can call it an "augment", "assistant" or "weird intern" all we like, but it's inventors have a goal of making us the assitant and it the master. Are we training our own replacement?
Jim: the stated goal of the language COBOL was to be able to replace expensive programmers with business people who could all write programs to do what they want. That was around 1960. Turns out that introducing COBOL meant it created... COBOL engineers.
Looking back through the history of computing, there have always been (and always will be) pushes to "replace" expensive devs so people can create programs without them. The latest such push a few years ago was "no code" approaches (press buttons to build apps). But UI-defined programs, things like Microsoft Access, to some extent Excel etc have all done this.
In an interesting change, all these things did not reduce demand for developers (people who can write machine interpretable code to translate business expectations). Though you can argue tools have replaced entire disciplines like webmasters in the 90s.
It's hard to predict how things evolve: and there are many lofty promises made by vendors. This episode was about what works, today, and how.
I Hope you're right Gergely, but I personally don't think generativeAI can be drawn parallel to any previous technological innovation, it changes the game too much, so I do think there is a real risk that it could extract far more than it creates in terms of jobs. In the short term it seems likely that some companies will reduce developer headcount and some senior devs will remain to assist the machine and act as QA ~ not the same job but at least they'll still have one I suppose.
That's already what we are doing, cajoling the machine so that it outputs what we want. :-)
But for now the code is the interface. If tomorrow GenAI will be the interface then it won't change much, instead of writing the code we'll configure and fine tune the AI.
What's funny is that for the LLMs to work you need to feed them code. They can't generate the code without having as input a HUGE mass of code. Basically, you need to know how to code, and feed it to them, so that they can code for you. And even if the models are already trained, they are static, and still need to be constantly updated with new code, which is very expensive to do. As said in the podcast, they are good with very few widespread languages, but as soon as you start doing something that is not on their list, they are lost and need to be retrained with original content.
Also, if a cost reduction happens then it will allow for more production. Some small companies, and startups, will be able to create their products without spending millions which will actually increase the demand for developers.
Many different perspectives, for sure, it will be interesting to see what comes next. As of today, I tried many tools, and most of them are useful for only 10% of my tasks, and as soon as they hallucinate I actually waste more time than writing the code by myself.
Yes, AI code generation really is very limited, despite the hype. I noticed openAI has a new feature where you can port code from one language to another - but in many cases that makes no sense. Eg you can't port a SPA built in React to Python without losing all of the front-end interactivity because of course Python is a server language. In terms of how these tools will affect the future of software development my 2 main concerns are: 1. It will be an excuse to replace lots of developers even though it is not actually up to the task (there will be massive fallout down the road) 2. The job of software development will become a low-paid, meaningless task of just babysitting the AI - tidying up its mistakes. Compare this to what is happening to writers, who are being turned into mere AI editors: https://www.bbc.com/future/article/20240612-the-people-making-ai-sound-more-human
Solid first episode! I strong agree with the advice to experiment with different language models to build some intuition for the general strengths and weakness, notably hallucinations, of LLMs.
To this end I'd recommend playing with https://lmarena.ai/ as means of experimenting w/ different LLMs. Each query is answered by two different, randomly chosen LLMs and we the user rate which answer is better. The models are initially anonymous and their identity is only revealed after voting. It is fascinating to regularly see small, largely unknown models match performance of frontier LLMs, although we can also see them fail spectacularly on other queries.
Hey there,
I noticed that you didn't mention Cursor! It's the best IDE for AI codegen, we've reccommended it to all our readers.
I just subbed to your Substack, good stuff. I went ahead and recommended you to our readers, who I teach about AI code generation. I hope it helps more people find your stuff.
Have a good weekend!