I build a pretty popular LLM tool. I think learning when/how to use them is as big a mental hurdle as it was learning to google well or whether something is googlable or not.
In the realm of coding here are a few things its really good at:
- Translating code, generating cross language clients. I'll feed it a golang single file API backend and tell it to generate the typescript client for that. You can add hints like e.g "use fetch", "allow each request method to have a header override", "keep it typesafe, use zod", etc
- Basic validation testing. It's pretty good at generating scaffold tests that do basic validation (Opus is good at writing trickier tests) as your building.
- Small module completion. I write an interface of a class/struct with it's methods and some comments and tell it to fill in. A recent one I did looked something like (abbreviated):
type CacheDir struct { dir string, maxObjectLifetime: Duration, fileLocks sync.Map }
type (cd *CacheDir) Get(...)
type (cd *CacheDir Set(...)
type (cd *CacheDir) startCleanLoop()
Opus does a really good job generating the code and basic validation tests for this.
One general tip: you have to be comfortable spending 5 minutes crafting a detailed query assuming the task takes longer than that. Which can be weird at first if you take yourself seriously as a human.
Note that I hadn't been able to do much of this with GPT-4 Turbo with with Claude Opus it really feels capable.
Just to answer to the turbo aspect, I've seen a big downgrade in quality when comparing 4 to 4-turbo, and even the new preview which is explicitly supposed to follow my instructions better. So I'm running a first pass through 4 and then combinging it with 4-turbo to take advantage of the larger context window and then running 4 on it again to get a better quality output.
I'm sure you know what your talking about, but pushing the point that what is "best" or worth talking about is something that changes like every month does not really help defend against the case that most of this is just hype-churn or marketing.
I'm not pushing what to talk about so much as pushing the point not to talk about stuff that is obsolete and starting to smell.
It's that hype-churn marketing that is a motivating factor for the groups to innovate, much like Formula 1. It might be distasteful, but that doesn't mean it isn't working.
>- Small module completion. I write an interface of a class/struct with it's methods and some comments and tell it to fill in. A recent one I did looked something like (abbreviated):
Are they considerably better than existing non-AI tools + manual coding for this? In VSCode and Visual Studio, when working with an interface in C# for example, I can click two context menus to have it generate an implementation with constructors, getters, & setters included, leaving only the business logic code to write manually. You've mention you have to describe to the AI in comments, and then I assume spend time on a step to verify the AI has correctly interpreted your request & implemented.
I can definitely see the advantage for LLMs when writing unit tests on existing code, but short of very limited situations, I'm really finding it difficult to find the 55% efficiency improvements claimed by the likes of GitHub's AI Copilot.
That sounds crazy useful and I think speaks most to the maturity of C# and Microsoft's commitment to making it so ergonomic. I'm pretty curious about that feature, I'd love something similar for C++ in VS Code, but thus far I've been doing a pretty similar Copilot flow to the parent comment. It's nothing groundbreaking, but a nice little productivity boost. If I had to take that or a linter, I'd take the linter.
Visual Studio (not VSCode) has this for C++, though it can be a bit finicky. It’s infinitely better than AI autocomplete, which just makes shit up half the time.
In the realm of coding here are a few things its really good at:
- Translating code, generating cross language clients. I'll feed it a golang single file API backend and tell it to generate the typescript client for that. You can add hints like e.g "use fetch", "allow each request method to have a header override", "keep it typesafe, use zod", etc
- Basic validation testing. It's pretty good at generating scaffold tests that do basic validation (Opus is good at writing trickier tests) as your building.
- Small module completion. I write an interface of a class/struct with it's methods and some comments and tell it to fill in. A recent one I did looked something like (abbreviated):
type CacheDir struct { dir string, maxObjectLifetime: Duration, fileLocks sync.Map }
type (cd *CacheDir) Get(...)
type (cd *CacheDir Set(...)
type (cd *CacheDir) startCleanLoop()
Opus does a really good job generating the code and basic validation tests for this.
One general tip: you have to be comfortable spending 5 minutes crafting a detailed query assuming the task takes longer than that. Which can be weird at first if you take yourself seriously as a human.
Note that I hadn't been able to do much of this with GPT-4 Turbo with with Claude Opus it really feels capable.