That said, performance and security considerations can be automated by a reusable harness and I think within six months it will just be built in to things like claude code and codex the same way you might tack on a performance/security section to your project's AGENTS.md.
A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorization, validation, etc. just because it's trivial to forget one of them. Yet now the LLM can come up with a lintable/verifiable way to ensure every route has the whole kit by construction.
Almost every way I feel like I can provide unique value to the automated AI workflow (aside from deciding what to build at all, and taste) feels like it can be automated by better markdown.
The checklist half automates cleanly, you can lint that every route calls authorize. It won't catch authorize being handed the wrong policy, which is the one that ships. And when I put a second model on review duty, the common failure is it agreeing with the first model's misreading of the spec, almost word for word. So I'd expect the forgetting class of bugs to mostly go away and the misread-the-spec class to sit exactly where it is.
LLMs trivially catch these things in review, and if you find agents agreeing incorrectly with each other (weak models?), maybe you need to be doing adversarial review.
It's pretty much solved, though I only use sota models.
I think for these convos, we need to see concrete fail cases so we can see what you're talking about and whether the truth matches up with the claim.
A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorization, validation, etc. just because it's trivial to forget one of them. Yet now the LLM can come up with a lintable/verifiable way to ensure every route has the whole kit by construction.
Almost every way I feel like I can provide unique value to the automated AI workflow (aside from deciding what to build at all, and taste) feels like it can be automated by better markdown.