$ eval --tools
Where AI coding tools help, and where they cost you time
The argument about whether these tools work is less useful than knowing which side of the line a given task sits on. Here is how we decide, after using them daily on production systems.
Updated 2026-09-10
The short answer
AI coding tools reliably save time on well-specified, self-contained work: boilerplate, test scaffolding, format conversion, unfamiliar syntax and first drafts of code you will review anyway. They reliably cost time on large refactors across unfamiliar codebases, subtle concurrency and state bugs, and anything where being confidently wrong is expensive - because reviewing plausible-but-wrong code is slower than writing it.
Where the time actually comes back
- Boilerplate and scaffolding, where the shape is known and the details are tedious
- Tests, especially the unglamorous edge cases people skip when writing them by hand
- Working in a language or framework you know conceptually but not idiomatically
- One-off transforms - parsing a log format, converting between structures, throwaway scripts
- Explaining unfamiliar code, which is often faster than reading it cold
Where it quietly creates work
- Refactors across a codebase the model cannot hold in context - it produces locally plausible, globally wrong changes
- Concurrency, caching and state bugs, where the wrong answer looks exactly like the right one
- Anything depending on undocumented internal behaviour it has never seen
- Security-sensitive code, where confident wrongness is the entire risk
How to tell before you start
- Could you specify the task fully in a paragraph? If not, the model cannot either
- Would you catch a subtle error in review? If not, the risk outweighs the speed
- Is the blast radius contained? Generated code in a leaf module is very different from generated code in auth
- Are you using it to avoid understanding something you will need to understand later? That debt comes due
Common questions
Do these tools make junior engineers redundant?
They change what juniors should be learning, not whether you need them. The tools are strongest at exactly the work juniors used to cut their teeth on, which makes deliberate teaching more important, not less. Someone still has to be able to tell when the output is wrong.
Should generated code be reviewed differently?
Yes. Human error and model error have different shapes. People make careless mistakes in obvious places; models make confident mistakes in plausible places. Review generated code assuming it looks right and may not be.