AI coding tools sell one number: more code, faster. It demos beautifully and it sells even better. It is also close to meaningless on its own. A team can double its lines-of-code output and be worse off - more surface area to review, more places for bugs to hide, more churn to clean up next quarter.
The question every engineering leader is actually being asked right now is different: is the AI-assisted code any good, and is it staying good? Velocity without a quality axis isn't progress - it's faster technical debt.
Output is not the same as outcome
The "feature factory" critique has always been that teams optimize for shipping features rather than moving metrics. AI makes that failure mode cheaper and therefore more tempting. When generating a plausible-looking implementation costs almost nothing, the constraint moves from can we build it to should we have, and did it work. If your dashboard only shows how much shipped, you've automated the wrong half of the job.
Two axes, always
Every unit of work has at least two dimensions worth tracking together - never one without the other:
| Axis | What it answers | How to measure |
|---|---|---|
| Throughput | How much shipped? | PRs merged, cycle time, deploy frequency |
| Quality | Was it good? | Review depth, rework rate, bugs traced back to the change |
Looking at throughput alone is how teams end up "shipping more" while the on-call rotation quietly gets worse. Looking at quality alone hides the teams that are meticulous and stalled. You need both on the same screen, over time, so a spike in one that comes at the expense of the other is impossible to miss.
The AI-vs-human split
Here is the measurement almost nobody reports credibly. Tag each pull request by who really authored it - a human, a human assisted by an agent, or an agent working mostly on its own - and then compare both axes across those buckets. You do this with commit trailers, bot accounts, or explicit labels; the mechanism matters less than the discipline.
Once you have it, you can answer the questions leadership is actually asking:
- Are our agents pulling their weight, or generating rework we pay for later?
- Which parts of the codebase do they help with, and which do they quietly make worse?
- Is our AI spend showing up as durable, shipped value - or just activity?
Most engineering-intelligence tools can only correlate AI adoption with team-level trends; they can't tell you what the AI actually wrote. That line-level attribution is exactly what the Measure pillar is built for, and it's the difference between an opinion and a number.
Rework is the tell
If you only add one quality signal this quarter, make it rework - code rewritten or reverted shortly after it merged. AI-generated code is especially prone to it: confident, plausible, and occasionally wrong in ways that read as correct. A rising rework rate in the AI-authored bucket is the earliest, clearest signal that speed is being borrowed against quality. The Rework Rate Calculator is a two-minute way to see where you stand.
Start with one repo
You don't need a platform migration to begin. Connect a single repository, score its pull requests and commits, split them by author type, and watch the trend for a month. The baseline itself is usually the insight - and once you can see it, the conversation shifts from "how much are we shipping" to "is it working." That's the whole game.
Measure what you ship - for real.
Put your team’s AI web-development on the outcome loop.
Get early access