Guide
AI Code Quality
AI code fails by looking right, not by looking broken. Here is how to judge whether an AI-generated change is correct, durable, and safe to own - one PR at a time.
AI can now produce code that compiles, passes a happy-path test, and reads convincingly - none of which guarantees it is correct, durable, or safe to ship. The failure mode of AI-generated code is not gibberish; it is plausibility. It invents an API that looks real, handles the case you demonstrated but not the adjacent one, or silently drops an error path. A confident reviewer skimming a clean-looking diff is exactly the reader that plausible-but-wrong code defeats.
Quality is not a property of the model or the prompt - it is a property of the merged artifact and the system it enters. The right unit of assessment is the pull request and its downstream behavior, not the chat transcript that produced it. This guide gives you a concrete rubric for judging a single AI-authored change, and the small set of trend signals that tell you whether your AI-assisted output is getting better or quietly accumulating risk.
The goal is not to slow AI down. It is to keep the speed while refusing to trade away ownership. Code your team cannot explain, cannot test, and cannot safely change later is not an asset - it is a liability that arrived quickly.
1. Separate "it works" from "it is correct"
A passing run proves the code did the right thing once, on the input you tried. Correctness is about the whole input space - including the cases the AI never saw. Interrogate the change against the inputs the demo skipped.
- Boundaries: empty collections, zero, negative numbers, max-length strings, timezone edges, and the off-by-one at the end of a range.
- Nulls and absence: missing fields, optional dependencies, and "not found" vs. "found but empty" - AI frequently conflates these.
- Concurrency and ordering: does the logic assume single-threaded, in-order execution that your system does not guarantee?
- Error paths: what happens when the network call fails, the row is locked, or the parse throws? Swallowed exceptions are the classic AI tell.
If you cannot state the invariant the code is supposed to preserve, you are not reviewing it - you are approving it. See code review best practices for a structured pass.
2. Verify every API, import, and dependency is real
Models hallucinate plausible interfaces: a method that should exist, a config key that reads sensibly, a package name one character off from the real one. These pass a quick read because they are designed to look right.
- Confirm each symbol exists in the version you actually ship - click through to the definition, do not trust the name.
- Watch for typosquat and abandoned packages. A newly added dependency is a supply-chain decision, not a convenience. Justify it.
- Check the arguments and return shape against the real signature, not the one the AI assumed.
Run the PR quality estimator to flag unusually large or dependency-heavy diffs that deserve a slower read.
3. Read for durability, not just today
Durable code is code the next engineer - human or agent - can safely change in six months. AI optimizes for producing a working answer now, which often means duplicated logic, missing abstractions, magic values, and comments that restate the code instead of explaining the why.
- Fit: does it follow existing patterns and boundaries, or invent a parallel one that fractures the codebase?
- Coupling: did it reach across a module boundary it should not know about?
- Naming and intent: can you infer why this exists without the prompt that generated it?
- Dead scaffolding: unused parameters, speculative options, and "just in case" branches the AI added and no one asked for.
Tech debt from AI compounds quietly because volume is high. Track it with a tech debt calculator rather than trusting a gut feel.
4. Insist on tests that constrain behavior
AI-written tests have a specific weakness: they tend to assert what the code does rather than what it should do. A test generated alongside buggy code will happily lock in the bug. Read tests as adversarially as production code.
- Does the test express the requirement, or just mirror the implementation? Mirror tests pass even when the logic is wrong.
- Are the meaningful edge cases covered - the ones from step 1 - or only the happy path?
- Would the test fail if you broke the code? Mentally (or literally) mutate a line and check the test catches it. Over-mocked tests often catch nothing.
A change with high line coverage and no behavioral assertions is worse than no tests, because it manufactures false confidence.
5. Scope the security and blast radius
Match review depth to consequence. A copy tweak and a change to auth, money movement, data deletion, or a public API are not the same risk, even at the same line count.
- Untrusted input: is user or third-party data validated and escaped before it hits a query, shell, template, or filesystem path? Injection is the highest-frequency AI slip.
- Secrets and permissions: no hardcoded keys, no widened scopes, no auth checks quietly removed to make a test pass.
- Data exposure: new endpoints, logs, or error messages that leak more than intended.
- Reversibility: is this change easy to revert, or does it run an irreversible migration? Prefer changes you can undo.
Feed the diff through an AI code review pass to surface these classes automatically, then confirm the high-severity findings by hand.
6. Require a human who can own and explain it
The non-negotiable gate: a named engineer understands the change well enough to debug it at 3am without the model. If the answer to "why does this line exist?" is "the AI wrote it," the change is not ready, regardless of test status.
- No approval without comprehension. Speed from AI is real only if a person still owns the result.
- Keep diffs small enough to review honestly. A 2,000-line AI PR is not reviewed; it is waved through. Push back on scope.
- Preserve the trail: commit messages and PR descriptions should explain intent, not just narrate the diff. See conventional commits.
Ownership is the line between AI as leverage and AI as a source of code no one can maintain.
7. Measure the outcome, not the volume
Per-PR review keeps individual changes safe; trends tell you whether your AI-assisted process is healthy. Lines of code and PR count are vanity metrics - AI inflates both while telling you nothing about quality.
- Revert and hotfix rate on AI-heavy changes vs. the baseline - the clearest durability signal.
- Rework: how much recently merged code gets rewritten soon after. High churn on new code means it was not durable.
- Change failure rate and time to restore - the stability half of DORA metrics.
- Review depth: are large AI PRs getting real scrutiny, or rubber-stamped in seconds?
Comparing AI-authored and human-authored outcomes side by side is the honest way to know if the tooling is paying off - see AI vs. human code quality and how FeatureFactory measures shipped AI code.