Fix AI-Generated Code Without Breaking Existing Tests

The suite was green at 6 pm. You asked your editor's AI to add a feature, accepted the diff, ran the tests, and now fourteen of them are red. Three are in files the AI never mentioned.

The tempting moves are all bad. Delete the failing tests. Loosen an assertion until it passes. Paste the stack trace back into the chat and hope the next patch lands. Each of these trades a real signal for a quiet screen, and you pay for it later when a client finds the bug in production.

This post is a narrow workflow for one moment: a working codebase, an AI-generated change, and a suite that just went red. It assumes you use VS Code or a VS Code-based editor like Cursor, and that you have git. It isn't a general debugging guide (we have a checklist for debugging AI-generated code in VS Code for that). It has one rule: the old tests are right until you can prove otherwise. Six steps follow, in order.

Treat the existing tests as the spec, not the obstacle

A test suite is a written record of what your code promised to do. Every assertion is a decision someone made, maybe you, six months ago, maybe after a bug report at midnight. When an AI change breaks one, assume the change is wrong, not the test.

That sounds obvious. It stops being obvious when you're tired and a deadline is close. AI assistants nudge you the wrong way too. Ask one to "make the tests pass" and some will happily edit the test instead of the code, because both satisfy the instruction. The instruction was the problem. The goal isn't green. The goal is green with the same promises intact.

Set three ground rules before you touch anything:

• No test is deleted, skipped or marked expected-fail to get the suite green.

• No assertion is loosened (exact value to "truthy", equality to "contains") without a written reason you could defend to a client.

• A test changes only when the requirement itself changed, and that decision is yours, not the AI's.

The third rule has an escape hatch on purpose. Sometimes the feature you asked for really does change behavior, and an old assertion encodes the old behavior. Fine. But you make that call deliberately, in a separate commit, with the test diff visible on its own. If you can't say which requirement changed, the test is still the spec and the code is still wrong.

Write those three rules in a note at the top of your terminal or in a commit message draft. You'll be tempted to break them within the hour.

Step 1: Establish a baseline on the last known-good commit before touching anything

Before you read the AI's diff, find out what "green" actually meant. Stash or commit the AI's work on a branch, then check out the last commit you trust and run the full suite there.

git switch -c ai-change-wip
git add -A && git commit -m "wip: AI change, suite red"
git switch main
npm test    # or pytest, go test ./..., whatever you use

This takes two minutes and answers a question people skip: was the suite green before the AI touched it? Often it wasn't. A flaky test, a missing environment variable, a database that was never seeded on this laptop. If three of your fourteen failures also fail on the baseline, they aren't the AI's fault, and chasing them inside the AI's diff will waste your evening.

Record the baseline numbers: total tests, passed, failed, and the names of anything already red. Paste them into a scratch file. Now you have a clean subtraction: failures on the AI branch minus failures on the baseline equals the regressions you own.

Two habits make this step cheap. First, commit before you ask an AI for a change, so the known-good point always exists. Second, run the suite once before you start, not just after. Students and early-career developers skip this most often, because a project that "worked yesterday" feels like a baseline. It isn't. Only a test run is a baseline.

If the baseline itself is red, fix or quarantine that first, in its own commit. Mixing old failures with new ones is how you end up blaming the wrong change.

Step 2: Inspect the AI's diff for changed units, defaults, signatures, shared helpers and edited tests

Now read the diff. Don't skim it. Read it with a short list of suspects, because AI-generated regressions cluster in the same few places.

Changed units. Seconds became milliseconds, rupees became paise, UTC became local time. The code still compiles and the types still match. Only the number is wrong by a factor of a thousand.

Changed defaults. A function parameter that defaulted to false now defaults to true. A timeout went from 30 to 5. A page size shrank. Defaults are invisible at call sites, so tests that never pass the argument break silently.

Changed signatures. An argument reordered, a return type widened from a value to a value-or-null, a sync function made async. Callers the AI didn't open still use the old shape.

Edited shared helpers. One tidy-up inside a utility used by forty files. The AI saw the helper and one caller. You have forty callers.

Edited tests. Run git diff --stat and look for test files in the list. If the AI touched tests during a task that wasn't about tests, read those hunks first. A changed expected value is either a legitimate requirement change or a quiet cover-up.

git diff main..ai-change-wip --stat
git diff main..ai-change-wip -- '*test*' '*spec*'

Write down every suspect with a file and line. Don't fix anything yet. A list of five suspects is a much better starting point than a vague feeling that "the AI broke something around auth."

Step 3: Reproduce the regression with the smallest failing test, using git bisect run if needed

Fourteen red tests is noise. Pick the one with the smallest scope, usually a unit test with the shortest stack trace, and make it your only target. Run it alone, repeatedly, until it fails the same way every time.

npx jest path/to/file.test.js -t "exact test name"
pytest tests/test_billing.py::test_token_expiry -x

If it passes alone but fails in the full suite, you've found something different and more interesting: shared state, test ordering, or a helper with a side effect. Note that and move on to a different failing test for now.

When the AI's work spans many commits (an agent session that committed ten times, say) and you can't tell which one broke the test, let git do the search. git bisect run runs a command at each step and narrows to the first bad commit by binary search.

git bisect start
git bisect bad ai-change-wip
git bisect good main
git bisect run npx jest path/to/file.test.js

With ten commits, bisect needs about four test runs. With a hundred, about seven. It works because your baseline from Step 1 gives it a trusted good endpoint. When it finishes, it prints the first bad commit. Run git bisect reset to return to your branch.

The skill here is making the failing check small and fast. A bisect over a ten-minute suite is miserable. A bisect over a single two-second test is a coffee break. If no single test isolates the failure, write a ten-line script that does and bisect on that.

Step 4: Minimize the change by reverting hunks one at a time

You now know which commit or branch breaks the test, but a single AI commit can still touch eight files. Shrink it. The goal is the smallest edit that, if reverted, turns the target test green.

Use interactive patch mode to revert one hunk at a time:

git checkout -p main -- src/billing/

git shows each hunk and asks whether to discard it. Take the suspects from your Step 2 list in order. Revert one, rerun the single failing test, and note the result. If it goes green, you've found the culprit hunk. Put it back with git checkout ai-change-wip -- <file> and study it. If it stays red, keep that revert only if the hunk was unrelated noise, and move to the next suspect.

This is slower than asking the AI "why is this failing", and it's much more reliable, because it replaces a guess with an experiment. Most regressions turn out to live in one to three lines: a flipped default, a unit conversion, a renamed argument. The other 200 lines of the diff are fine, and you keep them.

A concrete rule for the fix itself: change the code, not the test, and change as little as possible. If the AI multiplied a duration by 1000 and the contract says seconds, remove the multiplication. Don't add a compensating division three layers down. A compensating patch makes the tests pass and makes the next reader miserable.

When the fix is a single hunk, commit it alone with a message that names the cause: "restore seconds unit in token expiry". Your future self will thank you.

Step 5: Add a regression test, rerun the relevant tests, then the full suite

The old tests caught this bug, but only by luck of coverage. They didn't tell you the name of the broken promise. A failure called "expected 3600, received 3600000" is a symptom. Write a test that names the promise.

test("token expiry is expressed in seconds, not milliseconds", () => {
  const t = createToken({ ttl: 1 });
  expect(t.expiresAt - t.issuedAt).toBe(3600);
});

Pick the narrowest level that would have caught it: a unit test on the function you fixed, not an end-to-end flow. Make sure the new test fails on the AI's version of the code and passes on your fix. A regression test that never failed proves nothing. You can confirm by temporarily reverting your fix, running it, and watching it go red.

Then run in widening circles:

1. The single test you were targeting.

2. The whole file or module around it.

3. Every test that imports the file you changed.

4. The full suite.

Compare the full-suite result against your Step 1 baseline. The target isn't just "zero failures". It's "no failures that the baseline didn't already have, and the baseline's own failures untouched". If a new test appears red in the full run but passed in isolation, you may have introduced ordering problems with shared state. Fix that before you merge, not after.

One more check: run git diff main --stat and confirm no existing test file shows a deleted or weakened assertion. Only additions.

Step 6: Review before accepting, and where an agent that runs and self-fixes fits

Before you merge, read the final diff once more as a reviewer, not the author. Ask three questions. Does every changed line trace to something you asked for? Did any test file change, and if so, is each change a new test rather than an edited one? Can you explain, in one sentence, why the regression happened? If you can't answer the third, you'll see it again.

The workflow above is manual on purpose. It teaches you where AI changes go wrong. But the middle steps are mechanical, and that's where an agent that runs its own work earns its keep. OxCode, which works in VS Code and other VS Code-based editors, plans a change, writes it, runs what it writes in a sandbox, and fixes its own bugs before it calls a task done. Take the classic case: a task adds a token expiresAt field, and the first draft stores milliseconds where the existing test expects seconds. The suite goes red inside the run, the agent sees the mismatch, and corrects the unit before you ever see a diff. That's an illustrative example, not a guarantee for every change, which is why Steps 1 to 5 still matter when you review the result.

Check the evidence before you trust any agent's "done". Our guide on whether a coding agent actually runs tests lists what to look for: test output in the run log, not just a claim. OxCode also publishes its public run records for its 82% SWE-bench Verified and Terminal-Bench 2.1 results at oxcode.ai/benchmarks, so you can inspect how it behaves on real tasks.

Accept the change only when the suite matches the baseline, the new regression test exists, and you can explain the cause.

Conclusion

A red suite after an AI change isn't a crisis. It's the cheapest bug report you'll ever get, because your own tests wrote it for you. Spend it well: baseline first, read the diff with a suspect list, shrink the failure, revert hunks, and leave behind a test that names the broken promise.

Then make the loop shorter. Install OxCode from the VS Code Marketplace, hand it your next feature with the full test suite in the repo, and watch the run log. The free plan gives you 15 runs a day, enough to try it on one real regression. If the suite is green when it says done, keep it. If not, you have six steps to fall back on.