Preparing a Production Bug for an AI Coding Agent

An AI coding agent is only as good as the bug report you hand it. Feed it a stack trace and a vague description, and it will produce a plausible-looking fix that addresses the symptom rather than the cause. This piece covers three things: what runtime context to attach before you ever type a prompt, how to scope a reproduction case tight enough that the agent can't wander, and how to verify a proposed fix before it touches your main branch.

Runtime Context: What the Agent Actually Needs

Coding agents built on large language models reason over text. They don't have a mental model of your service topology, your database's current row counts, or the fact that your Redis cluster failed over twice last Tuesday. Every one of those facts has to be handed to them explicitly, or they get invented, filled in with the most statistically likely defaults from training data. That's how you end up with a fix that assumes PostgreSQL when you're running MySQL, or one that recreates a race condition because the agent assumed single-threaded execution.

Start with the stack trace, but don't stop there. Attach the actual log lines surrounding the error, not just the exception itself; timing matters, and an agent that sees a 200ms gap between two log entries will reason differently than one that sees 20 seconds. Include the relevant environment variables, the versions of your runtime and key dependencies, and, if the bug is concurrency-related, whatever thread or process information your APM tool captured. If you're using Datadog, Sentry, or New Relic, pull the full trace rather than summarizing it. Summarizing is where the important detail gets lost.

Database state matters more than most engineers think to mention. A bug that only reproduces when a table has more than a few thousand rows, or when a particular foreign key is null, needs that fact stated outright. Agents don't query your production database on their own (nor should they), so if the bug depends on data shape, you have to describe the data shape.

One more thing gets skipped constantly: recent deploys. If the bug appeared after a deploy, name the commit, or at minimum describe what changed. An agent working blind will search the entire codebase for a cause; an agent told "this started after we bumped the connection pool library" will look in exactly the right place.

Scoping a Reproducible Case

A bug report without a reliable reproduction step is a request for the agent to guess, and guessing is expensive, particularly in review time, which is the scarcer resource. The goal is a case that fails the same way every time, ideally one you can express as a failing test.

Write the failing test first, if you can. This isn't a novel idea, test-driven development has argued for this discipline for decades, but it matters more with an AI agent than with a human colleague, because the agent will treat the test as ground truth. If the test is wrong or too loose, so is the fix. A test that asserts "the function should not throw" gives the agent enormous latitude to satisfy the letter of the assertion while missing the actual defect. A test that asserts the exact return value, the exact number of database calls, or the exact sequence of emitted events gives the agent almost no room to cheat.

Strip the reproduction down to the smallest surface area that still triggers the bug. If the failure shows up in a checkout flow that touches inventory, payment, and shipping calculation, and you hand the agent the whole flow, it has three subsystems to search through instead of one. Isolate which subsystem actually fails, then hand over just that slice, along with enough surrounding context (interfaces, type definitions, the calling code) that the agent understands how the slice fits into the whole.

Flaky bugs deserve special mention. If a bug reproduces one time in ten, that intermittency is itself a piece of information, often the most important piece. Describe the failure by its actual rate rather than smoothing it into a reliable one. State the failure rate, describe what you've already ruled out, and if you suspect a race condition or a caching inconsistency, say so plainly rather than letting the agent discover it independently, which it may not.

Verifying the Proposed Fix

This is the step teams skip most often, and it's the one that determines whether the whole exercise saved time or just moved the debugging burden downstream to whoever reviews the pull request.

An agent's proposed fix should be evaluated against the same failing test that defined the bug, first and foremost. If the test now passes, treat that as a necessary condition rather than a sufficient one. Run the full existing test suite next; a fix that resolves one failure while silently breaking three others is a net loss, and agents, lacking full context on every downstream consumer of a function, produce this failure mode regularly. This is a structural fact about working from partial information, the same failure mode a new engineer produces on their first week touching an unfamiliar codebase.

Read the diff line by line, not just the summary the agent provides. Agents are prone to a specific bad habit: fixing the reported bug while also making unrelated "improvements," renaming variables, restructuring functions, adjusting error handling elsewhere in the file. Each of those changes is a new surface for regression, introduced without being asked for. A tight, reviewable diff that touches only what's necessary is a sign of a well-scoped prompt; a sprawling diff is a sign that the agent didn't have tight enough boundaries and started reasoning outward.

Check the fix against the original runtime context, not just the test. If the bug depended on a specific data shape or a specific timing window, confirm the fix actually addresses that condition rather than a nearby symptom. A classic failure here: an agent adds a null check that stops the crash but doesn't address why the value was null to begin with. The exception disappears; the underlying data corruption continues, quietly, until it surfaces somewhere else in a form that's harder to trace back.

Finally, deploy the fix to a staging environment that mirrors production closely enough to matter, and watch it under real or replayed traffic before merging to main. This is standard practice for any fix, human-authored or otherwise, but it carries extra weight here: an AI agent has no accountability for what happens after merge, no pager that goes off at 2 a.m. The engineer attaching their name to the pull request carries that, regardless of who wrote the diff. Treat the agent's output the way you'd treat a fix from a capable contractor who has never seen your production environment: promising, often correct, and never assumed correct until proven so.