[태그:] AI debugging

Imagine entering a quantity of 0 in a note app and finding that it will not save. You send the error screen to an AI, which replies, “It looks like an input validation problem.” That is a possible cause. Even “I've fixed it” remains a claim until you check the actual change and repeat the steps that caused the failure.

What else should go into the report? A comparison of three kinds of error report was prepared to explore that question, but no measured run of the AI model being tested took place. A route with no additional charges and isolation that would keep the AI from seeing the answers were among the unverified requirements. This article covers what the preparation established, why execution stopped, and untested reporting and verification suggestions. It does not establish which approach is faster or more accurate, or the minimum number of fields needed.

Start with a report you can check again

Claude Code's official bug-fixing guidance recommends providing the error, reproduction steps, a stack trace (the recorded sequence of calls), and whether the failure is intermittent, among other details. That is workflow advice, not a measurement of each item's effect. Bug-fixing guidance

This template is an untested editorial suggestion, not a proven minimum. It was not evaluated with readers or the AI model under test, and preparation did not measure whether any line improves a diagnosis. The plan bundled time, operation, code and ID together, so it could not establish each item's contribution. More information should not be assumed to improve a diagnosis.

Fill it out in a local note before sending anything externally. As this article's selection rule, leave out passwords, tokens, personal information and the data sent to a server (request bodies). Do not paste everything in the developer console, whole log files, HAR network exports, .env configuration files or chat histories. A “sanitized” export label is not proof that all content is safe to share. Separately, OWASP advises against directly logging sensitive values such as access tokens and passwords, and recommends inspecting logs. Logging guidance That guidance concerns logging; the narrower rules for selecting material to send to an AI are this article's suggestions.

What I did: which screen, what I entered, and what I clicked
Expected result / actual result:
Steps to reproduce / does it happen every time:
Visible error message: the safe portions, exactly as shown
Relevant screenshot: when layout, color, a blank screen, etc. matters
Time: include the time zone and source (my clock / app record)
Related record: operation, error code, and event ID only if shown by the app or its records
What I have already tried:
Request: say when the evidence is insufficient; give a possible cause, what remains unknown, and a check that could disprove it. Do not claim source inspection, execution or a fix that did not actually happen.

Inspect the outgoing text and image pixels directly. Exclude cookies, authorization values, URL queries and local paths; also check browser borders, usernames and other windows. Crop images to the relevant area, use a generic filename, and remove and check metadata (information attached to the file). If a stack trace is needed, select only a relevant frame after checking and removing sensitive content. Omit any line or screen you are not sure is safe to share. Mark omissions with a label such as [removed], clearly separate from the original text. Omit the whole line if removal changes its meaning or the marker itself cannot be assessed safely. Preserve the remaining safe text in its original language and put explanations beside it. This process does not guarantee that all sensitive information has been removed.

If there is no record, write “No related record available.” Do not invent a code or ID to fill the gap. An ID can help match related records when they carry the same label; it does not explain the cause. Each report in condition C, described below, contained just one synthetic event, so this was not a test of matching records or of an ID's benefit. The local event ID is not a W3C trace ID or traceparent. The W3C Trace Context specification prohibits personal or sensitive information in traceparent and tracestate and discusses privacy risks from linking trace records across components, even when an ID contains no personal information. Trace Context

What to check after “I've fixed it”

The prepared zero-input fault illustrates a verification procedure. Observing that 0 was rejected cannot, by itself, distinguish “0 was treated as missing” from “0 was explicitly forbidden.” Independently of the AI's explanation, inspect the rejection condition in the actual code before and after the change. If reading code is difficult, a suggested first step is to ask for the exact before-and-after change and the line that rejected 0. That explanation still needs checking against the actual change; the behavior checks below cannot by themselves prove the internal cause.

Write down the expected behavior and checks before the change, and preserve them. Do not judge a repair solely by checks the AI invents after making it. The following list checks behavior for this zero-input fixture. Inspect the whole change for another rejection path, removed checks or deleted failure reporting.

Then repeat the same input sequence and check each behavior: 0 saves, positive numbers save, negative numbers are still rejected, and nonnumeric values are still rejected. Reopen the saved data and check that the existing note remains. These checks were included in the local preparation, but they are not results from verifying an AI-generated patch. Reversing a known fault during preparation cannot be presented as a model's repair.

The simulated-timeout fixture needs separate guards: retain the contractual 120 ms allowance, save an 80 ms result once, time out at 121 ms, and never save a late result. The storage fixture must remove unnecessary data while retaining the 4096-byte limit; genuinely oversized required data must fail without deleting existing content. Arbitrarily enlarging a limit to pass is not a repair of this contract.

What would a comparison need to hold constant?

The prepared app validates note input, generates a preview, and saves to a local file. There is a working baseline and three faulty copies, each differing from it by one line: one rejects 0 as if it were missing, one allows too little time for a simulated operation, and one adds unnecessary data that exceeds a test storage limit. These do not test real network failures or browser storage quotas.

All three conditions were designed to receive the same reproduction steps and expected and actual results. Condition A also adds its image to that shared explanation.

Every prepared error panel said The operation could not be completed. In the zero-input case, C includes validateNote and INPUT_REJECTED. If an AI merely repeats “the input was rejected during validation,” it has not identified the faulty condition inside the app. C's readable operation names and codes deliberately favor it on failure-stage identification. For the other two faults, they are loadPreview / OP_TIMEOUT and persistNote / LOCAL_WRITE_FAILED. Under the planned scoring, repeating these labels could earn stage credit but no root-cause credit. The bundle also cannot isolate the effect of the timestamp or ID individually.

OP_TIMEOUT alone cannot distinguish an allowance that is too short from work that is too slow, and LOCAL_WRITE_FAILED does not identify unnecessary data exceeding the storage limit. Codes are clues to check, not proof of a mechanism. Whether these codes reduce unsupported claims was not measured.

The A materials for the simulated-timeout case match those for the save-failure case, and the B materials also match across those two cases. Both reports start with a fresh local app containing an existing synthetic note, then specify entering Demo note and quantity 1 and choosing Preview and save once. Both expect the new note to remain after reopening and the existing note to survive; both report a failure panel and an unsaved new note. Unlike the zero-input case, these two cases share even the reproduction context, so neither A nor B supplies evidence that could distinguish them. This is a limitation of the prepared joint preview-and-save action and withheld failure stage, not of error reports generally. A/B was a secondary delivery-format control for a monochrome text panel; it did not test failures where layout or color carries essential evidence.

The plan called for the same AI model under test across 3 faults × 3 conditions × 2 repeats of each case in fresh sessions: 18 runs in total. Each needed a fresh session without access to the known answers, source code, or previous responses. Failure stage and specific cause would be scored separately. One predetermined repair example with source access was set apart, because giving the model source access would change a comparison meant to use only the initial reports. This plan was not executed.

Even a completed set would repeat three fixed synthetic faults with one model, not sample independent real-world bugs. It could not establish population accuracy, statistical significance, performance on other apps or models, the effect of each added field, or a minimum sufficient set. C was longer as well as more informative, so any difference would not separate those factors. Scorers could not be kept fully unaware of conditions when responses mentioned images or logs. Without verified separation from source code, known answers and earlier responses, the planned report-only comparison would not be valid. This article reveals the mechanisms, so any future blinded experiment would need to reassess its materials and session separation.

What preparation established—and what it could not

On October 10, 2026, 12 offline tests passed. They checked the working and faulty copies, restoration, reopening saved data, preservation of an existing note, and related behavior. They included local synthetic export checks that rejected planted fake secrets. These were not tests of a live model transmission and did not establish that real logs, arbitrary secrets, image pixels or metadata could be safely anonymized.

The record shows that text in images created that day matched the prepared messages. After inspecting the image containing a planted fake secret, the reviewer excluded it from material to be sent. Later capture attempts with file-access and network restrictions failed. Earlier images and local checks did not validate the capture route that later failed.

In an independent review on October 18, 77 offline tests of the preparation tools passed. This was the set counted in that review, not a cumulative score to add to the earlier 12. The review still judged the setup not ready to run. Unresolved questions included whether programs started by the test tool could reliably be stopped even if they detached and kept running in the background; whether execution used the existing subscription; whether the AI was blocked from known answers and other tools; and whether remaining capacity and a route with no additional charges were confirmed. Screen capture under the restrictions also remained unverified.

More time would not supply that evidence by itself. An operational decision on October 25 set the scope to an article about the method without proceeding to measurement. Neither the 18 diagnostic runs nor the planned model repair was executed. Counts of passing offline tests are not AI diagnostic scores.

The practical suggestion is straightforward: keep the reproduction steps and evidence you have checked for safety before sharing, and ask for a way to check the AI's proposed cause. Record actual changes and reproduction results separately. When layout or color is the problem, preserving the relevant visual context is also advisable. This design, built around text-based error panels, cannot establish the general value of screenshots.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.