[카테고리:] English

We tested one AI-built practice sign-up form using automation, twice per condition. On the evaluated routes, the initial build already reached completion but did not display the entered values. One code revision added that information. This was not a test of user success rates or AI’s general capabilities. Actual 200% zoom was not tested, and part of the keyboard route in the Korean-language form remained unevaluated.

This is a conceptual illustration, not a test screenshot or evidence of successful zoom operation.

Guidance and our distinctions: reaching, operating and enlarging

W3C’s Understanding Keyboard explains that functionality must be operable through a keyboard interface. In this article, we distinguish reaching a control, seeing where keyboard input will act—its “focus”—and actually operating it.

Understanding Resize Text explains that text, except captions and images of text, must support enlargement up to 200% without losing content or functionality. Our narrower-page test cannot substitute for that enlargement test. WAI’s Easy Checks also explains that accessibility barriers can remain after preliminary checks. These are explanatory guides, not a conformance verdict on this form.

What we observed: reaching completion versus displaying the values

The intended route on September 27, 2026 was details → review → completion. Automation left the name blank and entered invalid as the email to trigger rejection, then corrected the fields to 연습 참가자 (“Practice Participant”) in the Korean-language form or Demo Participant in English, and demo@example.com. It used the form’s own Back action, attempted to change the session from Morning to Afternoon and submitted only where that change was confirmed. All details were synthetic.

B0 is the initial build; B1 is the build after one code revision. The recorded generation configuration was the Codex CLI command-line tool, gpt-6-astra, with high reasoning. The prompt requested semantic HTML and visible field labels. A separate directory was used, but the generation response alone did not independently verify which earlier context or actual model settings the tool used.

Automation used headless Chrome 153.0.8010.53—without a visible browser window—on macOS, with Node v24.19.0 and Playwright. It read the page’s internal structure, the DOM, to help find targets; this was not a person navigating by sight. The pointer condition also clicked the selector and then used the pointer-equivalent program command selectOption to choose a value. No keyboard run used a pointer to get past a blocked action: 0/8 keyboard runs in each build. Visual verification of the key sequences was partial.

Each table row represents two repetitions in each build. The reference viewport was 1280×900 CSS pixels, the units used for page layout; the narrow viewport was 640×900. In the records, M100 means pointer interaction and typing, K100 means keyboard at the reference width, and K200 names the narrow keyboard condition, with no actual zoom or text enlargement.

The criteria check the DOM for the name, email, Afternoon selection, and local-demo and nothing-sent statements on the completion screen. Counts meeting those criteria are neither screen-arrival counts nor human success rates. Earlier input, error recovery and Back navigation were tested even in the Korean-language keyboard conditions.

Both repetitions in each row had the same outcome. Of 12 assigned runs per build, eight had completion screens evaluated. B0 reached that screen in all eight, but it did not show the name, email or Afternoon selection. It did show the local-demo and nothing-sent statements. The plan and evaluation criteria required the final values to appear again, but the original generation prompt omitted that requirement. B0 had not disobeyed its instructions. B1 added the information and met the criteria in 8/8 evaluated runs. The demonstrated change was added information, not the removal of keyboard or zoom barriers.

In each build, DOM checks confirmed invalid-input rejection, correction, initial review and Back navigation in 12/12 runs. The automation used predetermined corrected values; this does not show that a person understood the error guidance. Values were retained at 44/44 checkpoints in each build: eight paths that changed the selection × four checkpoints, plus four Korean-language keyboard paths that never selected Afternoon × three checkpoints. Afternoon review was reached in 8/12 runs per build: 4/4 pointer-equivalent runs and 4/4 English-language keyboard runs that typed the prefix Af to choose Afternoon.

In both builds, the selection change remained unevaluated in 4/4 Korean-language keyboard runs. Automated ArrowDown (Down arrow) and End keys did not change the value. The tool did not support sending Korean (Hangul) prefix-selection input, so it skipped that attempt. After Space, ArrowDown and Enter, the details page advanced to review with Morning still selected, and the run stopped. This was not final submission; the later Alt+ArrowDown fallback never ran. The record classifies Afternoon selection and dependent completion as not tested because of testing-tool/environment limitations. That is a classification, not a conclusive causal diagnosis: it establishes neither an app keyboard defect nor correct operation.

The test tool changed, and some screens remain unseen

The table reports results obtained after three revisions to the test tool. Three earlier batches stopped in 12/12 runs each—36 stops in total—and were excluded from the denominators. The first contained six cases where the tool could not map 시간 (“Time”) to either a section or the selector, plus six selection-change stops. The second had 12 selection-change stops. The third had eight completion-information mismatches and four unresolved Korean-language selection operations. Those eight mismatches observed the same unmet criterion later scored in B0; they were not added to the scored results. Exclusion does not mean every observed mismatch was a false app failure.

The tool was changed to prefer the selector; the app’s duplicate “Time” names were not changed. Their effect on voice operation, for example, remains untested. The earlier email value 오후 (“Afternoon”) was classified as a tool error: keys continued in the email field after focus moved, though the trace does not expose exactly when it moved. It was not counted as app data loss. The generating AI’s own initial browser check also failed to launch and did not verify the screens.

The visual record comes from the then-coordinator, Claude (Opus 5.5), inspecting four of ten shortlisted images: English completion, the English narrow-width selector and completion, and the first Korean narrow-width error state. In the last image, the name field’s focus ring and both errors were visible. The restart ring on English reference-width completion was partly cut off at the bottom, but the coordinator judged it identifiable. The narrow-width selector’s ring also touched the bottom edge. Six images, including B0 completion, and readability, focus visibility and obscuration in other states remain unverified. B0’s missing values were a DOM finding, not part of those four image judgments. Horizontal overflow—content spilling beyond the page width—was detected in 0/910 B0 measurements and 0/902 B1 measurements. That does not establish readability in every state.

The experiment used one of three allowed app repairs and two of four generation requests. Under the frozen priority—completion information, lost input/error information, then focus/layout—the evaluated B1 omission was resolved and the remaining stops were classified as tool/environment limitations. B2 and B3 were not run. The limited image review also identified no barrier judged to require another repair. Unviewed screens, Korean-language selection and the ring-edge observations remain open; this is not a zero-defect finding.

Generation and repair response time totaled 6 minutes 33 seconds; the span from the first build to the end of B1 testing was 1 hour 52 minutes 59 seconds. Total working time, human task time and productivity were not measured. The incremental spending cap was USD 0; actual billing and token costs were unverified. The separately retained English-language experiment record links the prompts, excluded batches and evidence for each repetition.

Editorial judgment and a checklist for recording results

My editorial judgment is to specify the values needed on the completion screen in the initial generation request. Here, a planned requirement was not passed to the builder; after an explicit request, those values appeared on the evaluated routes. A change in this one implementation does not guarantee the same benefit for readers.

Actual zoom and intermediate levels, the separate test of content reflow at 320 CSS pixels, screen-reader speech, the Korean input method that turns keystrokes into Korean characters, other browsers and mobile devices, and disabled users’ experiences remain untested. Checking the nothing-sent statement was not a network audit. The deferred Grok planning and interview reviews were completed and adjudicated on October 5, 2026, using pre-experiment material. They were retrospective editorial reviews, not independent validation of the run.

The following checklist for recording results condenses the planning procedure for this article. Its effectiveness has not been validated, and it is not a certification checklist. Mark Yes, No or Not tested, and note the screen, values and point where progress stopped. The first item, actual zoom, was not tested in this experiment.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.

If you copy an app's code and its screen loads, is the move complete? This synthetic-app simulation on one computer checked two things separately: running in another folder, and using records and files with the original-service route blocked.

One synthetic app was copied to a separate folder on the same Mac. A local program stood in for the original service, and some packages deliberately omitted components. These were two reset repetitions of one app, not a real provider's export, a move to another computer or a general success rate. The app was authored in an Astra AI-agent assignment session. The measured runs made no model calls, and AI itself was not being evaluated.

What the documentation says: there is more to move than code

Lovable's documentation describes hosting the part of an app that people see elsewhere while retaining its Cloud service for supporting functions such as data. Supabase's documentation explains that database backups exclude the actual files held in its Storage file service. That does not mean the files cannot be moved. Support for these two statements was rechecked on October 5, 2026. This experiment did not export either product.

What the experiment recorded: the same code in three packages

This article uses five categories: code, records, settings, file contents and services. Settings include service addresses; services include programs that supply records and files. A dependency is something the app still needs to work.

The test ran on September 27, 2026, with 20 records and 3 attachments. All three packages used the same code. A contained only code and settings pointing to the original service. B added a database snapshot and settings for local records, while still fetching files from the original service. C also included all three files' contents and relative settings for local records and files. It did not receive an extra service program. The results checked the deliberately constructed omissions; they did not uncover unexpected vendor defects.

Each condition began with fresh copies and fresh browser state, with cache disabled and service workers bypassed. A local forwarding route either allowed requests through or immediately returned HTTP 503 (“service unavailable”) without contacting the original service. “Blocked” below means this condition. It is different from cancellation or switching off Wi-Fi. Behavior with caching enabled, timeouts, connection refusal, partial failures or authentication errors was not tested. The second repetition reversed the order to check for state carrying between conditions. This was 3 packages × 2 states × 2 repetitions: 12 conditions for one app, not 12 independent apps.

One separate baseline check ran only a disposable copy of the original service. Before and after restarting that service, records matched 20/20 and file contents and record-to-file links each matched 3/3. Editing the note field of record R05 changed only the disposable source database. The unused destination database copy and preserved originals stayed unchanged. That edit was not carried into later conditions.

In the Records column, a record counts only if all 7 fields match, out of 20 records. List and detail views were checked separately and gave the same count in every condition. A 20/20 result includes matching values and types, with no missing, extra or duplicate IDs. It cannot identify dependencies outside those seven fields. File contents means matching contents fetched through the download routes, out of 3. Match counts were the same initially and after restarting the copied app. After a successful edit, only the designated R05 note change was expected on restart.

A's records and files under allowed access, and B's files under allowed access, came from original-service responses, not contents held in the destination package. B's records and C's records and files were supplied by their local configuration. “Destination” means the copied app's folder and database. Requests count app forwarding attempts to the original service across the initial observation, edit and restart observation. Diagnostic requests checking the route's allowed and blocked states before each condition are excluded.

C's 0/0/0 counts audited app forwarding requests. In all 6 blocked conditions, none of these requests reached the original service. This was not an audit of every browser transmission or every possible path for reading original files. Operating-system-level browser isolation had been removed, and a separate browser-blocking check has no execution evidence. Complete isolation, including for C, was not established.

In 12/12 conditions, the initial and restarted page returned HTTP 200 with the app title. That alone does not mean the functions worked. In A with access allowed, the R05 note edit was saved only to the original service's disposable database. It remained visible after restarting the copied app, but had not been saved at the destination. The write location was determined from both databases' before-edit, after-edit and post-restart states and request traces, not from the screen alone. In A with access blocked, the list, details, files and edit all returned HTTP 503. The 0/20 result means inaccessible records, not deleted records; the failed save was not loss of a previously saved record.

In B with access blocked, all 3/3 record-to-file links remained, but none of the three files' contents could be retrieved (0/3): 메모.txt (Korean for “memo”), table.csv and bytes.bin. A name or link is different from the file itself. File links also matched 3/3 in the other conditions, except A with access blocked: the records were inaccessible, so their links could not be checked. Edits in B and C went only to the destination database and persisted after restart. No condition changed either database beyond the designated note, and the preserved originals stayed unchanged.

C's result covers only these records, files, the note edit and restart. Record creation, deletion and attachment replacement were not tested. Login, payments, email, background jobs and runtime AI features were neither implemented nor tested.

B's six requests were three file requests initially and three after restart. The summary record does not itemize the paths making up A's 52 and 51 requests, so they are not asserted to be identical call sets. Before each condition, one allowed diagnostic reached the original service. A blocked condition's log therefore still contains that earlier access, separate from the table's app requests.

Preparation failures and what remains unverified

The first pre-run check was put on hold because the permitted network and browser scope conflicted with the test requirements. After clarification, three separate attempts stopped for configuration, record-integrity and browser-execution problems respectively. Each measured 0/12 conditions; none was counted as an app-function failure. The fourth attempt after clarification observed all 12 conditions. The experiment record preserves the detailed errors and baseline preparation history.

The program running the tests was corrected before measurement, while the app, data, packages, order and scoring rules stayed unchanged. However, operating-system-level browser isolation was removed, so blocking the browser's own background traffic was not proven. There is no execution record for a separate check of whether the browser blocks a local port outside its permitted list; its result and duration are unknown. This is a different check from the observed HTTP 503 blocking on the original-service route. Part of the intervening relocation of temporary test folders is also undocumented.

The environment was macOS 27.0 arm64, Python 3.9.6, SQLite 3.54.0, Chrome 153.0.8010.53, Node v24.19.0 and the installed Playwright module. The final automated run took about 42 seconds, including the baseline; that is not how long an app migration takes. Total build and correction time, beginner migration time, total cost, model identifier, CLI version and subscription usage were not measured. This does not show that moving an app costs nothing or improves productivity.

The test did not establish portability to another machine or operating system; real providers' export permissions, difficulty or costs; public deployment; production security; legal ownership or licenses; or actual cancellation. Moving browser-held values, saving files through an operating-system download dialog and opening them in a separate application were also not tested.

My judgment: record write locations and file contents

In my view, “Can I see the edit?” needs two further questions: “Where was it saved?” and “Are the file contents still available without the original-service route?” C cannot establish portability or success rates for apps in general. Intentionally retaining the original service is also an option.

This is an initial checklist. Preserve the original account and data, and begin by comparing one declared read function on a copy with the expected contents. A code copy can still write to the original service: test edits only after isolating even the connected service's data into disposable test copies. If you cannot establish that separation, do not write; record “unknown.” Beginner usability has not been tested. The checklist does not certify a successful move or show that cancelling the original account is safe.

For each row, mark the planned location as “local/original service/unknown.” Always record whether it was checked without the original-service route (yes/no/not done), and which database received an edit (location/not done/unknown). This is not an instruction to cancel or block a live service. If you cannot inspect routing and storage in a test environment, independent operation remains “unknown.” Inclusion in an export and operation without the original service are separate judgments.

Record the contents compared, whether fetched file contents match, and whether an edit to disposable data survives restart. If you checked 3 items, write “3 items checked”; do not turn that into a claim that everything matches. Mark unknowns as “unknown” and name whom to ask.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.

Here, “before it broke” means T0: a recovery point recorded and verified in advance. It does not include everything done right up to the failure. “Exactly” means that the specified code and settings files match byte for byte, and that the database schema, complete contents, and designated behavior match the reference.

In one bounded experiment, a small task list with invented data received one deliberately specified defect; each of two recovery methods was tried twice. This was neither a real user incident nor a reproduction of an accidentally AI-generated app defect. Restoring code alone left incorrect data; restoring code, data, and settings together matched T0 but removed two later legitimate changes. These observations establish no success rate or general rule.

What the official documentation says about restoring code and data

Git restore restores selected files from a specified source. Some projects do track database files in Git, and ignore rules do not apply to files already tracked.

SQLite’s backup interface creates a database copy. integrity_check checks structural integrity and some constraints; foreign-key checking is separate. An ok result means only that this check found no problems. That it does not guarantee a match with earlier values is an inference from the check’s scope. These manuals were checked on October 5, 2026.

This experiment’s design

In this experiment, data and settings were excluded from Git history from the outset, and writes controlled by the script were stopped to record the components together. SQLite journal mode was delete; writes by arbitrary external processes were not independently ruled out. Simply copying a live database file during a transaction can produce an inconsistent copy. This experiment used SQLite’s backup interface; pausing writes was its coordination constraint, not a universal requirement of that interface.

Recorded observations: a stopped first attempt and a separate rerun

The experiment ran on September 27, 2026, Korea Standard Time (UTC+09:00), using macOS 27.0 arm64, Git 2.54.0, Python 3.9.6, and SQLite 3.54.0. Astra (Codex) wrote and ran the experiment script. It used local invented data and a fixed script, with no model inference during measurement. Method A restores code only; B restores code, data, and settings.

The first attempt stopped before installing the defect because the code copy had inherited read-only permissions. This was a preparation defect in the experiment script, not a failed app recovery. Of the two planned trials for each method, A and B each completed 0/2. The healthy-state check and restoration rehearsal each passed 11/11 checks, but neither was a recovery from a fault. The faulty state, recovery time, and loss were unmeasured. Before the procedure was frozen, the editing tool blocked one attempt to create files outside its allowed folder; no files were written by that attempt. A different creation method was then used.

The first attempt ended under a no-retry instruction. The user subsequently authorized one separate second attempt. Before that execution, the copy’s write-permission correction and fresh workspace were documented. App code, expected values, and check rules stayed unchanged, and the first attempt’s evidence was preserved. The second attempt’s healthy-state check and restoration rehearsal again each passed 11/11 checks.

Recorded observations: state changes in attempt 2

The comparison covered app.py and .gitignore, the schema and all ID-ordered rows of tasks(id,title,done), and the list-title and show-completed settings in settings.json. Below, 0 means incomplete and 1 means complete.

Adding task 4 and changing the title were legitimate work. The restriction that made “complete task 2” affect only that task was deliberately removed. Running the action once set all four completion flags to 1: task 2 was the requested target, task 3 was already complete, and tasks 1 and 4 changed unintentionally. The faulty-control record shows 4/11 checks passing: schema, integrity, current-list output, and current-settings output. Integrity still returned ok. Task names and the setting for showing completed tasks were unchanged.

Recorded observations: what each method restored

The procedures deliberately restored different components. These runs checked whether that expected contrast occurred in this implementation; they did not establish which method is superior. Each trial used a fresh copy of the same faulty state, in order A1/B1/B2/A2. Each method completed its planned 2/2 trials.

The 11 checks cover code, settings, data, and behavior; their pass count is not a recovery percentage. Even a legitimate title edit counts as a mismatch with T0. A passed code, schema, integrity, current-list/settings output, and behavior checks; it failed the historical-row and T0 settings/list comparisons.

Both A trials repaired only the code, leaving the incorrect completion values in place. The A restoration deleted nothing further. Task 4 remained incorrectly marked complete by the fault, so it was not “preserved intact.” The title edit survived.

Both B trials matched T0’s specified code and settings bytes, database schema, and complete ID-ordered records. At the same time, two pieces of legitimate work disappeared: task 4 and the title edit. Matching the recovery point and preserving later work were different outcomes. Neither method recovered the last legitimate pre-fault state: four tasks with flags 0,0,1,0 and the title Demo v2.

Behavior was tested on a separate copy seeded with the predetermined values (0,0,1)—tasks 1 and 2 incomplete, task 3 complete: after completing task 2, the values had to be (0,1,1) and remain so after reopening. A and B each passed this check in 2/2 trials; the unrepaired faulty copy used for comparison failed. This probe did not test the restored four rows or task 4, and did not modify the restored data being compared. Passing a behavior check is different from restoring historical data. In attempt 1, the backup’s database file hash (a content fingerprint) differed despite matching logical contents. Neither database file-byte identity nor all future behavior is claimed. Expected values were fixed separately from app output, but the same script restored and verified the fixtures. There was no independently implemented verifier, and the complete original trial copies were not archived.

These timings record an automated procedure, not a speed comparison. Values are rounded to six decimal places. Total time runs from the start of restoration to the end of verification and was measured separately, so rounded parts may differ from the total in the last digit. Preparing the separate trial copies took 0.007964, 0.007863, 0.009005, and 0.009480 seconds in order A1/B1/B2/A2. Automated preparation, recovery point capture, and control checks took 0.314130 seconds. This was an automated experiment in which even the interval from T0 to the fault was only 0.077574 seconds.

For the first attempt, the recorded timestamps show 0.214146 seconds from execution start to the failure; aggregate preparation time was unmeasured. These figures do not represent human working time or establish a speed advantage. The preparation figures come from the timing record; full details are in the experiment record.

My assessment and an untested recovery-point card

My assessment is that it matters to record what was compared and what remained, changed, or disappeared together. Whether the template below improves recovery or reduces losses has not been tested.

Recovery-point card
Recorded time and time zone: ___ / Scope of paused writes: ___
Code version and location: ___ / Settings copy and location: ___
Data copy, location, and backup method appropriate to this database: ___
What to compare against (saved in advance, separately from app output): file inventory and contents, table structure and complete values, specified behavior ___
Result of restoring into a rehearsal copy: ___
Legitimate work since the recovery point and acceptable losses: ___
Post-restoration comparison: code ___ / complete data ___ / settings ___ / behavior on a separate copy ___
Work remaining, altered, or missing: ___ / What could not be verified: ___

For example, B’s entry would be “matches T0’s three tasks and Demo / later task 4 and title edit lost.” “Unknown” is an acceptable entry. Without evidence of the later work’s values, this comparison cannot establish the loss. A record that a task existed can still miss an altered value. Pause and investigate if comparisons, control checks, or the declared scope disagree. This card does not replace a safe backup procedure for a database an app is writing to, or a service-specific recovery procedure.

One computer, one defect, and two trials per method cannot establish reliability, an optimal backup interval, product or model superiority, or productivity. Novice completion time and usability, actual cost, remaining subscription capacity, the authoring model’s backend, and CLI version remain unverified. Production, hosted, concurrent-write, and distributed environments; database schema changes; undoing actions already sent to outside services; losing a backup with the same failed disk; and selectively re-applying legitimate later work were not tested. Nor was a legitimate edit to an existing completion flag followed by a conflicting faulty overwrite. The operating system, file permissions, file timestamps, and future behavior were outside the comparison. Review provenance and its limitations are in the editorial conference record.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.

A fictional equipment-sharing club's January policy allows seven-day borrowing periods. A replacement policy takes effect in July and extends them to fourteen days. If you ask about the current borrowing period, is an exact quotation of the old seven-day rule enough? The wording is real, but it does not answer the question for the date that matters. All dates and policy values are invented. One model received six documents and twelve paired Korean/English questions, once per language and condition. No current-period answer used the superseded seven-day rule in this run, but some correct answers had inadequate quotations.

The test separates three checks: whether a source exists, whether its quoted passage supports the claim, and whether the rule applies to the question's date, item and scope of exceptions.

Two distinctions borrowed from earlier evaluations

The ALCE study evaluates answer correctness separately from citation quality. The SQuAD study on unanswerable questions examines answering when evidence exists and abstaining when it does not. SQuAD is an extractive reading task, unlike this generative-answer test. The borrowed ideas are narrower: separate correctness from citation quality, and answering from abstention. Dates, exceptions and conflicts are distinctions in this experiment’s design. The studies do not establish this model’s performance.

The experiment: the same documents, two requests

On September 27, 2026, the experiment used six synthetic documents and an answer key written before the runs. The documents covered the old and current policies, a camera exception, pickup instructions, and two conflicting projector-fee notices. Every request received all six short, deliberately legible documents in the same order. Searching for documents and order effects were not tested. The current policy explicitly replaces the old policy and retains the camera exception. The planning draft’s sentence pointing out the fee conflict was absent from the executed corpus.

Condition A was a basic citation request. Condition B added checking rules. Both required answers based only on the supplied documents, with document and clause IDs plus short verbatim quotations. These were typed references and quotations, not a test of a native citation feature. B added effective-date, explicit-replacement and exception checks; a rule against treating a later publication date alone as overriding; handling of unresolved equal-authority conflicts and missing information; quotation-support checks and omission of unsupported claims; and answering supported parts.

Test scope: Responses reported the model as claude-sonnet-5. The Claude command-line tool sent the documents to the model provider. Twelve questions were written in Korean and English; each was asked once per language and condition, in a new process and session (12×2×2=48). All 48 planned runs were attempted and produced 48 valid responses; there were zero invalid, skipped or retried runs. English responses had a 150-word limit including citations; Korean responses had a 600-character limit including spaces and citations. These were unequal budgets, but all outputs finished under their caps: the largest were 117 words and 371 characters, with no technical truncation. Another AI assistant, Codex, made rubric judgments against the prewritten answer key. The tested model did not grade itself, and no human scored the outputs independently.

The record describes pre-run changes from the proposed claude-sonnet-4-6, restoration of the request wording, and neutral wording for Q10. The revision timing lacks independent timestamp attestation. Exact CLI/OS versions, billing evidence and complete internal hook/memory isolation are also unverified. The separately retained experiment record contains summaries, correction history and paths to the original inputs, outputs, answer key and scoring files.

The English results come from runs using English documents and questions; they are not translated Korean results. Each cell counts questions meeting that row’s criterion out of those asked: all required facts for direct facts and applicable rules; both values and sources, an unresolved conflict and no definitive fee for conflicts; and explicitly stating that information is absent without inventing a value or turning pickup rules into return rules for missing information. Conflict and missing-information questions are shown individually.

These counts describe this single run, not a success rate or guarantee for real documents.

All four combinations correctly gave Monday–Friday, 10:00–16:00 for pickup in Q1, and the reservation code to bring in Q2. They distinguished the current fourteen-day standard loan, the seven-day rule applicable on June 15, 2026, and the three-day camera exception. These are calendar days, including weekends and holidays.

English B's Q8 response correctly said that one camera plus two standard items totals three and fits the item limit on the reference date, September 27, 2026. It omitted the frozen key’s additional explicit requirement: the maximum of one camera. The scores below therefore measure completeness against the required facts; this was not wrong arithmetic or an invented policy. The key prohibited stock or approval guarantees, and no response made one.

Across the eight questions the documents could answer (Q1–8), English A met all required facts on 8 of 8 and English B on 7 of 8. Korean A and B each met them on 8 of 8. None of the four combinations refused an answerable question: 0 of 8 each.

For the conflict questions, two notices issued on the same dates by the same body and both marked as in effect specified fees of five and eight fictional credits, with no precedence rule in the supplied collection. Every response showed both values and withheld a definitive fee. Korean B's Q9 response nevertheless omitted the shared issuer and active-status details. Handling a conflict correctly and explaining it completely are separate checks.

No response invented a late fee or treated pickup instructions as permission to return an item on Sunday. English A's Q12 response did add an inference about a Sunday due date. Counting weekends makes this plausible, but the documents do not fully specify when counting starts or whether a Sunday due date moves to another day. Codex scored the extra claim as only partly supported. The missing-information criterion still passed; that pass did not validate every added claim.

Correct answers did not always come with adequate quotations

English A's Q5 response correctly answered “seven days,” but its quotation was only “including weekends and holidays.” That passage does not contain the number seven. English B's Q8 response inserted an ellipsis into its camera-duration quotation. It preserved the duration’s meaning but failed this experiment’s verbatim-quotation criterion. Supporting a claim and reproducing the source exactly are separate checks.

Korean A’s Q3 also stated the correct replacement date but quoted only the duration sentence. English A’s Q8 supported an extra “three days” claim with a passage saying the exception remained applicable, which did not give the duration. Some A responses added correct pickup or camera facts with references but no quotations.

The columns distinguish semantic support from complete citations. B also instructed the model to omit unsupported claims, so claim selection itself could change the denominator. Answers contained different numbers of claims; these are not common-denominator performance comparisons or pooled rates. Codex assigned claim boundaries and support labels and later corrected some Q9 citation scores. The initial scores and reasons are preserved.

A document reference without a quotation did not earn full support credit on this measure. Statements that information or precedence was absent were checked separately against all six documents. Answers with no facts to cite were marked not applicable, not given automatic perfect credit.

A checklist for checking AI answers yourself

My takeaway is to compare the answer and the quoted passage side by side. These results do not establish that B is better or that an assistant should hold back an answer by default. The checklist below is an untested example format. Whether people detect more errors or check real documents more easily has not been measured.

When testing examples whose answers you already know, write down the expected answer before asking the AI. For everyday checking, you can leave that field blank and start with the source documents.

For a test, include a prewritten answer key and all four question types. Record quotation support, whether the rule applies, and any unnecessary refusal. Searching was not tested here, but my advice for tools that search documents is not to turn “the search did not find it” directly into “the documents do not contain it.”

You can try two new conversations with the same documents that are safe to share and the same question. Use the common request in both and add the complete B instructions to only one. This exercise uses the original wording; one attempt each cannot establish a winner. Its reference date and length cap belong to this experiment and need separate choices for another task.

Common request

Answer the question in English using only the six documents below. The reference date is 2026-09-27 unless the question specifies another date. Attach [document ID §clause ID] and a short verbatim quotation to factual claims. Do not use outside knowledge. Keep the answer within 150 words, including citations.

B’s additional instructions

Check effective dates, explicit replacement and the scope of exceptions. Do not treat a later publication date alone as overriding another rule. If equally authoritative documents conflict without a documented resolution, show both sources and state that the conflict prevents a definite answer. If necessary information is absent, state “Not stated in the supplied documents” without guessing. Check that each quotation actually supports its attached claim and omit claims it does not support. Answer any supported part of the question.

Providing every document was a favorable condition; 48 calls were not 48 independent real-world tasks. Both languages used the same English system instruction and one set of fictional values. Sensitivity to system language, values, length caps and order was not tested. Each question was run once per language and condition, using one model, short synthetic documents and related questions. It cannot establish language or product superiority, long-document or retrieval performance, or a general error rate. Both conditions already requested citations, and B changed a bundle of rules, so the test cannot isolate the effect of citations or of any individual rule. Human checking time was not measured. Whether the checklist saves time or improves productivity remains untested.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.

2026-02-30 looks like a date, but that day does not exist. 2028-02-29 does. An automated test was configured to check 36 subcases with made-up data in one AI-built sample app for listing supplies and space to prepare. This is a record of one build and one repair, not a failure rate for other apps.

Here, “handle” means accepting values that meet the experiment’s preset rules, refusing to save those that do not, explaining the correction, retaining other entries, and allowing one corrected save. The repaired version had no observed failures, but some interface outcomes remained unverified. That does not establish that the whole app was verified.

The evaluation rules were set before generation. Titles must contain 1–40 characters after ordinary leading and trailing spaces are removed. Notes may be empty and can contain 0–200 characters. The evaluation unit was Unicode code points, which do not always correspond to what a reader sees as one character. The test used A and the precomposed Korean syllable 가, each one code point; these boundary tests do not establish how the app counts other character sequences. Dates must be real calendar dates in YYYY-MM-DD format, within 2026–2028. Only four preset category/subcategory pairs are allowed. Matching trimmed title, date, category, and subcategory make a duplicate even if the note differs. When editing, the record itself is excluded. These are choices for this experiment, not standards for every app.

Guidance used: format, meaning, and correction

The OWASP Input Validation Cheat Sheet describes checking both an input’s structure and its meaning in context. Drawing on that distinction, this experiment separately checked the date’s format, calendar validity, and allowed year. Interface checks cannot replace checks by the code that saves the record. Neither the citation nor these results establish security.

W3C’s error-identification guidance explains that automatically detected input errors require identifying the affected item and describing the error in text. Its error-suggestion guidance calls for suggesting a correction when one is known, unless that would compromise security or the content’s purpose. These sources informed the assessment; actionable instructions, retained entries, and one corrected save were this experiment’s particular criteria. This was not an accessibility-conformance assessment.

Recorded observations: one build and one repair

The September 27, 2026 run asked the AI coding tool Codex CLI to build a preparation list with input, saving, and editing. The run plan designated gpt-6-astra, with reasoning set to high; an event-level model header did not independently confirm that identity. Local HTML, Python, SQLite, the category list, and a ban on executing entered text as code or formulas were specified from the start. Labels and error messages were in Korean. A text field replaced a date picker so exact invalid date strings could reach the app. The interface date results below concern that field; they neither show the same failure in a native picker nor recommend avoiding pickers.

The initial build, B0, did not receive the detailed evaluation rules. Its mismatches therefore cannot be described as failures to follow instructions it had been given. The same Codex session then received the rules, the complete scoring key containing every expected stored state, and failure records to produce repair R1. The initial request including setup constraints was 720 bytes; the repair request was 255,841 bytes. All records were synthetic; this is not an example of sending real records for repair. R1’s matrix and final replay reused the same frozen cases. They do not test generalization to unseen inputs or prompt effectiveness in two independent groups.

The automated test reset the app to two made-up records for each case and was configured to check 36 subcases through each route: the interface and direct save requests (API). Blocking a value on screen does not establish whether the saving code rejects it. Cases where exact interface entry or a later action could not be completed remained unverified. Browser prevention was kept separate from rejection by the saving code, and actual SQLite records were compared as well. The denominators below include unverified results. These are 36 conditions applied to one app, not 36 independent replications.

This table covers stored state, one of nine interface criteria, rather than overall quality.

On direct requests, 16 failed cases became met. On the interface, 14 failures became met and four previously unverified cases became observable: three edits reachable by record identifier and one blank-subcategory case that became submittable. The separate list-and-reopening criterion moved from 13 unverified cases in B0 to 13 met in R1. That is expanded observation coverage, not evidence that a person could not click B0’s edit button.

Of 23 applicable error-explanation cases, B0 had 2 met, 14 failed, and 7 unverified; R1 had 19 met, 0 failed, and 4 unverified. Recovery after correction had the same denominator: B0 0/13/10 and R1 18/0/5, in met/failed/unverified order. Zero observed failures in R1 does not mean every criterion was verified. These scores are not W3C conformance findings.

B0 saved the nonexistent dates 2026-02-30 and 2026-02-29 through both routes; 2026 is not a leap year. R1 rejected both. After the automated tester replaced each with the predetermined correction, 2026-02-28, R1 saved the corrected value. Both versions accepted 2028-02-29, the comparison date that should be accepted, unchanged. B0 also saved 03/04/2026 and 2026-2-03; R1 rejected them with format guidance. The first string leaves the order ambiguous: 4 March or 3 April. The corrected dates were chosen by the tester, not guessed by R1. Direct-request results separately checked the save boundary without relying on the text field.

B0 accepted both duplicate-creation cases; R1 rejected them. Both versions accepted titles made by repeating A or 가 40 times. B0 saved both 41-character values, while R1 rejected them without truncating and saving them. Both versions stored an empty note, the same title with a different date, and the literal text =1+1, which looks like a spreadsheet formula, unchanged. Direct requests with invalid categories or combinations were already rejected in B0. Among assessed items, no false rejection of valid values or change to unrelated records was observed.

Acceptance and stored values could differ. B0 accepted Workshop cards , meeting the handling criterion, but stored its edge spaces too, missing the expected stored value. R1 removed those ordinary edge spaces and disclosed the change. This explains why B0 had 21 direct-request handling passes but 20 stored-state passes.

When the title was empty and the date impossible together, B0’s browser stopped at the empty title. It did not meet the experiment’s rule requiring both problems to be explained on the first submission; downstream saving-code handling for that interface submission remained unverified. This is not a W3C violation finding. Direct requests were rejected in both versions. R1’s interface explained both problems and retained the note keep this note and selected values. The automated test corrected the two fields, saved once, and checked the reopened values. This was one confirmed scene within the 23 recovery cases.

Missing observations and the separate replay

Two early attempts were excluded from app assessment: one encountered a browser launch restriction; the other failed during reset because the test tool omitted a required creation timestamp. Each had 72 planned observations and zero valid app observations. Neither was an app input error. The counted runs followed two harness fixes, detailed in the appendix.

Even in R1, four invalid category values or combinations could not be entered exactly through the interface. Such widget prevention was a planned observation, not automatically an app failure. Save-boundary rejection for those inputs was checked through direct requests. One blank-subcategory case was rejected on screen, but the tool did not finish the keyboard correction. Direct-request results were not used to fill interface gaps. A further repair, R2, was omitted because there was no observed app defect to repair. A separate replay of four cases per route—normal creation, an impossible date, a leap day, and a valid edit—met the applicable criteria in all eight observations. These were previously disclosed cases, not a full retest or unseen inputs.

Editorial judgment: check rules, storage, and recovery separately

The editorial judgment is that checking input rules, actual stored values, and recovery separately was useful in this case. Repairing one app in the same session cannot establish general AI reliability or productivity gains. To track the execution allowance conservatively, 76 minutes were charged through the final replay, including 25 minutes of preparation, failed attempts, repair, verification, and waiting. That is neither total human working time nor measured cost; subsequent record writing is additional. Token usage is also not a bill and is described separately in the appendix.

A small checklist for your own app

Use a practice copy or a new empty app containing no real data. As in this run, use your own computer’s local address, 127.0.0.1, and a separate synthetic database; avoid shared or production servers and real login credentials. This reusable form was assembled during editing; it is not proven to save time or establish overall quality. Start with one field.

• Field name / required or optional / allowed format and range: ______
• Value to accept / unusual but valid value / value to reject: ______ / ______ / ______
• Changes the app may make before saving (e.g., removing edge spaces) / duplicate rule: ______ / ______
• One row for each value: input / expected result / actual result: ______ / ______ / ______
• Does the app identify the affected field and problem in text? Yes / No / Unknown
• Does it explain how to correct the value under this app’s rules? Yes / No / Unknown
• Does it retain other entries? Yes / No / Unknown
• Can the affected fields be edited and then saved once after correction? Yes / No / Unknown
• Does creation add one record, and does editing change only the same record? Yes / No / Unknown
• Are the reopened values and unrelated records as expected? Yes / No / Unknown

Do not count “Unknown” as a pass. This checks known correction paths; it is not a W3C conformance checklist. Reopening is an interface check, not proof that the display matches the database. The automated test read storage separately and recorded values that could not be entered through the interface and outcomes that could not be verified.

Evidence appendix: full conditions and verification scope

Records for closer scrutiny of the single-app observations are retained separately; the materials include English prose and machine-readable files. The full experiment record, frozen inputs, B0 adjudication, and R1 adjudication preserve exact requests, values, correction procedures, screens, and before-and-after storage evidence. Counts below are subcases per version, per route, totaling 36. Each version has 36 interface and 36 direct-request slots, for 72 planned observations. “Accepted” describes input handling; it does not mean every criterion, including reopening through the interface, was met.

The evaluation used a four-part duplicate key, but no valid control changed only the subcategory to isolate its role; the components were not independently tested. C04 checked removal of U+0020 edge spaces from Workshop cards with the remaining fields preserved. Repeated internal spaces, whitespace-only notes, and other edge-space characters were not separately tested.

Allowed category pairs are supplies/paper, supplies/pens, space/chairs, and space/tables. Notes are excluded from duplicate checking. Title trimming permits only ordinary spaces (U+0020) at the two ends; it does not permit arbitrary changes to case, internal spaces, or how Korean characters are encoded, composed or decomposed.

Denominators by criterion

Each result cell below is met / failed / unverified. Applicable counts include unverified cases. Interface-only criteria, such as explanations, preservation, and recovery, do not apply to direct requests; each has 36 “not applicable” cases on that route. Do not combine the criteria into a success rate.

For the interface, not-applicable counts are 23 for display, 35 for trimming disclosure, and 13 each for explanation, preservation, ability to correct, and recovery. Three direct-request criteria have 36 applicable cases each: stored state changes from B0 20/16/0 to R1 36/0/0; acceptance/rejection from B0 21/15/0 to R1 36/0/0; unrelated-record preservation is 36/0/0 in both versions.

The test harness’s method of locating records by ID did not match B0’s interface, leaving three edit cases and reopening assessments unavailable. That is not evidence that a person could not click the edit button. In B0, downstream app handling after browser prevention also remained unverified for the empty-title-only and combined-error cases. Unchanged storage met the stored-state criterion without proving downstream handling. C04 had the opposite distinction: handling met, stored state failed. For R1’s four cases that could not be entered exactly through the interface, stored state, handling, explanation, preservation, ability to correct, and recovery remained unverified. For the one blank-subcategory case, ability to correct and recovery remained unverified. Later observations did not retroactively fill earlier gaps.

B0’s two C02 cases and C20 had the expected corrected database but no reopening evidence. They belong to the 10 unverified recovery cases, not the 13 failures. Those 13 failures initially saved inputs that should have been rejected; a later diagnostic correction did not erase the initial failure.

The separate final replay covered C01, the first C08 value, C09, and C19: four cases per route. Stored state, handling, and unrelated-record preservation each met 4/4 per route; interface display met 3/3; explanation, preservation, ability to correct, and recovery each met 1/1. There were no failures or unverified results; the remaining criteria were not applicable. Correction-save actions numbered 16 interface and 23 direct-request actions for B0, 18 interface and 23 direct-request actions for R1, and one through each route for the replay. An action does not necessarily mean an actual write, and it does not erase an initial failure.

A separate read-only environment check met 1 out of 1 criterion. The date 2028-02-29, saved under ko-KR / Asia/Seoul (Korean language settings and Seoul time), remained unchanged in the list, reopened field, and database when read under en-US / America/Los_Angeles (US English settings and Los Angeles time). There were zero save actions. Locale and timezone changed together, and the interface text remained Korean, so this does not establish English localization or either factor’s individual effect.

What was not measured or fully recorded

• The experiment recorder was Astra, an AI agent. These results come from automated testing and adjudication of retained records, not direct observation of human non-developers using the app.
• Before the counted B0 run, the harness was changed to remove Chrome’s loopback sandbox wrapper and launch installed Chrome directly. The app’s OS socket sandbox remained, but browser request interception does not prove OS isolation of background traffic. Reset was then amended to supply both required timestamps, created_at and updated_at, fixed to 2026-09-27T00:00:00+00:00. Neither fix changed app code, frozen inputs, expected values, or denominators.
• The environment was Mac/Darwin, headless Chrome through Playwright (without a visible browser window), version 153.0.8010.53, Node 24.19.0, SQLite 3.54.0, and a 1280×800 viewport. Retained metadata did not establish the exact macOS, Python, Playwright, or Codex CLI versions, remaining subscription usage, or actual billed amount. The designated model and settings come from the run plan; they were not independently confirmed by a model header in the events themselves.
• Recorded elapsed times were 7 minutes 37 seconds for generation, 5 minutes 55 seconds for repair, about 66 seconds for the valid B0 test, about 74 seconds for R1, and about 9 seconds for the replay. These do not establish a production-time figure excluding preparation and verification, or total human working time. The 76-minute charge is conservative budget accounting that includes 25 minutes of preparation and subsequent waiting. B0 recorded 183,610 input tokens (including 162,560 cached) and 14,479 output tokens; R1 recorded 890,898 input tokens (including 800,256 cached) and 11,173 output tokens. Separately recorded reasoning-output counters were 848 and 1,652; these were not summed into a cost. Zero cost, time savings, and comparative productivity were not measured. Budget accounting used the revised 65 minutes previously charged and 45 remaining; the replay log’s noncumulative balance was not used. This does not prove adherence to every originally estimated time window.
• This involved one designated model, one generation, and a same-session repair given the complete scoring key and known failures. Independent repetitions, other models, browsers or devices, actual non-developers’ usability, satisfaction or understanding, and comparisons of error-message timing were not tested. The builder’s reported automated tests—five for B0 and 45 for R1—are separate from this experiment’s observations collected apart from the builder. The cases share one app and evaluator rules; they are not statistically independent samples.
• Korean input-method composition (typed letters assembling into a syllable), decomposed Hangul (syllables stored as separate letter parts), emoji and counting what a reader sees as one character, mobile devices, native date pickers, screen readers, and broader locale/timezone coverage remain untested. No security or accessibility certification was performed.
• Concurrent saves or duplicate races, transient writes during saving or atomicity, disk failures, persistence after restart, deployment, authentication, access isolation, file import/export, and spreadsheet-formula safety were not verified. They fall outside the immediate before-and-after storage comparison.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.

The AI-built packing-list app could add items, mark them as packed, and delete them. It opened with six example items and kept the list after a refresh. The evaluation criteria called for an empty starting list that reset on refresh—but neither expectation was in the initial request.

A small test using made-up data on September 27, 2026 compared one short-request run (A) with one brief-first run (B). It examined when expectations were shared, not productivity or time savings. Both runs used Korean requests and test inputs. This English edition reports the same experiment, not a separate English-language test.

What the sources say: what counts as done?

The Scrum Alliance glossary describes acceptance criteria as testable conditions for meeting user expectations. This article uses these as checks for one feature, stating an observable result: “Submitting only spaces shows an error message and leaves the list unchanged.”

The formal Definition of Done in the 2020 Scrum Guide establishes a shared understanding of work that meets the product’s quality measures. In this article, the title’s “done” refers to a checklist for one feature, not that formal concept. Neither source establishes that these practices make AI-assisted work faster.

Recorded observations: the same feature, different information

Each run started from an identical HTML skeleton and empty CSS and JavaScript files and produced a local browser app. Both runs were asked for a personal travel packing list with add, mark-as-packed, and delete actions and a comfortable layout. Appearance was not scored. B also received purpose, user, actions, screen states, exceptions, exclusions, eight checks, and instructions to report observations for each check. Both conditions allowed clarification questions, and A ran first. A separate AI operator agent (Claude Opus 5.5) evaluated the results and sent the change request, knowing the intended outcomes. A represents late disclosure of known expectations, not a typical beginner.

The record lists Codex CLI 0.153.4, the requested model gpt-6-astra, and medium reasoning effort. The actual responding model identifier was not retained, so whether both runs used the same model cannot be established. Development took place on macOS 27.0; the operator agent checked the pages separately in Chrome 153.0.8010.53 (headless, with a 1280×900 viewport). Both implementation agents failed to launch their own browser checks. B also retried with --no-sandbox, which disables the browser’s sandbox protection; that attempt failed too. B reported every screen check as 확인하지 못함 (“could not verify”). A reported passes in a simulated environment rather than a real browser. The table below comes from the operator’s browser checks, not the implementation agents’ reports. Requesting observations did not itself secure browser evidence.

The original test strings are preserved here: 물병 means “water bottle,” 양말 means “socks,” and 책 means “book.” The English glosses explain the inputs; they were not substituted during testing.

A contained code to trigger the browser’s own warning bubble, so it did have an error-handling mechanism. The automated check searched page text and could not detect this kind of warning. The saved screenshot could not establish whether a readable message appeared at that instant.

Both first deliveries matched the checks for trimming the extra spaces before and after 양말 , marking 물병 as packed and then unmarking it, and deleting one of two 양말 entries while leaving one. These details were never requested in A, so their match cannot be credited to the brief. The <b>책</b> check was also never requested in A. Submitting it displayed those characters literally without creating an HTML bold element. That last check is limited to this input, not a comprehensive security assessment.

A then received one message describing the expected and observed results for four items: the empty start, total count after adding, blank-input error display, and reset on refresh. The first two shared one cause: the six example items. Its next delivery started empty, displayed an error message on the page, and removed automatic saving. These changes followed the disclosure of previously unstated expectations. They do not show that adding items was broken or that saving a list was inherently wrong. Empty start and no saving were policy choices for this app; neither policy was tested for usefulness to travelers. In the retained checks after the checker adjustment, A’s second delivery and B’s first met all eight criteria. However, the retained checking code differs from the version locked in before the comparison. The operator reported keeping the criteria, order, and pass rules unchanged, but the before-and-after code evidence is incomplete.

The time record leaves out original authoring effort

Both requests were written beforehand. Their original authoring time is absent below, and the elapsed gap is neither the duration of the one correction exchange nor a saving.

A revision exchange here means one change-request message followed by the agent’s response, not each individual issue in that message.

Preparation consisted only of copying an existing request and registered as approximately zero minutes at the log’s one-second resolution. The time originally spent writing the requests and prepared answers to possible questions, setting up the shared environment, and checking the verification tools is unknown. Active operator effort was not measured separately either. These figures cannot be treated as total production time or used to calculate a savings percentage. Subscription resources were consumed, but actual additional charges were not independently verified.

Limits of the record

The verification record also has gaps. A’s first automated check misidentified page elements and was invalid. Its output was overwritten by a corrected check, leaving no copy of the invalid result. The operator also built a reference app beforehand to test the checking script, which departed from the plan. Isolation was incomplete: B saw A’s folder name and enumerated unrelated instruction-file paths. There is no recorded read of the other app’s source contents. Both runs ended after final verification before their 60-minute per-run limits, but full compliance with the procedure cannot be claimed. The separately retained experiment record preserves the detailed evidence and limitations.

My view: what I would put in the next request

A matched the four expectations after one message disclosed them; B received no change request. That cannot answer the title with a general “yes.” I suggest recording expectations alongside observations as a way to explain a change. The runs differed in several ways at once: information quantity, request length, disclosure timing, and the detail of checking instructions. The evaluator knew the intended outcomes and checked A first. Whether both used the same model is unconfirmed, and the time contribution of A’s extra filters, progress display, and illustration is unknown. With one run per condition, the results cannot establish the brief’s effect or a general reduction in revisions. Eight criteria are not eight independent experiments.

The following is a brief you can copy and fill in for one feature. Its effectiveness has not been tested, and the effort required for a beginner to write it was not tested. The examples reflect choices made for this practice app. This is an English adaptation of the Korean edition’s template, not an additional experimental prompt.

Feature/purpose: What should this feature help someone do?
User/device: Who will use it, and where?
Actions: What will they click or enter?
Screen states: What appears at first, after success, and after invalid input?
Exceptions: For example, what happens with blank input, duplicate names, and refresh?
Out of scope: For example, login, saving, and synchronization.
Acceptance criteria — input/action → expected result:
  Accepted input: Submit 물병 to an empty list → one item, not yet marked as packed;
    input field clears. (물병 means “water bottle.”)
  Rejected input: Submit three spaces → readable error; list unchanged.
Completion report: Give run instructions and actual observations for each check.
  Identify anything not verified; do not claim a pass from code inspection alone.

If the AI could not verify a check, open the page and try the input yourself. A change request can record input or action → expected → observed. A written condition can still be read differently, and the criteria themselves may miss important behavior. This experiment did not assess Korean character composition (IME), longer text, mobile use, keyboard behavior, accessibility, or maintenance, and passing its practice checks does not establish readiness for real use. When new requirements emerge, the criteria can change too.

AI assisted with research and drafting. The author reviewed the final manuscript and directed the revisions.

When building software by describing the desired behavior to AI—vibe coding—what should we check after a screen is generated? This article revisits nine separate experiments and prerequisite checks from this blog. It adds no new runs or connected production workflow. The year dates the records; it does not measure a market-wide shift or improvement over last year.

If a record survives a page refresh, has it really been saved? In a separately recorded browser demonstration within the storage experiment, three storage spaces inside the browser—sessionStorage, localStorage, and IndexedDB—returned the record after a hard reload. Memory-only storage and a condition that discarded writes (drop-write) were empty. Passing the reload check did not establish server-side storage. The demonstration is supported by a preserved observation record; this synthesis did not separately verify it against raw browser captures. Reloading a page, restarting the server program, and closing and reopening the browser are three different operations.

What these records cover

The experiments and prerequisite checks date from September 5–20, 2026. Their records were cross-checked in the repository on September 27. Verification and editing of this article mean comparison with the preserved records, not rerunning the experiments; the results remain limited to the environments and inputs used then.

The sections on reuse, delegation and acceptance, and interface checks and untested handoff are groupings for reading these records. They are not levels of proficiency or stages of a workflow tested from beginning to end. The records include actual executions, stops during prerequisite checks, and a local rehearsal with no service calls. Adding up their passes would not produce a meaningful success rate.

Observed: reuse

The storage experiment included a local server and SQLite database run, separate from the browser demonstration above. SQLite held the server-side records in that experiment. Requests carried synthetic identity headers—made-up user identifiers instead of a real login. The specified create, update, and delete operations, plus one restart of the server program, agreed with independent SQLite checks. These passes cover the checks that ran; the deployment check did not run.

Opening the app on another device, deploying it to an external server, and closing and reopening the browser were not tested. The interface in a fresh browser profile with no prior data, backups, and login and security for a real public service were also outside the scope. The browser reload result and the database evidence must stay separate because they checked records in different places after different operations.

The reusable-procedure experiment checked whether a copied task-instruction file loaded in a fresh conversation. It did. But successful loading did not establish more accurate answers. All 12 main answers passed the factual checks across ordinary requests, requests with the procedure included in the message, and requests explicitly invoking the tool's skill feature for calling a saved procedure. Two separate implicit-invocation diagnostics are not included in those 12 answers. All passed, so no factual-score advantage was observed; that does not establish equivalence between the approaches or show that the file had no benefit.

The comparison used two synthetic Korean-language cases, run once per condition and executed and scored by AI. It cannot establish long-term memory, a learning effect, or time savings.

My takeaway is to make “it works again” specific. Where was the data found, and after which kind of restart? Did the procedure load in a new conversation, and was any improvement in the answer measured separately?

Observed: delegation and acceptance

In the planted-bug experiment, the set of tests generated from the business rules alone passed only two of five executable checks on a known-correct version of the app. That made the set ineligible for defect scoring; it is not a score of “zero bugs found.”

The suite generated with access to both the rules and the app code passed all eight clean-baseline checks and detected three defects planted by the experimenter and hidden from the generating model. Those were actual detections. Each condition involved only one generation, however, so the comparison establishes neither a superior condition nor that code access caused the detections.

Reusing the code-aware set of tests after a feature addition exposed a different limit. The same five of eight checks failed on both the correctly updated app and the version containing a new defect. The checks required the result to have exactly the old structure, so they rejected a legitimately added note field too. Those failures did not establish that the suite could distinguish a regression—a defect introduced by a later change—from a correct update.

A separate check against the expected behavior did distinguish the note-lifecycle defect—a removed item's note remaining when it should disappear—but the generated suite cannot take credit for that result. Its initial detection of the three planted defects still stands.

The record on scheduled agents—AI set up to run tasks on a timetable—supplies a different kind of evidence. It was a local rehearsal that replayed predefined cases, with no real scheduling or AI model calls. A verification program rejected a draft report bound to the wrong data snapshot and set it aside (quarantined it). The rejection came from a verifier applying fixed rules. This was not an observation of AI recognizing incorrect input or of a live scheduling service working reliably.

In both records, producing a result and establishing grounds to accept it were separate operations. My reading is that a test set first needs checking against a known-correct app, and “Did the test fail?” needs a follow-up: “Did it distinguish a correct change from an incorrect one?” For a report, the corresponding questions are which input produced it and who accepted it against which criteria.

Observed: interface checks and untested handoff

The conversational-app experiment displayed an app within a chat interface in the official local reference host on the test computer. It responded to the tested Korean and English inputs, and the total changed from 90 to 100. Stored values also returned when a new view opened within the same running local process and session. These were successful interactions, but they did not establish storage that survives a server restart or transfers to another device.

An unsupported language code produced an actual failure. The server fell back to English with a warning, while the interface fell back to Korean without one. That mismatch belongs alongside the successful input and restoration checks. Three other checks had invalid procedures, and six were not tested. Neither category counts as an app failure. The results also do not establish behavior inside a commercial chat service or prove that the model actually received the content the app attempted to send to it.

The generative-design experiment used one run from a text specification and one from a wireframe drawn and annotated directly on a digital canvas. The wireframe was an experimental input made of shapes and text on a canvas. It did not test a drawing on paper or a novice's experience of making one. Both outputs met the predefined checks for three screens, two navigation paths, and three edits. Text ran first, and the outputs used different selection controls and some different checks for empty input. The pair cannot rank one input method above the other.

The edits also used different mechanisms. The first changed an element directly; the next two regenerated parts of the relevant screens. The final instruction required changing a button's destination while retaining its label. The result followed that instruction, leaving the Korean label 공간 목록으로—“To the space list”—at odds with where the button actually led. The mismatch followed the experiment's instruction; it was not a failure to carry out the requested edit. This does not establish whether the tool noticed the semantic conflict.

For me, the useful question here is how following an instruction relates to what a screen communicates. Even when an edit matches the request, the label and destination can be checked separately. Export was not performed, so transferable code and production readiness were not established. Interaction was observed in the reference host and in the design canvas, but no tested process connected those two cases through deployment or transfer to a phone.

Three records that stopped before execution

The following records did not observe products failing during use. Each stopped while checking a different prerequisite for starting its experiment.

Shut-down simulators listed in the phone inventory are not evidence of a physical-phone run. One route had unproven prerequisites; another was ineligible in that environment. This does not mean no route exists in any environment.

These stops support no verdict on the products' safety or availability, in either direction. The conversational app's language mismatch, by contrast, was a failure observed during execution. Keeping “unknown” separate from “failed” is part of explaining this synthesis accurately.

My judgment: questions to attach to the next request

These records covered reusable procedures, storage, checks on results, and interface behavior. Handoff or deployment connecting the cases was not tested. Reading them together is an editorial interpretation, not evidence of a market transformation or a complete, validated operating workflow.

The records suggest these questions for a subsequent task:

• After which operation, and in which storage location, was the result checked?
• Does the set of tests pass a known-correct version of the app?
• After a feature change, does the check distinguish a correct update from a faulty one?
• Which input and criteria does the person or program accepting the result compare?
• After an edit, does clicking the button lead where its label says?
• Were the intended environment and handoff tested?
• Were stops caused by unmet prerequisites kept separate from product failures?

These are editorial suggestions drawn from the records. Their effectiveness at preventing real-world problems was not measured.

The map pairs each retained observation with what was not tested. Links lead to the earlier published articles for further reading, not raw evidence or verification of current product capabilities. The observations this synthesis relies on, and their limits, are in this article's text and table, not in the links.

AI was used to assist with research and drafting. The author independently verified and edited the final article.

It would be convenient if a directly drawn digital wireframe could immediately become a clickable prototype. This experiment, however, did not use a drawing on paper or a photograph. The comparison used a directly drawn digital annotated wireframe, made by placing shapes and text one by one on a blank local canvas, and a text specification describing the same requirements. For the image route, only the screen-size, accessibility, and local-simulation conditions shared by both routes were added as text. The question was deliberately narrow: not “Which is better?” but “How well can the two inputs implement and revise the same three screens, connections, and validation rules?”

What Google says Stitch can do

Google announced that Stitch can take natural-language and image prompts and generate UI designs and frontend code. It later described a canvas that can use images, text, and code as context, support iterative changes through natural language, and turn multiple connected screens into a prototype. Google has also announced several handoff routes, including CSS/HTML and Figma. But an official description of an available feature is not evidence that a particular input will work as intended. The results below are therefore limited to the direct observations recorded on September 20, 2026.

Direct observation: one run per route, with no retry or repeated run

The experiment used two separate signed-in Stitch projects that the interface showed as private. On the Stitch web interface, the device type was set to App and the model mode to Balanced. The text was entered first and the wireframe second. Each route received one baseline generation and each of the three predefined edits, E1–E3, exactly once. That made eight actions in Stitch in total, with no retry or repeated run.

For this Korean-language test specification, both baseline results created the three screens, six core controls, and two navigation paths. Within the predefined scope, the error states for no time selected, an out-of-range headcount, and a fully booked time slot—and recovery from each—also worked. Both previews passed limited checks of keyboard operation, accessible names for controls, and behavior at 200% browser zoom. However, when the headcount field was emptied, the text result immediately put 2 back, so its empty-value error could not be tested (not_tested). The wireframe result passed the empty-headcount check. The same selection requirement was also implemented differently: the text result used a drop-down menu, while the wireframe result used two radio buttons.

The edits were not all applied in the same way. In E1, the button-label change was applied directly to the relevant element in both routes. In contrast, the added validation message in E2 and the changed screen connection in E3 were partial regenerations that created new design nodes—separate generated versions in the canvas. No unintended changes were seen in the non-target text, paths, or states that were checked, but the experiment did not inspect every pixel or hidden state.

E3 raised a particularly important caution. Both results met the E3 requirement: the button now led to the second screen and kept the selected time and headcount. Yet the button in both results was still labeled 공간 목록으로 (“to the space list”). The behavior changed, but the label no longer matched where the button led. Passing a behavior check and having a label people can understand turned out to be separate matters.

The usage display read 0 / 400 before the run and 12 / 400 at the final observation. Some intermediate values updated late, so the record does not calculate credits per action or assume that the final figure included the last action. No monetary charge, purchase prompt, or trial prompt appeared during the run.

When the image was uploaded, Stitch displayed a notice that uploaded images receive a stable public URL for HTML export. That condition was accepted only for the one independently privacy-checked synthetic wireframe used in this experiment. This observation does not tell us whether the project or prototype appears in a public listing, whether the URL can be discovered, how long the asset is retained, whether it remains confidential, or whether it is used for models. Until the exposure and data-handling terms have been checked separately, it is safer not to upload sensitive or identifiable material.

What the result means: observing a prototype is not verifying handoff

In this single run per route, both inputs met the predefined requirements for the connected prototype and the three edits. But because there were no repeated runs and the generated controls differed, the evidence does not establish that either input method is better. It also does not support a success rate. The keyboard, accessible-name, and zoom checks were only limited observations of the preview, not a full accessibility audit.

The interface included export controls, but they were not used. No files were downloaded, the actual frontend files and dependencies were not inspected, and the code was not run locally. Developer handoff—turning the design into files a developer can actually use—therefore remains not_tested: it was not tried, so it neither passed nor failed. Official announcements and visible buttons cannot fill that evidence gap.

A checklist for trying a similar tool

• Define the same screens, text, paths, and error conditions for both inputs before you begin.
• Specify both what should change and what must not change.
• Try empty values, out-of-range values, normal recovery, keyboard operation, and zoomed states yourself.
• Distinguish a direct change to an element from partial regeneration.
• Before entering anything, check how credits are counted and what the image-URL and data-handling terms say; do not upload sensitive information until those terms are clear.
• Record preview success separately from file-handoff verification.

Generative UI canvases are intriguing tools for trying out a connected flow in a preview. But even after the result appears to work, you still need to record whether its labels match its actual behavior and which steps remain untested. The most useful output of this experiment was not a winner’s table. It was that record of where the evidence stops.

AI was used to assist with research and drafting. The author independently verified and edited the final article.

I stopped before generating the app because neither of the two preselected phone routes had been shown to be ready. As a result, I never reached the stage of running anything on a physical phone.

This is therefore not a story about a prompt-built app succeeding or failing on a phone. It is a report on limits: what should be checked before generating code, so that a preview is not mistaken for evidence from a physical phone.

“I made an app” hides five different stages

A generated screen and a convincing preview can feel like a finished app. Once a physical phone becomes the standard, however, the process separates into five stages:

1. Generate: an AI turns the requirements into source code.
2. Build: package the code in a form the phone can accept.
3. Install or open: install an app-specific development version, or open the project inside the already-installed Expo Go app.
4. Reach the phone: the exact code version just generated appears on a physical phone.
5. Validate usability: repeat the flow with fingers, the real keyboard, long Korean text, and defined reopen conditions.

Expo's build-with-AI tutorial describes an official path from an AI coding agent to opening a project on a phone. Its development guide explains how a physical device can open a project from a development server using a QR code. However, Expo's workflow overview makes clear that Expo Go is a limited environment for learning and prototyping. Opening a project there is not evidence of a store-ready build.

The physical-phone test I planned

The planned test app was an offline Korean checklist with no server, login, personal data, or permission request. The frozen plan called for adding, editing, completing, and deleting fixed, predefined items across three runs.

It would then check whether the save button could be tapped five times with the keyboard open; whether the main tap targets met the finger-size guidance of 44×44 points on iOS or 48×48 dp on Android guidance; whether long Korean text and text mixing Korean, Latin letters, and numbers displayed without breaking. Switching the app to the background, reloading its screen, and reopening it three times after a full quit were treated as separate items.

Those numbers are not results. They are criteria frozen before execution. No physical-phone screen, tap record, or saved-state result was collected. The planned exploratory feedback from two people was also not collected.

What the pre-run check found

On September 13, 2026, I checked only the presence of local tools and configuration without changing the environment.

A simulator—a virtual phone on the computer screen—can show some of how screens look, but it cannot answer this article's physical-phone question. It cannot stand in for finger operation, the phone's keyboard, or the state of a physical device. I did not boot one to fill the evidence gap.

One route was unproven; the other was ineligible in this environment

The first route required Expo Go to be already installed on a phone and usable over the same LAN. Whether Expo Go was on the phone at all, whether its version matched the project, whether it would connect directly over the same network without an external relay, and whether the screen could be recorded without personal information appearing—all of this remained unknown.

The second route required an already prepared Android USB-debugging setup. The current environment did not contain the Android SDK, ADB, or a usable Java runtime, so using that route would have required installing or configuring tools. I also did not check whether a phone was set to trust this computer.

The decision was do not start, because the ability to run the test was not proven (NO_GO_CAPABILITY_NOT_PROVEN). This is different from a generated app being blocked on its way to a phone or its function turning out wrong. There was no first generated code version to test at all.

Why I did not manufacture a substitute

Running the Expo command or installing Android tools might have opened the next step, but it could also download packages and change the environment. Unlocking, connecting, pairing, trusting, or granting permissions on a phone is a separate device action. Build and distribution services that require an account or signing, and app-store routes, were outside this test's safety boundary.

I also did not make a static screen image or a mock app on the computer. Such things could illustrate an interface, but they would not show that the exact prompt-built revision ran on a physical phone. A QR code, build log, Expo Go home screen, or simulator image also cannot stand in for evidence that the generated app's screen was actually reached on a physical phone.

Three checks to make before generating

If you plan a similar test, separate these three questions first:

• Route: Is an app such as Expo Go already installed? Can you build the project for an SDK version it supports? Can you record whether the connection is direct on the same network or passes through an external relay?
• Boundary: Where will you stop if the path asks for login, payment, signing, store access, a new permission, or device trust?
• Evidence: Did the exact code version just generated reach its app screen on a physical phone? Only then should function, keyboard, touch, Korean text, and reopen behavior be assessed.

The smallest next condition is a limited check on a physical phone that already has Expo Go: confirm its version and a connection over the same local network, with no installation, login, payment, or new permission. After that, a project built for a matching SDK version and the ability to record the screen without personal information appearing would also have to be shown. If Expo Go is not already present, that condition simply is not met; this is not an instruction to install it. These are conditions for a future run, not actions performed for this article.

The title question therefore still has neither a “yes” nor a “no.” Even a future pass would apply only to one recorded phone and setup on that date. The lesson this halted test supports is narrower: evidence for the physical-phone route has to come before the generate button is pressed, and when that evidence is missing, the honest result is to stop rather than invent an app outcome.

AI was used to assist with research and drafting. The author independently verified and edited the final article.

Trusting a scheduled run should not mean that its report merely sounds plausible. In this article, trust means being able to reconstruct which input was used, whether the job stayed within its declared authority, whether its result passed the check, and whether it stopped when required.

The reason for this distinction is simple. A registered schedule, a finished run, a written report file, and a report you may trust are four different states. OpenAI’s Scheduled tasks guidance, checked on September 12, 2026, separates the saved prompt from the trigger and describes conditions for desktop tasks that use local files. It also recommends narrow access and review of early runs. The checked page does not establish output correctness, general retry or overlap behavior, a per-run spending cap, or universal mid-run cancellation. I therefore rehearsed the operating controls with synthetic data and a logical clock—a fixed event list that advances without waiting for future wall-clock time—instead of enabling a real schedule.

Yesterday’s report can still look convincing

No real AI was called. The central test used a canned candidate to produce a deliberately stale but internally plausible report. The verifier expected snapshot SYN-20260902; the candidate actually read SYN-20260901. The report based on the older file did not look obviously wrong.

Writing, however, was not acceptance. The verifier compared an independently frozen expected snapshot ID and input hash with the identity reported by the candidate. Its verification record contained binding_mismatch. The candidate never reached the accepted directory; it moved to quarantine, while the result a report reader would see was a rejection record labeled rejected_binding_mismatch.

This is a planted control, not a reliability rate. It shows that this verifier rejected the known stale input. It does not show that an AI recognized staleness.

What was tested without a real schedule

The experiment used a canned deterministic candidate and made no external model call. Its purpose was to inspect the recurring-job wrapper and verifier, not a real AI’s writing ability. A driver, standing in for the trigger, advanced through a finite list of logical events without waiting for future wall-clock time. The candidate received the selected synthetic JSON and run identifiers. A separate verifier used independently frozen bindings and a content oracle to check input identity and exact counts and changes. Only when that check passed did a writer promote the staging result to an authoritative accepted artifact.

Declared authority, enforced channels, and recorded observations were kept separate. The task specification declared allowed inputs and outputs. The local wrapper exposed no network, browser, subprocess, scheduler, or provider capability to the candidate. The ledger recorded decisions visible to that wrapper. This does not prove operating-system isolation or a vendor sandbox. Those restrictions applied to the candidate; the wrapper itself wrote files. The run was read-centered, not read-only, and wrote baseline, staging, verifier, accepted, rejected, quarantine, ledger, inventory, and summary artifacts inside one isolated run root.

Results across the 12 cases

Each frozen case was scored against its own expected outcome or outcomes, not averaged into a trust score.

• Accepted: the fresh baseline, changed current snapshot, and new-day no-change report passed verification.
• Blocked before candidate work: missing and malformed sources failed, and redelivery of the same operation created no second artifact.
• Conflict and retry: an overlapping request with a different binding was recorded as a conflict while the original binding remained accepted. A planted transient post-staging failure was quarantined, then accepted after one simulated retry. A persistent error stopped at the retry ceiling; a non-retryable error stopped with its separate stopped_nonretryable disposition.
• Separate stops: C10’s planted case-local budget refusal occurred before start with zero candidate calls and produced stopped_local_budget. Future-run prevention, active-local interruption, and simulated read-policy revocation each left a distinct record.

All 12 logical cases matched their frozen expected dispositions. Candidate invocations were the planned 11, below the whole-series hard ceiling of 20. C10’s pre-start refusal did not consume or prove that ceiling. The finite replay took 25.229 milliseconds, and the hash-chained 47-event ledger verified. That timing describes a small local fixture and canned candidate, not the speed of a real reporting job.

“Stop” was not a single button

Preventing a future run was confirmed by a marker persisted before the due event and by the absence of later work. Interrupting an active local step was confirmed by a request and acknowledgement, quarantined partial output, and no promotion. Simulated local read-policy revocation evaluated the new policy hash before source access and created no cached fallback. None of these tested a vendor queue, remote cancellation, or account-permission revocation.

Why the cost record has five lines

• Provider calls: 0 at the wrapper boundary
• Additional-charge observation: not_checked, because no billing surface was accessed
• Usage value: unavailable
• Local invocation ceiling: 20, controlled only by this driver
• Provider-enforced cap: not verified

That record does not support a “zero cost” claim. The local ceiling of 20 candidate invocations was not a provider budget cap. You can enforce only the limits you actually control, and the record must say so.

Seven questions before enabling recurrence

Before enabling a recurring automation, try answering each of these in one line:

1. Exactly which input does it read?
2. Which actions are allowed and forbidden?
3. Where are candidates kept separate from accepted results?
4. What evidence defines success?
5. Which spending or invocation limit can you actually enforce?
6. Which errors may retry, and how many times?
7. How do you separately prevent future runs, interrupt an active step, and revoke reading?

A blank answer is an operating-design gap, not a model-intelligence problem. The conclusion here is deliberately conditional: this local wrapper and verifier behaved as specified on the frozen synthetic cases. NOT TESTED: AI output behavior; real scheduling; vendor trigger, retry, overlap, cancellation, or delivery behavior; OS or vendor isolation; distributed exactly-once delivery; zero cost; a provider spending cap; or readiness to activate a real job.

AI was used to assist with research and drafting. The author independently verified and edited the final article.