跳到主要内容

Coding Agent Browser Testing: What to Check Before You Trust a UI Patch

By Youguang

I made the complete anonymous fixture available so you can reproduce both browser states: download ui-verification-lab.zip.

I built the ui-verification-lab repository as a bounded synthetic experiment. It contains no production code, no real user data, and no real orders. Every identifier — demo-001, the editor and viewer roles, the localhost routes — is fake and disposable. The only reason it exists is to make one uncomfortable sequence of results reproducible in writing.

The sequence starts like this: two Node tests pass, the static build completes cleanly, the known Retry endpoint returns HTTP 200, and I sit there looking at a document whose root scrollWidth is 632 pixels inside a 390-pixel viewport. The green checks still missed the failure. And yet the page was broken in the most visible way imaginable: it overflowed horizontally by 242 pixels on a mobile-width screen.

That gap — between what deterministic tests can see and what a rendered page actually shows — is the reason this article exists.

Why green tests are not a UI receipt​

When a Coding Agent modifies frontend code, it typically produces some combination of a passing test run and a successful build. Both are valuable. Neither tells you anything about these questions:

  • Does the page fit the viewport it was designed for?
  • Does the button a user is supposed to press actually call the right route?
  • Does the browser console show an error the tests never touched?
  • Is session state from a prior run leaking into the next one?

These questions live in different evidence layers. A logic-only Node test runs JavaScript logic in isolation; it does not open a real browser, paint a layout, or fire a real network request to a real handler. A static build verifies that the bundler finishes; it does not verify that the finished output renders correctly at any given screen width. An HTTP handler test verifies that a route returns the right status code when called directly; it does not verify that a button on screen calls that route rather than a different one.

The synthetic experiment made all four of those evidence gaps concrete at once, deliberately, so they could be resolved one by one with recorded evidence.

The broken state, named​

The broken fixture served at this local pattern:

http://127.0.0.1:4173/?mode=broken&role=editor

I opened it in a disposable browser session named ui-lab-broken, set the viewport to 390 × 844, and took an accessible page snapshot to confirm the intended control was present before touching anything:

agent-browser --session ui-lab-broken open \
  'http://127.0.0.1:4173/?mode=broken&role=editor'
agent-browser --session ui-lab-broken set viewport 390 844
agent-browser --session ui-lab-broken snapshot -i

The snapshot confirmed a Retry status check button was reachable. Then I measured the viewport and document widths:

agent-browser --session ui-lab-broken eval \
  '({ innerWidth: window.innerWidth, scrollWidth: document.documentElement.scrollWidth })'

Result:

{
  "innerWidth": 390,
  "scrollWidth": 632,
  "role": "Editor",
  "result": "Retry has not run."
}

Window.innerWidth is the layout viewport's interior width in pixels. Element.scrollWidth measures content width including content not visible because of overflow. When the root document's scrollWidth is 632 and innerWidth is 390, the page has 242 pixels of invisible horizontal overflow. In this fixture, that mismatch directly confirmed that the long identifier the Coding Agent introduced was not wrapping.

The two Node tests said nothing about this. They cannot; they have no viewport.

Following the one Retry interaction​

Before clicking anything, I cleared the network and console records so that only evidence generated by this specific action would appear afterward:

agent-browser --session ui-lab-broken network requests --clear
agent-browser --session ui-lab-broken console --clear
agent-browser --session ui-lab-broken click '#retry-button'
agent-browser --session ui-lab-broken wait 300
agent-browser --session ui-lab-broken network requests --filter '/api/orders/' --json
agent-browser --session ui-lab-broken console --json
agent-browser --session ui-lab-broken errors --json

The clear-before-action step matters. Without it, any console error or network call that happened during page load mixes with the evidence from the button click. Clearing first makes the record a receipt for exactly the action under test.

The result was unambiguous. The visible status on the page remained at Needs attention. The network record showed one synthetic POST — but not to the valid route. It went to:

POST /api/orders/demo-001/retry-status   →   HTTP 404

The valid Retry endpoint is POST /api/orders/demo-001/retry. The Coding Agent had wired the button to the wrong path. The console showed one controlled synthetic retry error. The page error list was empty. None of this appeared in the test run because the test called the handler directly; it never asked a browser button which URL it intended to use.

This is the exact failure mode that the existing HTTP tests did not catch: the correct handler exists and passes its test, while the button on screen quietly points somewhere else.

Session state carried over​

I checked storage in the disposable fixed-editor session. The disposable editor session had stored the fake value editor when the page loaded. Reopening the fixed route in that same session — without a role parameter — retained editor. That is expected behavior for local storage, but it is also a trap: if an agent reruns verification in the same session it used for a prior broken run, it may observe behavior that depends on leftover state rather than a clean start.

I opened a separately named disposable viewer session for the isolation check:

agent-browser --session ui-lab-fixed-viewer open \
  'http://127.0.0.1:4173/?mode=fixed&role=viewer'

The viewer session stored and displayed fake value viewer, independent of the editor session. The two sessions did not share state. This is the behavior Playwright describes for browser contexts: each context is an isolated clean-slate environment with independent local storage, session storage, and cookies. Multiple contexts can represent different users in one scenario.

The fixture's recorded result does not test real authentication, authorization, cookies, or user switching. It demonstrates the recorded browser-session boundary in a synthetic scenario. That is a more limited claim, and it is the accurate one.

The fix and the green run​

The bounded fix addressed three things: the long identifier was given a wrapping rule, the button was wired to the valid fake Retry route, and the verification rerun used a separate disposable session rather than the prior broken one. I ran the same command sequence against:

http://127.0.0.1:4173/?mode=fixed&role=editor

Measurement after Retry:

{
  "h1Count": 1,
  "innerWidth": 390,
  "mainCount": 1,
  "result": "Synthetic retry succeeded.",
  "role": "Editor",
  "scrollWidth": 390,
  "status": "Ready"
}

Root scrollWidth matched innerWidth at 390. The visible status was Ready. The network record showed one synthetic POST to the valid local route, HTTP 200. The console after clearing before the action: no new messages. The page error list after the action: none.

One H1 and one main are narrow structural observations; they were recorded as post-action structural checks, not as a substitute for an accessibility audit. An empty console list and an empty error list are evidence items, not proof that the page is free of all defects.

Here is the before/after comparison:

Evidence fieldBrokenFixed
Route?mode=broken&role=editor?mode=fixed&role=editor
Viewport390 × 844390 × 844
Session preconditionDisposable, freshDisposable, fresh
ActionClick #retry-buttonClick #retry-button
scrollWidth before Retry632390
Visible status after RetryNeeds attentionReady
Network: route called/api/orders/demo-001/retry-status/api/orders/demo-001/retry
Network: HTTP status404200
Console after clearOne synthetic retry errorNo new messages
Page errors after actionNoneNone
Session isolation (editor/viewer)Not separately verifiedIndependent state confirmed

The evidence contract​

The value of the clear-before-action discipline is that it turns a browser check into a statement with a known scope. Without it, "I checked the browser" is vague. With it, you can say: in a fresh disposable session, at 390 × 844, after clearing network and console records, one click on this specific element called this specific route, received this HTTP status, left this visible state, produced these console messages, and produced these page errors.

A compact contract for each browser verification run:

FieldWhat to record
RouteExact synthetic or approved test route
ViewportWidth × height used for the check
Session preconditionFresh context, stored role, or other expected starting state
ActionThe click, form submission, navigation, or reload under test
Expected stateThe visible result and allowed request outcome
Observed stateWhat rendered after the action
Console / networkNew messages and relevant request method, route, and HTTP status
Visual evidenceScreenshot or width measurement when it answers a real question
DispositionPass, fail, needs investigation, and remaining limits

This contract can be renamed or compressed to fit a team's process. The important thing is that it leaves an artifact — a file, a comment, a structured note — that shows what was checked, not just that a check was done. An agent summary that says "verified in browser" without these fields is not a receipt; it is a promise.

What this result does not prove​

The recorded green run is real and bounded. It does not prove:

  • Cross-browser rendering. One Chromium run at one viewport is one data point. It says nothing about Safari, Firefox, other screen widths, or zoom levels.
  • Accessibility. Recording one H1 and one main is a structural observation. It is not a WCAG audit, a keyboard-navigation test, or a screen-reader test.
  • Performance. No timing, no bundle size, no paint metric was collected.
  • Security. Separate disposable sessions with fake roles do not test real authentication or authorization. The session boundary was demonstrated; the authorization boundary was not.
  • Correctness of the underlying logic. The fixture's handler returns fake HTTP 200. A real handler might return 200 for the wrong reason. Browser verification confirms what the UI receives and how it responds; it does not substitute for logic tests that verify what the handler computes.
  • Full responsive quality. One 390-pixel snapshot in Chromium is not a responsive test suite.

These limits are part of the receipt, not disclaimers added afterward. Every evidence packet should name what it covers and what it leaves open.

Before you accept the next UI patch​

The synthetic experiment is finished, but the situation it represents — an agent modifies a frontend component, tests pass, and something in the rendered page is quietly wrong — is not synthetic at all. The next time a Coding Agent produces a UI change, run through this checklist before accepting it:

  • Layout check at the target viewport. Compare root scrollWidth with clientWidth at the intended minimum width. Excess indicates horizontal overflow.
  • Accessible snapshot before clicking. Confirm the element the agent describes as "the button" or "the link" is actually reachable in the rendered page before testing it.
  • Clear before the action. Zero out the console and network log before the specific interaction under test.
  • Identify the route actually called. After the click, inspect the network record. Confirm the method, path, and HTTP status match what the agent described — not just that a request went out with a 200.
  • Read the console record. An empty list after clearing is evidence. A list with one error is also evidence. Both are more useful than a check that never cleared first.
  • Check the page error list. This is a separate surface from the console. An uncaught exception may appear here without appearing as a logged console message.
  • Use a fresh session for each role. Do not reuse a session that touched a broken route. Name your sessions after their purpose; the name makes the precondition explicit.
  • State what the check does not cover. Add a disposition line that names remaining limits.

The evidence they produced was more specific than the test run, more specific than the build log, and more specific than any agent summary that does not include recorded values.

References​