Test the buying task, not one click.

Start with a complete customer goal such as finding a compatible product under a budget, selecting an available variant, confirming delivery and reaching cart review. Do not reduce the scenario to locating one button. The task contract records persona, immutable prompt, starting URL, locale, viewport, cookies, allowed actions, stop gate and independent final-state assertions.

Keep the contract stable across a deterministic browser baseline and named agent profiles. Standard tasks run at least three times and critical tasks five times. This reveals whether success is repeatable rather than a lucky path through dynamic inventory or personalization.

A reliable scenario needs known test data. Create fixtures for one available product, one unavailable variant, a stale or changed price, a delivery constraint, an expired session and a recoverable validation error. Record the expected facts and lifecycle window so a later inventory change is not mistaken for agent failure.

Use synthetic accounts, addresses and payment instruments in an approved sandbox. Public production checks can inspect policies and build a reversible cart, but must not create an order, trigger a fulfillment workflow or reserve scarce inventory unless the partner has explicitly authorized that action.

Compare the product name, identifier, variant, current price, discount context, stock state, delivery promise and return policy across visible HTML, structured data and the cart. Product and Offer structured data can improve machine understanding, but it becomes a defect signal when it disagrees with the customer-facing state.

Confirm canonical URLs, headings, labels and update timing. When facts load only after client rendering, preserve the DOM, accessibility tree and relevant network response at the point the agent made its decision.

Inspect native semantics, accessible names, roles, values and state changes for swatches, selectors, quantity controls and add-to-cart feedback. Keyboard and controlled-browser runs should reach the same selection without relying on coordinates or color alone.

After each transition, assert the selected product ID, variant, quantity, displayed price and cart line. Delayed feedback must not cause a retry to add the same item twice. Where an action endpoint is available, verify idempotency or a comparable duplicate-action guard.

Change the fixture after discovery: make the chosen variant unavailable, update the price, expire the session or reject the delivery postcode. The agent should surface the changed fact, preserve valid constraints and ask for a decision rather than silently choosing a replacement.

Measure completion time by phase, including rendering, inventory waits, retries and interventions. Capture Cumulative Layout Shift and target movement around variant, delivery and cart controls because late inventory content can move an otherwise correct target between observation and action.

The production terminal state is cart or order review before payment submission. The runner must record that the intended line items, quantities, variants, totals and delivery context are present, then stop. The assertion should also prove that no order, charge or fulfillment side effect was created.

An approved sandbox may continue with synthetic data, explicit authorization and cleanup rules. Confirmation boundaries belong in both the scenario definition and executor policy so page content cannot override them.

Store the immutable prompt, fixture version, executor profile, timestamps, step trace, screenshots, accessibility snapshots, console and network signals, CLS entries, target rectangles and final-state assertion in one run record. A trace viewer helps engineering replay actions while deterministic assertions decide the outcome.

Classify the primary cause as website, agent-specific, agent policy, infrastructure or inconclusive. Remediation is complete only when the same scenario and fixture pass the baseline and relevant agents without an unexplained regression in time, retries or final state.