Session replay needs a verified outcome.

A session replay should reconstruct the observable path from an immutable goal to a terminal state. It records actions, navigations, tool calls, waits, policy events and visible agent messages. It does not expose hidden chain-of-thought, and the agent's own completion message is not a success criterion.

Each run therefore ends with deterministic assertions against the website or sandbox state. The replay helps a reviewer locate friction and root cause; the assertion report decides whether the expected outcome exists.

Observability differs by executor. Use a small capability ladder and record the maximum level actually available, not the level the product wishes it had.

The primary user-facing duration runs from goal delivery to independent terminal verification. A separate run duration includes queueing, evidence flush and cleanup. Monotonic clocks measure elapsed time; wall-clock timestamps only correlate systems.

Keep startup, navigation, network, DOM settling, tool waits, policy approval, retries and unexpected human intervention visible. Compare agents only across the same scenario, fixture, release, locale, viewport, network profile and capture overhead. Publish latency beside success rate so a fast failure cannot look efficient.

Standard CLS describes unexpected visual movement during a document lifecycle. For an agent, the critical question is whether a control moved between perception and action. Record the nearest step, affected element, before and after rectangles, screenshots and the suspected trigger for each relevant shift.

Keep standards-based CLS separate from custom action-window shift sums, target displacement and misclick counts. A shift after recent input may be excluded from CLS while still disrupting the next agent action. Unsupported browser measurements must be labeled unsupported, never stored as zero.

Prefer synthetic fixtures and masked regions before capture. Remove credentials, tokens, cookies, payment data, financial or medical records and direct identifiers from screenshots, events, DOM snapshots, URLs and network payloads. A client report should link only to sanitized artifacts with verified hashes.

Set raw and sanitized retention periods, authorized viewers and deletion rules before testing. Evidence with pending or rejected redaction stays out of reports. Broad production recording with uncontrolled personal data is not an acceptable shortcut for a sandbox.

A browser-neutral manifest should index the normalized event stream, step traces, screenshots, video, DOM and accessibility snapshots, console and network logs, performance trace, layout-shift log and final assertion report. Each artifact needs a stable ID, timestamp, source, hash, redaction state and retention rule.

Controlled Playwright traces are useful because one viewer aligns actions, screenshots, DOM snapshots, console, network and timing. Preserve a normalized event stream as well so historical results remain reviewable outside a single tool. Run standard scenarios three times and critical ones five times to separate a reproducible defect from a one-off event.