The evidence chain
Evidence is not a folder of screenshots. It is a linked chain from the audit manifest to a run, step trace, artifact, assertion and finding. Stable IDs and timestamps let a reviewer reconstruct what happened without relying on the agent's own explanation.
Critical runs should preserve the highest capture level the executor permits. A controlled browser can usually provide synchronized video, screenshots, DOM snapshots, accessibility trees, console logs, network logs and performance traces. A hosted agent may expose only video and observed events. The report states that limitation instead of inventing missing data.
Every artifact records its environment, source run, capture time, content type, integrity hash, redaction state and retention class.
Session replay
A useful replay aligns visual recording with normalized events. Reviewers can move from a visible hesitation or unexpected navigation to the exact action, selector or accessible name, URL transition, assertion and related technical trace.
The replay manifest should include:
- goal delivery, first action and first relevant result markers;
- each planned and unplanned interaction;
- retries, recovery attempts and human interventions;
- stop-gate or sandbox commit marker;
- independently verified terminal state;
- capture limitations, masking rules and recording overhead.
Video alone cannot prove that a backend state changed. Final-state assertions should use an independent source when available, such as an order fixture, account state, API response or controlled database read.
Completion time
The primary timing measure is task_elapsed_ms: time from goal delivery to the independently verified terminal state. It excludes planned approval waiting, cleanup and recording overhead when those values can be separated.
Timing is decomposed so a team can see where work accumulated:
| Metric | What it explains |
|---|---|
| Time to first action | Agent interpretation and initial planning delay. |
| Time to first relevant result | Discovery, navigation and rendering delay. |
| Active interaction time | Time spent acting on controls. |
| Navigation and render wait | Site response and client rendering delay. |
| Recovery time | Time spent handling validation, expired sessions or unavailable state. |
| Time to verified outcome | End-to-end user-visible completion cost. |
Compare timing only across compatible scenarios, fixtures and executor profiles. Median, p75 and p95 remain diagnostic metrics even when efficiency is not part of sign-off. A fast incorrect run is still a failure.
CLS and target stability
Chromium runs collect standard Cumulative Layout Shift using the browser session-window algorithm. Entries associated with recent user input are excluded from CLS, as required by the metric definition. A raw sum that includes those entries is stored under a different name and is never presented as CLS.
Standard CLS is page-level. Agent interaction also needs local context, so the audit records target displacement and layout shifts around each planned click or input. These custom measures do not use Core Web Vitals thresholds. They affect readiness only when evidence connects movement to a missed action, wrong action, retry or exceeded project-specific budget.
Lab CLS, field p75 CLS from RUM and action-window target movement remain separate values. An unsupported measurement has an explicit unsupported state, not a numeric zero.
Redaction and retention
Evidence is reviewable only after sensitive content is controlled. Redaction applies before an artifact enters a report-safe bundle.
- Credentials, tokens and session identifiers are removed completely.
- Email, phone, address and customer identifiers use stable aliases where correlation is required.
- Payment values remain synthetic; only approved last-four display may remain in sandbox evidence.
- Medical, financial and identity documents are excluded from screenshots by default.
- DOM and accessibility captures receive the same review as images and video.
- Access is role-based and logged, with a purpose-limited retention period.
If safe redaction would destroy the evidence needed to support a finding, the artifact stays restricted. The public or customer-facing report receives a sanitized assertion summary and records the limitation.