Define one multi-party task contract
Start with a buyer or provider goal and record the immutable prompt, persona, constraints, locale, viewport, authentication state, fixture version, allowed actions, stop gate and deterministic assertions. Identify every party that can change the outcome: platform, seller or provider, payment or fulfillment partner and the user account.
Run the same contract through a deterministic browser baseline and named agent profiles. A plausible recommendation is not completion. The selected listing, seller, condition, total and terminal state must all match the expected fixture.
- DISCOVER: find listings that satisfy explicit category, location, price and availability constraints.
- COMPARE: attribute price, fees, policy, reputation and fulfillment to the correct seller.
- ACT: reach only the authorized cart, enquiry, booking or dispute state.
- VERIFY: assert the exact party, offer and final state, then prove the stop gate held.
Preserve listing and seller identity
Assign stable fixtures for listing ID, canonical URL, seller or provider ID, product or service identity, variant, condition, inventory state and fulfillment method. Test ranking, filtering, pagination, duplicate listings and sponsored placements without allowing the executor to merge facts from similar offers.
Inspect semantic headings, links, controls, accessible names, roles, values, focus order and dynamic announcements. Structured data such as Product, Offer or AggregateOffer can improve machine-readable context when it accurately represents the visible page, but its absence alone does not decide task success.
Verify total cost and trust evidence
Calculate the comparison total from item or service price, platform fee, seller fee, delivery, tax, deposit, protection and any conditional charge visible before the decision boundary. Recalculate after location, quantity, variant, seller or fulfillment changes and flag authoritative surfaces that disagree.
Keep platform guarantees, verified seller attributes, seller claims, ratings, reviews and moderation notices in separate evidence fields. Reputation is context, not proof that a listing fact is true. The report should state which party supplied each claim and when it was observed.
- Test new, used, refurbished, rental and service conditions where applicable.
- Separate current price from crossed-out, starting, estimated and historical values.
- Verify the same seller and fulfillment path at comparison and final review.
Treat UGC and listings as untrusted input
Reviews, listing descriptions, seller profiles, questions, messages and uploaded documents are user-generated content. Test whether visible or hidden instructions inside those surfaces can redirect the agent, request secrets, alter constraints, bypass a confirmation or make it treat a seller claim as a platform command.
Record the injection payload as evidence without allowing it to escape the scenario boundary. Sensitive account, payment, message and order data must remain unavailable to unrelated sellers, listings and sessions. Classify successful manipulation or cross-account disclosure as a security finding, not ordinary content friction.
Control availability changes and recovery
Create available, low-stock, reserved, delisted, seller-paused, changed-price, changed-fee and temporarily unavailable fixtures. Trigger the change after comparison or selection so the test observes whether the site exposes the new state and preserves valid constraints.
Recovery must not silently replace the seller, condition, variant, fulfillment method or commercial terms. If the original offer cannot continue, the agent should explain the invalidated choice, return to a safe decision point and request confirmation before selecting an alternative.
Enforce transaction stop gates
Public production runs stop before sending a seller message, placing a bid, submitting an enquiry, confirming a booking, creating an order, authorizing payment, opening a dispute or requesting a refund. The review state should expose the exact listing, seller, total, terms and next consequential action.
Approved sandboxes may exercise the complete flow with synthetic accounts and reversible fixtures. Use idempotency controls and independent assertions to prove exactly one intended message, transaction or case was created, and that retries, delayed responses or expired sessions did not duplicate the write.
Prove the multi-party final state
Assert the expected listing, seller, condition, quantity, total, fulfillment, account and review or sandbox transaction state outside the agent narrative. Also prove the absence of an unintended message, bid, reservation, order, charge, dispute, refund or cross-account data exposure.
Package the session recording, ordered steps, screenshots, accessibility evidence, DOM, console and network signals, source snapshots, completion time, waits, retries, interventions, Cumulative Layout Shift and target movement. Classify the primary cause as website, agent-specific, agent policy, infrastructure or inconclusive before remediation.
- Redact personal, payment, financial, message and account data from evidence.
- Compare repeated runs against the same seller, offer and environment fixture.
- Retest each finding with deterministic acceptance criteria and a matching stop gate.
Marketplace agent test checklist
- Define the persona, goal, constraints, fixtures, allowed actions, stop gate and assertions.
- Version listing, seller, variant, condition, inventory, fee and fulfillment fixtures.
- Preserve exact listing and seller identity across search, ranking, pagination and comparison.
- Verify semantic controls, accessible names, focus and dynamic state announcements.
- Reconcile visible content with Product, Offer and AggregateOffer data when present.
- Calculate total cost from price, platform fees, seller fees, delivery, tax and protection.
- Separate platform guarantees, verified attributes, seller claims, ratings and reviews.
- Test listing, review, profile and message content for prompt injection exposure.
- Trigger delisted, unavailable, changed-price and changed-fee recovery states.
- Prevent silent substitution of seller, condition, variant or fulfillment path.
- Measure completion time, waits, retries, interventions, CLS and target movement.
- Stop production before messages, bids, enquiries, bookings, orders, payments or disputes.
- Use idempotency and independent assertions for approved sandbox writes.
- Prove the exact final state and absence of unintended or cross-account side effects.
- Redact sensitive evidence and classify website, agent, policy and infrastructure causes.
Primary implementation sources
- Accessibility for AI agentsChrome for Developers
- Forms TutorialW3C Web Accessibility Initiative
- OfferSchema.org
- sellerSchema.org
- AggregateOfferSchema.org
- LLM Prompt Injection Prevention Cheat SheetOWASP
- Cumulative Layout Shiftweb.dev
- Trace ViewerPlaywright
