Headless guide

Can AI agents shop a headless store?

Yes when product facts are in the HTML your server sends, which Next.js, Nuxt and Hydrogen do by default, but headless stores commonly lose what the old theme gave them for free: structured data, sensible robots rules, policy pages and plain links.

In a headless store your team owns the storefront: rendering, robots.txt, metadata, structured data and routing are all code. The commerce backend, whether Shopify, Medusa, commercetools or your own, only supplies the data.

Request a demo Check AI crawler rules free

Reviewed 21 October 2026 against official documentation. 21 sources

Example: what an agent runs into on headless stores

Common problems

An illustrative store, not a real result. Your own result shows only what is really on your store.

Rendering: Next.js and Nuxt render on the server by default. Data fetched in the browser after load isn't in the first HTML.
robots.txt: A file or route you own: app/robots.ts in Next.js, the robots module in Nuxt, a route in Hydrogen.
Structured data: Only what you add. Next.js recommends a script tag in the page; Medusa's starter ships none.
Agent protocols: Hydrogen adds UCP and MCP endpoints for Shopify-backed storefronts. Elsewhere, you build them.

What usually breaks on headless stores

Frameworks render well by default. What breaks is usually the part a theme used to handle, which nobody rebuilt.

01
Prices and stock load in the browser
Next.js renders pages as Server Components and prerenders client components into the first HTML, and Nuxt renders on the server unless a route turns it off. What's missing is data a component fetches in the browser after load, such as a live price or stock call, and any route with server rendering switched off.[1][7]
02
Structured data didn't survive the replatform
Next.js's own JSON-LD example has a name, image and description, but no offers, and Medusa's storefront starters (the archived nextjs-starter-medusa and its successor, dtc-starter) ship no JSON-LD and no app/robots.ts; their next-sitemap config isn't installed or run. Price, stock, shipping and returns markup exist only if your team adds them.[2][12][13]
03
A starter's robots.txt blocks your policy pages
Hydrogen's older skeleton template serves a robots.txt route modelled on Shopify's old defaults, with Disallow: /policies/ and Disallow: /search in the default * group. Every crawler without a group of its own follows it, so stores built on it tell most compliant crawlers to skip the refund and shipping policies.[9]
04
Metadata can land outside the head
Since Next.js 15.2, metadata from a dynamic generateMetadata can stream into the body for bots that Next.js doesn't treat as HTML-limited. Its default list names no AI crawlers, so their title and description tags may not arrive in the head.[4][5]
05
Links that only work with JavaScript
Navigation built from click handlers on spans or divs, instead of real links with an href, leaves an agent that reads the page without a way to reach collections, products or policies.

How to fix it on headless stores

All of these are a few lines in the storefront. The examples use Next.js App Router; the same ideas apply in Nuxt, Remix or Astro.

Check which AI crawlers your robots.txt allows, free

Render price and stock on the server, with markup

Fetch the product in the page itself, not in a browser effect, so price and stock are in the first HTML. Add Product markup with one Offer per variant, as Next.js recommends, in a script tag escaped against injection. Use revalidation, or swr and isr route rules in Nuxt, to keep prices fresh.[2][7][17]

app/products/[handle]/page.tsx

export default async function ProductPage({ params }) {
  const { handle } = await params
  const product = await getProduct(handle) // on the server, in the first HTML

  const jsonLd = {
    '@context': 'https://schema.org',
    '@type': 'Product',
    name: product.title,
    image: product.images,
    description: product.description,
    offers: product.variants.map((variant) => ({
      '@type': 'Offer',
      sku: variant.sku,
      url: `https://example.com/products/${handle}?variant=${variant.id}`,
      price: variant.price,
      priceCurrency: variant.currency,
      availability: variant.inStock
        ? 'https://schema.org/InStock'
        : 'https://schema.org/OutOfStock',
      hasMerchantReturnPolicy: { '@id': 'https://example.com/returns#policy' },
    })),
  }

  return (
    <main>
      <script
        type="application/ld+json"
        dangerouslySetInnerHTML={{
          __html: JSON.stringify(jsonLd).replace(/</g, '\\u003c'),
        }}
      />
      {/* Name, price, stock, delivery and a link to /returns, as text */}
    </main>
  )
}

Write robots rules per crawler, in code

Next.js builds robots.txt from app/robots.ts, with a group per user agent. A crawler that finds its own group ignores the * group, so repeat your private paths there. In Nuxt, the @nuxtjs/robots module takes the same groups, and blockAiBots: 'training' blocks training crawlers only.[3][8][18]

app/robots.ts

import type { MetadataRoute } from 'next'

const privatePaths = ['/cart', '/checkout', '/account', '/api/']

export default function robots(): MetadataRoute.Robots {
  return {
    rules: [
      { userAgent: '*', allow: '/', disallow: privatePaths },
      // A named group replaces the * group, so repeat the private paths.
      {
        userAgent: ['OAI-SearchBot', 'Claude-SearchBot', 'PerplexityBot'],
        allow: '/',
        disallow: privatePaths,
      },
      // Model training: opt out.
      { userAgent: ['GPTBot', 'Google-Extended', 'CCBot'], disallow: '/' },
    ],
    sitemap: 'https://example.com/sitemap.xml',
  }
}

3
Unblock policies in an inherited Hydrogen robots route
If your storefront started from Hydrogen's skeleton, open app/routes/[robots.txt].tsx and delete the /policies/ line from the general disallow rules, so crawlers can read your refund and shipping policies.[9]
app/routes/[robots.txt].tsx, generalDisallowRules
```
  return `Disallow: /cart
Disallow: /account
Disallow: /collections/*sort_by*
Disallow: /search/?*
${sitemapUrl ? `Sitemap: ${sitemapUrl}` : ''}`;
// Removed: Disallow: /policies/
```
4
Send metadata in the head to every bot
Next.js lets you choose which bots get blocking metadata in the head through htmlLimitedBots. Matching every user agent turns streaming metadata off, so all crawlers get title and description where they expect them.[6]
next.config.ts
```
import type { NextConfig } from 'next'

const config: NextConfig = {
  htmlLimitedBots: /.*/,
}

export default config
```
5
Use real links for navigation
Render collections, products, filters and policies as links with an href, through your framework's Link component. Client-side routing still works, and an agent reading the page can follow every path.

Agent protocols for headless stores

A headless storefront can implement the open agent commerce protocols itself. These are the ones announced so far, newest last.

Sep 2025Agentic Commerce ProtocolOpenAI and Stripe released ACP, an open standard for checkout between buyers, AI agents and businesses. Medusa's tutorial shows how to implement its product feed and checkout specs; until OpenAI grants access, the example only logs to the console.[14][19][20]
Jan 2026Universal Commerce ProtocolGoogle announced UCP, an open standard for the whole shopping journey, co-developed with Shopify, Etsy, Wayfair, Target and Walmart.[21]
Mar 2026Vercel's agent-ready commerce templateVercel started vercel/shop in March 2026, described as agent-ready commerce with llms.txt, Schema.org data, a sitemap and robots, and archived the Next.js Commerce template in October 2026, pointing to vercel/shop.[15][16]
Oct 2026Hydrogen 2026-10 works with any frameworkShopify said Hydrogen now works with any JavaScript framework, with WebMCP so in-browser agents can search the catalog and update the cart. Its request handlers document MCP and UCP endpoints and buy permalinks for agents.[10][11]

What Aiscovery checks on headless stores

From the outside, the way an agent sees your store. Nothing to install, and agents stop before login, cart, checkout and payment.

Request a demo Free AI crawler checker

What the robots.txt your code generates declares for each AI crawler, and whether it blocks policy or product paths.
Page basics on the HTML your server sends, read without running JavaScript: title, description, canonical, headings and language.
JSON-LD syntax on the pages the check reads, and AI-facing files such as llms.txt and sitemaps.
Reachability and redirects for our identified checker.
GPT-6 Luna and Gemini try to find a product, its price, stock, delivery and returns in a real browser, stopping before cart and checkout.

Questions about headless stores and AI agents

Is a Next.js store readable by AI agents?

Usually. Pages and layouts are Server Components by default, and client components are prerendered into the HTML, so the first response carries your content. What's missing is anything fetched in the browser after load, plus whatever structured data and robots rules you didn't write.

How do I block GPTBot in Next.js?

Add a rule for userAgent 'GPTBot' with disallow '/' in app/robots.ts. GPTBot is OpenAI's training crawler; OAI-SearchBot decides whether you appear in ChatGPT search, so leave it allowed if you want to be found.

How do I block only AI training bots in Nuxt?

With the @nuxtjs/robots module, set blockAiBots to 'training'. It adds a group that disallows training crawlers and leaves search crawlers alone. You can also write your own groups per user agent.

Does Hydrogen support AI agent checkout?

For Shopify-backed storefronts, Hydrogen's request handlers include MCP and UCP endpoints and buy permalinks for agentic commerce, and Hydrogen 2026-10 adds WebMCP so agents in the browser can search the catalog and update the cart.

Does Medusa's storefront starter include robots.txt and structured data?

No. When we checked in October 2026, neither the archived nextjs-starter-medusa nor its successor dtc-starter had app/robots.ts or JSON-LD, and their next-sitemap config wasn't wired up, so add all three. Medusa's agentic commerce tutorial covers OpenAI's product feed and checkout specs.

Do you need access to our code to test a headless store?

No. Aiscovery tests the storefront from the outside, the same way an agent does, so it works with any framework and backend.

Sources

Official documentation and announcements, checked on 21 October 2026. Platforms change often, so check the source before you change settings.

1.Server and Client ComponentsNext.js docs
2.How to implement JSON-LD in your Next.js applicationNext.js docs
3.robots.txt file conventionNext.js docs
4.generateMetadata and streaming metadataNext.js docs
5.Default HTML-limited bots list (html-bots.ts)Next.js on GitHub
6.htmlLimitedBotsNext.js docs
7.Rendering modesNuxt docs
8.Nuxt Robots module configurationNuxt SEO
9.Hydrogen skeleton: app/routes/[robots.txt].tsxShopify on GitHub
10.Hydrogen 2026-10 is now availableShopify developer changelog, 14 October 2026
11.Hydrogen request handlersShopify developer docs
12.Next.js Starter StorefrontMedusa docs
13.medusajs/dtc-starterMedusa on GitHub
14.Implement Agentic Commerce specificationsMedusa docs
15.vercel/shopVercel on GitHub
16.vercel/commerce (archived)Vercel on GitHub
17.Merchant listing structured dataGoogle Search Central
18.RFC 9309: Robots Exclusion ProtocolIETF, 2022-09
19.Stripe powers Instant Checkout in ChatGPTStripe, 29 September 2025
20.Agentic Commerce ProtocolOpenAI and Stripe
21.New agentic commerce tools and the Universal Commerce ProtocolGoogle, 11 January 2026

Platform names and logos belong to their owners. Aiscovery is not affiliated with them; we test any store from the outside.

See what agents see on headless stores.

Tell us your store and we'll show you, on a short call, what AI agents could and couldn't find there, and the fix for each.

Have a real workflow to test now? Apply as a launch partner

Can AI agents shop a headless store?

headless stores at a glance

What usually breaks on headless stores

Prices and stock load in the browser

Structured data didn't survive the replatform

A starter's robots.txt blocks your policy pages

Metadata can land outside the head

Links that only work with JavaScript

How to fix it on headless stores

Render price and stock on the server, with markup

Write robots rules per crawler, in code

Unblock policies in an inherited Hydrogen robots route

Send metadata in the head to every bot

Use real links for navigation

Agent protocols for headless stores

What Aiscovery checks on headless stores

Questions about headless stores and AI agents

Sources

Related guides

See what agents see on headless stores.