Does Cloudflare block AI agents on my store?

How Cloudflare sorts AI bots today

Cloudflare deprecated the single Block AI bots switch on September 15, 2026. In its place, it sorts AI traffic by behavior into three policies, each with its own setting. Since that date, a new domain is offered one of two presets when it is set up: a site without ads gets Allow for Search, Training and Agent; an ad-funded site gets Search allowed, Training set to Disallow AI Training and Agent set to Block on pages with ads. Bot Preference Sync is on in both.

Check the settings that can block AI agents

1. 1

   Step 1: Open the AI bot policies

   In the Cloudflare dashboard, pick the domain, open Security settings, filter by Bot traffic and choose Configure AI bot policies. Note the Search, Agent and Training values. For most stores, Search and Agent should be Allow.

2. 2

   Step 2: If you opt out of training, pick Disallow AI Training

   It publishes the no-training preference in your robots.txt through Bot Preference Sync and blocks training-only crawlers, while Googlebot, Bingbot and Applebot stay allowed for search. One gap: Bing doesn't read a robots.txt no-training rule yet (Microsoft targets early 2027), so Cloudflare points to the NOARCHIVE meta tag to state a training preference to Bing.

3. 3

   Step 3: Check Bot Fight Mode

   On the same Bot traffic list, look at Bot fight mode (Free plan) or Super Bot fight mode (Pro and above). Bot Fight Mode challenges traffic that matches bot patterns, and Cloudflare says WAF custom rules can't skip it. Its docs don't list which verified bots it exempts, so check the events in step 5.

4. 4

   Step 4: Open AI Crawl Control

   AI Crawl Control (formerly AI Audit) is on all plans. Its crawler table shows each AI crawler's action, allowed and unsuccessful requests and robots.txt violations. Open a crawler's metrics to see its status codes: a 403 means blocked. Cloudflare notes that unsuccessful requests may come from any rule or error, not only this page.

5. 5

   Step 5: Read the security events for one crawler

   Go to Security, then Analytics, then Events, and add a filter on user agent, for example OAI-SearchBot or ChatGPT-User. Expand a row to see which feature acted on it. Bot Fight Mode events are labeled Bot Fight Mode.

Lines Cloudflare may add to your robots.txt

With Bot Preference Sync (which replaces managed robots.txt), Cloudflare prepends its own block to your robots.txt, or creates the file if you have none. It contains a Content-signal line and Disallow rules for known AI training crawlers. Your existing rules are kept below it.

User-Agent: *
Content-signal: search=yes, ai-train=no, use=reference
Allow: /

User-agent: GPTBot
Disallow: /

• search covers building a search index and returning results. ai-input covers feeding content into an AI model to answer a question. ai-train covers training or fine-tuning.
• Cloudflare says robots.txt compliance is voluntary; the blocking itself happens in the bot policies and AI Crawl Control.
• use=reference (index, excerpt and link back) belongs to a content-use extension that Cloudflare is still testing.
• Google Search Console may report Syntax not understood for the Content-signal line. Cloudflare says it has "observed no impact on crawling rates or SEO" from these reports.

Signed agents: how Cloudflare recognizes shopping assistants

Agents that browse for a person often look like ordinary Chrome. To tell them apart, some sign every request with Web Bot Auth (HTTP message signatures). OpenAI says ChatGPT's cloud browser signs its requests with Signature-Agent set to "https://chatgpt.com", and lists the Cloudflare bot tag chatgpt-agent. Google is testing signatures for Google-Agent under https://agent.bot.goog, on some requests only.

Since July 1, 2026, Cloudflare counts signed agents as verified bots, labeled Direct (one operator) or Intermediary (a service many people drive, such as a shopping assistant). Verified bots are no longer automatically left out of blocking: your Search, Agent and Training policies decide, and for these assistants that is the Agent policy.

On Shopify, these settings aren't yours

Shopify says it uses Cloudflare to protect every store from bots, that it is always active and needs no action from you. Shopify also doesn't support putting your own Cloudflare proxy in front of a Shopify store, and asks for DNS only records. So a Shopify merchant can't change Cloudflare's AI bot policies; the control you do have is robots.txt.liquid. See how to allow ChatGPT and other AI agents on Shopify.

We couldn't find a Shopify statement on how its bot protection treats AI crawlers or signed agents, so we don't make a claim about it here. On WooCommerce or any store where you own the Cloudflare account, everything above applies. More on Aiscovery for WooCommerce.