# Does Cloudflare block AI agents on my store?

> Cloudflare's Search, Agent and Training policies, Bot Fight Mode and AI Crawl Control decide which AI crawlers reach your store. Where to check each one.

- Canonical URL: https://aiscovery.io/resources/does-cloudflare-block-ai-agents
- Markdown version: https://aiscovery.io/resources/does-cloudflare-block-ai-agents.md
- Last updated: 2026-10-21

It can, depending on your AI bot policies: since September 15, 2026, a new Cloudflare domain without ads starts with search crawlers, agents and training crawlers all allowed, while an ad-funded site starts with agents blocked on pages that show ads. Check the Search, Agent and Training policies under Security settings, then confirm in AI Crawl Control which crawlers actually received errors.

- Topic: Cloudflare
- Author: Aiscovery team
- Published: 2026-10-21
- Updated: 2026-10-21
- Reading time: 8 min read

## Key facts

- Cloudflare now groups AI bots into three policies: Search, Agent (acting for a person in real time) and Training.
- Each policy can be set to Allow, Block on pages with ads, or Block. Training also has Disallow AI Training.
- Cloudflare says Block on Training also stops Googlebot, Bingbot and Applebot, search included.
- Cloudflare says Bot Fight Mode cannot be bypassed with WAF custom rules.
- On Shopify, Cloudflare runs as part of the platform and merchants don't get these settings.

## How Cloudflare sorts AI bots today

Cloudflare deprecated the single Block AI bots switch on September 15, 2026. In its place, it sorts AI traffic by behavior into three policies, each with its own setting. Since that date, a new domain is offered one of two presets when it is set up: a site without ads gets Allow for Search, Training and Agent; an ad-funded site gets Search allowed, Training set to Disallow AI Training and Agent set to Block on pages with ads. Bot Preference Sync is on in both.

Cloudflare AI bot policies and what they mean for a store:

| Policy | What Cloudflare counts in it | Options | For a store |
| --- | --- | --- | --- |
| Search | Crawlers that collect or index content to answer questions about it later | Allow, Block on pages with ads, Block | Allow. This is how AI search tools find and cite your products. |
| Agent | Automated activity acting in real time for a person, such as chat fetch bots and browser-use agents | Allow, Block on pages with ads, Block | Allow. These are shoppers' assistants opening your pages. |
| Training | Crawlers that take content to train or fine-tune models, including mixed-purpose crawlers | Allow, Disallow AI Training, Block on pages with ads, Block | Your choice. Use Disallow AI Training rather than Block if you opt out. |

> **The setting that removes you from Google.** Cloudflare's September 2026 post says Block on Training now applies to mixed-use crawlers and stops Applebot, Bingbot and Googlebot from reaching your site, search included. Disallow AI Training leaves those mixed-use crawlers out of the block, keeps them allowed for search and publishes a no-training preference in robots.txt instead.

## Check the settings that can block AI agents

### 1. Open the AI bot policies

In the Cloudflare dashboard, pick the domain, open Security settings, filter by Bot traffic and choose Configure AI bot policies. Note the Search, Agent and Training values. For most stores, Search and Agent should be Allow.

### 2. If you opt out of training, pick Disallow AI Training

It publishes the no-training preference in your robots.txt through Bot Preference Sync and blocks training-only crawlers, while Googlebot, Bingbot and Applebot stay allowed for search. One gap: Bing doesn't read a robots.txt no-training rule yet (Microsoft targets early 2027), so Cloudflare points to the `NOARCHIVE` meta tag to state a training preference to Bing.

### 3. Check Bot Fight Mode

On the same Bot traffic list, look at Bot fight mode (Free plan) or Super Bot fight mode (Pro and above). Bot Fight Mode challenges traffic that matches bot patterns, and Cloudflare says WAF custom rules can't skip it. Its docs don't list which verified bots it exempts, so check the events in step 5.

### 4. Open AI Crawl Control

AI Crawl Control (formerly AI Audit) is on all plans. Its crawler table shows each AI crawler's action, allowed and unsuccessful requests and robots.txt violations. Open a crawler's metrics to see its status codes: a `403` means blocked. Cloudflare notes that unsuccessful requests may come from any rule or error, not only this page.

### 5. Read the security events for one crawler

Go to Security, then Analytics, then Events, and add a filter on user agent, for example `OAI-SearchBot` or `ChatGPT-User`. Expand a row to see which feature acted on it. Bot Fight Mode events are labeled Bot Fight Mode.

## Lines Cloudflare may add to your robots.txt

With Bot Preference Sync (which replaces managed robots.txt), Cloudflare prepends its own block to your robots.txt, or creates the file if you have none. It contains a `Content-signal` line and `Disallow` rules for known AI training crawlers. Your existing rules are kept below it.

**robots.txt: example from Cloudflare's docs**

```text
User-Agent: *
Content-signal: search=yes, ai-train=no, use=reference
Allow: /

User-agent: GPTBot
Disallow: /
```

- `search` covers building a search index and returning results. `ai-input` covers feeding content into an AI model to answer a question. `ai-train` covers training or fine-tuning.
- Cloudflare says robots.txt compliance is voluntary; the blocking itself happens in the bot policies and AI Crawl Control.
- `use=reference` (index, excerpt and link back) belongs to a content-use extension that Cloudflare is still testing.
- Google Search Console may report Syntax not understood for the `Content-signal` line. Cloudflare says it has "observed no impact on crawling rates or SEO" from these reports.

## Signed agents: how Cloudflare recognizes shopping assistants

Agents that browse for a person often look like ordinary Chrome. To tell them apart, some sign every request with Web Bot Auth (HTTP message signatures). OpenAI says ChatGPT's cloud browser signs its requests with `Signature-Agent` set to `"https://chatgpt.com"`, and lists the Cloudflare bot tag `chatgpt-agent`. Google is testing signatures for Google-Agent under `https://agent.bot.goog`, on some requests only.

Since July 1, 2026, Cloudflare counts signed agents as verified bots, labeled Direct (one operator) or Intermediary (a service many people drive, such as a shopping assistant). Verified bots are no longer automatically left out of blocking: your Search, Agent and Training policies decide, and for these assistants that is the Agent policy.

## On Shopify, these settings aren't yours

Shopify says it uses Cloudflare to protect every store from bots, that it is always active and needs no action from you. Shopify also doesn't support putting your own Cloudflare proxy in front of a Shopify store, and asks for DNS only records. So a Shopify merchant can't change Cloudflare's AI bot policies; the control you do have is `robots.txt.liquid`. See [how to allow ChatGPT and other AI agents on Shopify](https://aiscovery.io/resources/allow-ai-agents-shopify-robots-txt).

We couldn't find a Shopify statement on how its bot protection treats AI crawlers or signed agents, so we don't make a claim about it here. On WooCommerce or any store where you own the Cloudflare account, everything above applies. More on [Aiscovery for WooCommerce](https://aiscovery.io/platforms/woocommerce).

## Keep going

- [Check your robots.txt](https://aiscovery.io/resources/check-which-ai-crawlers-your-store-blocks): Read your rules the way each AI crawler does.
- [AI crawler checker](https://aiscovery.io/tools/ai-crawler-checker): See what your robots.txt declares for each AI company's crawler.
- [Aiscovery on WooCommerce](https://aiscovery.io/platforms/woocommerce): What commonly breaks for AI agents on WooCommerce stores.
- [Request a demo](https://aiscovery.io/demo): Watch named AI agents try key shopping tasks in your store.

## Questions store owners ask

**Q: Does Cloudflare block ChatGPT by default?**

A: Not on a store without ads. Since September 15, 2026, a new domain without ads starts with Search, Training and Agent all allowed; an ad-funded site starts with agents blocked on pages that show ads. Older domains keep whatever their policies say, so check them.

**Q: Will Disallow AI Training hurt my Google rankings?**

A: Cloudflare says Disallow AI Training keeps Googlebot, Bingbot and Applebot allowed for search. The Block option on Training is the one that stops them, search included.

**Q: I'm on Shopify. Can I change Cloudflare's bot settings?**

A: No. Shopify runs Cloudflare as part of the platform, and it doesn't support your own Cloudflare proxy in front of the store. Use robots.txt.liquid to state your policy.

**Q: How do I know if Cloudflare blocked a specific AI crawler?**

A: Open AI Crawl Control and check that crawler's status codes, where 403 means blocked, or filter Security Events by its user agent and expand a row to see which feature acted.

**Q: Is the old Block AI bots toggle still there?**

A: No. Cloudflare deprecated it on September 15, 2026 in favor of the Search, Agent and Training policies. Domains that had it set to Block or Block on pages with ads moved to Disallow AI Training.

## Sources

Every technical claim on this page comes from the vendor's own documentation, checked on the date shown as Updated. Crawler names and settings change, so check the source before you edit a live store.

1. [Block AI bots](https://developers.cloudflare.com/bots/additional-configurations/block-ai-bots/) (Cloudflare Docs)
2. [Have it both ways: stay discoverable in search while disallowing AI training](https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/) (Cloudflare Blog)
3. [AI Crawl Control](https://developers.cloudflare.com/ai-crawl-control/) (Cloudflare Docs)
4. [AI Crawl Control: get started](https://developers.cloudflare.com/ai-crawl-control/get-started/) (Cloudflare Docs)
5. [AI Crawl Control: manage AI crawlers](https://developers.cloudflare.com/ai-crawl-control/features/manage-ai-crawlers/) (Cloudflare Docs)
6. [robots.txt setting](https://developers.cloudflare.com/bots/additional-configurations/managed-robots-txt/) (Cloudflare Docs)
7. [Bot Fight Mode](https://developers.cloudflare.com/bots/get-started/bot-fight-mode/) (Cloudflare Docs)
8. [Verified bots](https://developers.cloudflare.com/bots/concepts/bot/verified-bots/) (Cloudflare Docs)
9. [Super Bot Fight Mode](https://developers.cloudflare.com/bots/get-started/super-bot-fight-mode/) (Cloudflare Docs)
10. [Security Events](https://developers.cloudflare.com/waf/analytics/security-events/) (Cloudflare Docs)
11. [ChatGPT Work's Cloud browser allowlisting](https://help.openai.com/en/articles/11845367-chatgpt-works-cloud-browser-allowlisting) (OpenAI Help Center)
12. [Authenticating requests with Web Bot Auth (experimental)](https://developers.google.com/crawling/docs/crawlers-fetchers/web-bot-auth) (Google for Developers)
13. [Protecting your store from bots](https://help.shopify.com/en/manual/intro-to-shopify/bots/dealing-with-bots) (Shopify Help Center)
14. [Troubleshooting issues with domains connected to Shopify](https://help.shopify.com/en/manual/domains/troubleshoot-issues-with-domains) (Shopify Help Center)

## More guides

- [How do I add llms.txt to Shopify or WooCommerce, and do I need it?](https://aiscovery.io/resources/add-llms-txt-shopify-woocommerce) (llms.txt)
- [Why can't AI agents find my delivery price?](https://aiscovery.io/resources/why-ai-agents-cant-find-delivery-price) (Product facts)
- [How do I make product variants and stock readable for AI agents?](https://aiscovery.io/resources/product-variants-stock-ai-agents) (Product facts)

## See what AI agents do in your store.

A short call: we run the site check on your store, show what robots.txt tells each AI crawler, and walk through a test run where named AI agents try key shopping tasks.

[Request a demo](https://aiscovery.io/demo)

Have a real workflow to test now? [Apply as a launch partner](https://aiscovery.io/partners)

---

This is the Markdown version of the page for AI agents. The HTML page at the canonical URL is authoritative. Site index: https://aiscovery.io/llms.txt