# Our checker, AiscoveryAudit

> What Aiscovery's checker reads, how fast it goes, how to opt out with robots.txt, and how to allow it in Cloudflare, Akamai, Fastly, Vercel or any firewall.

- Canonical URL: https://aiscovery.io/checker
- Markdown version: https://aiscovery.io/checker.md
- Last updated: 2026-09-26

When an Aiscovery customer checks their website, one identified visitor does the reading. It reads public pages only, at most one page per second, and follows robots.txt.

- [How to opt out](https://aiscovery.io/checker#opt-out)
- [Allow it in your firewall](https://aiscovery.io/checker#allow)

## What your server sees

| Request | When |
| --- | --- |
| GET /robots.txt | always first |
| GET / | 1 second later |
| GET /llms.txt | 1 second later |

Request headers:

```
AiscoveryAudit/1.0 (+https://aiscovery.io/checker)
```

No cookies, no login and no personal data. HTTPS only, with a valid certificate.

## Why it visits your site

Aiscovery customers check how their own websites work for automated visitors and AI agents. A check starts when a customer adds a website and allows checks. With automatic checks on, it repeats on their plan's schedule.

It is not a search engine or an AI crawler. It doesn't collect pages to train AI models or to answer questions.

### What it reads, and what it never does

- **robots.txt comes first:** Before anything else it reads /robots.txt. If your rules ask it not to read a page, it doesn't.
- **A few public pages:** Usually your homepage, a handful of pages the customer picked, and files meant for AI tools such as llms.txt. It never crawls your whole site.
- **One page per second:** One request at a time, at most one per second. A standard check makes up to 60 requests and ends within 2 minutes. No check goes over 200 requests or 5 minutes.
- **Read only:** Plain GET requests with no cookies. It never logs in, fills in forms, adds to cart or pays. Checkout and account pages are skipped.
- **Stops at your firewall:** It never solves CAPTCHAs or works around bot protection. If your firewall stops it, the check ends and the customer sees “Couldn't check”.
- **Always identified:** It always sends the same user agent. It never pretends to be a browser, a search engine or an AI company's crawler.
- **Small, same-site reads:** It reads at most 512 KB of any response and follows redirects only within your site.
- **AI agent runs use it too:** When a customer runs an AI agent task, agents that run in our browser load your pages through this same checker, with the same user agent, robots.txt rules and pace. They stop before login, cart, checkout and payment.
- **No fixed IP addresses yet:** We don't publish fixed IP addresses yet. Identify our checker by its user agent.

## How to opt out

Add these two lines to your robots.txt. Our checker reads robots.txt first, so it will read nothing else from your site. The customer sees that your robots.txt asked checkers like ours not to read it.

robots.txt:

```
User-agent: AiscoveryAudit
Disallow: /
```

To keep only some pages private, list those paths instead, for example Disallow: /private/.

### Stop every check of your site

Email abuse@aiscovery.io from an address at your domain. We add your site to our do-not-check list for every Aiscovery customer and stop checks within one business day.

We don't need a login or any other details. We don't share who asked for the checks.

[abuse@aiscovery.io](mailto:abuse@aiscovery.io)

## How to allow our checker

If a customer sees “Couldn't check”, a firewall or bot protection probably stopped our checker. Allow our user agent on your public pages. Your other protections can stay on.

Match requests where the User-Agent contains:

```
AiscoveryAudit
```

Anyone can copy a user agent. Allow ours for public pages only, not for login, account or admin pages. Then press Check again in Aiscovery.

### Cloudflare

1. In the Cloudflare dashboard, open your domain and create a WAF custom rule (Security → WAF → Custom rules, or Security rules in the newer dashboard).
2. Name it “Allow AiscoveryAudit”. Match the field User Agent with the operator contains and the value AiscoveryAudit.
3. Choose the action Skip, select the bot and managed protections to skip for this visitor, such as Super Bot Fight Mode, and deploy.

Bot Fight Mode on free plans can't be skipped by a rule. While it's on, our checker may still be challenged.

### Akamai

1. In Akamai Control Center, open the security configuration for your site.
2. In Bot Manager, add a custom bot that matches User-Agent contains AiscoveryAudit, and set its action to Allow.
3. If App & API Protector rules also apply, add an exception for the same user agent. Activate on staging, then on production.

### Fastly

1. In Fastly Next-Gen WAF, open your site and add a site rule of the request type.
2. Add the condition “User agent contains AiscoveryAudit” and choose the action Allow.
3. If you also block bots in VCL or Compute code, exempt the same user agent there.

### Vercel

1. In your Vercel project, open Firewall and add a new custom rule.
2. Set the condition User Agent contains AiscoveryAudit and choose the action Bypass. Save and publish the rule.

Attack Challenge Mode challenges every visitor that isn't a verified bot, including ours. Check again after you turn it off.

### Any other firewall or bot protection

1. Add an allow rule for GET requests where the User-Agent header contains AiscoveryAudit.
2. Put it before your bot, challenge and rate-limit rules, so they don't run for our checker.
3. On AWS WAF, use a string match on the User-Agent header with the action Allow, at a higher priority than Bot Control.
4. On hosted shops and site builders, look for bot protection or firewall settings. If you can't allow one visitor there, ask their support.

## Questions or concerns about our checker

Write to abuse@aiscovery.io with your website and the dates you saw our checker. We reply the same business day. For security reports, use our security page.

- [abuse@aiscovery.io](mailto:abuse@aiscovery.io)
- [Security page](https://aiscovery.io/security)

## Common questions from site owners

**Q: Is AiscoveryAudit an AI crawler?**

A: No. It checks how a site works for automated visitors on behalf of an Aiscovery customer. It doesn't collect pages to train AI models, to answer questions or to build a search index.

**Q: Will it slow down my site?**

A: It is very unlikely. It sends one request at a time, at most one per second, and a standard check ends within 2 minutes.

**Q: Why is it checking my site?**

A: An Aiscovery customer added your website to check it. If you'd rather it stopped, add the robots.txt lines above or email abuse@aiscovery.io. We then stop checks of your site for every customer.

**Q: Does blocking it harm my site?**

A: No. We tell the customer we couldn't check the site, not that it is broken. Rules that block every automated visitor may also block some AI agents, so review them when you can.

**Q: How do I know a request really came from Aiscovery?**

A: We don't publish fixed IP addresses yet. If something calling itself AiscoveryAudit ignores robots.txt or reads faster than one page per second, it isn't us. Please tell us at abuse@aiscovery.io.

## Seen our checker on your site?

Tell us what you saw, or read how Aiscovery checks websites.

- [Email abuse@aiscovery.io](mailto:abuse@aiscovery.io)
- [Read our methodology](https://aiscovery.io/methodology)

---

This is the Markdown version of the page for AI agents. The HTML page at the canonical URL is authoritative. Site index: https://aiscovery.io/llms.txt