When an Aiscovery customer checks their website, one identified visitor does the reading. It reads public pages only, at most one page per second, and follows robots.txt.
Request headers
No cookies, no login and no personal data. HTTPS only, with a valid certificate.
Aiscovery customers check how their own websites work for automated visitors and AI agents. A check starts when a customer adds a website and allows checks. With automatic checks on, it repeats on their plan's schedule.
It is not a search engine or an AI crawler. It doesn't collect pages to train AI models or to answer questions.
Before anything else it reads /robots.txt. If your rules ask it not to read a page, it doesn't.
Usually your homepage, a handful of pages the customer picked, and files meant for AI tools such as llms.txt. It never crawls your whole site.
One request at a time, at most one per second. A standard check makes up to 60 requests and ends within 2 minutes. No check goes over 200 requests or 5 minutes.
Plain GET requests with no cookies. It never logs in, fills in forms, adds to cart or pays. Checkout and account pages are skipped.
It never solves CAPTCHAs or works around bot protection. If your firewall stops it, the check ends and the customer sees “Couldn't check”.
It always sends the same user agent. It never pretends to be a browser, a search engine or an AI company's crawler.
It reads at most 512 KB of any response and follows redirects only within your site.
When a customer runs an AI agent task, agents that run in our browser load your pages through this same checker, with the same user agent, robots.txt rules and pace. They stop before login, cart, checkout and payment.
We don't publish fixed IP addresses yet. Identify our checker by its user agent.
Add these two lines to your robots.txt. Our checker reads robots.txt first, so it will read nothing else from your site. The customer sees that your robots.txt asked checkers like ours not to read it.
To keep only some pages private, list those paths instead, for example Disallow: /private/.
User-agent: AiscoveryAudit
Disallow: /Email abuse@aiscovery.io from an address at your domain. We add your site to our do-not-check list for every Aiscovery customer and stop checks within one business day.
We don't need a login or any other details. We don't share who asked for the checks.
abuse@aiscovery.ioIf a customer sees “Couldn't check”, a firewall or bot protection probably stopped our checker. Allow our user agent on your public pages. Your other protections can stay on.
Match requests where the User-Agent contains
AiscoveryAudit
Anyone can copy a user agent. Allow ours for public pages only, not for login, account or admin pages. Then press Check again in Aiscovery.
Bot Fight Mode on free plans can't be skipped by a rule. While it's on, our checker may still be challenged.
Attack Challenge Mode challenges every visitor that isn't a verified bot, including ours. Check again after you turn it off.
Write to abuse@aiscovery.io with your website and the dates you saw our checker. We reply the same business day. For security reports, use our security page.
No. It checks how a site works for automated visitors on behalf of an Aiscovery customer. It doesn't collect pages to train AI models, to answer questions or to build a search index.
It is very unlikely. It sends one request at a time, at most one per second, and a standard check ends within 2 minutes.
An Aiscovery customer added your website to check it. If you'd rather it stopped, add the robots.txt lines above or email abuse@aiscovery.io. We then stop checks of your site for every customer.
No. We tell the customer we couldn't check the site, not that it is broken. Rules that block every automated visitor may also block some AI agents, so review them when you can.
We don't publish fixed IP addresses yet. If something calling itself AiscoveryAudit ignores robots.txt or reads faster than one page per second, it isn't us. Please tell us at abuse@aiscovery.io.
Tell us what you saw, or read how Aiscovery checks websites.