What the proposal covers
The llms.txt proposal defines a Markdown file at a site root or subpath. It provides concise context and curated links to detailed, preferably Markdown-friendly resources. Version 2 requires only one H1, then allows a summary, guidance and link sections in a defined order.
The proposal also recommends standard link relations so clients can discover Markdown alternatives and the llms.txt file that describes a path. The file is intended to stay concise while detailed content remains behind selected links.
It does not replace web foundations
Robots.txt communicates access preferences, sitemap.xml enumerates canonical indexable URLs and accessible HTML serves people, search engines and browser agents. llms.txt has a different role: it curates context for agents that choose to read it.
A stale file can be worse than no file because it creates another conflicting source. Generate it from the same content inventory where possible, then verify links, statements and product status during every material release.
How llms-full.txt fits
Some sites also publish llms-full.txt as an expanded text representation. It can be useful, but it is not a substitute for the concise discovery role of llms.txt, and teams should not present it as a universal requirement of the proposal.
Large files need extra attention to context size, duplication, stale sections and sensitive information. Include only public content and preserve links back to canonical pages.
Measure usefulness without inventing an SEO claim
Check whether relevant clients request the file, follow its links and reach correct answers. Test representative questions with the file as a starting point, but separate retrieval success from citation and downstream task completion.
Do not assign an automatic readiness penalty when the file is absent. Do not promise visibility gains solely because it exists. Record presence, validity, freshness, coverage and observed use as separate signals.
llms.txt checks for an agent-readiness audit
- Request llms.txt and llms-full.txt, recording status, content type and redirects.
- Validate the Markdown structure and all linked resources.
- Compare product facts, status and canonical URLs with rendered HTML.
- Check freshness, duplication, accidental private content and context size.
- Run representative retrieval questions and record observed outcomes separately from presence.
Primary sources
- The llms.txt file, version 2llms.txt proposal
- Lighthouse llms.txt auditChrome for Developers
- Lighthouse agentic browsing scoringChrome for Developers