Resources

Machine-readable content

Treat llms.txt as a maintained content map, not a ranking switch.

A concise Markdown guide can help agents reach authoritative content. Its value depends on accuracy, useful links and consistency with canonical HTML.

Aiscovery Research7 min readLast reviewed: August 25, 2026

What this helps you evaluate

  • What the current llms.txt proposal requires and recommends
  • How llms.txt differs from robots.txt and sitemap.xml
  • Which quality and maintenance checks belong in an audit

What the proposal covers

The llms.txt proposal defines a Markdown file at a site root or subpath. It provides concise context and curated links to detailed, preferably Markdown-friendly resources. Version 2 requires only one H1, then allows a summary, guidance and link sections in a defined order.

The proposal also recommends standard link relations so clients can discover Markdown alternatives and the llms.txt file that describes a path. The file is intended to stay concise while detailed content remains behind selected links.

It does not replace web foundations

Robots.txt communicates access preferences, sitemap.xml enumerates canonical indexable URLs and accessible HTML serves people, search engines and browser agents. llms.txt has a different role: it curates context for agents that choose to read it.

A stale file can be worse than no file because it creates another conflicting source. Generate it from the same content inventory where possible, then verify links, statements and product status during every material release.

How llms-full.txt fits

Some sites also publish llms-full.txt as an expanded text representation. It can be useful, but it is not a substitute for the concise discovery role of llms.txt, and teams should not present it as a universal requirement of the proposal.

Large files need extra attention to context size, duplication, stale sections and sensitive information. Include only public content and preserve links back to canonical pages.

Measure usefulness without inventing an SEO claim

Check whether relevant clients request the file, follow its links and reach correct answers. Test representative questions with the file as a starting point, but separate retrieval success from citation and downstream task completion.

Do not assign an automatic readiness penalty when the file is absent. Do not promise visibility gains solely because it exists. Record presence, validity, freshness, coverage and observed use as separate signals.

llms.txt checks for an agent-readiness audit

  • Request llms.txt and llms-full.txt, recording status, content type and redirects.
  • Validate the Markdown structure and all linked resources.
  • Compare product facts, status and canonical URLs with rendered HTML.
  • Check freshness, duplication, accidental private content and context size.
  • Run representative retrieval questions and record observed outcomes separately from presence.

Primary sources

Test the signal in a real journey.

Technical presence matters only when it improves a verified outcome. Connect the signal to a task, evidence and final-state assertion.