How Anthropic Adopted a New Web Format Built for AI Tools, Even Without Official Backing

The most interesting signal about whether a new web standard is worth adopting is whether the companies building the underlying technology adopt it themselves. In the case of llms.txt, that signal is mixed in a way that is actually useful. Anthropic, the company that builds the AI assistant Claude, has published an llms.txt file on its own website. Anthropic has also not officially confirmed that Claude uses these files when reading the web. The gap between those two facts is the whole story.

What llms.txt Is

llms.txt is a small markdown file that lives at the root of a website, in a place AI tools can find easily. It works as a curated map of the site, listing the most important pages with a short description of each one. The point is to give AI assistants a clean, opinionated guide to the content a brand wants surfaced and cited, rather than leaving the AI to piece things together from a normal web page that is full of navigation, ads, and scripts.

The format is written in markdown, which is a simple plain-text format used widely on the web. The file is short by design, usually fewer than one hundred links, with a one-line summary of the site at the top and section headings that group the linked pages by topic.

A minimal llms.txt looks like this:

# Anthropic

> Anthropic is an AI safety company. We build Claude, a family of AI assistants, and publish research on building reliable, interpretable, and steerable AI systems.

## Docs

- [Claude API documentation](https://docs.anthropic.com/): Reference for building applications with the Claude API.
- [Prompt engineering guide](https://docs.anthropic.com/en/docs/prompt-engineering): How to write effective prompts for Claude.

## Product

- [Claude](https://claude.ai/): The consumer chat product.
- [Pricing](https://www.anthropic.com/pricing): API and product pricing.

## Company

- [Research](https://www.anthropic.com/research): Published research from Anthropic.
- [Responsible scaling policy](https://www.anthropic.com/responsible-scaling-policy): How we evaluate and deploy increasingly capable models.

The # line is the site name, the > line is the one-sentence summary, and each ## section groups related pages. That is the entire spec.

The Anthropic Example

Anthropic.com hosts an llms.txt file at the top level of the site, formatted using the same proposed structure other companies have adopted. The file lists the company’s most important pages, including documentation, product information, and the technical references developers use to build with Claude. It is short, well structured, and reads like the work of a team that takes the format seriously, even without an official commitment to its use.

This is significant because Anthropic is one of the three or four companies that would actually decide whether llms.txt becomes an industry standard. They build a major AI assistant, and their system is one of the ones the file is theoretically meant to inform. If anyone has standing to dismiss the format, Anthropic does. Instead they have shipped the file, and they have done it cleanly.

What This Says About the Format

The honest read on llms.txt today is that no major AI company has officially confirmed that their systems use these files when reading websites. That includes Anthropic, OpenAI, and Google, the companies behind Claude, ChatGPT, and Gemini. Independent analyses of large samples of websites have found no measurable correlation between having an llms.txt file and being cited more often by AI assistants. The format is a proposal, not an enforced standard.

What Anthropic’s implementation suggests is that the question is not whether llms.txt works today. The question is whether the cost of shipping it is low enough that the upside of being early outweighs the wait-and-see approach. Anthropic appears to have decided yes. A long list of other technical companies have come to the same conclusion. The file takes about an hour to write, lives at the root of the site, and requires almost no maintenance, which is a very low cost for the chance to be indexed first if broader support lands.

How to Think About Hedge-Bet Decisions in AI Search

Most marketing decisions are evaluated against current performance data. AI search is one of the few areas where that approach can work against you, because the channels and signals are still being defined. A new format ships, a few major companies adopt it, the rest of the industry waits for measurable proof, and by the time the proof arrives the early adopters have already been indexed and cited for months.

llms.txt is the cleanest current example of this dynamic. The file does not have a published click-through study behind it, but it does have a long list of sites adopting it, including the company that builds one of the most cited AI assistants in the world. The cost of shipping is roughly the same as the cost of writing this paragraph, while the cost of being last to ship if support is officially announced is meaningfully higher.

The AEO market gap: real demand, almost no one ready.

Hedge-bet logic: llms.txt sits in the high-upside, low-cost quadrant.

Months after this article first ran, Stripe shipped a structured, product-organized version of the file that took the same idea further, and Vercel tied the format to a measurable acquisition channel when ChatGPT referrals became a real share of new signups.

Where This Sits in an AEO Strategy

Answer Engine Optimization, or AEO, is the practice of structuring a site so it is clear, citable, and machine readable for the AI tools that now generate answers instead of returning the traditional list of search results. AEO sits alongside SEO and Generative Engine Optimization (GEO) as the three layers of modern search visibility. llms.txt is one signal in that broader effort, alongside structured data, consistent entity information, structured Q and A content, and clean information architecture. Each layer reinforces the others. None of them are sufficient on their own.

Three layers of modern search.

The Anthropic example is a useful reference because it removes some of the speculation. If the company that builds Claude is willing to ship the file without an official commitment to use it, the case for waiting is weaker than it looks.

Audit Your AI Readiness

Most brands are still operating on a pre-AI assumption that traditional search is the only channel that matters. That is becoming a less safe assumption every quarter. Get an AEO audit to see whether your site is set up to be cited by AI assistants, including your llms.txt, your structured data, your entity signals, and your overall machine readability.