What is llms.txt, and how much does it matter?
llms.txt is a proposed Markdown file at the root of a website that gives AI assistants a short, readable summary of the site and links to its most useful pages. It's a convenience for language models, not a rule they must follow and not a ranking signal. It's cheap to provide, so it's reasonable to have one, but don't expect it to change your traffic on its own.
Where it came from
The format was proposed by Jeremy Howard in September 2024 and is documented at llmstxt.org. The problem it addresses is real: web pages are full of navigation, scripts and layout that a model has to wade through, and a model answering a question often has room for only a small amount of text. A single plain file that says "here's what this site is, and here are the pages worth reading" is easier to use than a crawl.
What the file contains
It lives at /llms.txt and is written in Markdown. The structure, in order:
- An H1 with the site or project name. This is the only required part.
- A blockquote with a short summary, a sentence or two saying what the site is.
- Optional plain paragraphs or lists with key details: what you offer, who it's for, important facts.
- H2 sections listing links, each entry a Markdown link to a page, optionally followed by a colon and a short note.
- An H2 called
Optional, by convention, for links an assistant can skip when it needs to keep things short.
A small business example
For a two-person physiotherapy clinic, a useful file might read: an H1 Hollis Physio; a blockquote saying it's a sports injury clinic in Headingley, Leeds, open weekdays 7am to 8pm; a short list of services with durations; a Pages section linking to services, prices, the team and booking, each with a one-line note; and an Optional section linking to the FAQ and the privacy policy. Facts go in as text. Marketing adjectives add nothing a model can use.
llms.txt vs robots.txt
| robots.txt | llms.txt | |
|---|---|---|
| Purpose | Says which parts of the site crawlers may fetch | Summarises the site and points to the best pages |
| Audience | Search and other crawlers | Language models and AI assistants, at the moment they read the site |
| Status | An internet standard (RFC 9309) | A community proposal |
| Controls access? | Asks compliant crawlers to stay out of paths | No; it grants and blocks nothing |
| Format | Plain directives | Markdown |
If you want to keep AI crawlers out, that's a robots.txt job, covered in how to create a sitemap and robots.txt. llms.txt is the opposite gesture: an invitation to read a tidy summary.
How much weight to give it
Be realistic. Google's guidance on AI features in Search says you don't need to create new machine-readable files or AI text files to appear in them; ordinary, crawlable, helpful pages are what count. Some AI tools and coding assistants do read llms.txt when they're pointed at a site, and some documentation sites publish one for that reason. Treat it as a small, low-cost courtesy that may help an assistant describe you accurately, not as an SEO tactic.
The fundamentals matter more: pages whose text is readable without scripts, a clear title and heading on each page, and your key facts (what, where, when, how much) written as text. A model can only summarise what's there.
Writing your own, step by step
- Write the H1 and a one-sentence summary that a stranger would understand.
- List the five to ten pages that answer most questions, with a short note for each.
- Add the facts people ask about: location, hours, prices or price ranges, how to contact you.
- Put secondary pages under
## Optional. - Save it as UTF-8 plain text at
/llms.txtand open it in a browser to check it displays as text. - Update it whenever your pages or facts change; a stale summary is worse than none.
What Anticode does
Every published Anticode site serves its own /llms.txt alongside robots.txt and sitemap.xml. It's built from the pages as published, so it follows your edits rather than going stale, and it includes your separate pages and CMS entries. Open it on your live address to see what an assistant would read. Because the summary comes from your pages, the way to improve it is to improve the pages' facts.
Limits
Anticode generates the file; it doesn't offer a hand-written override. It can't make any AI assistant read the file, cite your site or recommend you, and nobody can promise that. For the wider picture of being cited by assistants, see how to get cited by AI search.
Questions people ask
Do I need an llms.txt file?
No site needs one to appear in search or AI answers. It's an optional summary that some tools read. If your platform makes one automatically, keep it; if not, writing one takes twenty minutes.
Does llms.txt replace a sitemap?
No. A sitemap lists every page for crawlers; llms.txt summarises the important ones for language models. They do different jobs.
Can llms.txt block AI from using my content?
No. It doesn't restrict anything. Use robots.txt rules for the crawlers you want to keep out, knowing that compliance is voluntary.