llms.txt: what the file can do and whether it is worth it
The llms.txt is a proposal, not a standard with guaranteed effect. It costs about an hour, does no harm, and its main benefit may lie somewhere other than the name suggests.
In short
- The llms.txt is a Markdown file in the root directory telling a language model in a few lines what the website is about and which pages are the most important.
- It is a community proposal, not a binding standard — there is no guaranteed evaluation by the large providers.
- The effort is around an hour. It cannot do harm, and the side effect is often worth more than the file itself.
- It replaces neither robots.txt nor sitemap.xml — the three files answer different questions.
What goes in it
An llms.txt sits at /llms.txt and is written in Markdown — readable by people and by models alike. It typically contains:
- A heading with the name of the company or project
- One or two sentences on what it is about
- An annotated list of the most important pages, grouped by area
- Optionally a section of secondary pages
# Example Ltd > We build marketing processes for owner-led service businesses > with 5 to 30 people. Based in Zurich, founded 2021. ## Basics - [What we offer](https://example.com/services/): scope, process and price range of our services - [About us](https://example.com/about/): founding, team, location - [Prices](https://example.com/prices/): ranges and what they depend on ## Articles - [Building marketing without a team](https://example.com/blog/setup/): a plan for the first 90 days - [What automation costs](https://example.com/blog/costs/): four cost levels with a worked example ## Optional - [Legal notice](https://example.com/legal-notice/) - [Privacy](https://example.com/privacy/)
The difference from robots.txt and sitemap.xml
| File | Answers | For whom | Standing |
|---|---|---|---|
| robots.txt | What may be read? | all crawlers | established standard |
| sitemap.xml | Which pages exist? | search engines | established standard |
| llms.txt | What matters and what is it about? | language models | proposal, inconsistently supported |
The third row is the decisive point: a sitemap.xml lists everything but weights nothing. An llms.txt weights and explains — that is the actual difference, and the reason the proposal exists.
Worth knowing
The most valuable part of an llms.txt arises in the writing, not in the serving. To produce it you have to decide: which ten pages are genuinely the most important? And how do you describe in one sentence what each of them is about?
Answering those two questions regularly uncovers that an important page is missing, that two pages say the same thing, or that your own focus cannot be named in a sentence. Those findings are useful regardless of whether the file is ever evaluated.
Is the effort worth it?
Yes, if …
… you need an overview of your most important pages anyway, your website has many pages and the focus is not recognisable at first glance, or you offer documentation intended to be read by machines.
No, if …
… your website has twelve pages and the focus is obvious. The file then supplies no information that would not be apparent in two clicks anyway.
Only afterwards, if …
… the basics are missing: readable HTML with no JavaScript dependency, clean headings, short summaries, structured data. Those work for certain; the llms.txt works perhaps.
The side effect
The list of the ten most important pages with one sentence each is at the same time a good foundation for internal linking and for onboarding new staff.
Producing it nearly always surfaces the same thing: the page hardest to describe in one sentence is usually the one that needs reworking. If you cannot say which question a page answers, it probably answers none.
That is why the hour spent on the llms.txt is well invested even if not a single machine reads it. It is a stocktake in file form.
Creating it in four steps
- Settle the ten most important pages. Important means: they answer your audience's most frequent questions — not: they have the most traffic.
- Write one sentence per page saying which question it answers. Do not repeat the title.
- Group by area and add an "Optional" section for legal and secondary pages.
- Serve it as
/llms.txt, with the content typetext/plainortext/markdown, and update it along with the next website rebuild.
Help me create an llms.txt for our website. Our details: - Company and location: [details] - What we offer, for whom: [description] - Our pages (address, title, what it covers): [paste list] Tasks: 1. Choose the at most ten most important pages – important meaning they answer our audience's most frequent questions. Justify the selection briefly. 2. Write one sentence per page saying which question it answers. Do not repeat the title. 3. Name the pages from my list where you cannot manage that – those probably need reworking. 4. Name pages that cover the same ground. 5. Output the finished llms.txt in Markdown, with an "Optional" section for legal pages. Do not invent pages or addresses.
In closing
The llms.txt is a low-stakes bet: an hour of effort, no risk, uncertain effect. If you have the basics done, take it along; if they are still ahead of you, start there.
The certain part of the benefit lies in the making: naming ten pages and writing a sentence for each is a stocktake that regularly uncovers gaps and duplications — regardless of whether the file is ever read.
Common questions
What is llms.txt?
A Markdown file at /llms.txt telling a language model in a few lines what a website is about and which pages are the most important — with one explanatory sentence each. It is a community proposal, not a binding standard.
Is llms.txt actually evaluated?
There is no public commitment from the large providers to evaluate the file. Anyone promising an effect cannot demonstrate it. Creating the file is therefore a low-stakes bet — an hour of effort, no risk, uncertain effect.
What is the difference from sitemap.xml?
A sitemap.xml lists all pages but weights none and explains nothing. An llms.txt names only the most important ones and describes in one sentence per page which question it answers. The robots.txt in turn governs what may be read at all — three files, three different questions.
Who is an llms.txt worth it for?
For websites with many pages whose focus is not recognisable at first glance, and for providers of machine-readable documentation. On a website with twelve pages and an obvious focus it supplies no information that would not be apparent in two clicks anyway.
What does it achieve regardless of evaluation?
The making itself. Naming ten pages as the most important and writing a sentence for each regularly uncovers that an important page is missing, that two say the same thing, or that a page cannot be described in one sentence — and that last one usually needs reworking.
Marketing that sets itself up
The Studio Engine beta is live. Claim your spot and help shape it from the start.
Join the beta →