llms.txt Generator
Generate an llms.txt file (per llmstxt.org spec) to tell LLM crawlers (ChatGPT / Claude / Gemini) about your site structure. Form input or bulk import from sitemap.xml.
Sections (H2)
Bulk import from sitemap.xml
※ extracts <loc> URLs only. Title guessed from URL last segment
llms.txt
⬇ llms.txtPlace the generated file at site root: https://example.com/llms.txt
📖 Where people get stuck
Builds a Markdown index of your site for language models, following the format proposed at llmstxt.org, with bulk import from sitemap.xml. This is a proposed convention, not a W3C or IETF standard — nobody guarantees which crawlers, if any, will read it. Rather than something you publish expecting a result, it is closer in practice to the exercise of producing one page that explains what your site actually contains.
| Case | What happens | What to do |
|---|---|---|
| Titles imported from the sitemap are just URL slugs | A sitemap.xml contains only <loc> elements, that is, URLs, so titles can only be guessed mechanically from the last path segment: /tools/cidr/ becomes cidr. That produces the real problem here — an llms.txt that is a list of URLs with no descriptions has almost no value as an index, because no reader, human or model, can tell which entry to look at. At that point it says nothing that sitemap.xml did not already say. |
Write one sentence per line saying what the page lets you do — [CIDR calculator](/tools/cidr/): work out host counts and address ranges for a subnet. Adding what the title alone cannot convey is the whole point of the format. And you do not need to list every page; in fact trimming to the twenty or fifty that are real entry points works far better as an index, because a thousand-line list conveys no priority at all and effectively tells the reader to read everything. The test for inclusion is whether the entry helps someone see what the site covers. |
| You published it and nothing seems to read it | First, seeing no effect is normal: reading llms.txt is entirely at the crawler discretion and there is no mechanism to tell you it happened. But before that, a surprising number of sites are simply not serving the file. Common causes are an SPA fallback returning index.html for /llms.txt, a CDN that does not expect a .txt extension and answers 404, or a Content-Type of application/octet-stream that turns it into a download. Being able to see it in your browser is not the same as a crawler being able to fetch it. |
Check it with curl -I https://example.com/llms.txt: you want a 200 and a Content-Type of text/plain or text/markdown. Also run curl -s ... | head -5 to confirm the body is not HTML — an SPA fallback shows up instantly that way. While you are there, check that robots.txt does not blanket-block the site root. Browsers put caches and service workers in the path, so make the judgement with curl, not the browser. |
| Treating it as a replacement for robots.txt | llms.txt is not access control. It is a catalogue of what you would like read; omitting a page does not block it, and listing one does not constitute consent to training on it. The mistake runs the other way too — neither publishing it makes models read your site, nor does withholding it keep them away. Crawlers still go by robots.txt and the actual link structure, exactly as before. |
If you want to control AI crawlers, do it in robots.txt, where you can name them individually: User-agent: GPTBot, ClaudeBot, PerplexityBot, Google-Extended. The two files serve completely different purposes, so having both is the correct setup — robots.txt says what may be crawled, llms.txt says what is worth looking at within that. Note that the list of bot names keeps growing, so whatever you write today will be out of date; plan on revisiting it. |
llms.txt and llms-full.txt are different files. The first is an index: URLs with a line of description each. The second is a full-text edition including the body of every page, which can run to several megabytes. Build the first one, and consider the full-text version only if you genuinely have technical documentation you want ingested. The real operational problem is staleness: a hand-written index always rots, and an llms.txt full of dead links is worse than none at all, because it hands the reader a wrong map. The only arrangement that survives is generating it from sitemap.xml in CI and keeping only the descriptions under human control. Alongside that, check for dead links on every deploy — generated does not mean correct, and pages you merged or deleted leave their links behind.
📖 How to Use
-
1
Fill in site metaSite name (H1) and summary required. Long description optional
-
2
Add sections & linksGroup meaningfully like "Docs", "Examples", "Optional". Each link is [Title](URL): description
-
3
Download & place at site rootOnce https://example.com/llms.txt returns 200, LLM crawlers will pick it up
❓ Frequently Asked Questions
What is llms.txt?
How is it different from llms-full.txt?
Different from robots.txt?
🐛 Found a bug or issue with this tool?
Free to use, no signup. Even just the steps to reproduce are helpful. Reports go directly to the operator and help us fix issues.
Thanks for your report!
Your report has been delivered to the operator and will be used to improve the tool.