llms.txt explained: what it does, and what it doesn't
A small text file has picked up a big reputation. Here is what llms.txt actually is, what no AI company has promised about it, and how to write one properly if you decide it's worth fifteen minutes.
ARAgile Ranker TeamSearch & AI Visibility deskPublished28 Jul 2026Updated2 Aug 2026Read7 minReviewed bySearch & AI Visibility desk
On Google
does llms.txt help you rank in ai answers
AI Overview
llms.txt is a proposed file that points AI systems to a site's key pages. It is not a ranking factor, and major AI providers have not committed to reading it, so crawler access and clear content matter more.
agileranker.com
Example answer
Skim · 2 minFull · 7 minShare
The quick answerCopy
llms.txt is a plain-text Markdown file at your domain root that tells AI systems which pages best represent your site. It is a proposed convention, not a ranking factor, and no major AI model guarantees it reads one. Write one if it's cheap for you, but fix crawler access first.
llms.txt is a curated reading list for AI systems, served at yourdomain.com/llms.txt.
It is a proposal from 2024, not a standard. No major AI company has committed to using it.
It controls nothing. Blocking or allowing AI crawlers is still the job of robots.txt.
Documentation-heavy and software sites benefit most; local businesses should do the basics first.
Only list pages that exist, are public and say the same thing as the file.
01
What is llms.txt?
llms.txt is a Markdown file served at the root of your domain that gives AI systems a short, curated map of your site: who you are, what you do, and which pages explain it best. It was proposed in September 2024 by Jeremy Howard of Answer.AI.
The reasoning is practical. A language model reading a normal web page has to wade through navigation, cookie banners, scripts and footer links before it reaches the content. A clean Markdown summary with links to the pages that matter is easier to use, especially when an assistant fetches your site live to answer a question.
The proposal also suggests offering Markdown copies of key pages at the same address with .md added. A popular variant, llms-full.txt, bundles the full text of a site into one file. That variant came from documentation tools rather than the original proposal, which tells you how early and informal the whole convention still is.
02
Does llms.txt help you rank in AI answers?
Not on any public evidence. No major AI company has committed to reading llms.txt when it builds answers, and Google has said publicly that it does not use the file for Search, which includes AI Overviews and AI Mode.
You may see bots request /llms.txt in your server logs. That shows something fetched it, not that it changed whether your brand was cited. Whether ChatGPT, Gemini or Perplexity names you still depends on the same things as before: can their crawlers reach your pages, is your content clear enough to quote, and do other trusted sites describe you consistently.
So treat llms.txt as cheap housekeeping, not a strategy. If someone sells it to you as the key to AI visibility, ask them which engine has confirmed it uses the file. Practices change quickly in this area, so check each provider's own documentation rather than relying on blog posts, including this one.
03
How is llms.txt different from robots.txt?
robots.txt controls access: it tells crawlers which URLs they may fetch, and the major AI crawlers say they respect it. llms.txt controls nothing; it only suggests what is worth reading. To allow or block AI crawlers, robots.txt is still the tool.
AI crawlers have their own user-agent names, such as GPTBot and OAI-SearchBot for OpenAI, ClaudeBot for Anthropic and PerplexityBot for Perplexity. Google-Extended is a separate control that governs whether Google may use your content for its Gemini models; it does not affect whether you appear in Google Search or AI Overviews.
The most common problem we find on audits is not a missing llms.txt. It is a site that has carefully written one while its robots.txt, firewall or CDN bot-protection setting quietly blocks the AI crawlers it wants to reach. Some security services now block AI bots by default, so check yours before anything else.
Want this done for you?
Our AI Search Optimisation team can take it on — or start with the free AI visibility check and see where you stand on Google and five AI engines. No obligation.
Start with an H1 carrying your business name and a one-paragraph summary in a blockquote. Then add H2 sections listing your most useful pages as Markdown links, each with a one-line note. An optional section at the end marks links that can be skipped.
Pick the pages that answer the questions buyers actually ask an assistant: what you do, who it's for, where you operate, what it costs, and how to contact you. A list of every blog post defeats the purpose. Ten to thirty well-chosen links is plenty for most businesses.
Write the summary as plain fact. “Example Interiors is a residential interior design studio in Pune, working on apartments and villas across Maharashtra since 2016” is useful. “We are passionate about transforming spaces” tells a model nothing it can repeat.
Services or product pages, one link each
Pricing page or a clear cost explainer
About page with founding year, location and team
Your main FAQ page and key policies
Contact page with address and service areas
05
Which businesses should bother writing one?
Software companies and anyone with substantial documentation gain the most, because many developer tools and coding assistants can be pointed at a docs site's llms.txt. For a local or service business, it is a short job worth doing only after the basics are in place.
If you sell an API, a SaaS product or anything people configure, your documentation is often read by machines on a user's behalf. A tidy llms.txt, and Markdown copies of key docs, make that reading more reliable. This is where most real adoption has happened so far.
If you run a clinic in Bhopal or an accountancy practice in Leeds, your effort is better spent on a complete Google Business Profile, consistent business details across directories, and pages that answer buyer questions directly. Add llms.txt afterwards; it costs little and does no harm.
06
How do you stop llms.txt from going stale?
Tie it to your site changes. Whenever you launch, rename or retire a service page or change prices, update the file in the same release. A file that points to dead pages or old prices is worse than having no file.
The risk with any hand-written summary is drift. Six months after launch the site has moved on, but the file still describes last year's services. If an assistant does read it, you have handed it outdated facts with your own name on them.
Add the file to whatever checklist you already use for site updates, and check every link once a quarter. If your CMS or SEO plugin can generate it automatically from your page list, use that and review the output rather than writing it from scratch.
07
How do you write and publish an llms.txt file?
1Check crawler access firstOpen yourdomain.com/robots.txt and your CDN or firewall settings. Make sure the AI crawlers you want, such as GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot, are not blocked.
2Choose the pages that matterList the ten to thirty pages that best answer buyer questions: services, pricing, about, FAQs, policies and contact. Leave out tag pages, thin posts and anything behind a login.
3Write the headerStart with “# Your Business Name”, then a blockquote line beginning “>” with a factual one-paragraph summary: what you do, for whom, where, and since when.
4Group the linksAdd H2 sections such as “## Services” and “## Company”. Under each, write “- [Page title](full URL): one-line note on what the page covers”.
5Mark the skippable pagesPut lower-priority links, such as older articles, under a final “## Optional” heading so tools short on space know what to drop.
6Publish and testUpload the file to your domain root as plain text, open yourdomain.com/llms.txt in a browser to confirm it loads, and click every link once.
08
What are the common mistakes?
01Treating it as a ranking lever
No engine has confirmed it uses the file to choose sources. Put your effort into crawl access, clear answers and consistent business facts first.
02Listing pages AI crawlers can't reach
Linking to pages that robots.txt, a login or a bot-protection rule blocks sends readers into a dead end. Test each link logged out.
03Claims that aren't on your site
If the file says “24-hour service” and your pages don't, you've created a contradiction. The file should summarise your site, not add to it.
04Dumping every URL
A list of five hundred blog posts is a sitemap, not a guide. Curate, and use the Optional section for anything secondary.
09
Questions people ask
Will llms.txt stop AI companies training on my content?
No. It gives no instructions about training or access. To opt out of specific AI crawlers, use robots.txt rules for their user agents, and check each provider's documentation for what each crawler is used for.
Can adding llms.txt hurt my SEO?
There's no sign that it does. Search engines treat it as an ordinary text file. The only real risk is publishing outdated or contradictory facts about your business, which is why it needs maintaining.
Does it have to be Markdown?
The proposal specifies Markdown because language models read it easily and it stays readable for humans. Keep it to headings, a blockquote summary and bulleted links; no HTML or styling is needed.
Should I also create llms-full.txt?
Only if you have extensive documentation that developers or their tools read. For a typical business site, a short llms.txt pointing to good pages is enough, and a full-text bundle is one more thing to keep current.
How do I add one on WordPress?
Several popular WordPress SEO plugins can now generate an llms.txt file from your pages. Use the generated version as a draft, remove low-value links and check the summary is accurate before switching it on.
Sources
The /llms.txt file proposal — llmstxt.orgllmstxt.org/
Cite this article: Agile Ranker Team (2026). llms.txt explained: what it does, what it doesn't, and how to write one. Agile Ranker Insights. Updated 2 Aug 2026.
AR
Written by
Agile Ranker TeamSearch & AI Visibility desk · Agile Ranker
The Agile Ranker team that runs search, AI visibility, paid media and content for clients in ten markets. Notes are checked against current platform documentation before publishing.
Reviewed by the Search & AI Visibility desk
Included with every plan
A free 5-page website for your business.
A fast, AI-readable site that turns the traffic we bring into enquiries — built to the same standard as our paid web work.
5 pages, your choiceHome, About, Services, Key service, Contact
Free with any monthly plan from $499. Domain, hosting and paid tools are bought by you and stay in your name; shops and booking systems are quoted separately. The 48-hour clock starts once your plan is active and the launch checklist is done.
yourbusiness.comLIVE
HomeAboutServicesKey serviceContact
Your logo
Book a call
Schemallms.txtGA4Fast on mobile
FREEwith any plan
48h
Keep reading
Three notes that pick up from here.
Each one opens with its quick answer — skim first, read if it's yours.