Does llms.txt Help Your Site Get Cited by AI Engines
Amit Malvi
Founder, SearchAxe. Former Senior SEO Manager, Invideo AI
Published Updated 11 min read
Quick answer
Does llms.txt help your site get cited by AI engines? Not on the evidence available in October 2026. Google says Search ignores the file, no major AI company says its crawlers use it to choose sources, and SE Ranking's study of 300,000 domains found no link between having one and being cited. It costs little to publish, so add one if coding agents read your docs, but not as a citation tactic.
On this page
- Does llms.txt help your site get cited by AI engines?
- What have Google and the AI companies said about llms.txt?
- What does the evidence say about llms.txt and AI citations?
- Why don't AI engines rely on llms.txt to choose sources?
- When is an llms.txt file worth publishing?
- How do you implement llms.txt safely?
- What should you do instead of relying on llms.txt?
Key takeaways
- llms.txt is a September 2024 proposal for a Markdown reading list at your site root, not a standard any search engine has adopted.
- Google's own guide says Google Search ignores llms.txt, so it neither helps nor harms your Google rankings or AI features.
- SE Ranking's November 2025 study of 300,000 domains found llms.txt had no measurable effect on how often AI systems cite a site.
- OpenAI, Anthropic and Perplexity publish llms.txt files for their own developer docs, which points to agents reading docs, not to citation ranking.
- Citations come from crawlable, indexed pages that state your facts clearly, so fix those before spending time on llms.txt.
No, llms.txt does not help your site get cited, at least not on any evidence published so far. If you lead growth at an AI startup and llms.txt has turned up in a Slack thread, a growth newsletter or a vendor proposal as a quick fix for AI visibility, this is the evidence-based answer. Below is what the file is, what Google and each AI company have said about it, what the one large study found, when the file is still worth having and how to publish it without creating new problems.
The format is simple. The llms.txt proposal requires only an H1 with the site or project name, then allows a blockquote summary, optional notes and H2 sections that list links with a one-line description each. A section named Optional marks links an agent can skip when it needs a shorter context. The proposal also suggests serving a clean Markdown copy of each page at the same URL with .md appended.
Does llms.txt help your site get cited by AI engines?
No published evidence shows that it does, and the way AI search works explains why. Engines cite pages they retrieve from a search index, whether Google's, Bing's or their own crawler's. An llms.txt file does not add pages to any of those indexes, and no engine has said it uses the file to rank or select sources. A page that is crawlable, indexed and clearly written can be cited without one, and a page missing from the index will not be cited with one.
Google states the mechanism directly. Its guide to optimizing for generative AI features says these features use retrieval-augmented generation that relies on Google's core Search ranking systems to pull pages from the Search index. That is the path to a citation in Google AI Overviews and Google AI Mode, and llms.txt is not on it. We cover the cross-engine version of this in how AI search engines decide which sources to cite.
What have Google and the AI companies said about llms.txt?
Google is the only company that has addressed llms.txt in its search documentation, and a June 15, 2026 changelog entry records the note saying Search ignores the file. Chrome's Lighthouse llms.txt audit checks only that the file does not return a server error. OpenAI, Anthropic and Perplexity have not said their crawlers read it, and Anthropic's crawler help article does not mention it, although all three publish one for their own developer documentation. We found no public statement from Microsoft, and its October 2025 guide to AI search answers does not mention it.
| Company | What it has said or done | Source |
|---|---|---|
| Google Search | Search does not use llms.txt; creating one will neither harm nor help visibility or rankings; it is fine to keep one for other services | AI optimization guide, note added June 15, 2026 |
| Google Chrome | Lighthouse includes an llms.txt audit under agentic browsing; it fails only on a server error and marks a 404 as not applicable | Chrome for Developers, Lighthouse llms.txt audit |
| OpenAI | Its crawler documentation does not describe llms.txt as an input; the docs site points readers to its own llms.txt index and .md page copies | Overview of OpenAI crawlers |
| Anthropic | Its crawler help article does not mention llms.txt; its developer docs publish an llms.txt index | Anthropic crawler help article, platform.claude.com/llms.txt |
| Perplexity | Its crawler documentation does not describe llms.txt as an input; its docs tell AI agents to use its llms.txt index | Perplexity crawlers page |
| Microsoft | No public statement found; its guide to AI search answers does not mention llms.txt | Microsoft Advertising blog, October 2025 |
The Google wording is unusually blunt. The guide lists llms.txt under things you can ignore, and says creating such files "will neither harm nor help" your visibility in Google Search. John Mueller made the same point earlier. In an April 2025 Reddit thread reported by Search Engine Journal, he said none of the AI services had said they use llms.txt, that server logs show they do not even check for it, and compared it to the keywords meta tag. In June 2026, Search Engine Journal reported him arguing that a file in which every site describes itself cannot help a system tell sites apart.
The vendor files tell you who llms.txt is for. OpenAI's crawler page points readers to its documentation index in llms.txt and to .md copies of each page, Anthropic's developer docs publish one, and Perplexity's crawler page carries a note telling AI agents to use its llms.txt index. These are documentation sites aimed at coding agents and developers, not marketing pages trying to earn citations.
Find out which AI engines recommend you
A free consultation on where you appear today across AI engines and search.
Schedule a Free ConsultationWhat does the evidence say about llms.txt and AI citations?
The largest study we found shows no effect. SE Ranking modelled citation frequency across 300,000 domains and found that llms.txt added noise rather than signal. Adoption was also low and did not rise with traffic, which is the opposite of what you would expect if the file worked.
The same study found adoption of 9.88% among the lowest-traffic sites and 8.27% among sites with more than 100,000 visits. A correlation study cannot prove the file never matters, and engines change. What it does show is that the sites AI engines cite most did not get there by publishing llms.txt, so it should not sit near the top of your list.
Why don't AI engines rely on llms.txt to choose sources?
Because it is a claim the site makes about itself, and engines can read the site directly. A file that says your product is the leading AI video editor carries no more weight than a homepage headline saying the same thing. Engines judge a page by what it says, whether it answers the question, and whether other sources agree, and none of that comes from llms.txt.
There is also a spam problem. Any signal a site controls completely gets gamed, which is how the keywords meta tag lost its value. Google's guide makes a related point about scale: writing separate content for every possible fan-out query to manipulate AI answers breaks its scaled content abuse policy. A file built to tell AI systems what to say about you falls into the same category of self-description engines have learned to discount.
When is an llms.txt file worth publishing?
It is worth publishing when software agents already read your site to get something done, which in practice means developer documentation and API references. Mueller carved out exactly this case, an automated system that is already on your site and needs directions. If developers use coding assistants to write against your API, a clean index of your docs with Markdown copies can save the agent from parsing navigation and scripts.
- Worth it: an API product, SDK or developer platform with public docs that coding agents fetch.
- Worth it: a large docs site where the useful pages are buried under versioned or auto-generated ones.
- Harmless but low value: a marketing site with a dozen pages that are already crawlable.
- Not worth it: any plan that treats llms.txt as the fix for missing AI citations.
If your docs are where most of your citable content lives, the bigger gains are indexability, canonical structure and clear reference pages, which we cover in SEO for an AI product documentation site.
How do you implement llms.txt safely?
Keep it short, keep it true and keep it from creating duplicate pages. The file should list pages that already exist and say nothing those pages do not say. The risk is not the file itself but the Markdown copies the proposal suggests, because Google says it may crawl and index many kinds of files beyond HTML.
# Clipwise
> Clipwise is an AI video editor that turns long recordings into captioned short clips for TikTok, Reels and YouTube Shorts.
## Docs
- [Quickstart](https://www.example.com/docs/quickstart.md): Upload a video and export your first clip
- [API reference](https://www.example.com/docs/api.md): Endpoints, authentication and rate limits
- [Webhooks](https://www.example.com/docs/webhooks.md): Events sent when a clip finishes rendering
## Product
- [Pricing](https://www.example.com/pricing): Plans, limits and what each includes
- [Integrations](https://www.example.com/integrations): Supported storage and publishing tools
## Optional
- [Changelog](https://www.example.com/changelog): Release notes by monthPublishing llms.txt without side effects
- List only public, indexable pagesNever link gated, staging or logged-in URLs. The file is public and anyone, including competitors, can read it.
- Copy facts, do not add themEvery claim in the summary should match your homepage and product pages, so the file never contradicts what engines find on the site.
- Serve it as plain text with a 200 statusChrome's Lighthouse audit fails the file only on a server error, so a broken route is worse than having no file.
- Keep Markdown copies out of search resultsIf you serve .md versions of pages, send an X-Robots-Tag noindex header on them so they do not compete with the HTML pages in Google and Bing.
- Update it with releasesAdd it to the docs release checklist so renamed or deleted pages do not leave dead links behind.
- Check your logs before judging itGrep for requests to /llms.txt and note the user agents. That tells you who actually reads it, which is more useful than any claim about it.
What should you do instead of relying on llms.txt?
Work on the inputs engines do use. Every one of these has a documented mechanism behind it, which llms.txt does not.
- Let the search crawlers in. OAI-SearchBot, Claude-SearchBot, PerplexityBot and Bingbot need access, as we explain in should you allow GPTBot, ClaudeBot and PerplexityBot.
- Get indexed in Google and Bing. ChatGPT search and Copilot draw on Bing's index, and Google's AI features draw on Google's.
- State your product facts on the page. Name the category, the job it does and who it is for in plain words near the top.
- Mark up those same facts with structured data, covered in which schema markup helps AI search engines understand your product.
- Earn mentions on the third-party pages engines already cite for your category.
This is the work an LLM SEO program should spend its first months on. If a proposal leads with llms.txt, ask what evidence it rests on.
Frequently asked questions
Is llms.txt an official web standard?
No. llms.txt is a proposal Jeremy Howard published on llmstxt.org in September 2024. Google, OpenAI, Anthropic, Perplexity and Microsoft have not said their crawlers use it to select sources, and Google says its Search ignores the file. Several AI companies publish an llms.txt for their own developer docs, which shows it is useful as a docs index for agents, but that is different from being a standard engines rely on.
Will adding llms.txt hurt my Google rankings?
No. Google's guide to optimizing for generative AI features says creating llms.txt or similar files will neither harm nor help your visibility or rankings in Google Search, because Google Search ignores them. The one practical risk is separate: Markdown copies of your pages, which the proposal suggests, could be crawled and indexed alongside the HTML versions, so add a noindex header to them if you publish them.
Does llms.txt replace robots.txt?
No. They do different jobs. robots.txt tells crawlers which URLs they may fetch, and the major AI crawlers from OpenAI, Anthropic and Perplexity document that they follow it. llms.txt is a reading list with no access rules at all. Listing a page in llms.txt does not allow a blocked crawler to fetch it, and leaving a page out does not block anything.
How can I tell whether AI bots read my llms.txt file?
Check your server access logs for requests to /llms.txt and look at the user agents making them. Compare those agents with the crawler names OpenAI, Anthropic and Perplexity document, and confirm the IP addresses against the ranges each company publishes, since user agents can be faked. In April 2025 John Mueller said server logs showed AI services were not even checking for the file.
Which kind of startup should publish llms.txt?
A developer-facing startup with public API docs, SDK guides or a large reference site gets the most from it, because coding agents and AI assistants fetch documentation to complete tasks. OpenAI, Anthropic and Perplexity all publish one for their own docs. A startup with a small marketing site gains little, since its pages are already easy to read, and should spend the time on crawler access and clearer product pages.
