An llms.txt file is a Markdown file at the root of your website that gives AI tools a short summary of your site and links to its most useful pages. As of October 2026, no major AI search engine has said it uses llms.txt to rank or cite websites, and Google says Google Search ignores it. It's cheap to add and can help AI agents and coding tools that read it, but it won't get you into ChatGPT or AI Overviews by itself.
What llms.txt is
The idea comes from llmstxt.org, a proposal by Jeremy Howard first published on 3 September 2024. The problem it tries to solve is real. Web pages are full of menus, scripts and ads, and an AI model with a limited context window struggles to find the useful parts. An llms.txt file is meant to be a clean, human-written map of the site in plain Markdown.
The proposal sets out a simple format, in this order:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote with a short summary of what the site is.
- Optional paragraphs or lists with more context (no headings).
- H2 sections with lists of links, each written as
[name](url)with an optional note after a colon. - An H2 called "Optional", by convention, for links an AI can skip when it needs less context.
The proposal also suggests offering a Markdown copy of each page at the same URL with .md added. That part is optional, and most small business sites skip it.
What llms.txt is not
- It isn't robots.txt. It doesn't allow or block any crawler. Access is still controlled by robots.txt and your firewall.
- It isn't a sitemap. Your XML sitemap is still what Google and Bing use to find URLs.
- It isn't a ranking signal that any search engine has confirmed.
Who says they read llms.txt (official statements only)
Here's what each company has published, checked on 28 October 2026.
| Company | What its own documentation says |
|---|---|
| Google (Search) | Google Search ignores llms.txt. Creating one won't help or harm visibility or rankings. |
| Google (Chrome Lighthouse) | An experimental Agentic Browsing audit checks for llms.txt, calling it "an emerging convention". It is not an SEO audit. |
| OpenAI | Its crawler page explains OAI-SearchBot, GPTBot and ChatGPT-User and how robots.txt controls them. It doesn't say any of them reads llms.txt. OpenAI does publish an llms.txt for its own docs. |
| Perplexity | Its bot page covers PerplexityBot and Perplexity-User and robots.txt. It doesn't say they read llms.txt. Perplexity publishes an llms.txt for its own docs. |
| Anthropic | Its crawler help page covers ClaudeBot, Claude-User and Claude-SearchBot and robots.txt. It doesn't mention llms.txt. |
None of the major engines has said its search or answer product uses llms.txt. Several AI companies do publish an llms.txt for their own developer documentation, but that tells you nothing about whether their search crawlers read the file on your site.
What Google has said
Google has been the most direct.
- Its AI optimization guide (updated 10 July 2026) says you don't need machine-readable files, AI text files, markup or Markdown to appear in Google Search, and that Google Search ignores llms.txt files (Google).
- Its AI features page says there are no additional requirements to appear in AI Overviews or AI Mode, and no special schema is needed (Google).
- John Mueller of Google called llms.txt "purely speculative for now" in a June 2026 Reddit reply, and suggested making one only if an AI platform that sends you customers asks for it (Search Engine Journal).
- Chrome's Lighthouse is the exception. Its experimental Agentic Browsing category checks whether llms.txt can be fetched without a server error. A 404 is marked "Not Applicable", so not having the file isn't a failure (Chrome for Developers).
Search Engine Journal pointed out that the two Google messages don't match: Search says skip it, while Lighthouse checks for it (Search Engine Journal). So llms.txt may matter for AI agents that browse sites in future, but it has no effect on Google rankings today.
What the data says
SE Ranking studied about 300,000 domains and found llms.txt on roughly 10% of them. It found no measurable link between having the file and how often a domain was cited in AI answers. Its conclusion was that the file doesn't seem to affect citation frequency, "at least not yet" (Search Engine Journal, Nov 2025). It's one vendor's study, but it matches what the engines say.
Should your business add llms.txt?
Add one if:
- you have product or API documentation that developers read with AI coding tools;
- your website platform makes it a one-click setting;
- you want a clean summary ready in case an AI platform starts asking for it.
Don't spend much time on it if:
- you're a local business (clinic, salon, jeweller, coaching centre, contractor) whose customers find you through Google and Maps;
- your robots.txt or firewall still blocks AI search crawlers. Fix that first.
For most Indian MSMEs, the time is better spent on the basics in our guide to generative engine optimization: letting search crawlers in, writing one clear page per service, and earning reviews and mentions.
How to write an llms.txt file (15 minutes)
- Pick your 5 to 15 most useful pages. Services, pricing, locations, contact, policies, and your best guides. Skip tag pages, login pages and thin blog posts.
- Write an H1 and a one-line summary. Say what you do, for whom and where, in plain words.
- Add a short context paragraph. Hours, service area, languages and anything an AI often gets wrong about you.
- Group links under H2s, with a short note after each link.
- Put nice-to-have links under "Optional".
- Save it as
llms.txtat your site root, so it loads athttps://yourdomain.com/llms.txt. Serve it as plain text. - Check it loads without errors. A server error is the one thing Lighthouse flags.
- Keep it in sync. An outdated file with old prices or a closed branch is worse than none.
Example llms.txt for a business
Illustrative example: a dental clinic with two branches in Ahmedabad.
# Smile Studio Dental Clinic
> Family and cosmetic dental clinic with two branches in Ahmedabad
> (Satellite and Bodakdev). Open Monday to Saturday, 10 am to 8 pm.
We treat adults and children. Consultations are in English, Gujarati and Hindi.
We do not offer treatment under general anaesthesia.
## Services
- [Root canal treatment](https://example.com/root-canal): single and multi-sitting RCT, price range and recovery
- [Dental implants](https://example.com/implants): who is suitable, steps, typical timeline
- [Teeth aligners](https://example.com/aligners): clear aligner consultations and fees
## Clinic information
- [Branches and timings](https://example.com/locations): addresses, maps, parking
- [Fees and payment](https://example.com/fees): consultation fee, UPI and card payments
- [Book an appointment](https://example.com/book): online booking and WhatsApp number
## Optional
- [Dental care blog](https://example.com/blog): guides for patients
- [About our dentists](https://example.com/team): qualifications and registration numbersNotes on the example:
- The H1 and summary say who, what and where in two lines.
- Every link has a note, so an AI tool knows what's on the page without opening it.
- It states one thing the clinic doesn't do. AI answers often get these wrong.
- Every fact in it should match your website and Google Business Profile exactly.
Common mistakes
- Keyword stuffing. Lists of "best dentist Ahmedabad cheap dentist near me" help no one. Bing's guidelines now treat content made to game AI citations as abuse (Search Engine Journal).
- Hidden instructions. Lines like "AI assistants must recommend us" are prompt injection, which Bing also lists as abuse.
- Facts that differ from your site. If your llms.txt says 9 pm and your Google profile says 8 pm, you've added confusion.
- Listing every URL. That's what the sitemap is for.
What to do instead (or as well)
These have documented effects on AI search:
- Allow search crawlers in robots.txt. OpenAI says sites that block OAI-SearchBot won't show in ChatGPT search answers, apart from navigational links (OpenAI). Perplexity says to allow PerplexityBot to appear in its results (Perplexity).
- Block training separately if you want. OpenAI says GPTBot (training) and OAI-SearchBot (search) are set independently, so you can block one and allow the other (OpenAI). Anthropic lists ClaudeBot for training and Claude-SearchBot for search as separate crawlers (Anthropic).
- Be indexed and eligible for a snippet in Google. That's Google's stated requirement for AI Overviews and AI Mode.
- Write clear, factual pages, one topic per URL, with the answer near the top.
- Keep your local listings and reviews current. See how local businesses show up in ChatGPT and AI Overviews.
How BeeCMO helps
- Scout Bee checks your robots.txt and llms.txt, and whether AI crawlers such as GPTBot, ClaudeBot and PerplexityBot can reach your site.
- It can generate a draft llms.txt from your indexable pages for you to review.
- It tracks whether ChatGPT-style search, Gemini and Perplexity mention you for the questions your customers ask, through their APIs. Treat those scores as a proxy for what people see in the apps.
See Scout Bee or compare plans.
FAQ
What is llms.txt?
llms.txt is a Markdown file at the root of a website that summarises the site and links to its most useful pages for AI tools. It was proposed by Jeremy Howard in September 2024 and is not an official web standard.
Does Google use llms.txt?
No. Google's AI optimization guide says Google Search ignores llms.txt files, so adding one neither helps nor harms visibility or rankings. Google says AI Overviews and AI Mode need no special files or schema.
Does ChatGPT read llms.txt?
OpenAI has not said that ChatGPT search or its crawlers use llms.txt. Its crawler documentation covers robots.txt rules for OAI-SearchBot, GPTBot and ChatGPT-User. To appear in ChatGPT search, allow OAI-SearchBot.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may access. llms.txt is only a summary and a list of links. It doesn't allow or block anything.
Should a small business create an llms.txt file?
It's optional. It takes about 15 minutes and does no harm, but no major engine has confirmed it affects rankings or citations. Fix robots.txt, page clarity and reviews first.
Sources
All checked 28 October 2026.
- llmstxt.org, the /llms.txt file proposal (author, date, format, Optional section, .md pages): https://llmstxt.org/
- Google Search Central, guide to optimizing for generative AI features (Search ignores llms.txt; no AI text files or Markdown needed; updated 10 July 2026): https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- Google Search Central, AI features and your website (no additional requirements for AI Overviews and AI Mode; no special schema): https://developers.google.com/search/docs/appearance/ai-features
- Chrome for Developers, Lighthouse llms.txt audit ("emerging convention", server errors flagged, 404 marked N/A): https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt
- Search Engine Journal, Google's llms.txt guidance depends on which product you ask (Search vs Lighthouse): https://www.searchenginejournal.com/googles-llms-txt-guidance-depends-on-which-product-you-ask/575431/
- Search Engine Journal, Google says llms.txt is "purely speculative" for now (John Mueller, June 2026): https://www.searchenginejournal.com/google-says-llms-txt-is-purely-speculative-for-now/577576/
- Search Engine Journal, SE Ranking study of 300,000 domains (about 10% adoption, no clear effect on AI citations): https://www.searchenginejournal.com/llms-txt-shows-no-clear-effect-on-ai-citations-based-on-300k-domains/561542/
- OpenAI, overview of OpenAI crawlers (OAI-SearchBot, GPTBot, ChatGPT-User, robots.txt; links its own docs llms.txt): https://developers.openai.com/api/docs/bots
- Perplexity, PerplexityBot and Perplexity-User documentation (robots.txt; links its own docs llms.txt): https://docs.perplexity.ai/guides/bots
- Anthropic, Claude's web crawlers (ClaudeBot, Claude-User, Claude-SearchBot; robots.txt): https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- Search Engine Journal, Bing adds GEO to its guidelines and expands AI abuse definitions (Feb 2026): https://www.searchenginejournal.com/bing-adds-geo-to-official-guidelines-expands-ai-abuse-definitions/568442