Is llms.txt worth it? What Google and the data say

An llms.txt file is a short, plain-text guide to your website, written for AI tools. This post answers one question: is it worth publishing one, given what Google has said and what the data shows?

What llms.txt is

The idea comes from Jeremy Howard of Answer.AI and fast.ai, who published it on 3 September 2024. His proposal (version 2 was updated on 10 August 2026) asks sites to add a Markdown file at /llms.txt that gives "brief background information, guidance, and links to detailed markdown files."

The format is simple. The only required part is an H1 with the name of the site. After that you can add a short blockquote summary, some plain text, and lists of links, each with a short note. The proposal says the file is meant to help agents while they use a site ("inference"), not to feed model training: "Our expectation was that llms.txt would mainly be useful for inference rather than training."

What it isn't matters just as much. Despite the name, it works nothing like robots.txt. As Ahrefs puts it, it "controls nothing and blocks nothing."

llms.txt in one look

It is

• A Markdown file at /llms.txt
• A short summary of your site with links
• Written for AI agents and coding tools

It is not

• A robots.txt-style rulebook
• A ranking signal for Google Search
• A way to make AI tools cite you

What Google has said

Google's position is now written into its own documentation, and it's clear.

Google's guide to optimizing for generative AI features on Search (updated 10 July 2026) has a "mythbusting" section. It lists llms.txt files among the things you can ignore:

"You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them."

It then adds: "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."

Google's documentation updates page lists a 15 June 2026 entry, "Clarifying guidance on llms.txt files," which says the note was added "to address questions from the community."

Google's people had said similar things earlier. At a Search Central Live Deep Dive event in Bangkok in July 2025, Gary Illyes told attendees, "To get your content to appear in AI Overview, simply use normal SEO practices. You don't need GEO, LLMO or anything else." (We know of reports that he also said Google wouldn't use llms.txt, but we couldn't open that report, so we don't rely on it.)

John Mueller has been more blunt. As Search Engine Journal reported on 15 June 2026, he said on Google's Search Off the Record podcast that a self-written file can't help AI systems pick between sites, because "by design, can't trust what is here as a way of differentiating between different websites." He did see one use: "If someone is already on your website, maybe some kind of automated system is helpful." Earlier, SEJ reported he compared llms.txt to the keywords meta tag.

There's one wrinkle. Google's Chrome team added an experimental llms.txt check to Lighthouse, and its documentation (updated 5 May 2026) calls the file "an emerging convention." It says providing one is "optional at the moment," and marks the audit "Not Applicable" if the file is a 404. That's about helping agents browse a site. It's separate from how Google Search ranks pages.

What studies of real traffic found

Opinions are cheap here, but there is some data. We opened both studies below and read their methods.

Ahrefs, 15 June 2026. Ahrefs looked at all 137,210 domains in its Web Analytics product that received traffic in May 2026, and checked each for an llms.txt file and who requested it. It found:

• 28% of the domains publish one. Ahrefs says its customers lean technical, so treat that as an upper bound.
• Of the roughly 38,000 domains with a valid file, 97% saw no requests for it in May. The remaining 3% (about 1,100 domains) received all the traffic.
• Of the requests that did reach files, 96% came from bots, and 19.5% came from named AI tools. Another 12% came from tools that audit or study llms.txt.
• Ahrefs found zero requests from AI bots for llms.txt files that didn't exist. In other words, AI tools aren't going around hunting for the file.

llms.txt in Ahrefs' data (May 2026)

• Domains with a file28%
• Files never requested97%
• Requests from bots96%
• Requests from named AI tools19.5%

Ahrefs' own caveats matter. A "fetch" doesn't mean a tool read or used the file. The study counts requests, not citations. And it didn't check whether files followed the spec.

SE Ranking, 7 November 2025. SE Ranking analyzed nearly 300,000 domains. Only 10.13% had an llms.txt file. Using statistical analysis and a machine learning model, it found no relationship between having the file and how often a domain was cited in AI answers. Removing the file from its model improved the model's accuracy. The authors say their results depend on the dataset and model they used.

Both studies come from SEO software companies. They measure different things (requests to the file, and citations in answers), and both point the same way.

What it costs to publish one

Not much. The proposal says the file should stay small enough to fit in an AI tool's context, and ours is about 4 KB. The costs are small but real:

• Writing it. Mostly deciding what to leave out.
• Keeping it true. A stale file describes services you've dropped or prices that changed. Ahrefs also warns that agents are built to trust the file and that some researchers are already probing it for prompt-injection tricks. That's a reason to keep the file to plain facts and links, and to control who can edit it.
• Attention. If you do it for "AI visibility," you'll wait for a result the data doesn't support.

Website platforms are starting to generate these files for you, which lowers the cost further. Ahrefs mentions Wix as an example.

Why we publish one anyway

Magfellow publishes an llms.txt file. We do it as a low-cost courtesy, with no promise that it changes anything.

Here's what's in it, so you can judge it for yourself. It has a name and a one-paragraph summary of who we are. It explains how buying works, in six short steps. It states the 365-day placement guarantee and says that some publishers label paid content, and that we tell clients before they approve a site. It lists our services and industries, our country pages, our case studies, blog and reviews, and our policies. It only repeats what's already on our site, so it needs updating whenever the site changes.

We don't link to the file from our pages. We also don't sell "llms.txt optimization" or any service that promises AI citations, and we don't think anyone should. We have no evidence the file does anything for us, and the studies above give us none.

Is llms.txt worth publishing? Our recommendation

Our view: treat llms.txt as optional housekeeping. Don't treat it as an AI visibility tactic.

• If you run developer documentation or an API, publish one. That's what the format was built for, and Ahrefs found coding agents among the tools that fetch it. Several AI companies serve one for their own developer docs. (We checked on 6 October 2026: OpenAI's, Anthropic's and Google's Gemini API docs all do.)
• If you run a normal business site, it's fine to skip it. If you publish one, keep it short, true and low-effort.
• If someone sells it as a way to get cited by AI, ask for proof. Google says its Search ignores the file, and the studies we found show no link to citations.

What to do instead of worrying about llms.txt

The effort is better spent on things Google's guide says do matter: unique, helpful content; pages Google can crawl and index; and honest reasons for other sites to talk about you. We cover the last point in brand mentions vs backlinks and expert quotes and AI answers.

If you decide to publish one

• Keep it shortA summary and the pages that matter
• Only plain factsNothing instruction-shaped, nothing you wouldn't put on the site
• Match your siteUpdate it when services, prices or policies change
• Control who edits itA stale or altered file misleads every tool that reads it
• Don't expect resultsCheck your logs before investing more time

For the wider question of how AI tools choose their sources, see how AI assistants pick sources. And if you'd like to talk about where links and mentions fit into your plan, our services page shows what we do.

Sources

• llmstxt.org: The /llms.txt file, v2 (published 3 September 2024; modified 10 August 2026)
• Google Search Central: Optimizing your website for generative AI features on Google Search (updated 10 July 2026)
• Google Search Central: Documentation updates (entry of 15 June 2026)
• Chrome for Developers: Lighthouse llms.txt audit (updated 5 May 2026)
• Search Engine Roundtable: Google says normal SEO works to get into AI Overviews (Gary Illyes, Bangkok, 23 July 2025; reported 24 July 2025)
• Search Engine Journal: Google's Mueller says llms.txt can't help LLMs differentiate sites (15 June 2026)
• Search Engine Journal: Google's llms.txt guidance depends on which product you ask (20 May 2026)
• Ahrefs: We analyzed 137K sites, 97% of llms.txt files never get read (15 June 2026)
• SE Ranking: LLMs.txt study, nearly 300,000 domains (7 November 2025)
• Magfellow's llms.txt

Checked on 6 October 2026.