LLMs.txt is a plain-text file designed to help AI systems understand your website, but no major AI provider has confirmed that they use it. Ahrefs analysed 137,000 sites and found that 97% of LLMs.txt files never get read. The SEO Framework tracked 57 AI bots over 6 months; zero accessed LLMs.txt.
Google's June 2026 guidance explicitly states the file does not affect Search rankings or AI Overviews. If you want AI visibility, invest in technical SEO, topical authority, schema markup, and brand citations instead. This article breaks down all the evidence and shows you what actually works.
Introduction
LLMs.txt is everywhere right now. SEO plugins are shipping it. Twitter threads are calling it essential. And it feels like every GEO guide published in 2026 includes a section on setting it up.
But there's a question nobody seems to be asking:
Does any AI bot actually read it?
The answer, backed by data from 137,000+ websites, multiple independent experiments, and Google's own official guidance, is no. Not in any way that matters.
In this article, I'll walk you through the actual evidence, explain why the hype got ahead of reality, and share what's genuinely driving AI visibility right now based on real-world projects where I've helped generate 1.5M+ clicks and thousands of organic leads.
What Is LLMs.txt?
LLMs.txt is a plain text file you place at the root of your website, similar to robots.txt or sitemap.xml. The concept was proposed by AI researcher Jeremy Howard in September 2024 to solve a specific technical problem: LLM context windows are too small to process most websites in full, and converting complex HTML into clean text for AI systems is messy and imprecise.
The idea was straightforward: create a structured Markdown file at yoursite.com/llms.txt that lists your most important pages with brief descriptions.
AI crawlers could then read this file and understand your site quickly without parsing JavaScript, navigation menus, and ad code.
Think of it as a curated table of contents designed specifically for AI consumption.
The concept made sense on paper, which is exactly why it spread so fast.
Why LLMs.txt Went Viral in the SEO Space?
The pitch was compelling: if Google has robots.txt, why shouldn't ChatGPT have LLMs.txt?
That analogy resonated with SEO professionals and business owners who were already anxious about AI search visibility. Within months, Yoast SEO added LLMs.txt generation. Rank Math followed. All in One SEO enabled it by default. WordPress plugins dedicated entirely to LLMs.txt started appearing.
Suddenly, it felt like not having an LLMs.txt file meant falling behind.
But here's what got lost in the rush: the analogy doesn't hold up. Robots.txt works because Google publicly committed to respecting it decades ago. Every major search engine reads it. There's documented, measurable behaviour.
LLMs.txt has none of that. No major AI provider, not OpenAI, not Google, not Anthropic, not Meta, has publicly committed to reading or acting on the file.
The concept was promising. The adoption was premature.
The Evidence: Four Studies, One Consistent Result
The strongest argument against LLMs.txt isn't opinion. It's data. Four independent studies, conducted by different teams using different methodologies across different timeframes, all arrived at the same conclusion. Here's what each one found.
Study 1: Ahrefs 137,000 Sites Analysed
The most comprehensive study to date comes from an Ahrefs case study, published in mid June 2026. They analysed server logs and live traffic across 137,000 domains.
The headline finding: 97% of LLMs.txt files never get read by anyone.
Among the 3% that did receive visits, the breakdown tells its own story. GPTBot was the most active AI bot, followed by Claude-Code, but both were primarily accessing developer documentation sites, not business websites.
Around 12% of all LLMs.txt requests came from the SEO industry itself: GEO tools, LLMs.txt checker tools, and researchers studying adoption patterns.
The Chrome Lighthouse LLMs.txt audit (more on this later) produced roughly 1 in 1,000 fetches.
One finding stood out: zero requests came from AI bots for LLMs.txt files that don't already exist. AI bots never go looking for the file. They only occasionally encounter it when it's already there.
Key Data Point
Of the 3% of LLMs.txt files that received any visits, the majority were developer documentation sites, not business websites, e-commerce stores, or service brands. Around 12% of all requests came from the SEO industry itself: GEO tools, checker tools, and researchers. The audience for LLMs.txt is, ironically, the people debating LLMs.txt.
Study 2: The SEO Framework - 6 Months, 57 AI Bots, Zero Reads
The SEO Framework, a well-known WordPress SEO plugin, ran a controlled experiment from October 2025 to April 2026. They hosted a website with an LLMs.txt file and tracked every server log entry for six months.
| Metric | Result |
|---|---|
| Total website hits | 4,900,000+ |
| AI bot hits (all bots combined) | 180,219 |
| AI bot hits on LLMs.txt | 0 |
| Unique AI/LLM bots identified | 57 |
| Bots impersonating Google crawler | 23 |
Nearly five million total hits. Over 180,000 from AI bots. And not a single AI bot accessed the LLMs.txt file. Not ChatGPT. Not Claude. Not Perplexity. Not Meta. Not Apple. None.
Study 3: OtterlyAI 90-Day GEO Experiment
OtterlyAI ran a focused 90-day experiment monitoring AI bot behaviour on their website after implementing LLMs.txt.
Out of 62,100+ total AI bot visits, only 84 requests targeted the LLMs.txt file. That's 0.1% of all AI bot traffic.
To put that in perspective, the average page on the site received around 265 AI bot visits during the same period. LLMs.txt received 3x fewer AI bot visits than the average content page on the site.
Study 4: Reboot Online 3-Month Controlled Test
Reboot Online took a different approach. They published new landing pages on two test websites and added LLMs.txt files containing references to those pages with no other internal or external links pointing to them.
The idea was to test whether AI bots would use LLMs.txt to discover new content.
After three months, zero AI bots visited the LLMs.txt file on either website. Meanwhile, other pages on the same sites received regular AI bot traffic.
The conclusion was clear: AI bots don't use LLMs.txt for content discovery.
Google's Official Position (June 2026)
On May 15, 2026, Google published its first official AI optimisation guide titled "Optimising your website for generative AI features on Google Search." On June 15, Google added a dedicated section titled "LLMs.txt" after what it described as widespread misinformation in the community.
Google's statement was unusually direct: LLMs.txt files are not needed to appear in Google Search, and they will not positively or negatively impact your visibility or rankings.
Google also debunked several related tactics that were being promoted as "AI SEO hacks", including content chunking for AI, exact-keyword obsession for AI Overviews, and artificial brand mentions.
Their summary was blunt: answer engine optimisation and generative engine optimisation are still, fundamentally, SEO.
Two days later, Google updated the guidance to acknowledge that maintaining LLMs.txt for other AI services is "completely fine" but reiterated that Google Search ignores the file entirely.
AI SEO Prompts Pack - Free Resource
Skip the unproven tactics. Get ready-to-use prompts for keyword research, content optimisation, technical SEO audits, and GEO - the same frameworks I use across 14+ live projects to generate real organic leads.
The Lighthouse Confusion
Adding to the mixed signals, Chrome Lighthouse 13.3.0 (released May 7, 2026) introduced an "Agentic Browsing" audit category that checks for the presence of an LLMs.txt file.
This created real confusion. Google's search team says the file doesn't matter for rankings. Google's Lighthouse team checks for it in audits.
The distinction matters: the Lighthouse audit is explicitly marked as experimental and carries no score. It's a forward-looking signal about agent readiness, not a ranking factor. Treating it as a to-do list item is a misreading of its purpose.
More importantly, Google's actual investment in agent-to-website interaction isn't LLMs.txt. It's WebMCP (Web Model Context Protocol), a live interface that lets AI agents interact with a website directly, rather than reading a static description file. WebMCP was announced in Chrome Canary in February 2026 and featured at Google I/O 2026.
The difference: LLMs.txt describes a site. WebMCP lets an agent operate it. Google is clearly betting on the latter.
So Who Is Actually Accessing LLMs.txt?
The Ahrefs study and the SEO Framework experiment both tracked this. The answer is revealing.
Curious SEO professionals are manually checking the file to see if competitors have implemented it.
GEO and AEO tools are scanning for adoption patterns.
Security scanners and unknown crawlers.
Fake Google bots: the SEO Framework experiment identified 23 instances of bots using a Googlebot user-agent string but originating from non-Google IP addresses.
The people reading LLMs.txt files are, overwhelmingly, the people talking about LLMs.txt.
What Actually Drives AI Visibility Right Now?
If your goal is to appear in ChatGPT, Perplexity, Google AI Overviews, or Claude's search results, LLMs.txt isn't your lever.
I've worked on projects that generated 7,800+ organic leads for a US multi-location business, scaled a UK B2B brand from zero to 3,911 organic leads, and ranked a brand-new website to position one in India and position two in the USA within 20 days, all through organic systems, not unproven tactics.
Here's what actually influences AI visibility based on real-world execution:
1. Technical SEO Fundamentals Still Come First
AI bots are actively crawling websites; the data confirms this with hundreds of thousands of hits. The question isn't whether they visit. It's whether your content is structured cleanly enough for them to extract value when they do.
Clean site architecture, fast load times, proper heading hierarchy, crawlable internal links, and mobile responsiveness aren't optional. These are the baseline requirements for both Google and AI systems.
2. Topical Authority Determines Who Gets Cited
ChatGPT, Perplexity, and Gemini pull from sources they consider authoritative on a given subject. Three blog posts and a homepage don't establish authority. A deep, interconnected content cluster around your core topic does.
Real-World Example
For a CAD platform with no prior organic authority, I built a content cluster targeting every stage of the user journey from awareness queries to comparison pages. Over 12 months, that system generated 27.5M+ impressions and scaled CTR from 2% to 5.5%, all through content depth and topical coverage, not shortcuts. Full breakdown: CAD Platform Case Study.
This is where GEO and traditional SEO work together. When you build content depth around a topic, addressing questions at every stage of the buyer journey, AI systems have more reasons to cite you across a wider range of queries.
3. Brand Mentions and Citations Build AI Training Signals
AI models are trained on web data. When other websites mention your brand, reference your content, or link to your pages, those signals get embedded into the model's understanding of who's authoritative on a topic.
Your off-page presence isn't just a Google ranking factor. It's increasingly an AI visibility factor too.
4. Structured Data (Schema Markup) Actually Gets Read
This is the closest thing to LLMs.txt that actually works, and it has years of documented adoption behind it.
Schema tells search engines and AI systems what your content represents in a structured, machine-readable format. FAQPage, HowTo, Article, Service, Organisation- these schema types help AI systems understand context and extract accurate answers.
Unlike LLMs.txt, schema markup has confirmed adoption by Google, Bing, and multiple AI platforms.
5. Conversion-Focused Content, Not Just Traffic-Focused Content
Traffic without conversion is a vanity metric. When you build content that answers questions your customers actually ask and structure it to move them toward a decision, you get both AI visibility and business results.
Every page should have a clear purpose: educate, build trust, or move the reader toward the next step. If a page doesn't do one of those things, it's not earning its place on your site.
Is Your Site Ready for AI Search?
Most websites fail on the basics: broken schema, thin topical coverage, zero entity signals. Run your site through this step-by-step checklist covering technical SEO, GEO readiness, and AI crawler accessibility.
The Bigger Lesson for Business Owners
If you're a founder or business owner reading this, please don't let your SEO agency upsell you on an LLM setup as a paid service. There's no evidence it delivers any measurable return.
The time and budget spent creating, maintaining, and updating an LLMs.txt file would be better invested in content that actually ranks, pages that convert visitors into leads, or automation systems that generate a predictable pipeline.
Chasing unproven trends is how businesses waste their SEO budget. Building systems that compound over time is how they grow.
“His deep understanding of SEO has helped us achieve great results. His ability to think beyond conventional SEO tactics and implement innovative approaches has been invaluable to our success.”
The data across every study is consistent. 137,000 sites analysed. Six-month experiments completed. Google's official guidance has been published. The result is the same everywhere you look: LLMs.txt has no measurable impact on SEO or AI visibility today.
Should You Add LLMs.txt Anyway?
Despite everything above, the honest answer is: it depends on your situation.
Yes, it might make sense if:
- You run developer documentation or an SDK, where AI coding assistants (Cursor, GitHub Copilot, Claude Code) actively retrieve docs in real time
- Your CMS plugin generates it automatically and requires zero manual effort
- You want to be positioned for a possible future where major AI providers start reading the file
No, it's not worth your time if:
- You run a business website, e-commerce store, local business, or service-based brand
- You're considering paying someone to create and maintain it
- You're treating it as a replacement for technical SEO, content strategy, or schema markup
The cost of adding LLMs.txt is low if it's automated. The cost of treating it as a strategy is high because it distracts from work that actually moves the needle.
My Take
Here's what I think this situation really reveals about the SEO industry.
We have a pattern of adopting new "signals" before anyone verifies whether they work. AMP was supposed to be essential for mobile rankings until it wasn't. IndexNow was supposed to accelerate indexing, but Google never supported it. And now LLMs.txt is being positioned as the key to AI visibility while every data set shows it doing nothing.
Rank Math added support. Yoast added support. Every AI SEO plugin shipped it. But they did it because the conversation around LLMs.txt was loud, not because AI bots actually use it. That's fear-driven feature shipping, not evidence-based strategy.
The fundamentals haven't changed. Build content worth citing. Structure it so machines can parse it. Earn authority through depth, consistency, and real-world results. Whether the traffic comes from Google, ChatGPT, or Perplexity, the websites winning are the ones doing the basics exceptionally well.
Until a major AI platform publicly confirms it reads and acts on LLMs.txt in production, its practical value remains zero for most websites.
Want a System That Actually Drives AI Visibility?
Book a 1:1 AI + SEO Growth Strategy Session. I'll audit your current setup, identify exactly where you're losing visibility in Google and AI search, and build you a roadmap based on what the data actually shows works.
Key Takeaways
- LLMs.txt is a plain-text file designed to help AI systems understand your website, but no major AI provider has confirmed that they use it.
- Ahrefs analysed 137K sites and found that 97% of LLMs.txt files never get read.
- The SEO Framework tracked 57 AI bots over 6 months, zero accessed LLMs.txt
- Google's June 2026 guidance explicitly states that LLMs.txt does not affect Search rankings or AI Overviews
- Google is investing in WebMCP for agent-to-website interaction, not LLMs.txt
- Focus on technical SEO, topical authority, schema markup, and brand citations for real AI visibility
- Don't pay for LLMs.txt setup as a service. Invest that budget in content and systems that convert
