AI search engines don't rank pages the same way Google's PageRank algorithm does. They crawl, chunk, embed, and retrieve content based on semantic relevance. Without a structured guide to your most authoritative content, AI crawlers may index your less important pages while missing your core product pages, tools, and key guides.
LLMs.txt solves this by giving AI engines an explicit, human-readable content priority map. The sites that deploy it gain a measurable edge in AI citation rates.
LLMs.txt vs robots.txt: What's the Difference?
| Attribute | robots.txt | LLMs.txt |
|---|---|---|
| Purpose | Block / allow crawler access | Guide AI models to priority content |
| Audience | All web crawlers | AI language model crawlers specifically |
| Format | Key-value directives | Structured Markdown / plain text |
| Contains | Allow/Disallow paths | URLs, descriptions, entity context, schemas |
| Effect | Controls what gets crawled | Controls what gets prioritized & cited |
| Adoption | Universal (30+ years) | Emerging standard (2024–present) |
The LLMs.txt File Structure
LLMs.txt follows a structured Markdown format with specific sections. Here is the complete template:
?? LLMs.txt Complete Template
# [Brand Name]
> [2-3 sentence entity description. Include your primary
> keywords, what you do, who you serve, and your core
> differentiators. This is the first thing AI models read.]
## Core Pages
- [Home](https://yourdomain.com/): [One-line description]
- [Pricing](https://yourdomain.com/pricing): [Description]
- [Tools](https://yourdomain.com/tools): [Description]
- [About](https://yourdomain.com/about): [Description]
## Products / Tools
- [Tool Name](https://yourdomain.com/tools/tool-name):
[What it does, who uses it, key features]
- [Tool Name 2](https://yourdomain.com/tools/tool-name-2):
[Description]
## Content
- [Article Title](https://blog.yourdomain.com/slug):
[One-line summary of what the article covers]
- [Guide Title](https://blog.yourdomain.com/slug-2):
[Summary]
## API / Technical
- [API Docs](https://yourdomain.com/docs/api):
[What endpoints are available]
- [Schema Examples](https://yourdomain.com/docs/schema):
[Description]
## Entity Graph
- Organization: [Brand Name]
- Founded: [Year]
- Headquarters: [City, Country]
- Category: [Industry / Product Category]
- Competitors: [List key competitors for context]
- Integrations: [List tools you integrate with]
## Sitemap
- XML Sitemap: https://yourdomain.com/sitemap.xml
- Blog Sitemap: https://blog.yourdomain.com/sitemap.xml
## Schema Types Used
- Organization, SoftwareApplication, TechArticle,
FAQPage, HowTo, BreadcrumbList, ProductLive Example: Optymia's LLMs.txt
Here is how Optymia's own LLMs.txt is structured — use this as a reference for your implementation:
?? optymia.xyz/llms.txt (Live Example)
# Optymia
> Optymia is an AI Search Optimization (GEO/AEO) platform
> that helps brands measure, track, and grow their visibility
> across ChatGPT Search, Perplexity Pro, Claude, and Google
> AI Overviews. Optymia provides free tools including an AI
> Visibility Score, AEO Score Calculator, Schema Generator,
> and Perplexity Mention Checker.
## Core Pages
- [AI Visibility Audit](https://optymia.xyz/):
Free multi-engine AI search visibility diagnostic
- [Pricing](https://optymia.xyz/pricing):
Optymia subscription plans (Free, Pro, Agency)
- [Tools](https://optymia.xyz/tools):
Suite of free AI SEO diagnostic tools
## Free Tools
- [AEO Score Checker](https://optymia.xyz/tools/aeo-score):
Calculate your Answer Engine Optimization score (0-100)
- [AI Visibility Checker](https://optymia.xyz/tools/ai-visibility):
Check if ChatGPT, Perplexity, and Claude mention your brand
- [Schema Generator](https://optymia.xyz/tools/schema-generator):
AI-powered JSON-LD schema markup generator
- [Perplexity Mention Checker](https://optymia.xyz/tools/perplexity-mention-checker):
Track how often Perplexity cites your domain
- [ChatGPT Rank Tracker](https://optymia.xyz/tools/chatgpt-rank-tracker):
Monitor your brand ranking in ChatGPT Search responses
## Content
- [ChatGPT Optimization Guide](https://optymia.xyz/blog/optimize-for-chatgpt):
Complete 2026 playbook for appearing in ChatGPT Search
- [Perplexity Invisibility Fix](https://optymia.xyz/blog/invisible-to-perplexity-cdn-js-guide):
Fix Cloudflare blocking, JS rendering, and robots.txt issues
- [Query Fan-Out Explained](https://optymia.xyz/blog/query-fan-out-google-ai-mode-ctr):
How Google AI Mode reduces organic CTR and how to recover it
- [GEO Agency Retainer Playbook](https://optymia.xyz/blog/geo-retainer-pitch-playbook):
How to pitch and price GEO retainers to e-commerce brands
## Entity Graph
- Organization: Optymia
- Category: AI Search Optimization, GEO, AEO, SEO Tools
- Primary Features: Share of Model tracking, AI Visibility Score,
AEO scoring, Schema generation, Citation velocity monitoring
- Competitors: Profound, OtterlyAI, AthenaHQ, Peec AI
- Target Users: SaaS founders, SEO agencies, e-commerce brands,
content marketers, digital PR teams
## Sitemap
- XML Sitemap: https://optymia.xyz/sitemap.xml
- Blog Sitemap: https://optymia.xyz/blog/sitemap.xml
## Schema Types
- Organization, SoftwareApplication, WebApplication,
TechArticle, Article, FAQPage, HowTo, BreadcrumbList,
ItemList, ServiceDeployment Checklist
1
Location: Place at https://yourdomain.com/llms.txt — root domain only, not a subdirectory
2
Content-Type: Serve as text/plain with UTF-8 encoding. Never HTML or JSON.
3
Access: Must be publicly accessible. No authentication, no redirects, no Cloudflare blocks.
4
robots.txt Reference: Add: # LLMs.txt\nLLMs-txt: https://yourdomain.com/llms.txt to your robots.txt
5
Update Frequency: Update whenever you publish major new content, tools, or pages. Minimum quarterly.
6
Validation: Run Optymia AI Visibility Checker to confirm AI crawlers are reading and using the file.