llms.txt generator
Generate, preview, and serve a dynamic /llms.txt file from SEO Booster Tools for AI crawlers and answer engines.
Tools overview · Entity Map (Pro) · Markdown URLs (Pro) · AI Readiness
Admin path: SEO Booster → Tools → llms.txt
llms.txt tells AI where to start. Entity Map (Pro) tells AI who you are.
The llms.txt tool curates which public content AI systems should prioritize. When enabled, SEO Booster serves /llms.txt dynamically via a WordPress rewrite rule, not as a static file in your theme folder (unless you deliberately add a physical file that may shadow the virtual route).
Enable and configure
| Setting | Purpose |
|---|---|
| Publish / serve | Registers rewrite for /llms.txt |
| Discovery signals | Optional LLMS: line in robots.txt, front-page alternate link, HTTP Link header |
| Introduction | Short site description at the top of the file (Markdown blockquote); optional AI suggest |
| Post types | Which public types appear in the generator |
| Max links | Cap on listed URLs (keep focused, e.g. 10–30 pillar pages) |
| Directory rules | Include / exclude / auto for URL path prefixes |
| Cache TTL | Seconds to cache generated content |
Save settings from the tab. First enable may flush rewrite rules once.
FAQ section
Turn on Include FAQ blocks from your SEO plugin in llms.txt. The extractor reads FAQ blocks from Yoast SEO, Rank Math, All in One SEO, SEOPress, and The SEO Framework (not Yoast/Rank Math only). Set Max FAQ items to keep the file focused.
Directory scanner
The directory rules table lists public URL path prefixes. For each directory choose auto, include, or exclude. Auto follows your post-type and max-link curation. Include forces that prefix in. Exclude drops it even if pages would otherwise qualify.
AI bot crawl gaps
When AI bot tracking has mapped content hits, the AI bot crawl gaps table lists pages with significant bot traffic that are not in the current llms.txt selection.
- Pin forces that page into the curated file
- Unpin returns it to normal curation
Empty table means no gaps, or bot tracking has no mapped content hits yet.
Preview and download
- Preview, AJAX-rendered content as crawlers would see it
- Download, save a snapshot for offline review
Preview updates when settings or included posts change. Post saves invalidate the cached content (configurable TTL).
What appears in the file
Typical structure:
- Site title and short description
- Curated links to important posts/pages (title + URL + optional summary)
- Optional sections for documentation or policy pages you include
- Pro: Structured knowledge links to Entity Map (
/entitymap.json/ HTML) when Entity Map is published and linked from llms.txt
Builder-specific and system post types (attachments, reusable blocks, template libraries) are excluded from the picker by default.
Beyond llms.txt: Markdown and Entity Map (Pro)
A curated link list is a starting index. On Pro, the same Tools tab includes Markdown discovery (*.md, /index.md, optional /llms-full.txt). Entity Map publishes structured organization and content relationships as /entitymap.json and /entitymap.html. Free installs keep llms.txt; Pro unlocks Markdown settings, the Entity Map Tools tab, and optional Structured knowledge section above.
AI Readiness connection
Sitewide AI Readiness site checks include llms.txt availability. Enabling and serving the file can clear related readiness items on the next analysis. Entity Map status is covered separately when Pro is available.
Physical file warning
If a real llms.txt exists in the web root, it may take precedence over the virtual endpoint depending on server configuration. Remove or rename conflicting static files if the dynamic route should win.
Related
- Entity Map (Pro)
- Settings Stats and Tools tabs, cache and maintenance
- AI Bots overview, crawler traffic vs human referrals
- llms.txt for WordPress (editorial guide)
- AI search readiness for WordPress
Browse topics