AI search tool
llms.txt Generator
Create a concise llms.txt draft that points AI systems toward your most useful public resources without implying guaranteed visibility.
What does this tool check?
This tool creates a conservative llms.txt draft from a public domain, site name, description, and canonical resource links. It structures the output as readable text with labeled sections, while warning against private, redirected, blocked, or low-value URLs. llms.txt is treated as an emerging optional convention, not a Google requirement or a visibility guarantee.
What does the result mean?
The draft is a starting point for manual review. It does not prove that a crawler supports llms.txt or that listed pages will be crawled, indexed, ranked, included, or cited.
What should I fix first?
- Remove private, redirected, blocked, duplicate, and non-canonical URLs.
- Keep only authoritative public product, documentation, support, pricing, and policy pages.
- Publish at /llms.txt as UTF-8 plain text, then validate every link.
Sources and last review
Last reviewed: .
What This Checks
The generator accepts a public domain and creates a starter llms.txt draft with a site heading, concise description, recommended resource links, and a caution that llms.txt is only an emerging discovery convention.
The result is meant to reduce avoidable crawl and extraction friction. It does not guarantee LLM inclusion, ranking, indexing, training use, or citation.
What llms.txt Is And Is Not
llms.txt is an emerging convention for publishing a concise map of resources that may be useful to AI assistants, answer engines, and other automated readers. A good file points to canonical documentation, product pages, policies, support material, pricing, API references, and other public pages that explain what the site does. It should be written for clarity, not for keyword stuffing.
The file is not an access-control mechanism, a ranking signal guarantee, or a promise that any model will crawl, train on, cite, or display your content. Treat it as a lightweight discovery aid that complements robots.txt, sitemap.xml, structured data, and readable page content.
Recommended File Structure
Start with a clear H1 that names the site or product. Follow it with a short description of the organization, audience, and most important public resources. Group links by intent: overview, product documentation, support, policies, changelog, pricing, API docs, and reference pages. Use absolute canonical URLs when possible so automated systems do not have to resolve ambiguous paths.
Keep the file focused. A short, curated file is usually more useful than a long dump of every URL. The generator creates a conservative draft and includes a reminder that llms.txt does not guarantee AI visibility.
What To Include And Avoid
Include pages that answer common questions directly, define the product or entity clearly, and are intended for public discovery. Avoid private endpoints, staging URLs, tracking-heavy links, duplicate variants, thin tag pages, or pages blocked by robots.txt. Do not include secrets, API keys, unpublished roadmaps, internal dashboards, or access-controlled documentation.
Common mistakes include publishing the file somewhere other than the site root, linking to redirected URLs, making unsupported claims about guaranteed LLM visibility, and listing pages that contradict the site canonical strategy.
llms.txt Compared With robots.txt And sitemap.xml
robots.txt communicates crawl policy for compliant crawlers. sitemap.xml lists canonical URLs for discovery and indexing workflows. llms.txt is different: it is a human-readable and machine-readable guide to high-value context. The three files should not contradict each other. If llms.txt promotes a page that robots.txt blocks or a sitemap excludes, the site is sending mixed signals.
Use the generator as a starting point, then validate the final file and check it against your crawler policy before publishing.
Publishing Workflow
Before publishing, review every generated link and remove anything that is not intended for broad public discovery. Confirm that each URL returns 200, self-canonicalizes or redirects cleanly to the canonical page, and is not blocked by robots.txt. If your documentation lives on a subdomain, decide whether that subdomain should have its own llms.txt file or whether the root file should point to the most important documentation sections.
After publishing, fetch the file in a browser and from a command-line client. Check that the response is plain text, the heading is readable, and the links resolve without login prompts. Then submit or inspect the same URLs through your normal SEO QA workflow. The file should be maintained like any other discovery surface: update it when product names, documentation paths, policies, or support resources change.
Example
# Example Site Public resources for understanding Example Site. ## Recommended resources - [Product overview](https://example.com/product) - [Documentation](https://example.com/docs) - [Pricing](https://example.com/pricing) - [Support](https://example.com/support) This file is an emerging discovery aid and does not guarantee indexing or citation.
Example Input
Domain: https://example.com Site type: SaaS tool website Key public pages: /, /features, /pricing, /docs, /support, /privacy
Example Output
# Example Site Example Site helps teams monitor AI search readiness. ## Product and company - [Product overview](https://example.com/) - [Features](https://example.com/features) - [Pricing](https://example.com/pricing) ## Documentation and support - [Documentation](https://example.com/docs) - [Support center](https://example.com/support) ## Policies - [Privacy policy](https://example.com/privacy) This file is an emerging discovery aid and does not guarantee indexing, ranking, model training, or citation.
Common Errors Detected
- Including private dashboards, staging hosts, login-only URLs, or unpublished internal documents.
- Listing every URL instead of a curated set of high-value canonical resources.
- Claiming that llms.txt guarantees LLM visibility, indexing, ranking, training inclusion, or citation.
- Pointing to redirected, duplicate, blocked, or non-canonical pages.
Recommended Fix Steps
- Start with canonical public pages that define the product, documentation, pricing, support, and policies.
- Remove any private, duplicate, redirected, blocked, or low-value URLs from the generated draft.
- Publish the final file at /llms.txt and keep the wording clear that it is an emerging convention.
- Validate the published file, then compare its links with robots.txt and sitemap.xml.
Before Requesting Indexing
Before submitting this page in Search Console, confirm that the page returns 200 on the canonical host, has a self-referencing canonical tag, appears in the reduced sitemap only when it is index-worthy, and contains enough visible text to stand on its own. Check that ad placeholders do not interrupt the main workflow, that structured data matches visible content, and that the page does not claim AI search visibility is guaranteed.
For a new domain, it is better to request indexing for a small group of strong pages than to push every thin route at once. Re-run the relevant AI Index Check tool after publishing and keep a record of changes so future crawler, schema, or content updates can be audited.
Recommended Workflow
Related Checks And Guides
Related AI Search Guides
Learn what llms.txt is, where it lives, what to include, and why it should be treated as an emerging AI discovery convention.
llms.txt Example for SaaS, Documentation and Tool WebsitesUse a practical llms.txt example for SaaS, documentation, and tool websites, with notes on what to include and avoid.
FAQ
Does llms.txt guarantee LLM visibility?
No. llms.txt is an emerging convention. It can make preferred resources easier to find, but adoption varies by system.
Where should llms.txt live?
Publish it at the root of your site, such as https://example.com/llms.txt.
What pages should I include in llms.txt?
Include canonical public pages that explain the product, documentation, pricing, support, policies, API references, and other high-value resources.
What should not be exposed in llms.txt?
Do not include private URLs, staging hosts, internal dashboards, API keys, unpublished plans, or pages blocked from public discovery.