# AI Index Check
> Tools and source-backed references for checking AI crawler access, llms.txt files, schema extractability, and AI citation readiness.

## Primary tools
- [llms.txt Generator](https://aiindexcheck.com/tools/llms-txt-generator): Generate a conservative llms.txt draft from canonical public pages.
- [llms.txt Validator](https://aiindexcheck.com/tools/llms-txt-validator): Validate the structure and links of an llms.txt file.
- [AI Crawler / robots.txt Checker](https://aiindexcheck.com/tools/ai-crawler-robots-checker): Test crawler policy for a specific URL and view matched robots directives.
- [Schema Extractability Checker](https://aiindexcheck.com/tools/schema-extractability-checker): Review JSON-LD validity, entity signals, and visible-content consistency.
- [AI Citation Readiness Report](https://aiindexcheck.com/tools/ai-citation-readiness-report): Review crawl access, extractability, source quality, freshness, entity clarity, structured data, and internal discovery.

## Crawler references
- [OAI-SearchBot](https://aiindexcheck.com/crawlers/oai-searchbot): OpenAI search discovery for ChatGPT search features.
- [GPTBot](https://aiindexcheck.com/crawlers/gptbot): OpenAI crawler for content that may be used to train generative AI models.
- [ChatGPT-User](https://aiindexcheck.com/crawlers/chatgpt-user): User-initiated ChatGPT and Custom GPT requests.
- [Googlebot](https://aiindexcheck.com/crawlers/googlebot): Google Search crawling and indexing.
- [Google-Extended](https://aiindexcheck.com/crawlers/google-extended): Control token for certain Gemini and Vertex AI uses outside Google Search.
- [PerplexityBot](https://aiindexcheck.com/crawlers/perplexitybot): Perplexity search and retrieval crawler.

## Methodology
- [AI Index Check Methodology](https://aiindexcheck.com/methodology): Scoring logic, crawler verification, limitations, sources, and correction policy.
- Tool results do not guarantee crawling, indexing, ranking, inclusion, or citation.
- Crawler roles and platform guidance link to primary documentation.
- Each crawler reference displays its manually maintained last verification date.

## Important
- llms.txt is an emerging convention and is not a Google Search or AI Overview requirement.