Source-backed crawler reference
OAI-SearchBot Robots.txt Checker
OAI-SearchBot is OpenAI's crawler token for automatic search discovery. Use robots.txt rules that target OAI-SearchBot when you want to allow or block OpenAI search crawling separately from GPTBot training access.
Last reviewed: for page guidance. Source verified: .
User agent
The robots.txt user-agent token to test is OAI-SearchBot. Use this exact token when checking allow and block rules for OAI-SearchBot.
User-agent: OAI-SearchBot
What OAI-SearchBot is used for
Use this reference when the question is whether OpenAI search discovery can retrieve a public page. It is not the same token as GPTBot, and it should not be treated as ChatGPT-User user-triggered browsing.
OpenAI documents OAI-SearchBot as the robots.txt token for managing automatic search crawl and Search opt-outs.
Search indexing, AI training, AI retrieval, shopping, or other use
Crawler policy surface. Review its documented purpose separately from unrelated search, training, retrieval, and dataset crawlers.
Impact on crawling, indexing, AI use, or retrieval
OAI-SearchBot may affect OpenAI search discovery paths, but allowing it does not guarantee crawling, indexing, inclusion, ranking, or citation in ChatGPT search features.
This is the OpenAI crawler relevant to search discovery, separate from model training and user-triggered requests.
OAI-SearchBot is used for OpenAI search discovery. GPTBot is documented separately for model training, and ChatGPT-User is used for user-initiated requests. A site can therefore allow search discovery while making a different policy decision about training.
How to allow OAI-SearchBot
Add an allow rule when this crawler should be permitted to request public pages. Test the deployed robots.txt file on the exact URL path, because a homepage allow can coexist with deeper disallow rules.
User-agent: OAI-SearchBot Allow: / User-agent: * Allow: /
What allowing OAI-SearchBot enables
Allowing OAI-SearchBot means the deployed robots.txt policy does not block the OAI-SearchBot token on matching public paths. It does not guarantee a visit, indexing, inclusion, ranking, training, or citation.
How to block OAI-SearchBot
Add a block rule only when the policy intent is to restrict this crawler. Blocking is a public directive for compliant crawlers, not authentication and not a ranking control.
User-agent: OAI-SearchBot Disallow: / User-agent: * Allow: /
What blocking OAI-SearchBot affects and does not affect
Blocking OAI-SearchBot asks compliant OAI-SearchBot requests not to fetch matching paths. It does not authenticate the page, remove it from indexes, or control other crawler tokens.
Common verification notes
- Test OAI-SearchBot separately from GPTBot and ChatGPT-User.
- Check high-value paths such as documentation, pricing, support, and guides.
- Confirm the final canonical host serves the robots.txt file being tested.
OAI-SearchBot verification and troubleshooting note
Verify OAI-SearchBot with its official source, then test the final canonical URL and the exact path in robots.txt. Server logs can help confirm whether the crawler requested the page after policy changes.
How to interpret a robots.txt checker result
An allow result means the matched robots.txt directive does not block this compliant crawler on the tested path. It does not guarantee a visit, indexing, inclusion, ranking, or citation. A block is a public crawler directive, not authentication or access control.
Recommended action: Allow it when ChatGPT search discovery matches your policy. A robots allow does not guarantee crawling, inclusion, or citation.
Official source
OpenAI documentation for OAI-SearchBot
This page bases crawler role descriptions on the linked owner documentation and labels limitations where robots.txt cannot prove downstream product behavior.
See the AI Index Check methodology for verification, scoring, limitations, and correction policy.