AI crawler policy
GPTBot Robots.txt Checker
Audit whether a site's robots.txt rules allow, block, or leave GPTBotunspecified. Use the result as a policy and crawlability check, not as a visibility guarantee.
Crawler Details
User agentGPTBot
VendorOpenAI
Policy surfacerobots.txt
FAQ
What does GPTBot do?
GPTBot is associated with OpenAI and web crawling that may support AI model improvement and search experiences.
Can robots.txt guarantee GPTBot behavior?
Robots.txt is a directive for compliant crawlers. It is useful for public policy signaling, but it is not access control.