AI crawler policy
Google-Extended Robots.txt Checker
Audit whether a site's robots.txt rules allow, block, or leave Google-Extendedunspecified. Use the result as a policy and crawlability check, not as a visibility guarantee.
Crawler Details
User agentGoogle-Extended
VendorGoogle
Policy surfacerobots.txt
FAQ
What does Google-Extended do?
Google-Extended is associated with Google and control signal for Gemini and Vertex AI training use.
Can robots.txt guarantee Google-Extended behavior?
Robots.txt is a directive for compliant crawlers. It is useful for public policy signaling, but it is not access control.