Source-backed crawler reference
Amazonbot Robots.txt Guide: Allow, Block, and Verify Crawling
Amazonbot is Amazon's web crawler token used across Amazon services. Site owners can express an allow or block policy for Amazonbot in robots.txt.
Last reviewed: for page guidance. Source verified: .
User agent
The robots.txt user-agent token to test is Amazonbot. Use this exact token when checking allow and block rules for Amazonbot.
User-agent: Amazonbot
What Amazonbot is used for
Use this reference when the site needs an explicit Amazonbot policy. Product-specific effects should be verified against current Amazon documentation.
Amazonbot can be controlled through robots.txt, but product-specific use should be checked against current Amazon documentation.
Search indexing, AI training, AI retrieval, shopping, or other use
Amazon-operated crawler for Amazon services. Product-specific search, shopping, AI, or other effects should be verified against Amazon documentation before making claims.
Impact on crawling, indexing, AI use, or retrieval
Amazonbot should not be reduced to a single AI search or training purpose without evidence. Treat it as a separate crawler policy decision.
Its role should not be reduced to a single AI search or training purpose without current official evidence.
How to allow Amazonbot
Add an allow rule when this crawler should be permitted to request public pages. Test the deployed robots.txt file on the exact URL path, because a homepage allow can coexist with deeper disallow rules.
User-agent: Amazonbot Allow: / User-agent: * Allow: /
What allowing Amazonbot enables
Allowing Amazonbot permits the Amazonbot user-agent token to request matching public paths if Amazon chooses to crawl them. It does not guarantee Amazon discovery, shopping visibility, model use, ranking, or citation.
How to block Amazonbot
Add a block rule only when the policy intent is to restrict this crawler. Blocking is a public directive for compliant crawlers, not authentication and not a ranking control.
User-agent: Amazonbot Disallow: / User-agent: * Allow: /
What blocking Amazonbot affects and does not affect
Blocking Amazonbot asks that Amazon crawler token not to fetch matching paths. It does not block Google Search, OpenAI crawlers, Anthropic crawlers, or other Amazon-related user agents that may be documented separately.
Common verification notes
- Use explicit rules when Amazonbot policy matters.
- Do not infer Google, OpenAI, or Anthropic policy from Amazonbot rules.
- Monitor current documentation and server logs for behavior changes.
Amazonbot verification and troubleshooting note
Verify Amazonbot against Amazon's current documentation and compare with server logs if Amazon access matters. Do not infer a training, shopping, or Search effect unless the current source confirms it.
How to interpret a robots.txt checker result
An allow result means the matched robots.txt directive does not block this compliant crawler on the tested path. It does not guarantee a visit, indexing, inclusion, ranking, or citation. A block is a public crawler directive, not authentication or access control.
Recommended action: Use an explicit allow or block policy only when it matches your business intent.
Official source
Amazon documentation for Amazonbot
The source identifies Amazonbot as an Amazon crawler. This page labels unverified product-specific effects instead of assigning a single AI or shopping purpose.
See the AI Index Check methodology for verification, scoring, limitations, and correction policy.