GoogleOther

Google · Training honors robots.txt vendor-doc verified July 2026

What it does

GoogleOther is Google's generic crawler for internal research and development — Google states it is NOT used for Search indexing, and Google's AI-training control is the separate Google-Extended robots token. We file it under 'training' as the closest bucket, but blocking Google-Extended (not GoogleOther) is how you opt out of Gemini training.

Generic Google crawler for internal R&D / one-off fetches; not a ranking crawler. Categorized under training as the closest bucket — blocking it does not affect Search.

Full UA: not captured in our July 2026 roster review — match log hits on the token GoogleOther (case-insensitive substring) and verify by IP before acting.

How to verify a hit is really GoogleOther

Verification method (July 2026): Published IP list: https://developers.google.com/static/search/apis/ipranges/googlebot.json + rDNS crawl-*.googlebot.com (fetched live 2026-07-19)

User-agent strings are freely spoofed, so a UA match alone proves nothing. Confirm the source IP belongs to Google — published IP-range JSON where available, otherwise RDAP/rDNS on the IP. Full method: verify by IP.

Allow or block in robots.txt

Match the exact token GoogleOther in robots.txt:

# Block GoogleOther site-wide
User-agent: GoogleOther
Disallow: /
# Explicitly allow GoogleOther
User-agent: GoogleOther
Allow: /
Compliant bots stop fast. Well-behaved crawlers stop within roughly one crawl cycle of a robots.txt Disallow (we observed GPTBot's hammering on a production site we operate stop within one cycle). Give a fresh directive about a day before concluding it's being ignored.

For the allow-search-block-training combined pattern (and the Allow-directive gotcha that silently kills carve-outs), see the robots.txt guide.

Vendor documentation: https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers

Frequently asked questions

What is GoogleOther?

GoogleOther is Google's training crawler. Content is copied into datasets used to train foundation models. Blocking costs you nothing in traffic today; allowing is a donation of your content to model weights with no citation or referral in return.

Does GoogleOther respect robots.txt?

Yes — Google documents robots.txt compliance for GoogleOther (vendor docs reviewed July 2026).

How do I block GoogleOther?

Add 'User-agent: GoogleOther' followed by 'Disallow: /' to your robots.txt. Use the exact token — substring variants of other tokens will not match.

Other bots

AnswerFootprint crawler analytics — bot roster compiled July 2026 from vendor crawler docs and published IP lists (21/28 vendor-doc verified). The analyzer runs 100% client-side — your logs never leave this browser. User-agent strings can be spoofed; treat UA matches as a first pass and verify by IP before acting. Errors in the bot data: report them.