What it does
Google-Extended never appears in access logs. It has no HTTP user-agent string — it exists only as a robots.txt User-agent: token. The actual crawling happens under Google's regular crawler user agents; this token controls whether the content those crawlers already fetch may be used for AI training.
NEVER appears in access logs. Google-Extended has no HTTP user-agent string — crawling happens under regular Googlebot UAs; the token only works as a robots.txt User-agent line controlling Gemini training / AI grounding use. A log analyzer must not offer it as a log-match pattern.
Full UA: none exists — Google-Extended is a robots.txt-only token with no user-agent string; any log line carrying it is a spoofer.
How to verify a hit is really Google-Extended
Verification method (July 2026): n/a — robots.txt control token only
Nothing to verify: Google-Extended never generates traffic of its own, so there are no hits to attribute. Any log line carrying this token is a spoofer by definition.
Allow or block in robots.txt
Add the token to robots.txt — this is the only place Google-Extended does anything:
# Block Google-Extended site-wide
User-agent: Google-Extended
Disallow: /# Permit AI use of crawled content
User-agent: Google-Extended
Allow: /For the allow-search-block-training combined pattern (and the Allow-directive gotcha that silently kills carve-outs), see the robots.txt guide. robots.txt governs whether Google-Extended may fetch; llms.txt is the separate, curated map AI assistants read once allowed in.
Vendor documentation: https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers
The Google family
Google operates 5 distinct tokens in this roster, and they do not block each other: a robots.txt line for Google-Extended does nothing to its siblings — each needs its own User-agent: entry (the robots.txt guide has combined patterns).
| Token | Role | robots.txt |
|---|---|---|
| GoogleOther | Training | honors robots.txt |
| Google-CloudVertexBot | User action | honors robots.txt |
| Google-Agent | User action | ignores robots.txt |
| Google-GeminiNotebook | User action | ignores robots.txt |
Frequently asked questions
What is Google-Extended?
Google-Extended is a robots.txt control token from Google — it has no user agent and never appears in access logs; it controls AI-training use of content crawled by Google's regular crawlers.
Does Google-Extended respect robots.txt?
Yes — Google documents robots.txt compliance for Google-Extended (vendor docs reviewed July 2026).
How do I block Google-Extended?
Add 'User-agent: Google-Extended' followed by 'Disallow: /' to your robots.txt. Use the exact token — substring variants of other tokens will not match.
Related bots
Same vendor, then other training crawlers — the full directory profiles all 34.