What it does
Google-Agent is Google's user action crawler. A live fetch made because a human asked the assistant about your page right now. Closest thing to a real visitor; several of these (Perplexity-User, meta-externalfetcher) ignore robots.txt by design because the request is user-initiated.
Agents hosted on Google infrastructure navigating the web and performing actions at a user's request. As a user-triggered fetcher it generally ignores robots.txt — blocking requires WAF/IP rules against the published ranges. Token embedded in a full Chrome-like UA string. New since our 2026-07-19 snapshot.
Full UA: not captured in our July 2026 roster review — match log hits on the token Google-Agent (case-insensitive substring) and verify by IP before acting.
How to verify a hit is really Google-Agent
Verification method (July 2026): Published IP list: https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers.json (fetched live 2026-08-22)
Strongest available check: the vendor publishes the crawler's egress IPs. Fetch the list, then confirm the hit's source IP falls inside it — a UA match from any other IP is a spoofer.
User-agent strings are freely spoofed, so a UA match alone proves nothing. Full step-by-step method: verify by IP.
Allow or block in robots.txt
Match the exact token Google-Agent in robots.txt:
# Block Google-Agent site-wide
User-agent: Google-Agent
Disallow: /# Explicitly allow Google-Agent
User-agent: Google-Agent
Allow: /For the allow-search-block-training combined pattern (and the Allow-directive gotcha that silently kills carve-outs), see the robots.txt guide. robots.txt governs whether Google-Agent may fetch; llms.txt is the separate, curated map AI assistants read once allowed in.
Vendor documentation: https://developers.google.com/search/docs/crawling-indexing/google-user-triggered-fetchers
The Google family
Google operates 5 distinct tokens in this roster, and they do not block each other: a robots.txt line for Google-Agent does nothing to its siblings — each needs its own User-agent: entry (the robots.txt guide has combined patterns).
| Token | Role | robots.txt |
|---|---|---|
| Google-Extended | Training | honors robots.txt |
| GoogleOther | Training | honors robots.txt |
| Google-CloudVertexBot | User action | honors robots.txt |
| Google-GeminiNotebook | User action | ignores robots.txt |
Frequently asked questions
What is Google-Agent?
Google-Agent is Google's user action crawler. A live fetch made because a human asked the assistant about your page right now. Closest thing to a real visitor; several of these (Perplexity-User, meta-externalfetcher) ignore robots.txt by design because the request is user-initiated.
Does Google-Agent respect robots.txt?
No — Google-Agent does not treat robots.txt as binding. Blocking it requires WAF or IP-level rules.
How do I block Google-Agent?
robots.txt cannot stop Google-Agent; block it with WAF or IP-level firewall rules instead.
How do I verify Google-Agent traffic by IP?
Fetch Google's published IP list for Google-Agent and confirm the hit's source IP falls inside it — a Google-Agent user-agent from any other IP is a spoofer.
Related bots
Same vendor, then other user-action fetchers — the full directory profiles all 34.