What it does
meta-externalagent is Meta's training crawler. Content is copied into datasets used to train foundation models. Blocking costs you nothing in traffic today; allowing is a donation of your content to model weights with no citation or referral in return.
Llama training + product improvement crawler. Observed live in our own logs (a production site we operate: meta-externalagent/1.1, 272 hits/24h on 2026-05-10) and honored our robots/403 combination. Lowercase token in the wild.
Full UA: not captured in our July 2026 roster review — match log hits on the token meta-externalagent (case-insensitive substring) and verify by IP before acting.
How to verify a hit is really meta-externalagent
Verification method (July 2026): IP ranges discoverable via Meta ASN 32934 (RDAP/whois); docs page fetch returned 400 this pass
User-agent strings are freely spoofed, so a UA match alone proves nothing. Confirm the source IP belongs to Meta — published IP-range JSON where available, otherwise RDAP/rDNS on the IP. Full method: verify by IP.
Allow or block in robots.txt
Match the exact token meta-externalagent in robots.txt:
# Block meta-externalagent site-wide
User-agent: meta-externalagent
Disallow: /# Explicitly allow meta-externalagent
User-agent: meta-externalagent
Allow: /For the allow-search-block-training combined pattern (and the Allow-directive gotcha that silently kills carve-outs), see the robots.txt guide.
Vendor documentation: https://developers.facebook.com/docs/sharing/webmasters/web-crawlers
Frequently asked questions
What is meta-externalagent?
meta-externalagent is Meta's training crawler. Content is copied into datasets used to train foundation models. Blocking costs you nothing in traffic today; allowing is a donation of your content to model weights with no citation or referral in return.
Does meta-externalagent respect robots.txt?
Meta claims meta-externalagent respects robots.txt, but compliance is not independently confirmed.
How do I block meta-externalagent?
Add 'User-agent: meta-externalagent' followed by 'Disallow: /' to your robots.txt. Use the exact token — substring variants of other tokens will not match.