What it does
OAI-SearchBot is OpenAI's ai search crawler. An AI search index that can cite and link your pages in answers (ChatGPT search, Perplexity, Claude search, Alexa, Siri). Blocking removes you from those answer surfaces — for most sites these are wanted crawlers.
Powers ChatGPT search citations; not used for foundation-model training. ~24h for robots.txt changes to take effect. Vendor now documents OAI-SearchBot/1.4 inside a full Chrome-like UA string. NOT substring-matched by 'GPTBot' patterns — separate token. OpenAI docs moved to developers.openai.com 2026-08 (old platform.openai.com path 301s).
Full UA: not captured in our July 2026 roster review — match log hits on the token OAI-SearchBot (case-insensitive substring) and verify by IP before acting.
How to verify a hit is really OAI-SearchBot
Verification method (July 2026): Published IP list: https://openai.com/searchbot.json (fetched live 2026-08-22)
Strongest available check: the vendor publishes the crawler's egress IPs. Fetch the list, then confirm the hit's source IP falls inside it — a UA match from any other IP is a spoofer.
User-agent strings are freely spoofed, so a UA match alone proves nothing. Full step-by-step method: verify by IP.
Allow or block in robots.txt
Match the exact token OAI-SearchBot in robots.txt:
# Block OAI-SearchBot site-wide
User-agent: OAI-SearchBot
Disallow: /# Explicitly allow OAI-SearchBot
User-agent: OAI-SearchBot
Allow: /For the allow-search-block-training combined pattern (and the Allow-directive gotcha that silently kills carve-outs), see the robots.txt guide. robots.txt governs whether OAI-SearchBot may fetch; llms.txt is the separate, curated map AI assistants read once allowed in.
Vendor documentation: https://developers.openai.com/api/docs/bots
The OpenAI family
OpenAI operates 4 distinct tokens in this roster, and they do not block each other: a robots.txt line for OAI-SearchBot does nothing to its siblings — each needs its own User-agent: entry (the robots.txt guide has combined patterns).
| Token | Role | robots.txt |
|---|---|---|
| GPTBot | Training | honors robots.txt |
| ChatGPT-User | User action | claims robots compliance |
| OAI-AdsBot | User action | claims robots compliance |
Frequently asked questions
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's ai search crawler. An AI search index that can cite and link your pages in answers (ChatGPT search, Perplexity, Claude search, Alexa, Siri). Blocking removes you from those answer surfaces — for most sites these are wanted crawlers.
Does OAI-SearchBot respect robots.txt?
Yes — OpenAI documents robots.txt compliance for OAI-SearchBot (vendor docs reviewed July 2026).
How do I block OAI-SearchBot?
Add 'User-agent: OAI-SearchBot' followed by 'Disallow: /' to your robots.txt. Use the exact token — substring variants of other tokens will not match.
How do I verify OAI-SearchBot traffic by IP?
Fetch OpenAI's published IP list for OAI-SearchBot and confirm the hit's source IP falls inside it — a OAI-SearchBot user-agent from any other IP is a spoofer.
Related bots
Same vendor, then other ai search crawlers — the full directory profiles all 34.