Agent directory ยท AI and search crawler
ExaSearchBot
Exa's crawler for its search engine, which AI apps and agents use to find pages.
- Run by
- Exa
- Type
- AI and search crawler
- Robots.txt name
ExaSearchBot- Verifiable
- Not yet
What ExaSearchBot does
Exa's crawler for its search engine, which AI apps and agents use to find pages.
As an AI and search crawler, it indexes pages so a search engine or AI search tool can find, cite and link to them. The kinds of AI agent that visit websites.
Should you allow it?
Allow it, unless you deliberately want to stay out of that search engine. Blocking it removes you from its results and answers. A decision guide for blocking or allowing AI crawlers.
How to allow or block ExaSearchBot
ExaSearchBot reads the ExaSearchBot group in your robots.txt. To block it everywhere:
User-agent: ExaSearchBot
Disallow: /To allow it everywhere, even if another rule blocks other bots:
User-agent: ExaSearchBot
Allow: /Robots.txt is a request, not a lock: well-behaved crawlers follow it, impostors don't. Robots.txt for AI agents.
How to tell if it's really ExaSearchBot
Our registry doesn't verify ExaSearchBot against published IP ranges yet, so treat a request that uses the name as a claim, not proof: anyone can send its user agent. Check Exa's documentation for how to verify it.
Does ExaSearchBot visit your site? Ghost Agent Lab reads your CDN or server logs and shows every request from ExaSearchBot and 1,566+ other known agents: what they read, what your site sent back, and whether each request was genuine. See agent traffic, or check whether AI agents can use your site for free.