Ghost Agent Lab Free AgentScore Start 30-day trial

Agent directory · AI and search crawler

PerplexityBot

Crawls the web so Perplexity can find, cite and link to sites in its answers. Perplexity says it isn't used to train foundation models.

Run by
Perplexity
Type
AI and search crawler
Robots.txt name
PerplexityBot
Verifiable
Yes

What PerplexityBot does

Crawls the web so Perplexity can find, cite and link to sites in its answers. Perplexity says it isn't used to train foundation models.

As an AI and search crawler, it indexes pages so a search engine or AI search tool can find, cite and link to them. The kinds of AI agent that visit websites.

Should you allow it?

Allow it, unless you deliberately want to stay out of that search engine. Blocking it removes you from its results and answers. A decision guide for blocking or allowing AI crawlers.

How to allow or block PerplexityBot

PerplexityBot reads the PerplexityBot group in your robots.txt. To block it everywhere:

User-agent: PerplexityBot
Disallow: /

To allow it everywhere, even if another rule blocks other bots:

User-agent: PerplexityBot
Allow: /

Robots.txt is a request, not a lock: well-behaved crawlers follow it, impostors don't. Robots.txt for AI agents.

How it identifies itself

A typical user agent:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

How to tell if it's really PerplexityBot

You can check a request that claims to be PerplexityBot against what Perplexity publishes:

Requests that use the name but fail these checks are impostors, usually scrapers trying to get past bot protection. Ghost Agent Lab checks every request this way and gives you rules to block impostors. How to verify AI crawlers.

Every Perplexity agent, and a robots.txt for all of them →

Does PerplexityBot visit your site? Ghost Agent Lab reads your CDN or server logs and shows every request from PerplexityBot and 1,566+ other known agents: what they read, what your site sent back, and whether each request was genuine. See agent traffic, or check whether AI agents can use your site for free.