Ghost Agent Lab Free AgentScore Start 30-day trial

Agent directory · AI and search crawler

DuckDuckBot

DuckDuckGo's crawler for its search engine.

Run by
DuckDuckGo
Type
AI and search crawler
Robots.txt name
DuckDuckBot
Verifiable
No

What DuckDuckBot does

DuckDuckGo's crawler for its search engine.

As an AI and search crawler, it indexes pages so a search engine or AI search tool can find, cite and link to them. The kinds of AI agent that visit websites.

Should you allow it?

Allow it, unless you deliberately want to stay out of that search engine. Blocking it removes you from its results and answers. A decision guide for blocking or allowing AI crawlers.

How to allow or block DuckDuckBot

DuckDuckBot reads the DuckDuckBot group in your robots.txt. To block it everywhere:

User-agent: DuckDuckBot
Disallow: /

To allow it everywhere, even if another rule blocks other bots:

User-agent: DuckDuckBot
Allow: /

Robots.txt is a request, not a lock: well-behaved crawlers follow it, impostors don't. Robots.txt for AI agents.

How it identifies itself

A typical user agent:

DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)

How to tell if it's really DuckDuckBot

DuckDuckGo doesn't publish IP ranges or a reverse DNS domain for DuckDuckBot, so a request can't be proven to come from it. Anyone can send its user agent, so treat the name alone as a claim, not proof. Ghost Agent Lab reports these requests as unverified rather than guessing.

Every DuckDuckGo agent, and a robots.txt for all of them →

Does DuckDuckBot visit your site? Ghost Agent Lab reads your CDN or server logs and shows every request from DuckDuckBot and 1,566+ other known agents: what they read, what your site sent back, and whether each request was genuine. See agent traffic, or check whether AI agents can use your site for free.