Applebot

What Applebot does

Apple's crawler for Siri, Spotlight and Safari suggestions.

As an AI and search crawler, it indexes pages so a search engine or AI search tool can find, cite and link to them. The kinds of AI agent that visit websites.

Should you allow it?

Allow it, unless you deliberately want to stay out of that search engine. Blocking it removes you from its results and answers. A decision guide for blocking or allowing AI crawlers.

How to allow or block Applebot

Applebot reads the Applebot group in your robots.txt. To block it everywhere:

User-agent: Applebot
Disallow: /

To allow it everywhere, even if another rule blocks other bots:

User-agent: Applebot
Allow: /

Robots.txt is a request, not a lock: well-behaved crawlers follow it, impostors don't. Robots.txt for AI agents.

How it identifies itself

A typical user agent:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)

How to tell if it's really Applebot

You can check a request that claims to be Applebot against what Apple publishes:

• IP ranges: search.developer.apple.com/applebot.json
• Reverse DNS: the address resolves to a name ending in applebot.apple.com, and that name resolves back to the same address

Requests that use the name but fail these checks are impostors, usually scrapers trying to get past bot protection. Ghost Agent Lab checks every request this way and gives you rules to block impostors. How to verify AI crawlers.

Other agents from Apple

• Applebot-Extended: AI training crawler

Every Apple agent, and a robots.txt for all of them →