Learning Center · Answers · Measurement
How do I know if AI agents visit my website?
Short answer
Look in your server or CDN logs, which record every request whether or not any JavaScript runs. Count requests by agent name, then verify each agent against its operator's published IP ranges or reverse DNS, because impostors copy well-known names. Track which agents visit, what they read and where they're blocked.
In more detail
Each log line shows which agent a request claims to be, where it came from, which page it asked for and the status code it got back. A 200 means it got the page, a 403 means it was blocked and a 429 means it was rate limited. Cloudflare Logpush, Vercel log drains and the access logs from Nginx or Apache are common places to start. Hosted platforms such as Shopify don't usually give you raw logs, so ask what bot reporting they offer.
A count by name is only a first look. Anyone can send a request claiming to be GPTBot or Googlebot, so split claimed agent traffic into verified, spoofed and can't tell. Treat requests you can't verify as unknown rather than fake, because blocking all of them can shut out agents you want.
The mix matters more than the total. Assistant fetchers such as ChatGPT-User are people asking about you right now, while a busy training crawler isn't a customer. Watch which pages agents reach, how often they're blocked, and how each agent's traffic moves week over week.
What to do
- Get access to your CDN or server request logs.
- Count requests by agent and by kind of agent.
- Verify agents against published IP ranges or reverse DNS.
- Track blocks, errors and the pages agents read most.
- Review the numbers weekly and look into sudden drops.
Read more
How does your own site do?
AgentScore checks your site in about a minute and ranks what to fix first.