Learning Center · Answers · Measurement

How do I know if AI agents visit my website?

Short answer

Look in your server or CDN logs, which record every request whether or not any JavaScript runs. Count requests by agent name, then verify each agent against its operator's published IP ranges or reverse DNS, because impostors copy well-known names. Track which agents visit, what they read and where they're blocked.

Answer · Last checked November 12, 2026 · Based on Why your analytics can't see AI agents, and how to measure agent traffic, How to tell if an AI crawler is real, Agent traffic isn't bot traffic

In more detail

Each log line shows which agent a request claims to be, where it came from, which page it asked for and the status code it got back. A 200 means it got the page, a 403 means it was blocked and a 429 means it was rate limited. Cloudflare Logpush, Vercel log drains and the access logs from Nginx or Apache are common places to start. Hosted platforms such as Shopify don't usually give you raw logs, so ask what bot reporting they offer.

A count by name is only a first look. Anyone can send a request claiming to be GPTBot or Googlebot, so split claimed agent traffic into verified, spoofed and can't tell. Treat requests you can't verify as unknown rather than fake, because blocking all of them can shut out agents you want.

The mix matters more than the total. Assistant fetchers such as ChatGPT-User are people asking about you right now, while a busy training crawler isn't a customer. Watch which pages agents reach, how often they're blocked, and how each agent's traffic moves week over week.

What to do

  1. Get access to your CDN or server request logs.
  2. Count requests by agent and by kind of agent.
  3. Verify agents against published IP ranges or reverse DNS.
  4. Track blocks, errors and the pages agents read most.
  5. Review the numbers weekly and look into sudden drops.

Read more

All answers

How does your own site do?

AgentScore checks your site in about a minute and ranks what to fix first.