Why AgentScore stopped marking you down for checking who's visiting

AgentScore tests whether your bot protection lets AI assistants in by visiting your site under their names: ChatGPT-User, Claude-User, Perplexity-User. But those visits come from our servers, not from OpenAI's, Anthropic's or Perplexity's. A site that checks where a visitor really comes from will rightly turn them away. Until this month, AgentScore marked that site down for it. Here's how we fixed that, and two other corrections the same work turned up.

The problem: good security looked like blocking

Anyone can put "ChatGPT-User" in a request. Scrapers do it all the time to get past bot protection. So the best bot protection doesn't trust the name: it checks that the request comes from the addresses the AI company publishes, or carries its cryptographic signature. A request that only claims the name gets turned away.

That's exactly what AgentScore's test requests are: they claim the name but come from our own cloud servers. A store with well-configured protection turned them away, and AgentScore reported "ChatGPT-User was blocked". The store was doing the right thing, and its score went down.

We found it while auditing our study of online stores. Of the 22 stores that failed the bot protection check in our first run, 5 had turned away only one request: the one claiming to be Googlebot. On 4 more, every request claiming to be a known crawler was turned away, Googlebot included.

• 22 stores failed the bot protection check in our first run
• 5 had turned away only a fake Googlebot
• 4 turned away every fake crawler, Googlebot included

The fix: Googlebot as a control

We can't send requests from OpenAI's addresses, so we can't see what the real ChatGPT-User gets. What we can tell is whether a site checks addresses at all. AgentScore now also visits your home page as Googlebot, once per scan. No store wants to block the real Googlebot, because it would vanish from Google search. So if a request claiming to be Googlebot from our servers is turned away, your protection is checking addresses, not names.

That gives three outcomes:

• Googlebot gets in, AI assistants don't Your protection blocks AI assistants by name. The check fails, as before.
• Both are turned away Your protection checks where visitors come from. We can't tell whether the real assistants get through, so the check says "not tested" and why. It never costs points.
• Only the fake Googlebot is turned away You verify Googlebot and let AI assistants in. The check passes.

The same rule applies to the two other checks that visit as an AI assistant: whether assistants can open your product, pricing and cart pages, and whether they're blocked at the cart and checkout.

One limit, stated plainly: a store that checks Googlebot's address but blocks AI assistants by name will now be marked "not tested" rather than blocking. We think that's the right trade. Marking good security as a failure was the bigger error.

Two more corrections

The same audit turned up two smaller problems, both now fixed.

Stores we didn't recognise as stores. AgentScore finds a product page by what product links usually look like, such as /products/…. Many stores use plain names instead, like /brass-desk-lamp, and were treated as ordinary websites, so they skipped every store check. AgentScore now also looks for product cards on your home page (a picture, a price and a link) and opens one to confirm it's a product page. The same sample of sites now gives 258 stores instead of 234.

Prices in some currencies. The check for prices in your page's HTML didn't recognise prices like "12,99 zł" or "1 299 Kč", so some stores were told their prices were missing when they weren't. It now recognises the currencies stores use most, with the symbol before or after the amount.

First run

• 234 stores recognised
• 9% blocked AI assistants on the home page
• 30% had no prices in the product page's HTML

After the fixes (scoring v7)

• 258 stores recognised
• 5% blocked AI assistants on the home page
• 11% had no prices in the product page's HTML

The prices correction is the one we're least proud of. "30% of stores hide their prices from agents" was a headline figure in our first study, and most of it was our bug. We've republished the study with a note saying so.

Also new: AgentScore checks your checkout

Most checkouts only show their form once something is in the cart, so AgentScore usually couldn't check whether your checkout fields are labelled, or whether you offer guest checkout. It now adds one item to the cart, the way a shopper would, and looks. It never fills in the form or places an order, and if it can't add an item, for example because a size has to be chosen first, the report says why. See how to make checkout work for AI shopping agents.

What this means for your score

• If your bot protection checks where visitors come from, those checks now say "not tested" with an explanation, instead of failing.
• If your product links don't look like /products/…, your scan may now include the product and checkout checks it skipped before.
• If you price in złoty, koruna, kronor or similar, your prices are now recognised.

Scores from different scoring versions aren't comparable, so run AgentScore again to see where you stand now. To see what each check does, read how AgentScore works.