Seven agent readiness mistakes to avoid

Most agent readiness problems aren't decisions anyone made. They're side effects of sensible work: a security rule, a design choice, a tracking setup. Here are seven mistakes that are easy to make and easy to miss, why each one matters, and what to do instead.

1. Blocking AI assistants along with training crawlers

Many sites decide not to let AI companies train on their content, which is a fair choice. The mistake is blocking everything with "AI" in its name, including the assistants that fetch a page because a customer asked a question right now. Those are different agents with different names: GPTBot collects training data, while ChatGPT-User and OAI-SearchBot fetch pages for ChatGPT users and search. Anthropic draws the same line between ClaudeBot and Claude-User.

# Opt out of AI training
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
Disallow: /

# Everyone else, including AI assistants and AI search
User-agent: *
Allow: /

Instead: decide on training and on assistants separately. See should you block AI crawlers? and robots.txt for AI agents. The same split applies to bot protection rules, which often block by category rather than by name.

2. Putting a CAPTCHA at checkout

A CAPTCHA at checkout is often added to stop card testing, and it works on bots. It also stops every AI agent at the very last step, after it has searched, compared, chosen a size and filled the cart. That's the most expensive place to lose a customer.

Instead: use your payment provider's fraud tools and rate limits on payment attempts, and keep challenges for traffic that is actually suspicious. AgentScore checks whether AI agents are blocked or shown a CAPTCHA at the cart and checkout. See checkout for AI shopping agents and bot protection and CAPTCHAs.

3. Prices that only appear with JavaScript

Many sites build product and pricing pages in the browser. A person sees the price a moment after the page loads. Many AI agents read the HTML your server sends and never run the scripts, so they see a product with no price, or a placeholder. They may skip you, or tell their user to check the site.

Instead: make sure prices are in the HTML, through server-side rendering or static generation, and in your product structured data. Check by viewing the page source, not the browser's inspector, which shows the page after scripts run. See prices AI agents can read and JavaScript-only content.

4. Icons with no names

A magnifying glass for search, a bag for the cart, a cross to close a pop-up. People know what they mean. Browser agents choose what to click by each control's name, the same way screen readers do, and an icon with no text or label has no name at all. To the agent, the cart button isn't there.

<button aria-label="Open cart">
  <svg aria-hidden="true">…</svg>
</button>

Instead: give every icon-only button and link a label, and use real <button> and <a> elements rather than clickable boxes. See buttons, links and forms agents can use and accessibility work is agent readiness work.

5. Publishing llms.txt and forgetting it

llms.txt is a proposed convention for giving AI a short guide to your site. It's quick to write, which is also why it goes stale: a file written at launch still links to last year's collections, a retired plan, or a returns page that moved. An out-of-date guide can be worse than none, because it points agents at the wrong answer.

Instead: give llms.txt an owner and add it to the checklist for site changes, alongside the sitemap. Note that AgentScore checks that the file exists, not whether what it says is still true, so that part is up to you. See how to write an llms.txt file.

6. Trusting user agents

A user agent is a name a visitor gives itself, and anyone can use any name. If you allow a list of AI agents by user agent alone, scrapers borrow those names to get in. If you block by user agent, you may turn away a real assistant while impostors simply change their name.

Instead: verify identity. The large operators publish their IP ranges, and many crawlers can be confirmed with a reverse DNS lookup. Newer agents can sign their requests with Web Bot Auth, which your CDN may already check for you. See how to tell if an AI crawler is real and why we sign every request our agents send. Ghost Agent Labs' Verification page checks every request that claims to be a known agent and shows the impostors.

7. Measuring AI agents with JavaScript analytics

Analytics tools such as Google Analytics count visitors by running a script in the browser. Most AI agents don't run it, so they never appear. A report that shows no AI agents isn't evidence that none came. It may mean they came and the tag didn't see them, or that they were blocked before the page loaded.

Instead: measure agents from your server or CDN logs, which record every request whether or not scripts run. Keep analytics for the people who click through from an assistant's answer, which it can see. See why your analytics can't see AI agents, AI referral traffic and agent traffic isn't bot traffic.

What these have in common

None of these mistakes shows up from inside the business. The site looks fine in a browser, the dashboards look normal, and the security team sees fewer bad bots. The cost lands on customers using an assistant, who quietly go elsewhere.

Mistake	Quick check	Usually fixed by
Assistants blocked with training crawlers	Read /robots.txt and your bot rules for AI categories	SEO, bot protection or CDN admin
CAPTCHA at checkout	Ask your security team what protects the checkout	Bot protection or CDN admin
JavaScript-only prices	View source on a product page and search for the price	Developer
Icons with no names	Tab through the header and listen with a screen reader	Developer
Stale llms.txt	Open /llms.txt and click every link	SEO or content team
Trusting user agents	Ask how "allowed" AI agents are identified	Bot protection or CDN admin
JavaScript analytics only	Ask where agent traffic numbers come from	Analytics or digital lead

A free AgentScore scan catches several of these in under a minute. For the journeys themselves, Ghost Agents show whether an agent can actually finish them.