Myths and facts
What people often believe about AI agents and websites, and what's actually true. Each fact links to the guide that explains it.
-
Myth Blocking GPTBot takes us out of ChatGPT's answers.
Fact GPTBot collects training data. ChatGPT's search and the fetches it makes when someone asks about you use different agents with their own names, so you can opt out of training and still appear in AI answers. The real risk is a long "block all AI" list that shuts out assistants and AI search too.
For marketing and seo. Read more: Should you block AI crawlers? A decision guide for leaders, robots.txt for AI agents: who to allow and who to block, Seven agent readiness mistakes to avoid
-
Myth AI agents load pages like a browser, JavaScript and all.
Fact Most AI crawlers and assistant fetchers read only the HTML your server sends and never run JavaScript. Prices, stock, reviews or menus that appear only after scripts load can be invisible to them. Browser agents do run JavaScript, but they are slow and impatient.
For developers. Read more: Why AI agents can't see your JavaScript-only content, Prices AI agents can read, What an AI agent sees when it visits your store
-
Myth If we rank well in Google, AI agents can read our site too.
Fact Good SEO is a head start, not the finish. Google runs JavaScript and is usually let through by bot protection, so a site can rank well while AI assistants see a nearly blank page or a challenge at the door. SEO checks rarely test how bot protection treats AI assistants.
For marketing and seo. Read more: SEO got you found. Agent readiness gets you chosen, Why AI agents can't see your JavaScript-only content
-
Myth Adding an llms.txt file is the main way to get seen by AI.
Fact llms.txt is a short guide that points agents to your key pages. It is cheap and worth adding, but it is a proposal that not every AI company reads, and AgentScore weights it lightly. It helps alongside clear pages, structured data and a sitemap, never instead of them.
For marketing and seo. Read more: How to write an llms.txt file, Reading your AgentScore report: what to fix first
-
Myth Our robots.txt allows AI, so agents can get in.
Fact robots.txt is only the first door. If your CDN, firewall or bot protection challenges "AI crawlers", the agent is stopped before it ever reads your robots.txt. Check your bot protection and CDN settings as well as the file.
For marketing and seo. Read more: robots.txt for AI agents: who to allow and who to block, Bot protection and CAPTCHAs: stop blocking the agents you want, Setting up your CDN for AI agents
-
Myth Bot traffic is a cost and a risk, so block as much as possible.
Fact "Bot" now covers very different visitors. An AI assistant reading your page is often answering one person's question about you right now, and a browser agent may be trying to buy. Treating them like scrapers turns those customers away, and nobody notices because analytics never see them.
For leaders. Read more: Agent traffic isn't bot traffic, The AI agents that visit your website, and what each one wants, Bot protection and CAPTCHAs: stop blocking the agents you want
-
Myth If AI agents visited, we'd see them in Google Analytics.
Fact Page-tag analytics mostly cannot see AI agents. Most agents never run the tracking script, analytics tools filter out known bots on purpose, and browser agents look like ordinary visitors. Server or CDN logs record every request, so that is where agent traffic is measured.
For marketing and seo. Read more: Why your analytics can't see AI agents, and how to measure agent traffic, Agent traffic isn't bot traffic
-
Myth A visitor calling itself GPTBot must really be OpenAI.
Fact A user agent name is a promise, not a proof. Scrapers often borrow the names of well-known crawlers to get past bot protection. Check where the request came from, using the operator's published IP ranges, reverse DNS or a signature, and treat requests you can't verify as unknown.
For developers. Read more: How to tell if an AI crawler is real, Agent traffic isn't bot traffic
-
Myth Accessibility and agent readiness are unrelated projects.
Fact Many browser agents read a page through the same structure screen readers use. Named buttons, labelled form fields and image descriptions help both. Accessibility doesn't cover everything agents need, such as robots.txt and bot protection, but if you run an accessibility program, put agent readiness alongside it.
For e-commerce. Read more: Accessibility work is agent readiness work, Buttons, links and forms agents can use: a checklist
-
Myth To be agent-ready, we need to build an MCP server.
Fact An MCP server can help when your catalog is hard to browse and your basics already work, but it is the next step, not a substitute. AgentScore weights it lightly and never fails a site for lacking one. Fixes to bot protection, named buttons and pop-ups carry far more weight and usually take less time.
For leaders. Read more: Should you build an MCP server yet?, MCP and WebMCP: giving AI agents a direct way into your site
-
Myth Smarter AI models will soon get past our CAPTCHAs anyway.
Fact A CAPTCHA exists to stop automated visitors, and an agent shopping for someone is one. When an agent meets a challenge it usually stops or hands the page back to the person. A challenge at checkout loses the sale at its most valuable step, and the fix is usually a bot protection setting.
For e-commerce. Read more: What AI agents can't do on websites yet, Bot protection and CAPTCHAs: stop blocking the agents you want
-
Myth Agent readiness is a technical job for IT to sort out.
Fact Agent readiness spans robots.txt and bot protection, page content, buttons and pop-ups, and checkout, and each has a different team. Each team sees its own part working while the failure shows up elsewhere. It needs one clear owner and a clear split of the rest.
For leaders. Read more: Who owns agent readiness?, SEO got you found. Agent readiness gets you chosen
Not sure which of these apply to your site? Take the ten-question self-assessment or run a free AgentScore scan.