Shoppers now ask AI assistants which brand to buy, and the answer may never send anyone to your site. This guide shows how to track what assistants say about you and your products with a fixed set of questions, how to record and compare the answers over time, and how to fix the pages behind the answers that are wrong.
Why track answers, not just traffic
Your analytics see people who arrive. They don't see the shopper who asked "Which coffee subscription is best for espresso?" and was told about three other brands. They also don't see the shopper who was told your returns window is 14 days when it's 30, and decided not to bother.
Two kinds of measurement fill that gap. AI referral tracking counts the visits and sales that do come from assistants. Agent traffic shows which AI agents fetch your pages. This guide covers the third: what the answers themselves say.
- Agent traffic Which AI agents fetch your pages, from server and CDN logs
- AI referrals Visits and orders that arrive from an assistant's answer
- Answers Whether assistants name you, how they describe you, and what they cite
Build a fixed question set
Answers only mean something if you ask the same questions every time. Write a list of 15 to 25 questions, keep it stable, and change it only on purpose. Group them by what they test.
| Type | Example for Northwind Coffee | What it tells you |
|---|---|---|
| Discovery | "Best coffee subscription for espresso at home" | Whether you're named at all when the shopper doesn't know you |
| Comparison | "Northwind Coffee or Ridgeline Roasters for decaf?" | How you're described next to a named competitor |
| Brand facts | "Does Northwind Coffee ship to Canada?" | Whether stated facts match your site |
| Product facts | "How much is Northwind's Kenya AA, and is it in stock?" | Whether prices and stock are current |
| Policy | "Can I return opened coffee to Northwind?" | Whether your policies are quoted correctly |
- Leave your name out of discovery questions. The point is to see whether the assistant brings you up unprompted.
- Write them as shoppers talk, with the constraints real shoppers add: a budget, a use, a location.
- Use your own data for ideas: site search terms, customer service questions and the search queries that already bring people to you.
Pick competitors and what to record
Choose two to four competitors that shoppers really compare you with, not the biggest names in the category. For every answer, record the same fields, so you can compare month to month.
- Date, assistant and settings (for example, web search on)
- Whether it names you, and in what position
- Which competitors it names, in order
- How it describes you, in a few words
- Any facts it states about you: prices, policies, stock
- Which pages it cites, and whether any are yours
A spreadsheet with one row per answer is enough to start. The fields that pay off most are the stated facts and the cited pages, because they tell you what to fix.
Ask the same way every time
Assistants don't give the same answer twice. The wording changes, the order of brands changes, and sometimes the brands do. A single answer tells you little. Make the method consistent instead:
- Ask each question several times, each in a fresh conversation, and record how many of the answers name you rather than whether one did.
- Sign out, or use a clean account. Memory and past conversations can change answers.
- Note whether web search was used. Answers with web search can cite and reflect your current pages; answers without it reflect what the model learned earlier. Anthropic's documentation for Claude's web search tool, for example, says responses include citations for the sources drawn from search results.
- Keep the cadence fixed, weekly or monthly, and log changes you make to your site on the same timeline, so you can see whether a fix moved anything.
A single answer tells you little. Ask the same questions, the same way, on a schedule, and count.
Treat small changes with caution: With a few questions asked a few times, a swing from one month to the next can be chance. Look for changes that hold over several checks, and across several questions, before you report a win or a loss.
Check facts and fix the source
The most useful output of tracking is a list of things assistants get wrong. Go through every stated fact and compare it with your site. Then trace each mistake to where it came from.
| What you see | Likely cause | Where to fix it |
|---|---|---|
| Not named, and no page of yours cited | Assistants can't reach or read your pages | Access: robots.txt and bot protection |
| Wrong or old price | Price missing from the HTML, or data that disagrees with the page | Prices AI agents can read |
| Wrong returns or shipping terms | Policy page unclear, or two pages that disagree | Policy pages assistants can quote |
| A competitor's review site cited instead of you | Your own page doesn't answer the question directly | Write a clear page that does |
| Wrong facts traced to another site | An outdated listing, directory or article | Ask that site to correct it, and state the fact clearly on yours |
Access comes first. OpenAI and Anthropic both say that blocking their search crawler or assistant fetcher can keep you out of their answers or reduce your visibility in them, and Google only shows pages in AI Overviews and AI Mode that are indexed and eligible for a snippet. After access, most fixes are the same work as agent readiness generally: facts in the HTML, structured data that matches, and one version of each policy. Our guide to how AI assistants decide which products to recommend covers what each vendor documents and why each step matters.
For Google's features there is one extra source of data: Google says sites that appear in AI Overviews and AI Mode are counted in Search Console's Performance report, under the Web search type (AI features and your website).
- Ask The same questions, several times each
- Record Named or not, competitors, facts, citations
- Spot Wrong facts, missing mentions
- Trace To the cited page or the page that's missing
- Fix Your own page first, then other sites
Tracking answers in Ghost Agent Lab
The Ghost Agent Lab app has an AI visibility page that does this for Claude. It's in beta, and results appear once it's switched on for your account. You add the questions shoppers might ask, one per line, or have Claude suggest some from your site, and you name your brand and up to three competitors. Each check asks Claude every question three times, with web search on, in a fresh conversation. It records whether the answer names you, where, how it speaks about you, which other brands it names and which pages it cites. After the first check it runs every Monday.
The page shows your share of answers (the share of answers that name you), how many questions name you at least once, your average position and how often answers link to your site, with a comparison against your competitors. You can open any question to read each answer and its citations. The answers come from Claude's API with web search on, so they're close to, but not exactly, what a signed-in person sees in the Claude app. In the beta you can track up to 25 questions per site, and other assistants aren't covered yet, so keep a manual check for those.
Where to start
- Write 15 to 25 questions across discovery, comparison, brand facts, product facts and policy. Leave your name out of the discovery ones.
- Pick two to four real competitors.
- Ask each question three times in fresh conversations on two or three assistants, and record the six fields for every answer.
- List every wrong fact and every question where you're never named, and trace each to a page.
- Fix your own pages first, starting with access. Run AgentScore to find what stops assistants reading you.
- Repeat on the same day each month, and report the trend alongside your agent readiness KPIs.