User agent

The name a visitor sends with every request to say what it is, such as a browser version or GPTBot. Easy to fake, so it shouldn't be trusted on its own.

Why it matters

User agents are how most analytics, logs and bot rules decide what a visitor is, so it matters that they can be faked. A rule that trusts the name alone lets scrapers in by pretending to be Googlebot, while a rule that blocks by name can turn away real assistants. Pair names with verification before you let a visitor in or count it as a real agent.

More terms in the agents

• AI agent Software that uses an AI model to do something on the web for a person or a company: read a page, answer a question, compare products or complete a task.
• Training crawler A bot that collects pages to train AI models, such as GPTBot or ClaudeBot.
• AI search crawler A bot that indexes pages so AI search and assistants can cite and link to them, such as OAI-SearchBot or PerplexityBot.
• Assistant fetcher An AI assistant fetching a page in real time because a person asked it something, such as ChatGPT-User or Claude-User.
• Browser agent An agent that opens your site in a real browser and clicks, types and scrolls like a person, to book, sign up or buy.

Every term, A to Z → · Read the glossary in full →