Why AI agents can't see your JavaScript-only content
Usually fixed by: Developer · Typical effort: weeks
Open a modern website with JavaScript turned off and you'll often see a logo, a spinner and not much else. That's what many AI agents see too. If your prices, products or policies only appear after scripts run, a large share of AI traffic never sees them.
How agents read a page
When a browser loads a page, it does two things: it downloads the HTML your server sends, then runs the JavaScript, which may fetch more data and build much of what you see. Agents differ in how far they go:
| Agent | Typically reads |
|---|---|
| Training and AI search crawlers | The HTML only. Running JavaScript for billions of pages is slow and expensive, so most AI crawlers don't. |
| Assistant fetchers (fetching a page because someone asked) | Usually the HTML only, and they have a few seconds at most. |
| Browser agents | The full page, including JavaScript, but they're slow and impatient. Long loading states and content that appears late still trip them up. |
Googlebot does run JavaScript, which is why many sites that rely on it rank fine in Google search. That has hidden the problem: the same site can be well indexed by Google and nearly blank to AI assistants.
How to check your own site
- View the source. In your browser, use View Page Source (not the developer tools inspector, which shows the page after JavaScript). Search for a product name or price you can see on the page. If it isn't in the source, it's added by JavaScript.
- Turn JavaScript off. In Chrome's developer tools, open the command menu, type "Disable JavaScript", and reload. What's left is roughly what an HTML-only agent sees.
- Fetch it from the command line.
curl -s https://yourdomain.com/your-page | grep -i "your price"shows exactly what your server sends. - Run AgentScore. It compares the text in your HTML with the text after JavaScript runs. It passes when at least 70% of the text is already in the HTML, and warns or fails below that.
What usually goes missing
- Prices and stock loaded from an API after the page appears, so the HTML says "Loading…" or nothing at all.
- Whole pages in single-page apps, where the HTML is an empty
<div id="root">and everything else is built in the browser. - Product lists with infinite scroll that load more items only as a person scrolls, with no paginated links to follow.
- Content inside tabs and accordions fetched only when clicked, such as specifications, sizing and shipping details.
- Structured data injected by a tag manager or script, which agents reading the HTML never see. See product data AI shopping agents can read.
- Reviews from third-party widgets that load in the browser.
How to fix it
The aim is simple: the important content should be in the HTML your server sends. JavaScript can still make the page interactive afterwards.
Server-side rendering (SSR)
The server builds the full HTML for each request, then JavaScript takes over in the browser. Modern frameworks support this directly: Next.js and Remix for React, Nuxt for Vue, SvelteKit for Svelte, and Angular's server rendering. If your site is a single-page app built with one of these, turning on server rendering for key pages is usually the biggest single fix.
Static generation
Pages are built to HTML ahead of time, at deploy. It's ideal for content that changes rarely, such as marketing pages, docs, policies and blog posts. Astro, Eleventy, Hugo and the frameworks above can all do it, often alongside server rendering for pages that change often.
Render the essentials, enhance the rest
You don't need to render everything on the server. Prioritize what agents ask about: product name, price, availability, key specifications, shipping and returns, and plan pricing. Personalized recommendations, recently viewed items and chat widgets can stay in JavaScript.
Pre-rendering as a stopgap
Pre-rendering services run your JavaScript ahead of time and serve the finished HTML to bots. It can work as a short-term fix, but it means maintaining two versions of each page, and it depends on correctly recognizing every agent. Google describes serving bots differently ("dynamic rendering") as a workaround rather than a long-term solution. Server rendering is better.
Smaller fixes that help
- Replace infinite scroll with, or add, real paginated links (
?page=2) agents can follow. - Put tab and accordion content in the HTML and hide it with CSS, instead of fetching it on click.
- Move JSON-LD from your tag manager into the page template.
- Include a summary of reviews (average rating and count) in the HTML, even if the full reviews load later.
- Give the page a meaningful
<title>and<h1>in the HTML, never only after JavaScript runs.
Once your content is in the HTML, the next question is whether agents can find their way around it and use it. See buttons, links and forms agents can use.