Articles

Why AI agents can't see your JavaScript-only content

Usually fixed by: Developer · Typical effort: weeks

Open a modern website with JavaScript turned off and you'll often see a logo, a spinner and not much else. That's what many AI agents see too. If your prices, products or policies only appear after scripts run, a large share of AI traffic never sees them.

Short answer

Can AI agents see content my site loads with JavaScript?

Often they can't. Many AI agents read only the HTML your server sends, so prices, products and policies that appear after scripts run may be invisible to them, even if you rank well in Google. Ask your developers to put product name, price, availability, key specifications, shipping, returns and plan pricing in the server's HTML first.

How agents read a page

When a browser loads a page, it does two things: it downloads the HTML your server sends, then runs the JavaScript, which may fetch more data and build much of what you see. Agents differ in how far they go:

AgentTypically reads
Training and AI search crawlersThe HTML only. Running JavaScript for billions of pages is slow and expensive, so most AI crawlers don't.
Assistant fetchers (fetching a page because someone asked)Usually the HTML only, and they have a few seconds at most.
Browser agentsThe full page, including JavaScript, but they're slow and impatient. Long loading states and content that appears late still trip them up.

Googlebot does run JavaScript, which is why many sites that rely on it rank fine in Google search. That has hidden the problem: the same site can be well indexed by Google and nearly blank to AI assistants.

view-source:example.com/product

<body>
  <div id="root"></div>
  <script src="/app.js"></script>
</body>
What an HTML-only agent gets. In a single-page app the server sends an empty shell: no product name, no price, no text to read.

How to check your own site

  1. View the source. In your browser, use View Page Source (not the developer tools inspector, which shows the page after JavaScript). Search for a product name or price you can see on the page. If it isn't in the source, it's added by JavaScript.
  2. Turn JavaScript off. In Chrome's developer tools, open the command menu, type "Disable JavaScript", and reload. What's left is roughly what an HTML-only agent sees.
  3. Fetch it from the command line. curl -s https://yourdomain.com/your-page | grep -i "your price" shows exactly what your server sends.
  4. Run AgentScore. It compares the text in your HTML with the text after JavaScript runs. It passes when at least 70% of the text is already in the HTML, and warns or fails below that.

What usually goes missing

  • Prices and stock loaded from an API after the page appears, so the HTML says "Loading…" or nothing at all.
  • Whole pages in single-page apps, where the HTML is an empty <div id="root"> and everything else is built in the browser.
  • Product lists with infinite scroll that load more items only as a person scrolls, with no paginated links to follow.
  • Content inside tabs and accordions fetched only when clicked, such as specifications, sizing and shipping details.
  • Structured data injected by a tag manager or script, which agents reading the HTML never see. See product data AI shopping agents can read.
  • Reviews from third-party widgets that load in the browser.

How to fix it

The aim is simple: the important content should be in the HTML your server sends. JavaScript can still make the page interactive afterwards.

Ways to get content into the HTML

  • Server-side rendering Full HTML built for each request
  • Static generation HTML built at deploy, for pages that rarely change
  • Essentials first Render what agents ask about, enhance the rest
  • Pre-rendering A stopgap: two versions of each page
Your options. Server rendering is usually the biggest single fix; pre-rendering can tide you over.

Server-side rendering (SSR)

The server builds the full HTML for each request, then JavaScript takes over in the browser. Modern frameworks support this directly: Next.js and Remix for React, Nuxt for Vue, SvelteKit for Svelte, and Angular's server rendering. If your site is a single-page app built with one of these, turning on server rendering for key pages is usually the biggest single fix.

Static generation

Pages are built to HTML ahead of time, at deploy. It's ideal for content that changes rarely, such as marketing pages, docs, policies and blog posts. Astro, Eleventy, Hugo and the frameworks above can all do it, often alongside server rendering for pages that change often.

Render the essentials, enhance the rest

You don't need to render everything on the server. Prioritize what agents ask about: product name, price, availability, key specifications, shipping and returns, and plan pricing. Personalized recommendations, recently viewed items and chat widgets can stay in JavaScript.

In the HTML from the server

  • Product name and price
  • Availability and key specifications
  • Shipping and returns
  • Plan pricing

Fine to leave in JavaScript

  • Personalized recommendations
  • Recently viewed items
  • Chat widgets
Prioritize what agents ask about. The facts a shopper's question depends on go in the HTML; extras can load later.

Pre-rendering as a stopgap

Pre-rendering services run your JavaScript ahead of time and serve the finished HTML to bots. It can work as a short-term fix, but it means maintaining two versions of each page, and it depends on correctly recognizing every agent. Google describes serving bots differently ("dynamic rendering") as a workaround rather than a long-term solution. Server rendering is better.

Smaller fixes that help

  • Replace infinite scroll with, or add, real paginated links (?page=2) agents can follow.
  • Put tab and accordion content in the HTML and hide it with CSS, instead of fetching it on click.
  • Move JSON-LD from your tag manager into the page template.
  • Include a summary of reviews (average rating and count) in the HTML, even if the full reviews load later.
  • Give the page a meaningful <title> and <h1> in the HTML, never only after JavaScript runs.

Once your content is in the HTML, the next question is whether agents can find their way around it and use it. See buttons, links and forms agents can use.

← All articles Test your site with AgentScore →

Can AI agents use your site?

Get your free AgentScore in under a minute. No sign-up needed.