Infinite scroll, “load more” and pagination for AI agents

Infinite scroll feels effortless to a shopper with a thumb. To many AI agents, it's a wall. Products that only load as someone scrolls may never be seen by an assistant that fetches the page, and a browser agent can waste its time scrolling and waiting. The fix isn't to drop infinite scroll or "load more". It's to put real, linked pages underneath them.

What agents see on a long list

Different agents meet a category page in different ways:

• Assistants that fetch pages, such as ChatGPT or Perplexity looking something up for a person, usually read the HTML your server sends. They don't scroll, and often don't run JavaScript. They see the first batch of products and nothing more.
• AI search crawlers also read HTML and follow links. If page two has no link, it doesn't exist for them.
• Browser agents can scroll and click, but each scroll is a step that takes time. They can't always tell whether more products are coming or the list has ended.

So a category of 300 products that shows 24 at a time, with more loading on scroll, can look like a category of 24. If the product a shopper asked for is number 61, the assistant may tell them you don't sell it.

Give every page of results its own URL

The foundation is the same for infinite scroll, "load more" and numbered pages: each chunk of results should exist at its own address, and that address should load those results when opened fresh.

https://northwind.example/collections/coffee
https://northwind.example/collections/coffee?page=2
https://northwind.example/collections/coffee?page=3

• Use query parameters or paths, not fragments. ?page=2 and /page/2 are separate URLs. #page=2 is not: servers never see what follows the #, so it can't return different results.
• Render the results in the HTML. ?page=2 should arrive with its products already in it, not an empty shell that fills in later. See why AI agents can't see your JavaScript-only content.
• Keep the order stable. Page two should show the same products whether you arrive from page one or open it directly. Random or personalized ordering makes pages unreliable to follow.
• Carry filters and sort along. ?roast=dark&sort=price-asc&page=2 should keep both. Our guide to site search and filters AI agents can use covers filter URLs in detail.

Google's guidance on pagination and incremental page loading takes the same approach, and it works for AI agents for the same reasons.

Link the pages together with real links

Unique URLs only help if agents can find them. Each page should link to the next one, and ideally to the previous one and a few page numbers, with ordinary <a href> links in the HTML.

<nav aria-label="Pagination">
  <a href="/collections/coffee?page=1">Previous page</a>
  <a href="/collections/coffee?page=1">1</a>
  <a href="/collections/coffee?page=2" aria-current="page">2</a>
  <a href="/collections/coffee?page=3">3</a>
  <a href="/collections/coffee?page=3">Next page</a>
</nav>

• Wrap the links in <nav> with a name like "Pagination", so agents can tell it apart from the main menu.
• Name arrow links in words ("Next page"), not just "›".
• Mark the current page with aria-current="page".
• Say how many products there are in text, such as "Showing 25–48 of 312". It tells an agent there's more to see.

Make "load more" a button, backed by a link

"Load more" is a good middle ground for shoppers, and it can work well for agents if it's built as two things at once: a real button for people and browser agents, and a real link for everyone else.

<a class="load-more" href="/collections/coffee?page=3">
  Load more products (264 remaining)
</a>

Start with a plain link to the next page. Then let JavaScript take over: intercept the click, fetch the next page, append the products, and update the address with history.pushState so the URL matches what's on screen. If the script doesn't run, or the agent doesn't run scripts, the link still goes to page three. If you prefer a <button>, keep the numbered page links alongside it so there's still a route.

• Give the button a specific name: "Load more products", not "More" or an arrow icon.
• Announce new results with a live region ("24 more products loaded") so the change is easier for agents and assistive tech to notice.
• Move keyboard focus to the first new product, or leave it on the button, but don't send it back to the top of the page.

Infinite scroll: keep it, but add pages underneath

If you want true infinite scroll, treat it as an enhancement over paginated pages. As each new batch loads, update the URL to the page number the shopper has reached. Keep the paginated links in the HTML, even if they're visually tucked away at the bottom. And make sure the footer is reachable: if the page grows every time someone nears the bottom, neither people nor agents can reach your contact, shipping and returns links.

Pattern	Fetching assistants and crawlers	Browser agents
Infinite scroll only	See the first batch only	Must scroll and wait, can't tell when it ends
"Load more" button with no URL	See the first batch only	Can click, if the button has a name
"Load more" link to ?page=2	Can follow every page	Can click or follow
Numbered page links	Can follow every page	Can jump straight to a page

What about rel="next" and rel="prev"?

For years, <link rel="next"> and rel="prev" in the page head were the standard way to tell search engines that pages belonged in a series. Google said in 2019 that it no longer uses them for indexing. Other search engines and tools may still read them, and they're part of HTML, so they do no harm. But don't rely on them: an agent following links needs the visible <a href> links in the page.

Two related head tags matter more:

• Canonical. Give each paginated page its own canonical URL. Pointing ?page=2 at page one tells crawlers to ignore the products on page two. See canonical URLs and duplicate pages.
• Robots meta. Don't mark paginated pages noindex by default. Pages that stay out of the index can, over time, get less attention from crawlers, and so can the products linked from them.

Back it up with a sitemap

Even with perfect pagination, some products sit deep in a long list. Your XML sitemap should list every product page directly, so crawlers don't depend on paging through categories to find them. AgentScore's "Valid sitemap" check confirms you have one. See XML sitemaps for AI agents. The sitemap is the safety net. The links are still the main route.

How to check your category pages

1. View the source of a category page and search for a product name from further down the list. Then look for a link to page two.
2. Open ?page=2 directly in a private window. It should show the second set of products, not the first and not an empty page.
3. Turn off JavaScript and try to reach the last page of a category.
4. Check the footer is reachable on a category page with infinite scroll.
5. Run AgentScore. It has no pagination check, but "Content loads without JavaScript" and "Key pages are linked from the home page" show whether HTML-only agents can see your content and reach your products.

For the bigger picture of how agents move around a site, see What is agent readiness?