Snippets and templates
Copy-ready code from our guides, for you or your developer. Every snippet is an example: swap in your own names, prices and addresses before it goes live, and read the guide it comes from first.
robots.txt and llms.txt
Allow assistants, block AI training robots.txt
Use when you want AI search and assistants to reach your store but not train on it; change the paths and sitemap URL.
# Allow AI search and assistants; opt out of AI training
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: meta-externalagent
Disallow: /
User-agent: *
Disallow: /cart
Disallow: /account
Allow: /
Sitemap: https://www.example.com/sitemap.xml
From robots.txt for AI agents: who to allow and who to block
Simple robots.txt that allows everyone robots.txt
Use as a starting point for a store with nothing to hide from crawlers; swap in your own sitemap URL.
User-agent: *
Allow: /
Sitemap: https://www.example.com/sitemap.xml
From robots.txt for AI agents: who to allow and who to block
llms.txt for an online store Markdown
Use to give AI assistants a short guide to your key pages; replace the summary, links and descriptions with your own.
# Northwind Coffee
> Northwind Coffee roasts and sells specialty coffee beans, ground coffee and
> brewing equipment online, shipping to the US and Canada.
Orders over $40 ship free in the US. We don't sell gift cards or wholesale online;
wholesale enquiries go through the contact page.
## Shop
- [All coffee](https://northwind.example/collections/coffee): Every bean we sell, with roast level, origin and price
- [Subscriptions](https://northwind.example/subscriptions): Recurring deliveries every 2, 4 or 6 weeks; pause or cancel any time
- [Brewing equipment](https://northwind.example/collections/equipment): Grinders, kettles and pour-over kits
## Help and policies
- [Shipping](https://northwind.example/pages/shipping): Rates, delivery times and countries we ship to
- [Returns](https://northwind.example/pages/returns): 30-day returns on unopened items and equipment
- [FAQ](https://northwind.example/pages/faq): Grind sizes, freshness, subscriptions and account questions
- [Contact](https://northwind.example/pages/contact): Email and live chat hours
## Optional
- [Our story](https://northwind.example/pages/about): Who we are and how we source our beans
- [Brewing guides](https://northwind.example/blog/guides): Step-by-step guides for each brewing method
Basic XML sitemap XML
Use if your platform does not generate a sitemap; list your own page URLs and real last changed dates.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://northwind.example/</loc>
<lastmod>2026-10-01</lastmod>
</url>
<url>
<loc>https://northwind.example/products/ethiopia-yirgacheffe</loc>
<lastmod>2026-10-07</lastmod>
</url>
<url>
<loc>https://northwind.example/pages/shipping</loc>
<lastmod>2026-08-14</lastmod>
</url>
</urlset>
Structured data
Product structured data with price and stock HTML
Use on every product page; replace the name, SKU, GTIN, price, stock and return policy with each product's real values.
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Product",
"name": "Ethiopia Yirgacheffe Whole Bean Coffee, 340 g",
"description": "Light roast single-origin coffee with notes of jasmine and lemon.",
"sku": "NW-ETH-340",
"gtin13": "0123456789012",
"brand": { "@type": "Brand", "name": "Northwind Coffee" },
"image": "https://northwind.example/images/ethiopia-340.jpg",
"url": "https://northwind.example/products/ethiopia-yirgacheffe",
"aggregateRating": {
"@type": "AggregateRating",
"ratingValue": "4.7",
"reviewCount": "212"
},
"offers": {
"@type": "Offer",
"price": "18.00",
"priceCurrency": "USD",
"availability": "https://schema.org/InStock",
"itemCondition": "https://schema.org/NewCondition",
"url": "https://northwind.example/products/ethiopia-yirgacheffe",
"hasMerchantReturnPolicy": {
"@type": "MerchantReturnPolicy",
"applicableCountry": "US",
"returnPolicyCategory": "https://schema.org/MerchantReturnFiniteReturnWindow",
"merchantReturnDays": 30
}
}
}
</script>
Organization structured data for your store HTML
Use on your home page; swap in your business name, address, contact details, social profiles and return policy.
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "OnlineStore",
"@id": "https://northwind.example/#organization",
"name": "Northwind Coffee",
"legalName": "Northwind Coffee Ltd",
"url": "https://northwind.example/",
"logo": "https://northwind.example/images/logo.png",
"description": "Specialty coffee roasted to order and shipped across the US.",
"email": "help@northwind.example",
"telephone": "+1-555-010-0199",
"address": {
"@type": "PostalAddress",
"streetAddress": "120 Harbor Street",
"addressLocality": "Portland",
"addressRegion": "OR",
"postalCode": "97201",
"addressCountry": "US"
},
"contactPoint": [{
"@type": "ContactPoint",
"contactType": "customer service",
"email": "help@northwind.example",
"telephone": "+1-555-010-0199",
"availableLanguage": ["en"],
"hoursAvailable": {
"@type": "OpeningHoursSpecification",
"dayOfWeek": ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"],
"opens": "09:00",
"closes": "17:00"
}
}],
"sameAs": [
"https://www.instagram.com/northwindcoffee.example",
"https://www.linkedin.com/company/northwind-coffee-example"
],
"hasMerchantReturnPolicy": {
"@type": "MerchantReturnPolicy",
"applicableCountry": "US",
"returnPolicyCategory": "https://schema.org/MerchantReturnFiniteReturnWindow",
"merchantReturnDays": 30,
"returnMethod": "https://schema.org/ReturnByMail",
"returnFees": "https://schema.org/FreeReturn",
"merchantReturnLink": "https://northwind.example/returns"
}
}
</script>
From Business and store details AI agents can trust: Organization and LocalBusiness data
BreadcrumbList structured data HTML
Use on product and category pages; match the names and URLs to the breadcrumb trail shown on the page.
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "BreadcrumbList",
"itemListElement": [
{ "@type": "ListItem", "position": 1, "name": "Coffee",
"item": "https://northwind.example/coffee/" },
{ "@type": "ListItem", "position": 2, "name": "Espresso",
"item": "https://northwind.example/coffee/espresso/" },
{ "@type": "ListItem", "position": 3, "name": "Dark Roast Beans" }
]
}
</script>
VideoObject with transcript JSON-LD
Use on pages with product or how-to videos; fill in your video's title, dates, length, URLs and transcript.
{
"@context": "https://schema.org",
"@type": "VideoObject",
"name": "Setting the Northwind Aero kettle for green tea",
"description": "How to fill the Aero to 1.7 liters and use the 80°C setting.",
"thumbnailUrl": "https://northwind.example/media/aero-tea.jpg",
"uploadDate": "2026-09-14T09:00:00-05:00",
"duration": "PT2M05S",
"embedUrl": "https://northwind.example/embed/aero-tea",
"transcript": "Hi, I'm Dana from Northwind. Today we're making green tea..."
}
ProductGroup for sizes and colors JSON-LD
Use when one product comes in several variants; list each variant's SKU, options, URL, price and stock.
{
"@context": "https://schema.org",
"@type": "ProductGroup",
"name": "Field jacket",
"productGroupID": "FJ-100",
"variesBy": ["https://schema.org/color", "https://schema.org/size"],
"hasVariant": [{
"@type": "Product",
"sku": "FJ-100-NAV-M",
"color": "Navy",
"size": "M",
"url": "https://northwind.example/products/field-jacket?variant=40318",
"offers": {
"@type": "Offer",
"price": "189.00",
"priceCurrency": "USD",
"availability": "https://schema.org/InStock"
}
}]
}
Product ratings and reviews markup JSON-LD
Use when product pages show reviews; the rating, count and review text must match what shoppers see on the page.
{
"@context": "https://schema.org",
"@type": "Product",
"name": "Ethiopia Yirgacheffe Whole Bean Coffee, 340 g",
"aggregateRating": {
"@type": "AggregateRating",
"ratingValue": "4.6",
"bestRating": "5",
"reviewCount": "318"
},
"review": [{
"@type": "Review",
"author": { "@type": "Person", "name": "Dana R." },
"datePublished": "2026-09-14",
"reviewRating": { "@type": "Rating", "ratingValue": "5", "bestRating": "5" },
"reviewBody": "Bright and floral as a pour-over. Arrived two days after roasting."
}],
"offers": {
"@type": "Offer",
"price": "18.00",
"priceCurrency": "USD",
"availability": "https://schema.org/InStock"
}
}
Buttons, forms and navigation
Give icon buttons a name HTML
Use for any button that shows only an icon, such as cart or close; write a label that says what it does.
<!-- An agent sees: button, no name -->
<button><svg>…</svg></button>
<!-- An agent sees: button "Cart, 2 items" -->
<button aria-label="Cart, 2 items"><svg aria-hidden="true">…</svg></button>
Field error linked to its input HTML
Use for form validation messages; change the field, ID and wording to say exactly how to fix the problem.
<label for="zip">ZIP code</label>
<input id="zip" name="zip" autocomplete="postal-code"
aria-invalid="true" aria-describedby="zip-error">
<p id="zip-error">Enter a 5-digit ZIP code, like 10001</p>
Form error summary with field links HTML
Use on checkout and sign-up forms with several fields; list each real problem and link to its field.
<div role="alert" tabindex="-1" id="error-summary">
<h2>There are 2 problems with your details</h2>
<ul>
<li><a href="#zip">Enter a 5-digit ZIP code, like 10001</a></li>
<li><a href="#phone">Enter a phone number with area code</a></li>
</ul>
</div>
Color picker built from real radio buttons HTML
Use for color or size swatches; keep a visible text name for each option and use your own values.
<fieldset>
<legend>Color</legend>
<label class="swatch">
<input type="radio" name="color" value="navy" checked>
<span class="dot" style="background:#1f2a44" aria-hidden="true"></span>
Navy
</label>
<label class="swatch">
<input type="radio" name="color" value="olive">
<span class="dot" style="background:#5b5e3a" aria-hidden="true"></span>
Olive
</label>
</fieldset>
Breadcrumb trail with real links HTML
Use on product and category pages; replace the links with your own category path to the current page.
<nav aria-label="Breadcrumb">
<ol>
<li><a href="/">Home</a></li>
<li><a href="/coffee/">Coffee</a></li>
<li><a href="/coffee/espresso/">Espresso</a></li>
<li><a href="/coffee/espresso/dark-roast-beans" aria-current="page">Dark Roast Beans</a></li>
</ol>
</nav>
Cookie banner with plain buttons HTML
Use as the pattern for your consent bar; link your own cookie policy and keep reject as easy as accept.
<div class="consent-bar" role="region" aria-label="Cookie choices">
<p>We use cookies to run the store and, with your OK, to measure ads.
<a href="/pages/cookies">Cookie policy</a></p>
<button type="button">Reject all</button>
<button type="button">Accept all</button>
<button type="button">Choose cookies</button>
</div>
Site search form that works without scripts HTML
Use for your store search box; point the action at your search page and keep the query in the URL.
<form role="search" action="/search" method="get">
<label for="q">Search products</label>
<input id="q" name="q" type="search" autocomplete="off">
<button type="submit">Search</button>
</form>
Servers, CDNs and headers
429 response with Retry-After HTTP
Use when your rate limiter turns requests away; set the wait time and message to match your real limit.
HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: text/plain
Too many requests from this client. Please wait 30 seconds and try again.
From Rate limits for AI agents: fair limits that don't turn customers away
503 maintenance page with Retry-After HTTP
Use during planned downtime instead of a 200 or 404; set Retry-After in seconds and your own return time.
HTTP/1.1 503 Service Unavailable
Retry-After: 1800
Content-Type: text/html
<h1>We're updating the store</h1>
<p>Back by 10:30 UTC. Orders placed before 10:00 are safe.</p>
Check a URL's status and redirects Shell
Use to see the final status code and redirect count for any page; replace the URL with one of yours.
curl -s -o /dev/null -L -w "%{http_code} %{num_redirects} %{url_effective}\n" \
https://northwind.example/beans/kenya
nginx log format that records user agents nginx
Use if your logs leave out the user agent or response time; add it to your nginx config and point access_log at it.
log_format agents '$time_iso8601 $remote_addr "$request_method $uri" '
'$status $body_bytes_sent $request_time "$http_user_agent"';
From Finding the errors AI agents hit, in your server and CDN logs
Measurement
Count status codes AI assistants get Shell
Use on a raw access log to spot blocks and errors for assistant visits; point it at your log file.
grep -E 'ChatGPT-User|Claude-User|Perplexity-User' access.log \
| awk '{print $9}' | sort | uniq -c | sort -rn
From Finding the errors AI agents hit, in your server and CDN logs
Weekly error rates by AI agent SQL
Use if your logs are in a SQL database; rename the table and columns to match your own log schema.
SELECT
CASE
WHEN user_agent ILIKE '%ChatGPT-User%' THEN 'ChatGPT-User'
WHEN user_agent ILIKE '%OAI-SearchBot%' THEN 'OAI-SearchBot'
WHEN user_agent ILIKE '%Claude-User%' THEN 'Claude-User'
WHEN user_agent ILIKE '%Perplexity-User%' THEN 'Perplexity-User'
WHEN user_agent ILIKE '%PerplexityBot%' THEN 'PerplexityBot'
WHEN user_agent ILIKE '%Googlebot%' THEN 'Googlebot'
ELSE 'everything else'
END AS agent,
COUNT(*) AS requests,
COUNT(*) FILTER (WHERE status IN (401, 403)) AS blocked,
COUNT(*) FILTER (WHERE status = 429) AS rate_limited,
COUNT(*) FILTER (WHERE status = 404) AS not_found,
COUNT(*) FILTER (WHERE status >= 500) AS server_errors,
ROUND(100.0 * COUNT(*) FILTER (WHERE status >= 400) / COUNT(*), 1) AS error_pct
FROM requests
WHERE ts >= now() - interval '7 days'
GROUP BY 1
ORDER BY requests DESC;
From Finding the errors AI agents hit, in your server and CDN logs
Count visits from each AI crawler Shell
Use on a combined format access log to see which AI agents visit; add any agent names you care about.
awk -F'"' '{print $6}' access.log \
| grep -oiE 'GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-SearchBot|Claude-User|PerplexityBot|Perplexity-User|Applebot|Amazonbot|meta-external[a-z]+|Bytespider|CCBot' \
| sort | uniq -c | sort -rn
From Why your analytics can't see AI agents, and how to measure agent traffic
Regex for AI assistant referral traffic Text
Use in your analytics to group visits referred by AI assistants; add new assistant domains as they appear.
(^|\.)(chatgpt\.com|chat\.openai\.com|perplexity\.ai|gemini\.google\.com|copilot\.microsoft\.com|claude\.ai)$