This article explains what happens when AXP is installed, how AI crawlers receive content, and what users and search engines see. It also answers common UX questions teams raise when planning for LLM-optimized content delivery.
AXP is an enterprise infrastructure layer that serves an AI-optimized mirror of your website to AI retrieval agents. It does not change your human experience and does not require re-platforming.
Does AXP serve content from an LLMs.txt file?
No. AXP does not use LLMs.txt as part of the platform. Modern consumer AI systems like ChatGPT, Perplexity, or Gemini do not reliably read or use LLMs.txt files. For this reason, AXP does not depend on or expose this file by default.
If a customer wants Scrunch to generate and serve an LLMs.txt file, we can do so, but it is optional and generally unnecessary.
If created, the file follows the standard convention:
https://example.com/llms.txt
What content does an AI agent receive when AXP is enabled?
When Cloudflare is configured to route AI user agents to AXP:
AI agents retrieve pages using the existing URLs already indexed in search, such as
https://www.example.com/treatments/faqCloudflare routes only the AI user agents (e.g., ChatGPT-User, PerplexityBot, googlebot-extended, Meta external agents) to the AXP origin.
AXP serves an AI-optimized version of the page, including:
Clean, server-rendered HTML with no JS dependency
Structured summaries and context blocks
Clarified entity definitions and claims
Removed or simplified dynamic components
Any approved page-level or template-level enhancements
Human visitors continue to receive your normal site, unchanged.
AXP only serves optimized content for AI retrieval agents that match the Cloudflare rule. It does not serve AI-optimized content to humans.
When an AI platform cites a page, what URL does it cite?
The same URL a human user sees.
AXP does not create new URLs and does not replace your public URLs. When ChatGPT cites your content, it cites the page it retrieved from web search. For example:
“Source: www.example.com/conditions/psoriasis-overview”
This protects the human UX. When a user clicks a citation, they should always land on the real site rather than the AI-optimized mirror.
Can a human load the AXP version of the page?
Not by default.
AXP does not expose new URLs and does not create a separate human-viewable version of your website.
However, the AXP version is not “private.”
A human could access it only by spoofing one of the AI user agents targeted in the Cloudflare rule. This is expected behavior, since the same content is being served to AI platforms.
What is the URL of the AXP origin?
Every AXP deployment receives a Scrunch-hosted origin, for example:
01k962hy8xj5qz29mfczbqsymc.agentxp.net
This origin only activates when Cloudflare routes qualified AI user agents to it. Before Cloudflare is configured, the origin does nothing.
How does Cloudflare decide what goes to AXP?
Customers deploy a Cloudflare Snippet that:
Rewrites requests from AI user agents to the AXP origin
Passes the original hostname through a header
Leaves all human traffic and search-engine indexing bots unchanged
A standard snippet rule includes AI retrieval agents such as:
ChatGPT-User
OAI-Searchbot
PerplexityBot
Meta external agents
Gemini’s googlebot-extended
Applebot
Mistral
Claude
ScrunchAI-Testbot
The rule can be customized to include dev or staging subdomains.
What happens before content is approved?
Until approved, AXP runs in pass-through mode. This means:
All AI agents receive the same page humans see
No enriched or optimized content is served
No changes affect your site until Scrunch receives approved content and configuration
Once content is approved, AXP begins serving optimized content immediately.
Does AXP change search indexing?
No.
Search engines such as Googlebot and Bingbot are not routed to AXP. They continue to crawl and index your existing site.
AXP only affects real-time AI retrieval bots, not search indexing bots.
Key UX takeaways
AI agents find your pages using your existing URLs through web search
AXP serves an optimized mirror only to AI agents, not to humans
AI citations always reference your real URLs
Humans clicking citations get your real site, not the AXP mirror
No new URL structure is created
No additional Cloudflare changes are needed after setup unless you want to expand the scope
AXP content must be approved before going live
Additional UX considerations
The AXP mirror should be treated as discoverable to any actor impersonating an AI bot
The optimized content is not AI-generated in real time. It is selectively curated and deployed in advance
AI platforms may quote or summarize AXP-served content to end users
The AXP layer is invisible to humans and preserves your current CMS, templates, and design
Enterprise teams can apply AXP to dev or staging domains before deploying to production