Overview
Scrunch’s Bot Traffic Analytics shows how LLM bots hit your site (e.g., ChatGPT, Perplexity, Gemini, Grok)—including volume over time, top pages, and recent requests. If your site is proxied by Cloudflare with an Enterprise subscription, you can use Logpush to send request logs straight to Scrunch over HTTPS, and no code is needed.
What You’ll See
Once your Logpush is connected, the Bot Traffic dashboard will show:
Total Bot Traffic in the last period
Bot traffic over time
Traffic distribution between Retrieval, Indexer, and Training LLM Bots
Comparison between the current period and the last period (%)
Top bot agents and when they were last seen
Top content pages accessed by LLM bots
Recent bot requests
A date filter to see data from the last 24 hours, last 7 days, or last 30 days
Scrunch AI's Bot Traffic Analytics feature allows customers to granularly track which AI platforms are consuming their content (and for what purpose) to enable a better understanding of how their content:
will be surfaced in AI platforms like ChatGPT
drives AI responses to relevant questions
and ultimately, how it influences AI to describe and recommend their brand,
products and services, and click through to their site(s).
Adding Your Website
Open the Scrunch app
Go to the Sites menu
You’ll see the list of your websites connected to Bot Traffic Analytics
Click + Connect Site
Select Cloudflare Enterprise as the platform
You’ll see an instructions page with your Webhook URL, Site ID, and API Key
Your page will look like this one:
ℹ️ Each site has its own endpoint and key. Don’t reuse them across different sites or integrations.
Integrating Cloudflare Enterprise Logpush
Step 1: Ensure your website is proxied by CloudFlare
EdgeWorkers will run only if your website is proxied by CloudFlare. You can check if the website is proxied by the DNS menu on the domain level.
Then, check if your website is proxied or not. Your website DNS is normally served by A or CNAME rules
⚠️ If the website is not proxied (orange cloud), then the next steps won't work
Step 2: Create a Logpush Job
Log in to your Cloudflare dashboard
Access a domain
Navigate to Analytics & Logs → Logpush and click on + Create a Logpush job
Choose HTTP Destination
On the next page, paste your Scrunch Webhook URL and append header params:
https://webhooks.scrunchai.com/v1/sites/{SITE ID}/platforms/cloudflare-enterprise/web-traffic?header_X-Api-Key={TOKEN}
Step 3: Selecting fields
On the dataset, pick HTTP Requests
Give a name to your Logpush job (without spaces or special characters)
Select the required fields to proceed with Scrunch Bot Analysis. The required fields are:
EdgeStartTimestampClientRequestHostClientRequestUserAgentClientRequestURIClientRequestPathClientRequestMethodEdgeResponseStatusEdgeEndTimestampClientIP
Click on Submit and wait to receive the next batch from CloudFlare Enterprise Logpush
Step 4: Advanced Settings
Click on Advanced Options and ensure the timestamp format is
rfc3339Sampling should be left at 100% so Scrunch AI can properly process all website activity
Step 5: Verify Traffic
Wait up to 5 minutes for your site to show as “Active” in Scrunch.
If you don’t see traffic, test with:
curl -v -H "User-Agent: ScrunchAI-Testbot" https://yourdomain.com
This will send a sample request to confirm logs are flowing.
👉 Once configured, your Cloudflare Logpush will continuously forward logs to Scrunch, giving you real-time visibility into how LLM bots access your website.
Troubleshooting and Tips
Don’t see any traffic?
Ensure the Logpush job is enabled
Revisit the required fields on the Logpush job configuration and confirm if all required fields are checked
Wait up to 5 minutes—Scrunch may take a short time to detect your configuration.
Tips for better results:
Select all fields from CloudFlare Enterprise Logpush
Repeat the process if you manage multiple sites