Skip to main content
All CollectionsTroubleshootingSitemaps
Sitemaps are not being downloaded
Sitemaps are not being downloaded

A guide for when our tool can't download your sitemaps

Updated over 2 weeks ago

The downloaded URLs do not match the total URLs in the XML sitemaps.

Why does this happen?

There are two common scenarios:

  1. Partial downloaded URLs

  2. No downloaded URLs

Partial Downloaded URLs

The downloaded URLs do not closely match the total URLs in the XML sitemaps.

The example below shows that the total number of downloaded URLs is 50% less than the total number in the sitemap index.

This impacts the inspection URLs that we can monitor.

It's common to have duplicate URLs if you submit multiple sitemaps from well-known content management systems.

Our tool automatically deduplicates URLs when we download sitemaps.

This means the downloaded URLs will never 100% match the total number of URLs within your sitemaps. But they should closely match.

No downloaded URLs

Our tool has not downloaded any URLs from selected sitemaps.

We can see that we have selected XML sitemaps from the Google Search Console account. But there have been zero downloaded URLs (which means we can’t calculate any other metrics).

How to fix this problem

There are three ways that you can fix this:

  1. Review your XML sitemaps

  2. Review your URLs within the sitemaps

  3. Whitelist Indexing Insights

Review your XML sitemaps

Check to make sure that the sitemap files are live.

The sitemap data in Indexing Insight is pulled from Google Search Console API. And one the big problems with the GSC sitemap report is that it doesn’t reflect real-time changes.

If pages are not being downloaded always recommend checking that:

  • XML sitemaps in Google Search Console are live

  • XML sitemaps in Google Search Console are valid

  • Sitemap index file in Google Search Console include the right XML sitemaps

We’re in the process of making changes to our sitemap report to make sure you have up-to-date information.

Review the URLs within the sitemaps

Check to make sure important URLs are included in the selected sitemaps.

The sitemap data in Indexing Insight is pulled from Google Search Console API. And the number of URLs in the GSC sitemap reports can be incorrect if changes are been recently made.

If pages are not being downloaded always recommend checking that:

  • The XML sitemaps selected includes the exact URLs you want to monitor

  • The XML sitemaps selected includes https:// versions of the URLs

  • The XML sitemaps selected are empty.

We’re in the process of making changes to our sitemap report to make sure you have up-to-date information.

Whitelist Indexing Insights

Check to make sure the XML sitemaps are whitelisted.

One common problem is that our tool can partially download URLs from your sitemaps and then get blocked by your website server (or CDN).

Strict server firewalls can block Indexing Insight.

We do have a solution for firewalls that block our crawlers. But for obvious security reasons, we don’t want to provide the answer on a public web page.

Please contact support@indexinginsight.com so we can provide a solution for your team.

Did this answer your question?