In case the crawl did not get any data or seems to be crawling for a long time without finding any URLs a change of the useragent or crawler country can sometimes help.

To check whether our crawler is able to crawl your domain and find any URLs in the current crawl have a look at the crawl history: (Project Settings -> Crawl history )


In case there are 0 URLs found after a decent amount of time into the crawl you can cancel the crawl and retry with a different User-agent and/or Crawler country:


The option "Crawler country" will change the server location from where the crawler will analyze.

The option user-agent will determine the name of the crawler, you can enter firefox as user-agent for example. Or if you want to make sure that you whitelist our service only you can give the crawler an individual name (e.g:crawler123abc)

This option is only available with Pro account or higher!

Did this answer your question?