There is a new Google Crawl Stats report that is replacing the old one, and providing a lot more actionable data.
The link to the new report is here: https://search.google.com/search-console/settings/crawl-stats
Alternatively you can go to Google Search Console -> Settings -> Crawl stats (open report)
After having an initial look around, it feels much more like a log file analysis.
It has urls broken down by response codes, which also allow you to dig in granularly and see which urls are being crawled.
One of the interesting things I noticed, was it flagged multiple 404 urls that Google was crawling, that WERE NOT included in the usual Coverage report – this suggests it will be very useful to granular technical audits, without requiring log file access.
You can also see how much of your crawl budget is used on Discovering new content vs Refreshing existing content.
You can see crawl by Googlebot type, including Smartphone, Image, Page Resource Load, and Desktop crawlers (from my initial checks).
It also gives you notifications if your host has problems, checking robots.txt fetch, DNS resolution, and Server Connectivity.
Do you need help with a technical audit?
We are sorry that this post was not useful for you!
Let us improve this post!
Tell us how we can improve this post?