Google's John Mueller said on Twitter that if he had to simplify crawling priority, the way Google prioritizes crawling the web, it wouldn't be based on the size of a website. Instead, it would prioritize how the web is crawled based on "the additional unique and important value that the pages offer".
He said this on Twitter after a few back and forth questions from some SEOs about why larger sites seem to be crawling faster.
If I had to simplify crawling priority, I wouldn't do it by size, but by the additional unique and important value that the pages offer.
– 🍌 John 🍌 (@JohnMu) December 7, 2020
This isn't really new, but it's good to post here.
Forum discussion on Twitter.
