Video transcription
CUTTS: Lots of questions from the UK. Lee Willis from Cumbria, UK asks, “Why does Google crawl/index blogs, specifically, sites notified by ‘WordPress XMLRPC pings,’ so much faster than a “normal” site submitting a revised Sitemap. What is the impact of that on the overall ‘quality’ of the index?”
Well, we always try to maximize the quality, the relevance, the accuracy of our index. And you’d want to make a distinction between crawling and indexing because Sitemap submission does not guarantee that we will crawl the URLs on that list.
It is very helpful to help us discover new URLs or to make canonicalization decisions, but we don’t guarantee you that if you submit a Sitemap we’ll go and crawl it. There have been some people who did some experiments where they saw that happened, but I’m not going to, you know, confirmed or deny that, and policy can always change on exactly how we do use Sitemaps submissions.
But crawling and indexing is different, so if you do a ping a lot the time Google will come and crawl you but often it’s Google blog search because if you’re doing those WordPress or web logs or feed burner pings, those pings are often, you know, the sort of things that are blogs, and, so, a blog search might come and crawl you five minutes later.
But then, if you show up, you might show up in the blog search core post, not in our main web index core post. So just because you get crawled, it doesn’t mean that you’re getting some sort of index boost or anything like that.
We do sort of try to rationally decide what’s the best quality of data, how do we get that, sometimes it’s crawling stuff immediately. Like, with blog search, you have a very fast, very real time sort of results.
And sometimes it’s, you know, taking Sitemaps and then that might result in crawling at a different pace or you might not give any boost at all. But we do use that information in lots of ways to try to help us improve canonicalization and help us try to improve the quality of our index.
So, you know, I wouldn’t say, “Oh, ping, that’s the way it automatically gets crawled,” or anything like that. If you make great content, you get to be well-known, we will probably crawl you relatively frequently and see updated content anytime you make a good change.
Quick Answer: It’s about great content and authority