Google confirms that unindexed pages can still be used to evaluate a website's core web vitals, which will soon become search ranking drivers.
In other words, pages that are intentionally kept out of the Google search index can soon be used to evaluate a range of metrics that affect search rankings.
This is a reasonable cause for concern for SEOs and website owners. The topic will be discussed extensively in the live stream of Google Search Central with John Mueller on December 4th.
In the context of Google, a question is asked that uses aggregated data to measure core web vitals. Google takes a group page, analyzes it and uses the data to calculate the core web vitals that are representative for the entire website.
This is how Google gets to the data found in the search console. The same rating method will be used when the Web Vitals Ranking Factor is introduced in the next year.
The question addressed to Müller asks how Google selects the page groups that are used to calculate the Core Web Vitals:
Read on below
“Core Web Vitals count for unindexed pages and things that are blocked in robots.txt. It's pretty interesting since the search console is obviously aggregated and you get a group of pages. How do you understand that this is a group if these are not indexed and you don't have the context? Or is it just based on the url path
The idea that Google could pull up URLs to evaluate a website's web vitals is worrying for websites with slow pages that are intentionally not indexed. Even if excluded from search results, the slow pages can hurt the site in Google's Core Web Vitals rating.
Mueller on the question of why unindexed pages are among the most important web vitalities
Mueller justifies this with the fact that Core Web Vitals were developed to measure the page experience and unindexed pages are still accessible to users on the website. Therefore, they should be included in the overall assessment.
However, he's not sure how the search console selects which pages to rate.
Read on below
"My general feeling is (unindexed pages are) something that is also part of your website. So if you are using extra features and they are not indexed, people will still see them as part of your website and say this website is slow or this site is fast. I don't know what Search Console is reporting on this in particular. I know that the Chrome User Experience reporting data on the Chrome developer page has some information about creating this grouping, but I don't know how detailed that information is are.
Also, one of the tricky parts is that when a page is something that shouldn't be indexed it's very hard for us to understand because of all the canonical decisions and all that. If you're only looking at a single page, it is sometimes it is not absolutely clear whether this page can be accessed directly, or the cookie set at the beginning has to be set in order to access the page … like everything that is involved there. So I imagine that it's always difficult to make up for that. "
One of the live participants then asks Mueller a follow-up question as to whether there is a way to exclude pages from the Google Core Web Vitals rating.
Can pages be excluded from the evaluation of the most important web vitals?
A live participant quickly jumps in with a corresponding follow-up question:
"Is there some way to make sure certain pages are not being used to gauge how well a website is doing Core Web Vitals?"
In response, Müller says clearly: "I don't have any good answers for you at the moment."
Not entirely satisfied with this answer, the participant would like to know whether there will be answers in the future.
Mueller says he isn't up to date with all of the information about the grouping, so he's hesitant to come up with suggestions. In general, however, Google tends to sort pages into groups based on URL and page content.
So it can be helpful to use folders to signal to Google which pages to recognize as a group.
Read on below
Here is the full quote from Müller:
"Maybe. I don't know at the moment what information is available about the grouping. So it is very difficult for me to say exactly what to look out for or what to ignore.
In general, when it comes to grouping between websites, when we try to group we try to do it based on the URL pattern on the one hand and based on the website's content on the other. I think that's how we group in the search console, for example. So if you have parts of your website that should be viewed as belonging together, I would definitely make sure that it is clear from the URL pattern perspective that they belong together.
For example, if you are specifically concerned about search pages, putting those pages in a folder with / search / can make it a little easier to understand that all of these search pages belong together. All of these product pages belong together. and all of these blog posts belong together. We may be able to treat them individually when it comes to Core Web Vitals. "
Read on below
Google uses limited data to assess key web vitalities
The last thing Mueller adds on this topic is that Google doesn't have a lot of aggregated data for every website. For some websites, Google may only have a few data points available.
In cases where Google has used a limited amount of site data, grouping pages by URL structure may not affect which ones are selected as a group.
"The other thing to keep in mind is that we don't have a lot of aggregated data for every website. We may have only a few data points for some websites. So we only have (information about the Core Web Vitals), and in that case we may not be able to use that grouping even if you use url structures to break things up neatly because we just don't have enough data for those individual groups. "
Mueller says more information about page grouping may be available at a later date. In order to manage expectations, there will be 100% no way to choose which individual pages to use to calculate Core Web Vitals.
Read on below
“This is something I suspect we'll probably have a bit more information over time, but it will still be something that isn't 100% that you can tell ignore this page and get on this page to include a website. "
Listen to the following page grouping questions and answers in the video below: