Do you know what a mobile crawl of your site looks like?

If you follow the world of Google and search at all, you’ve heard about Google’s intent to switch to a mobile-first index. In this post, I’m going to briefly review the main points we know about Google’s plans for this new index, but I’m going to go further and detail what we found in a mobile-specific crawl we did of one website.

Based on this data, I’ll also talk about the implications of the switch and some of the challenges that Google faces with this process.

Key points about a mobile-first index

If you’ve already read about Google’s impending switch to a mobile-first index, you can jump down to the site analysis below. If not, here are some of the key statements from the Google announcement:

Today, most people are searching on Google using a mobile device. However, our ranking systems still typically look at the desktop version of a page’s content to evaluate its relevance to the user. This can cause issues when the mobile page has less content than the desktop page because our algorithms are not evaluating the actual page that is seen by a mobile searcher.

This situation occurs when publishers are trying to make their mobile site more streamlined, so they don’t include all the content from their desktop version. It’s bad for Google, though, because they currently rank sites based on a desktop crawl — and they may be sending someone to a mobile page that doesn’t have the content the user is looking for.

While Google has traditionally told people that content that is not immediately visible to users on page load will not be counted as much as other content, they’ve reversed direction on that position because of the needs of the mobile UX:

Be careful to design this in a way that will still allow Google to see the content. Basically, if it’s not in the rendered Document Object Model (DOM), Google can’t see it.

If you have a responsive site or a dynamic serving site where the primary content and markup is equivalent across mobile and desktop, you shouldn’t have to change anything.

While this is true, don’t overlook this key part of the phrase: “where the primary content and markup is equivalent across mobile and desktop.”

Make sure to serve structured markup for both the desktop and mobile version. … When adding structured data to a mobile site, avoid adding large amounts of markup that isn’t relevant to the specific information content of each document.

This post made it really clear that structured data remains a big part of Google’s plans. Structured data makes Google’s job easier. While they don’t offer ranking benefits for using it, some types of markup do alter the presentation of your content in the SERPs (in a good way), and it increases Google’s chances of correctly interpreting the content on your pages.

Sites do not have to make changes to their canonical links; we’ll continue to use these links as guides to serve the appropriate results to a user searching on desktop or mobile.

Thanks for small favors! Actually, Google rightly recognizes that it would be a nightmare for them to try to get publishers to reverse the direction of all these “switchboard tag” links.

Two other key points:

Google is trying to make the change as minimally disruptive as possible:

The mobile web does a lot less interlinking. What that means is that the mobile version of your site probably has far fewer links than the desktop version. So will Google stop using links? Here is what Gary Illyes had to say about that:

Some great posts with more commentary on this update:

1. Google begins mobile-first indexing, using mobile content for all search rankings
2. FAQ: All about the Google mobile-first index
3. Google’s Mobile First Indexing Change: Everything SEOs Need to Know by Jennifer Slegg
4. Mobile-First Unanswered Questions by Jenny Halasz

A case study of a mobile-first crawl

In my experience, those who work on designing mobile sites take a highly UX-centric approach to their designs. You know what? That’s a great thing! However, sometimes other aspects of site design get left off the table, such as the discoverability of the content (though I’d argue discoverability of content is a big positive for UX, too).

To illustrate the point, we did a detailed analysis of one site using our own in-house crawler, which we used to crawl both the desktop and mobile versions of the site. From a top-level view, the results were as follows:

Right away, we see that the mobile site has 1,021 fewer pages than the desktop version — so, roughly one-third of the desktop pages are “missing” on the mobile website. That’s a pretty big number! When we delve into why, we quickly find that files with “index.php” as part of the file name are where the mobile site is missing the most pages (1,016 fewer pages on the mobile site).

For this particular site, the index.php pages are the ones that define their category hierarchy, as well as their individual post pages. In looking into this further, I found that the desktop site had 705 different category pages, and the mobile site had only 54. Worse still, there is no category nav at all on the mobile site. In fact, the UX for navigating from one post to another looks like this (Note: This is not the real site, I’ve anonymized it):

When you click the button, it will show you the next few posts. Keeping in mind that this site’s blog has been active for a decade, the click depth gets quite deep:

Google is pretty determined when it crawls websites, but it’s unlikely that it’s going to travel more than 30 clicks from the home page to find content. After all, no user is going to go that far, so all that content is pretty much invisible now.

Another interesting observation from this audit is that, on the mobile version of the site, each comment on the blog posts shows on its own URL. This is not true for the desktop site. This is actually a case where the mobile site is creating more pages than the desktop site, and it’s not a good idea, as many of the comment pages are relatively poor in content.

Last but not least, exactly as Google would fear, the desktop version of this site has structured data markup, and the mobile version doesn’t.

Summary

There is much more that I could share, but I think you already get the point. Note that from all the info Google shared, there are other major issues to be concerned about that weren’t exposed in this particular audit. However, other sites we’ve looked at have had these problems.

Google may be determined to make their rollout of the mobile-first index “quality neutral,” but they are going to face some major challenges. In the public tweet discussions, there was some indication that they may use a mixed model (signals from desktop and mobile) for a while.

For one thing, I don’t think there is any danger of links being replaced as a signal. There is no other signal that captures the concept of, “I value this third-party web page so much that I want to show it to my users, and I am willing to have them leave my site as a result.”

Personally, I’d bet we’re months away from the mobile-first index being a reality, but I could be wrong. Nonetheless, it’s incumbent on us to give Google some help. This is a great time to start working on an audit of your mobile site to see how it will stack up once the switch-over occurs.

Get yourself ready in advance, and you might just pick up some rankings benefits if your competition is slower about it than you are.

The post Do you know what a mobile crawl of your site looks like? appeared first on Search Engine Land.