Understanding Mobile-First Indexing (1/3): The Immediate Impact on SEO

January 25, 2017

By: Cindy Krum

Google’s Mobile-First Indexing announcement has generated a lot of discussions, but no one seems too concerned just yet. This might be because Google has been touting various enterprise-wide ‘Mobile-First’ initiatives for over a year, but it could also be because Google appears to have changed their procedure for launching updates, or at least ‘mobile’ updates. If recent history is any indication of the future, Google’s new process looks somewhat like this:

• Publicly announce the changes ahead of time (sometimes even with dates and deadlines)
• Roll rankings adjustments out slowly (rather than all at once)
• Pre-announce adjustments and related updates as needed

This pattern began with the Mobile-Friendly update (Mobilegeddon) and continued with modifications related to app interstitial, HTTPS and deep linking as well as the various AMP announcements.

This new release strategy is likely due to the impact that machine learning has over the algorithmic evaluations, but the key benefit to SEO’s is that algorithmic changes no-longer disrupt the rankings immediately after going live. This slow-roll out has led to many of the recent updates being initially mocked for having minimal impact, but over time all of these changes will prove to be quite significant. The switch to Mobile-First Indexing will be no exception. 

This article is the first in a three-part series to help SEO’s understand the long and the short-term impact that Google’s change to Mobile-First Indexing will have on SEO strategies and Best Practices. It will begin with a brief history of mobile indexing, then it will outline how SEOs can make an existing site compliant with Mobile-First Indexing. In the second article, we will discuss the crucial distance that Mobile-First Indexing appears to allow between URLs and corresponding content, the increasing push to host content in Google’s cloud and how we can expect these concepts to impact the web as a whole in the future. The final article in the series will review the various development options that Google considers natively Mobile-First when they should be used and the larger themes that drawl them all together.

A Brief Look Back on Mobile SEO & Mobile Indexing

Historically, Google has struggled with different mobile indexes and different mobile crawlers. Briefly, during the days of mobile-only WAP sites, Google had a mobile-specific index, but as soon as color and images became common in mobile site designs, mobile search results began to include both mobile and desktop pages. When mobile ‘mDot’ subdomains were popular, mobile pages were indexed based on their relation to a desktop page which was generally determined by a type of server settings called ‘User-Agent Detection & Redirection’. Mobile pages with corresponding desktop page were difficult for Google to discover and index unless a sitemap had been submitted with the mobile URLs or someone had actively linked from a desktop page to the mobile-specific page. Even then, it was difficult for Google to determine when mobile pages should rank, because mobile content sometimes had to out-rank its desktop counterparts that had more history and positive SEO ranking signals, despite algorithmically looking like duplicate content.

Google’s Diagram for Proper User Agent Detection & Redirection: https://developers.google.com/webmasters/mobile-sites/mobile-seo/common-mistakes

To address the problem, Google suggested that web masters indicated when a desktop page had an alternate, mobile version by adding ‘rel=alternate’ and ‘rel=canonical’ links to the HTML of both pages. The ‘rel=alternate’ tag on the desktop page would be indexed, and trigger a mobile crawler to visit the mobile version of the page indicated in the tag. This way it could also be crawled and indexed. The mobile version of the page avoided being counted as duplicate content and passed its SEO value to the desktop version of the page, using the ‘rel=canonical’ tag, which linked back to the desktop version of the page.

That process was error-prone and resource-intensive. So to minimize the problem, Google began actively encouraging web masters to update their websites to Responsive-Design. Since the guideline was to build one Responsive website that would work on both mobile and desktop devices, Google retired the mobile-specific crawler and started crawling everything (desktop and mobile content) as a smartphone. (At this point crawling was mobile, but indexing was still based on old desktop standards.) While Responsive-Design solved some of the issues of complexity with crawling and indexing, it added new problems.

Using CSS media queries and JavaScript to adapt content for one URL to fit many different screen sizes tended to add to the latency of most websites dramatically – especially on mobile phones where the number of round-trip requests to the server for various pieces of content embedded in the HTML slowed the experience down considerably. Since then, Google has focused mobile SEO and development guidelines on speed, attempting to teach and persuade web masters to build clean and efficient sites with strong Critical Path Rendering in mind. (Critical Path Rendering is the practice of strategically ordering internal and external page elements to optimize load time.) The goal was to help make the sites faster and more usable, even on slow mobile networks. But eventually, Google recognized that consistently great Critical-Path Rendering was beyond the reach of most web masters and skilled development teams, especially when Responsive-Design became part of the standard requirements. Pages became larger and slower because of all the code needed to accommodate the extra variables needed to make the page Responsive. 

Thanks to Louis Holzman for the Illustration showing the difference between a web site and a web app: https://www.linkedin.com/pulse/web-application-vs-website-whats-difference-louis-holzman

Over time, Google also discovered that even when strong Critical Path Rendering was in play, mobile and desktop renderings of a Responsive Design page still struggled with intense caching and compression problems, often caused by the CDN (Content Distribution Network). These caching and compression problems were often difficult for SEOs and development teams to address and some were so significant that they made all the Critical-Path Rendering work nearly irrelevant anyway. While some smart web masters began using advanced rendering protocols to help speed things up (pre-cache, pre-fetch, and pre-render), most were just creating slower websites.

That gets us closer to the present time when users are demanding increasingly fast websites that look and behave more like their speedier native app cousins. They want ‘mobile web apps’ instead of ‘mobile web sites’. These demands put a massive strain on developers because all of the JavaScript that is required to make the websites look and feel more like apps. This creates even more delay in the load time of the site. To off-load some of the code, web masters are turning to selective-serving and progressive-rendering which leaves some page content on the server until it is requested by the user. JavaScript and HTML5 are used to request the content from the server but it is only retrieved as needed. This helps improve the load time of the site, but since this framework does not require unique content to have a unique URL, mobile web apps that are built without excess care for SEO can be very difficult for Google to index. It is the interplay of all of these complex elements that have created the need for Mobile-First Indexing. 

Fundamentals of Mobile-First Indexing

Mobile-First Indexing is Google’s next attempt to improve and standardize web development by incentivizing the change with a potential increase in search rankings. It is significant because historically, Google has based all of its indexing evaluations and most of its ranking evaluations on the desktop version of a page. While most algorithm changes relate to positive or negative ranking factors, Mobile-First Indexing is different because it alters the pre-algorithm evaluations of a page. Mobile-First Indexing will not generate a new, separate Mobile-First Index. Instead, it will change how content is added to the existing index. To put it simply: if your new web pages do not pass this initial evaluation, they may not be included in Google’s index, and remember, if a webpage is not included in Google’s index, it has absolutely no chance to rank at all. 

Don’t fret though. Google won’t overwrite well-indexed desktop content with mobile content that has fewer ranking signals and less information. That would be mobile-ONLY indexing, which is not what Google is announcing. If your website is indexed now, it will probably remain in the index, especially if it is are part of the 80% of the web that met the basic requirements to receive the now-hidden Mobile-Friendly designation. As a reminder, to be Mobile-Friendly, your content must have crawl-able JavaScript and CSS, and (ironically emphasized less in Google’s Mobile-Friendly tool evaluation) the content should be fast, properly sized, usable, and engaging when it is presented on a mobile phone. While the label is no longer presented in SERPs, the requirements remain and are likely quite important for Mobile-First Indexing.

Mobile-First Indexing may also be an indication that Google is becoming less dependent on traditional links and HTML URLs for ranking. Remember, mobile SEO strategies have never focused on links or link building to because rankings were driven primarily by the desktop site, where it was easier and more logical for developers to drive links. Mobile-First Indexing may allow the absolute location of content be more open to interpretation and nebulous. This concept will be discussed more in the second and final articles in this series. 

Mobile-First Updates for Existing Websites

For now, SEOs that want to update an existing website to ensure that it will remain indexed and relevant after the launch of Mobile-First Indexing face a simple challenge: With all of the historic mobile development options, the main thing SEOs must do is make sure that mobile devices are getting all the essential ranking and indexing instructions that have historically been included on the desktop pages. This especially means title tags, description tags, href=lang tags, canonical tags and Schema.org markup; It also includes all OG tags & Twitter cards, links to XML sitemaps and media sitemaps, robots instructions (on-page metas and robots.txt) and probably even links to a privacy policy page. In both cases, the task may also involve verifying the websites relationship with app-specific assets like the websites links to app association files so that deep links can be indexed and executed correctly.

The amount of work required will vary based on how the site has historically handled mobile traffic. If the site is currently built in Responsive-Design, minimal, if any, changes may be required. However, if the site is still built with separate mobile URLs on an ‘m.’ subdomain, or if it uses selective, dynamic or adaptive serving to send content to mobile devices, there will be more work to do, especially in terms of the different versions of the content. The best tools for an individual page or template verification are Google’s Mobile-Friendly tool, and Chrome Developer Tools viewing source code in the simulators using ‘inspect element‘ to get a clear picture of what might be captured during a crawl. For the large-scale investigation, SEOs should lean heavily on information that they can get from Google Search Console, and from tools like Screaming Frog, that allow you to crawl a site with Google’s smartphone crawler, extracting the tags that must be verified on each page.

Chrome Dev Tools, Evaluating a Responsive-Design Site: http://www.girliemac.com/blog/2016/08/16/developer-experience-matters/

Conclusion

Google has transitioned from launching updates that create significant jarring changes to now rolling out all changes gradually, over a period of time without major disruption. Their impact might not be as obvious right away, but Mobile-First Indexing is sure to have a profound impact on SEO and the rest of the digital landscape over time. This article is the first in a three-part series focused on Mobile-First Indexing. This article has focused on the immediate updates that SEO’s must consider in their strategy to bolster an existing website. The next article in this series will focus on the larger impact that Mobile-First Indexing will have on SEO strategies and the palpable shift towards the dissolution of ties between URLs, content and the device the content is presented on. It will also discuss the algorithmic shift towards Schema and feeds and cloud-hosting. The final article in the series will focus on the new group of Mobile-First site options that Google is favoring, presumably to help SEOs and web masters achieve strong, long-term Mobile-First Indexing. The World is going mobile and these strategies will ensure that your SEO efforts can keep up!

If you’d like to talk more about Mobile-First indexing, reach out to Cindy or the MobileMoxie team.