How Bad Was Google’s Deindexing Bug? – Moz
Skip to content
Moz logo
Menu open
Menu close
Search
Products
Moz Pro
Moz Pro Home
Moz Local
Moz Local Home
STAT
Mozscape API
Free SEO Tools
Competitive Research
Link Explorer
Keyword Explorer
Domain Analysis
MozBar
More Free SEO Tools
Learn SEO
Beginner’s Guide to SEO
SEO Learning Center
Moz Academy
SEO Q&A
Webinars, Whitepapers, & Guides
Blog
Why Moz
Agency Solutions
Enterprise Solutions
Small Business Solutions
Case Studies
The Moz Story
New Releases
Log in
Log out
Products
Moz Pro
Your All-In-One Suite of SEO Tools
The essential SEO toolset: keyword research, link building, site audits, page optimization, rank tracking, reporting, and more.
Learn more
Try Moz Pro free
Moz Local
Complete Local SEO Management
Raise your local SEO visibility with easy directory distribution, review management, listing updates, and more.
Learn more
Check my presence
STAT
Enterprise Rank Tracking
SERP tracking and analytics for SEO experts, STAT helps you stay competitive and agile with fresh insights.
Learn more
Book a demo
Mozscape API
The Power of Moz Data via API
Power your SEO with the proven, most accurate link metrics in the industry, powered by our index of trillions of links.
Learn more
Get connected
Compare SEO Products
Free SEO Tools
Competitive Research
Competitive Intelligence to Fuel Your SEO Strategy
Gain intel on your top SERP competitors, keyword gaps, and content opportunities.
Find competitors
Link Explorer
Powerful Backlink Data for SEO
Explore our index of over 40 trillion links to find backlinks, anchor text, Domain Authority, spam score, and more.
Get link data
Keyword Explorer
The One Keyword Research Tool for SEO Success
Discover the best traffic-driving keywords for your site from our index of over 500 million real keywords.
Search keywords
Domain Analysis
Free Domain SEO Analysis Tool
Get top competitive SEO metrics like Domain Authority, top pages, ranking keywords, and more.
Analyze domain
MozBar
Free, Instant SEO Metrics As You Surf
Using Google Chrome, see top SEO metrics instantly for any website or search result as you browse the web.
Try MozBar
More Free SEO Tools
Learn SEO
Beginner’s Guide to SEO
The #1 most popular introduction to SEO, trusted by millions.
Read the Beginner’s Guide
How-To Guides
Step-by-step guides to search success from the authority on SEO.
See All SEO Guides
SEO Learning Center
Broaden your knowledge with SEO resources for all skill levels.
Visit the Learning Center
Moz Academy
Upskill and get certified with on-demand courses & certifications.
Explore the Catalog
On-Demand Webinars
Learn modern SEO best practices from industry experts.
View All Webinars
SEO Q&A
Insights & discussions from an SEO community of 500,000+.
Find SEO Answers
August 7-9, 2023
Lock in Super Early Bird savings for MozCon
Snag tickets
Blog
Why Moz
Small Business Solutions
Uncover insights to make smarter marketing decisions in less time.
Grow Your Business
The Moz Story
Moz was the first & remains the most trusted SEO company.
Read Our Story
Agency Solutions
Earn & keep valuable clients with unparalleled data & insights.
Drive Client Success
Case Studies
Explore how Moz drives ROI with a proven track record of success.
See What’s Possible
Enterprise Solutions
Gain a competitive edge in the ever-changing world of search.
Scale Your SEO
New Releases
Get the scoop on the latest and greatest from Moz.
See What’s New
New Feature: Moz Pro
Surface actionable competitive intel
Learn More
Log in
Moz Pro
Moz Local
Moz Local Dashboard
Mozscape API
Mozscape API Dashboard
Moz Academy
Avatar
Moz Home
Notifications
Account & Billing
Manage Users
Community Profile
My Q&A
My Videos
Log Out
By: Dr. Peter J. Meyers
April 11, 2019
How Bad Was Google’s Deindexing Bug?
Search Engines
|
Advanced SEO
On Friday, April 5, after many website owners and SEOs reported pages falling out of rankings, Google confirmed a bug that was causing pages to be deindexed:
MozCast showed a multi-day increase in temperatures, including a 105° spike on April 6. While deindexing would naturally cause ranking flux, as pages temporarily fell out of rankings and then reappeared, SERP-monitoring tools aren’t designed to separate the different causes of flux.
Can we isolate deindexing flux?
Google’s own tools can help us check whether a page is indexed, but doing this at scale is difficult, and once an event has passed, we no longer have good access to historical data. What if we could isolate a set of URLs, though, that we could reasonably expect to be stable over time? Could we use that set to detect unusual patterns?Across the month of February, the MozCast 10K daily tracking set had 149,043 unique URLs ranking on page one. I reduced that to a subset of URLs with the following properties:
They appeared on page one every day in February (28 total times)The query did not have sitelinks (i.e. no clear dominant intent)The URL ranked at position #5 or better
Since MozCast only tracks page one, I wanted to reduce noise from a URL “falling off” from, say, position #9 to #11. Using these qualifiers, I was left with a set of 23,237 “stable” URLs. So, how did those URLs perform over time?
Here’s the historical data from February 28, 2019 through April 10. This graph is the percentage of the 23,237 stable URLs that appeared in MozCast SERPs:
Since all of the URLs in the set were stable throughout February, we expect 100% of them to appear on February 28 (which the graph bears out). The change over time isn’t dramatic, but what we see is a steady drop-off of URLs (a natural occurrence of changing SERPs over time), with a distinct drop on Friday, April 5th, a recovery, and then a similar drop on Sunday, April 7th.
Could you zoom in for us old folks?
Having just switched to multifocal contacts, I feel your pain. Let’s zoom that Y-axis a bit (I wanted to show you the unvarnished truth first) and add a trendline. Here’s that zoomed-in graph:
The trend-line is in purple. The departure from trend on April 5th and 7th is pretty easy to see in the zoomed-in version. The day-over-day drop on April 5th was 4.0%, followed by a recovery, and then a second, very similar, 4.4% drop.
Note that this metric moved very little during March’s algorithm flux, including the March “core” update. We can’t prove definitively that the stable URL drop cleanly represents deindexing, but it appears to not be impacted much by typical Google algorithm updates.
What about dominant intent?
I purposely removed queries with expanded sitelinks from the analysis, since those are highly correlated with dominant intent. I hypothesized that dominant intent might mask some of the effects, as Google is highly invested in surfacing specific sites for those queries. Here’s the same analysis just for the queries with expanded sitelinks (this yielded a smaller set of 5,064 stable URLs):
Other than minor variations, the pattern for dominant-intent URLs appears to be very similar to the previous analysis. It appears that the impact of deindexing was widespread.
Was it random or systematic?
It’s difficult to determine whether this bug was random, affecting all sites somewhat equally, or was systematic in some way. It’s possible that restricting our analysis to “stable” URLs is skewing the results. On the other hand, trying to measure the instability of inherently-unstable URLs is a bit nonsensical. I should also note that the MozCast data set is skewed toward so-called “head” terms. It doesn’t contain many queries in the very-long tail, including natural-language questions.
One question we can answer is whether large sites were impacted by the bug. The graph below isolates our “Big 3” in MozCast: Wikipedia, Amazon, and Facebook. This reduced us to 2,454 stable URLs. Unfortunately, the deeper we dive, the smaller the data-set gets:
At the same 90–100% zoomed-in scale, you can see that the impact was smaller than across all stable URLs, but there’s still a clear pair of April 5th and April 7th dips. It doesn’t appear that these mega-sites were immune.
Looking at the day-over-day data from April 4th to 5th, it appears that the losses were widely distributed across many domains. Of domains that had 10-or-more stable URLs on April 4th, roughly half saw some loss of ranking URLs. The only domains that experienced 100% day-over-day loss were those that had 3-or-fewer stable URLs in our data set. It does not appear from our data that deindexing systematically targeted specific sites.
Is this over, and what’s next?
As one of my favorite movie quotes says: “There are no happy endings because nothing ever ends.” For now, indexing rates appear to have returned to normal, and I suspect that the worst is over, but I can’t predict the future. If you suspect your URLs have been deindexed, it’s worth manually reindexing in Google Search Console. Note that this is a fairly tedious process, and there are daily limits in place, so focus on critical pages.
The impact of the deindexing bug does appear to be measurable, although we can argue about how “big” 4% is. For something as consequential as sites falling out of Google rankings, 4% is quite a bit, but the long-term impact for most sites should be minimal. For now, there’s not much we can do to adapt — Google is telling us that this was a true bug and not a deliberate change.
With Moz Pro, you have the tools you need to get SEO right — all in one place.
Start your free trial!
Read Next
How Helpful Was the Helpful Content Update?
Read this post
5 Things I Learned About E-A-T by Analyzing 647 Search Results
Read this post
Is TikTok Going to Replace Google?
Read this post
Comments
Please keep your comments TAGFEE by following the community etiquette
Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.
Moz logo
Contact
Community
Free Trial
Terms & Privacy
Jobs
Help
News & Press
Copyright 2022 © Moz, Inc. All rights reserved.