The third version of Google’s much-maligned Penguin algorithm was released back at the end of 2014. Since then, we have enjoyed relatively stable search results, at least where links are concerned. That all changed a few weeks ago, when the fourth version was released, shaking things up once more.
As a quick review, what actually is the Penguin algorithm? Penguin is a set of filters and changes to how Google ranks sites, based entirely on links. While we don’t think of penguins as predators in nature, Google’s version of the animal seeks out cases where links have been abused to rank a site higher than it deserves.
Before the first set of Penguin updates hit, links were either valuable or not. If a link you had pointing at your site was considered spammy by Google, they devalued it. It didn’t hurt your site beyond the loss of the value of that link, and it was easy enough to simply create a few more links to counteract that loss.
Penguin added a penalty on top of that. When a link was discovered and labeled spammy, Google would essentially make that link toxic to your site. Instead of simply being valueless, it would actively hurt your site until it was either removed or disavowed.
There are a couple of problems with Penguin as it was implemented in Penguin 3, though, and the update to Penguin 4 has much improved it.
The first issue is that the changes were only made when Penguin updated, which is pretty rare. After all, the gap between 3 and 4 was nearly two full years. Imagine being penalized in 2014 and having to wait until 2017 to see whether or not the results of your fixes actually fixed anything. People would abandon sites entirely rather than try to fix them, and I’m sure that’s not something Google wants.
The other issue is that of disavowal, and I’m going to actually dedicate a section to discussing it.
There’s some controversy in the SEO community as to whether or not using Google’s disavow tool helps solve Penguin issues. First, though, let’s talk about a few things that need defining before we can really get into this.
First, what is the disavow tool? The disavow tool is a tool provided by Google in their webmaster tools and search console. See, Google uses links to calculate your search ranking. Relevant links are beneficial, and the better the site they come from, the better they are. Conversely, irrelevant and spammy links are harmful, and the worse the site they come from, the more damaging they can be.
The disavow tool is a way to tell Google “hey, look, I don’t like these links either. Can you make them not count against me?” Ideally, Google will agree and make those links no longer harmful to you.
Why would Google put this tool out, though? If they made an algorithm dedicated to penalizing sites that spam links, why would they make a tool that allows those sites to simply reverse that penalty with no change in behavior or repentence? The answer is because of negative SEO.
So what is negative SEO? Well, in a post-Penguin world, bad links can count against a site. Now imagine I don’t like your site. I go to Fiverr and spend $50 buying 100,000 spammy links to your site. Now you have a huge number of links that are, by and large, irrelevant. Some of those links, though, will be bad enough to hurt you. You didn’t buy or build them, they’re just hurting you through no fault of your own. The rationale behind the disavow tool is to give webmasters like you a way out from under undeserved penalties caused by people like this hypothetical me.
The disavow tool is tricky to use. You need to build up a list of bad links you want Google to ignore, which means pulling a backlink profile and examining all of the links according to what Google wants out of a site. This is already beneficial to Google, because it gets you looking and learning about how they want links to work on the web.
Next, you need to submit this list of bad links to the disavow tool. You need to be careful not to disavow good links or you will lose their value as well. Then you wait, while Google decides whether they’re going to honor your disavow file. They don’t have to, and sometimes they don’t; sometimes it just takes a while to work.
Interestingly, Google’s examples indicate that they’re asking for comments on what you have done to remove bad links before resorting to the disavow tool. The idea is to make you try to remove bad links before you try to get Google to ignore them.
The question is, does using the disavow tool work? If it does, webmasters have no reason to go out of their way to ask for links to be removed. If it doesn’t, then the tool seems to serve no purpose. Has anyone tested it?
A Whiteboard Friday by Moz set forth some ideas for a test, and tested a dozen sites. Their conclusion was that merely disavowing links did not seem to totally remove Penguin, if it contributed at all. It was impossible to test; every site that claimed to just use the disavow file also had links disappear.
Of course, that doesn’t mean those sites actually removed links. They could have been removed by external forces, sites going down, scrapers being removed, URLs being changed, and so forth. 12 sites is also not a huge sample size.
When we scroll down to the comments of this Whiteboard Friday, we see many valid arguments and quite a bit of data suggesting that the disavow tool does in fact work. Personally, I choose the hybrid approach.
The ideal recover process for Penguin is to identify all of your bad links first. Then, go to each individual site and contact the webmaster, asking for the links to be removed. Many will not, of course. Some might ask for payment, which you can ignore. Some will already be gone by the time you get there. Some will happily remove the links or nofollow them for you. Then, once you’ve gotten as many removed as possible, compile the rest and submit the list to the disavow tool.
The main flaw with just using the disavow tool to recover from Penguin is that it’s easy to accidentally change your disavow file and “restore” links you had previously disavowed. When you upload a new file to the tool, anything that used to be on it but isn’t now is restored, assuming it still exists. You need to maintain consistency between files, which means record keeping.
Penguin has made some changes in their 4th iteration. Perhaps the number one most important change is that Penguin is now a real time update, as part of the core algorithm.
In the past, you would be forced to wait between Penguin index refreshes to see if your changes had any effect. This meant even if you disavowed your bad links immediately, or even if you got them all removed, you would have to wait the months it took for another refresh before you would see the difference. By making Penguin a real time update, you can see the effects of the changes as soon as Google processes them.
Of note here is that the disavow tool is not a fast, automatic tool. It requires a Google overview, short as it may be, and as such can take several weeks to take effect. Removing links is almost always going to be faster, except in cases where the owner of the site refuses to acknowledge your contact or remove your links. With real time updates, as soon as Google reindexes your page – which can be anywhere from hours to days after your last index point – you’ll see a lifting of the penalty. Assuming, of course, you got all of the links.
The second change is that Penguin is now a lot more granular. Rather than penalizing your site as a whole, they are focusing more on page-level penalties. A ton of spam links pointing at one subpage of your 1,000-post blog will nuke that one post from the face of search, but will leave the rest of your site largely intact. Oh, you’ll still feel the hurt, but it won’t be quite as devastating, and it leaves you less vulnerable to negative SEO.
Oh, and it turns out our predictions from way back in 2015 were pretty accurate.
One thing to note with Penguin recovery is that it comes in stages. Let’s say that you have a site at “100%” ranking, with a bunch of spammy links that are still giving you value because they have been indexed by Google but haven’t been flagged as bad just yet. This is where a site might look after having bought a link package.
Penguin audits the site and finds that a bunch of links are spammy. In penalty, the algorithm turns those links toxic. The site descends to 50% of its original value due to the downward pressure of those links.
The owner of the site immediately notices their penalty and correctly traces it to their link scheme. They remove a bunch of the links and disavow the rest. Google reindexes the page and parses the disavow file, and turns all of those links from toxic to neutral. The site will rise, but you have to remember, they also lost the value those links had before. The site might end up at 75% of its original value.
From the outside, this looks like the penalty is still in place, but it’s really not. What happened is that the artificial boost is gone, leaving the site at its true ranking. It would be more accurate to say that it was originally at 125%, was dropped to 75%, and disavowed links to get back to 100%. Still lower, but much more accurate.
One thing to note moving forward is that, with real time updates, Penguin is no longer a truly detectable algorithm. In the past, we’ve been able to trace when Penguin hits. We can analyze data – Moz is amazing for this – and can say “large numbers of links changed value and sites changed ranking because of it. This was a Penguin update, and it affected X amount of search results.”
These updates happened in discrete chunks of time. Specific days can be indicated as days Penguin updated. With Penguin 4, however, there’s no longer a Penguin Update Day. It’s much like Panda, since its conversion to real time; it’s simply how the search algorithm works now. Bad links will hurt your site much more immediately, though in much lower degrees of damage due to the page-level granularity of the algorithm. Conversely, removing bad links or disavowing them will have a much faster effect. You are penalized, and recover from penalties, much faster than ever before, and those penalties are not as bad as they used to be.
Is Penguin dead? This is an interesting question. Obviously, Google liked how it worked enough to make it part of the core algorithm. However, they also decided on a neutered version of it that doesn’t hurt sites as much for their spammy links. While this is a good bit of protection against negative SEO, it’s also sort of pulling the fangs out of the Penguin’s mouth. It makes it more accurate to the animal, but less influential to search as a whole.
The moral of the story here is that bad links are bad, and should be removed if possible. If they can’t be removed, they should be disavowed. Removing links will take a very short amount of time to recover, as quickly as Google indexes you site again. Disavowing them will take longer, up to two or three weeks, as they process your disavow file and apply the changes. Either way, the whole process works a lot faster than it ever has before.