How to Determine SEO Keyword Difficulty and If You Should Optimize

Evaluating keyword difficulty is one of the most-challenging stages of the keyword research process. Many available software solutions on the market (such as Ahrefs, Mangool Keyword Tool, SEMrush) offer the evaluation of keyword difficulty through their internal software metrics. Yet, none of them possesses the exact formula indicating when and when not to optimize your content for a specific keyword. To judge keyword difficulty correctly, you need to mix up the semantic, marketing and software information and be ready to use your guts during the evaluation process. But let’s have a closer look at the entire process.

What is Keyword Difficulty?

First of all, let’s try to briefly define keyword difficulty. Often referred to as keyword competition, keyword difficulty in search engine optimization (SEO) is an evaluation of how difficult it will be to rank for a specific keyword.  Determining keyword difficulty helps you decide whether or not to produce specific content for a specific keyword. At the same time, it should be mentioned that keyword difficulty is not necessarily a numeric metric. It is rather a mix of mathematical and humanistic evaluation of which keywords are in our competitive range.

Keyword Difficulty Software Approach

There are several software tools on the market that allow you to measure keyword difficulty. While they may be useful during the evaluation process, it is important to underline that they are only a part of the mix needed to determine how competitive a specific phrase is.

Here are the most useful keyword difficulty software tools:

1. Google Keyword Planner provides you with the search volume of the keyword, together with keyword competition. In this case, the competition is based on the quantity and quality of pay-per-click (PPC) advertising bids for the specific keywords. As keyword planner is based only on PPC metrics, it is not as relevant for SEO purposes. At the same time, evaluating the search volume and competition may come in handy while determining the competitiveness of a keyword. 

1. Ahrefs calculates the keyword difficulty based on a number of referring domains pointing to each of the pages present in the top 10 Google results. Thus it permits you to calculate an approximate number of referring domains needed to rank for a specific keyword. At the same time, it doesn’t take into account the strength of your website, or the relevance of your page for the specific keyword. Nor does it consider the relevance of linking pages either. That’s why Ahrefs is mainly valuable as a complementary metric.  You can read more about it here. 

• Ahrefs SEO Toolbar

For analysis purposes, we will use Ahrefs SEO Toolbar, which you can install both for FireFox and Google Chrome. You’ll need to create a free Ahrefs account to use this tool. The toolbar permits you to analyze specific search engine result pages, with the additional link data pertinent for each page and domain:

• Domain Rating (DR)
• URL Rating (UR)
• Estimated Organic Search Traffic (ST)
• Number of ranking keywords (KW)
• Number of referring domains (RD)
• Number of backlinks (BL)

1. SEMrush is another paid software that uses backlinks data to determine the keyword difficulty. It is useful, yet has similar limitations to Ahrefs. More info here. 
2. Mangools Keyword Tool is estimating the keyword difficulty based on the metrics provided by link-building tools like Moz and Majestic. Again, the calculation is mainly based on link-building metrics, rather than the relevance of the specific page. For more detailed information on how Mangools calculates keyword difficulty click here.

Key Takeaways: While there is software that estimates keyword difficulty, it only takes into account the link-building data, ignoring the relevance of the specific page. It can’t be used as definitive keyword-difficulty metrics. In order to properly evaluate keyword difficulty, you need to focus on something else.

3 Factors that Determine the Keyword Difficulty 

In order to evaluate the true difficulty of a keyword, you need to combine several factors that affect the positioning of the determined pages inside the search engine results pages (SERPs). These factors are:

1. The content of pages ranking in Google’s top 20 results (Yes, Google’s second page may be just as useful for finding valuable keywords, we will get back it a little later)
2. Search engine user’s intent (We need to know what type of content does Search Engine User expect after choosing specific keyword)
3. Page and domain authority and link popularity (Consider how many links does a specific page receive and how high is their quality)

Factor 1: The content of the Pages in Google’s top 20

Analyzing the content from the first two pages of Google allows you to evaluate your chances for ranking inside the top 10 quite precisely. The results from the first 10 positions give you an idea of what kind of content Google needs in order to generate traffic through its search engine. The pages from 11-20th positions are a good source of emerging content, which may help you discover new trends. Sometimes an emerging page from the second search page may give us more material for analysis than pages in top 10.

“Traditional” and “old-school” SEO focused on analyzing the typical SEO factors, like keywords in title tags, H1, H2, URL and keywords inside the content (especially in the introduction). At the same time, with the evolution of Google’s algorithm (especially after Panda and Hummingbird updates), this kind of analysis proves to be obsolete. 

Instead of analyzing keyword positioning you should look through the content of the page and judge what kind of content is published and what is the true quality of the copywriting  on the specific page. Is the content published by a trustworthy author? Is it sufficiently in-depth?  Does it focus specifically on your topic, or is it rather a generic article? These are all the questions one should ask yourself during the Google results analysis.

Example:

Let’s use the keyword  “office desks” as an example to explain the concept of content analysis. 

“Office Desk” SERP (USA, NYC) – First Three Results

• The examples of the analysis were created in May 2020, and they may vary from the current results of search engine result pages. They can also contain local NYC pages, as the research was done through an NYC IP address. Nevertheless, they are more than sufficient for explanatory purposes.

The No. 1 result (https://www.officefurniture.com/Office-Desks) managed to get the first position, in spite of having rather low domain rating (DR 42). The page contains a textual description of the “Office Desks” category, with a  variety of links to related subcategories. 

What is important here, is the presence of a long description of the category, which definitely is appreciated by Google:

“Office desks” SERP, second Page (USA, NYC)

It is also interesting to look at page two of Google results, where we can find several local websites offering office furniture in NYC. The domain rating of the websites is really low, though they manage to appear on Google’s second page. Chances are that with better link-building campaigns they may even appear in Google’s top 10 results.

Let’s look on the exemplary structure of https://d2officefurniturenyc.com/ . It is a classic presentation page, with textual content mixed up with images. 

This search result informs us that if we are promoting a local website specialized in office desks, we should not fear competition from the big online stores. Google provides a space for local office desk shops too. Of course, it should be noted that Google will show specific city results only for specific city-related IPs. In our case the IP comes from New York City, so Google presents NYC office desks stores.

Key Takeaways:

• It is important to look for the quality and the typology of the content shown in Google First 20 results. 
• Analysing the content of the pages from Top 10 10 may give you a hint on which type of content works best. 
• Keeping track of the results 11-20 gives you an idea of what kind of content may be promoted even with a very limited link-building budget.

Factor 2: Search Engine User’s Intent

While preparing the content to be ranked in Google’s first page, it is important to have an idea of what kind of content the user is expecting to find for a specific keyword. There are more intents out there, but for now we can now focus on the following two:

1. Commercial intent – a user wants to buy or is about to buy something. For example, “office desks” is a keyword closely related to commercial intent. The best way to satisfy commercial intent is to present an eCommerce website, or a company presentation website (just like the two “Office Desks” websites we analyzed in the first point: https://www.officefurniture.com/Office-Desks and https://d2officefurniturenyc.com/ )
2. Informational intent – a user is looking for information related to a specific product, for example “best office desks” keyword is a classic informational keyword. In this case, publishing a blog article, or a “ranking page” content would be the best way to proceed.

Example:

Let’s try to spot the difference between user intent for our two keywords: “office desks” and “best office desks”.

Keyword “office desks”

Comment: For the keyword “office desks” only e-commerce content is being presented, both on the first and the second page of Google results. The way for your website to appear among these results is to provide an in-depth eCommerce experience or a detailed presentation of your company, with the high-quality copywriting.

Keyword “best office desks”

Comment: In case of “best office desks” keyword the intent is to look for information and navigate through contextual content as well as through the images. As a result, blogs and review websites are the main types of content shown on Google’s SERPs.

Key takeaway: Check whether the type of content you are going to produce pertains to what users and search engines expect. Avoid choosing commercial intent keywords for informational content and vice-versa. Sometimes it may prove impossible to rank for a commercial keyword if all the content on the SERPs is informational.

Important: Don’t be an analysis slave. If you think you can propose some kind of innovative type of content which is not yet on Google Results, go for it!

Factor 3: Domain and Page Popularity and Authority

The third factor of the keyword difficulty is page and domain popularity and authority. In other words, you should be able to answer the following questions:

1. How many backlinks and referring domains does each SERP page receive?
2. How many backlinks and referring domains does each SERP domain receive?
3. What is the quality of these backlinks and referring domains for each page?
4. How topically related are the backlinks for each page? (If we are analyzing the keyword “office desks” then how many links come from websites that talk about “furniture?)

The good news is that the first three questions can be answered by Ahrefs software, a toolbar I strongly recommend. It can provide you with the number of backlinks (BL), referring domains (RD) and the general URL (UR) and domain rating (DR) that indicate the quality of the links coming to a specific page and domain.

The bad news is that there is no software that analyzes the content of the pages linking to every website in the world. Then, based on the toolbar data you can only analyze overall popularity and authority, but you aren’t taking into the consideration “topical” popularity and authority. That said, uniting the page and domain link and authority data together with the content analysis and search intent analysis is still more than enough to evaluate the difficulty of the chosen keyword.

Final Keyword Difficulty Analysis

Your main goal during the final analysis is to see how many low-authority (domain rating Ahrefs) websites are present in the first 20 pages of the SERPs. This should help you define the type and quality of content needed to rank in top 10 and discover how many quality links you’ll need to receive to generate organic traffic.

Example: ”Office Desk” Keyword, top four results

Comment: In case of “Office Desk” keyword, we see that even the lower domain-authority website like https://www.officefurniture.com/Office-Desks (domain rating 42) can beat highest domain-authority website (Amazon) https://www.amazon.com/Home-Office-Desks/b?ie=UTF8&node=3733671 (domain rating 96) . You should notice that the URL Rating of the first result is 35, while the second result has 40 URL Rating. 

Conclusion

That said, in case of the content focused specifically on office furniture and link-building campaigns from this particular niche, it is possible to outrank big players such as Amazon or Wayfair. In order to rank in the top 10 of the “office desks” keyword, you need to have a website that is extremely relevant for your keyword (like https://www.officefurniture.com), with a proper copywriting on the category page (like https://www.officefurniture.com/Office-Desks). You might also need approximately 100 referring domains pointing to your page and 1000 referring domains pointing to your domain, with the overall domain rating of at least 42. 

It is always advised to match the content analysis with the page and domain popularity and authority analysis. Which websites with an authority lower or equal to yours are ranking in top 20? What kind of content structure do they have? Can we beat the websites with even higher domain authority by producing more relevant content and linking it from more relevant pages?

These are questions to be answered during keyword difficulty analysis.