Rankings Compared: Scraped Rankings vs Average Rankings

Written by and published

There’s been a lot of debate lately about the necessity of ranking data and the validity of where that ranking data comes from.

I wanted to write a balanced and honest look at the usefulness of rankings and compare the data that comes from Google Webmaster Tools (GWT) and scraped data that comes from providers like AuthorityLabs and Moz.

What are rankings good for?

In the world of SEO, if you don’t rank, you don’t have (organic, free) traffic. Rankings have their place, and monitoring rankings can provide several benefits:

  • Knowing the position for campaign-related keywords
  • Correlating campaign efforts with changes in rankings
  • Correlating algorithm changes with changes in rankings
  • Comparing your site rankings with competitors
  • Analyzing rankings with other data points for advanced tactics

It’s important to point out that rank monitoring is not essential for doing effective SEO. It may be essential for some advanced analysis and tactics using formulas on a spreadsheet, but it’s not necessary for most activities.

An SEO needs to optimize a site for search engines, find relevant and authoritative sites and build relationships. Sprinkle in a little content creation and social sharing, and you sum up – at least at a high-level – what modern SEO looks like.

Ranking well for certain terms doesn’t tell SEOs they’re doing a good job. Organic traffic increases and conversions from that traffic tell SEOs they’re doing a good job.

Regardless, many SEOs still want their ranking data, and their clients want it reported to them.

A brief history of Raven and scraped rankings

When Raven first began tracking ranking results, we got the data straight from Google’s own API. Then one day they said, “Nope!”

That’s when we – and every other ranking results provider – had to turn to scraping. We didn’t want to, but we didn’t have any other choice if we were going to continue providing that data.

Google never really like being scraped and has made it increasingly harder to do over time. So much so that it spawned companies like AuthorityLabs, whose main speciality is scraping Google to get ranking results.

It became so time-consuming and difficult for us to get ranking results that we decided to use AuthorityLabs instead, so we could refocus our time on making Raven better.

AuthorityLabs was and is very good at what it does. So much so that it has become what I believe is now the largest and most reliable provider of scraped rankings in the world – providing millions upon millions of daily ranking results to customers and software vendors worldwide.

In late 2012, the AdWords API Compliance team notified us that we had to remove all scraped data derived from Google from our platform. They specifically listed AuthorityLabs and SEMRush. It turned out that for whatever reason, Google was using its AdWords API as leverage to get software companies like Raven to stop using data that was scraped from Google.

Based on the information we had from reliable sources, we determined that this was the beginning of a much bigger (albeit very slow-going) fight against companies using scraped data. We didn’t see it as an AdWords issue; we saw it as a Google issue.

The long-term success of the Raven platform ultimately relies on having a positive and healthy relationship with Google. We use and rely on its APIs to provide value to our customers. We asked ourselves what the product would look like if we defied Google’s request. For example, what if our access to the Google Analytics API were revoked?

It was both a difficult and easy decision to make.

If you doubt our concern, consider this. Google can at any time revoke access to the software provider you authorized with, or revoke your Google account from accessing its APIs. That means if you’re using Moz for scraped ranking results and you’ve authorized Google Analytics, Google can at any time either blacklist Moz from using its API (revoking all access) and/or remove your ability to authorize Google APIs – with the former being the most likely scenario.

GWT ranking results vs. scraped data

Scraped data certainly has a lot going for it. The biggest pros include the ability to get:

  • Specific universal rankings
  • Competitor rankings

However, there are some things it simply can’t get that only Google can provide. For example, scraped rankings cannot provide…

  • Query impressions, clicks and CTR
  • Keywords that Google is testing with your site
  • Average position based on real-life searches

If the campaign for your client includes a push for videos, you’ll want to know if the SERPs are displaying a video result and/or a page result. It’s also nice to know, analyze and keep tabs on where your competitors rank for the same terms you’re competing against.

However, aside from video-centric results, scraped rankings aren’t performance-based, nor are they insightful (at least not on the surface).

Unlike scraped data, GWT lets you know how your search queries actually perform in real-life searches. Real searches are personalized. They include a person’s search history, Google+ connections and other factors that in turn determine what the user will see.

Google makes sense of this diversity by reporting on the average position of a search query. Then it tells you how many times the result has been seen, how many people are clicking on it and its click-through-rate – something that’s close to impossible to perfectly match up with scraped data – or Google Analytics, thanks to (not provided).

Another argument for scraped rankings is that SEOs can see keywords that are barely ranking, especially long-tail keywords. Personally, I don’t find that data to be nearly as useful as GWT‘s.

In fact, GWT does something much more interesting and insightful with its results – something many people claim is an example of how bad the data is when it’s actually awesome!

GWT includes keyword phrases ranging anywhere between position 1 to the upper 500s and includes phrases even if their impressions and clicks are less than 10.

That data should be used as valuable insight from Google. For example, any keyword that falls within that range tells me that Google recognizes those phrases as being related to content on my site. More importantly, it tells me that those phrases are being tested in their SERPs – valuable information that scraped data cannot tell me.

Additionally, if keyword phrases have a high rank but few impressions and clicks, that tells me I may need to focus my campaign on more popular phrases or improve my landing page’s meta data and content.

Testing the data

One of biggest complaints I’ve read about GWT‘s ranking data is that it’s inaccurate. In my experience – aside from occasional anomalies that may occur with clicks and impress that are <10 – the data is quite accurate.

I compared two different sites with 8-9 keyword phrases that would fit a typical targeted campaign. I recorded results from GWT, AuthorityLabs and Moz, and also hand-checked the results in a browser I never use (Opera, which I installed just for this test).

The first site was a mobile computing site with a campaign to rank for phrases related to backing up your Mac. GWT, AuthorityLabs and Moz all had their hits and misses, but most were close enough to the hand-checked results. Three of the ranking results that were the most off from GWT also had impressions and/or clicks that were less than 10, so that was to be expected.

Keyword Phrase GWT Rank AL Rank Moz Rank Manual Check
backup for mac 20*
backup for macbook 9* 15 12 13
best backup for mac 11 19 17 11
best backup for macbook 5* 5 7 5
best cloud backup for mac 5 8 7 4
cloud backup for mac 19 16 19 16
mac book backup 3* 12 6 7
mac cloud backup 11 12 11 11

* Impressions and/or clicks <10

Next, I tested Raven’s Schema Creator site. In this test AuthorityLabs and Moz were spot on with my manual checks. GWT had three ranking results that were slightly different, most likely attributed to the fact that it reports the average ranking for real-life searches.

Keyword Phrase GWT Rank AL Rank Moz Rank Manual Check
schema markup 8 8 8 8
schema generator 1 1 1 1
schema creator 1 1 1 1
microdata generator 4 4 4 4
schema.org generator 2 4 4 4
schema 9 12 12 12
google schema 7 9 9 9
schema tool 1 1 1 1
schema maker 1 1 1 1

Which do you need?

There are some SEOs who will always need scraped data. If your methodology includes a quasi-scientific approach to reverse-engineering Google’s SERPs, then you’ll probably always want and need scraped data. The same is true if you need to know where your competitors rank at all times or need to know the exact type of universal result.

But most clients simply want to know how well their site is performing. And most modern SEOs are looking for the real proof that their efforts are paying off – proof that looks like organic traffic and conversions.

That’s why we added results-oriented data – like traffic and goals from GWT and Google Analytics – to Raven’s new ranking report. Combined with rankings data that comes directly from Google (just like your Google Analytics data) and provides unique insights, we think it’s the perfect ranking performance report for clients.

Update: As Remko van der Zwaag mentioned in the comments, you can filter results in GWT by locale. The article originally listed local/geo data as an advantage of scraped data, which is incorrect.

Discuss post at Raven Community