Understanding Correlation and Causation: We Should Never Have A Bad SEO Study

One of the best things about working in SEO is the community, and that it’s so active and diverse. SEO studies and research pieces are also great, and help the industry evolve.

There are however, a lot of SEO studies published that will all best intentions, make the mistake of confusing correlation with causation.

Notably, some recent examples of these studies include:

  • How a single, no-follow backlink increase organic traffic, but disregarded the changes made to the title tag, and changed copy on the page…
  • Various ranking factor studies, such as direct website visits and time on site being prominent ranking factors…
  • Most things written by Mr Patel, that he hasn’t stolen from other SEOS
  • A talk at MeasureCamp Manchester, were a senior SEO claimed bounce rate was a big ranking factor…
  • A talk at BrightonSEO 2017, where it was claimed that a Mommy blogger outreach campaign increased Google My Business activity for a chain of restaurants bearing the name of a high-profile celebrity chef…

That being said, not all studies suffer with not knowing the difference between correlation and causation, and this can lead to some bad SEO advice.

What do they mean?

Firstly, lets look at what the two terms mean:

Correlation is a statistical measure (expressed as a number) that describes the size and direction of a relationship between two or more variables. A correlation between variables, however, does not automatically mean that the change in one variable is the cause of the change in the values of the other variable.

Causation indicates that one event is the result of the occurrence of the other event; i.e. there is a causal relationship between the two events. This is also referred to as cause and effect.

The difference between the two types of relationship is easy to identify and isn’t a mistake that should be made. An action or event can cause another, or it can correlate with another.

This relationship is not only important for studies, but also when understanding why a site loses traffic. For example, I worked on a site migration (domain name change) in late November 2017, and the site was not optimised – it was full of spun doorway pages and not all of them were in Google’s index. The migration went well, but then in December 2017 the Maccabees update hit websites with doorways pages.

So in this instance, the SEO migration did not cause the site performance to drop, but it did correlate with a Google update that appears to have targeted issues the site was rife with.

Why are correlation and causation important?

The objective of most research and SEO studies is to identify to what extent one variable has impacted another, or how one variable relates to another.

By understanding the difference between the two, noticing a correlation can provide just cause for further research into whether or not the relationship between two variables is causational.

How can causation be established?

Causality is an area of statistics, and as a log of SEO studies have shown, is often misunderstood – as some believe that because data may show a correlation, it means there is definitely an underlying causal relationship between the variables.

The most effective way of establishing causality between variables is to perform a controlled study, and split the sample/population (in this instance a website) into two groups that are the same, and have one as the control and subject the other to variable changes. This obviously isn’t possible in the real world.

A change in how we report

What’s important is to understand that everything we do as SEOs is theoretical. We need to ensure we document our research methodologies and present findings in a structured manner, and not claim that anything is causational.

A lot of issues are down to the language we use, we call things “ranking factor studies” and use sensational headlines like “how you can achieve 503,740,973 more visits with these three things.

We should adopt a scientific standard as an industry, on how to write a research paper – and use this in our approach, to prevent click-bait blog posts and bad SEO advice, that will damage people’s businesses.

Published by

Dan Taylor

I'm Dan, and I'm an award winning SEO consultant and technical lead based in the United Kingdom. I work with brands around the world, ranging from SaaS, fintech and retail, to travel brokers, agencies and airlines.

3 thoughts on “Understanding Correlation and Causation: We Should Never Have A Bad SEO Study”

  1. Having an industry format for how we publish studies and research, even in blog posts will improve the industry as a whole and prevent sensationalist, mis-leading advice from those who just market themselves, talk a hundreds of conferences and don’t actually do any work.

  2. I don’t think many people who know what correlation is don’t realize this, but I suppose the example could help someone act on this knowledge.

Leave a Reply

Your email address will not be published. Required fields are marked *