This post started out as an experiment to test if you could rank a new page using suggestions from Google’s new QuestionHub tool which rolled out to the US market in January 2021.
It was whilst researching QuestionHub to determine if it was potentially useful for SEOs and editorial teams to capture additional relevant users via organic search, that I noticed that there was a huge number of unanswered questions around bias in Google’s SERP results.
There was a bunch of questions around if Google was biasing its search results mainly in favour of politically left-leaning websites, with a few questioning whether on the contrary it biased right-leaning websites. There was also a bunch of consumers asking why certain left-wing or right-wing websites were effectively being censored from their search results. The last set of questions, were less about bias and more about why Google search wasn’t as good as it was in recent years.
In this post, I’ve looked for evidence of bias for and against both sides and have published what I have found.
At this stage, I want to declare I have no political affiliations or left or right wing leaning at all.
I have presented the data I have found after a few hours of research. I am sure there are bigger organisations, with deeper pockets, out there who have conducted even more thorough research into concerns of bias and censorship by search engines (not just Google).
I have just set out to explore the Authoritas international database of historical Google search results to see whether it helps me answer any of the multitude of unanswered questions that consumers have on this topic.
I am sure there’s even greater depth of data that I could look into – but unless the EU, FCC or Competition Commission cares to hire me to do that then this will have to suffice for now. Of course, if Google wanted to hire us and give us access to a firehose of its SERP results then we’d be delighted to look into this independently in extreme detail at scale!
If you feel strongly about this issue, then by all means have your say in the comments and feel free to share hard facts and data demonstrating bias from Google (or other search engines), but please keep your strongest opinions for Twitter – after all this is a blog predominantly aimed at search marketing professionals.
What is QuestionHub?
It’s a fairly new tool from Google which shows you questions from users of Google search that Google has determined as ‘unanswered’. If you would like to know more then read my full review of Google’s QuestionHub tool covering how to access it, how to use it, why you might use it and exploring Google’s motivation in making this tool accessible to us all.
If you are only interested in whether there’s any evidence to support bias in Google’s search results then read on.
What questions are Google’s users asking about bias in its search results?
After querying for ‘Google Search’ multiple times on QuestionHub, I retrieved a list of 351 questions that Google determined are ‘unanswered’ on Google.
I went through all of these questions and removed the downright strange, bizarre and funny phrases. Here are a few of my favourites;
“Google’s search results are restricted to earth service that being said cheaper to establish our own search engine that operates with in Starfleet?”
“Why does the phrase contentious marshmallow not appear in a Google search?”
“Is it a good idea to Google search terms you’ve come up with while dreaming?”
“How can I permanently turn off this parental control on my Google search I’m 43 years old?”
“Why do Google staff let their wives sleep with other men?”
I also removed innumerable phrases about what I’m calling ‘Visibility Bias’.
Obviously, there are many questions about why a site is not visible in Google search, why spammy sites rank and how to get a site to rank on page 1 (many of these are of course not unanswered given that SEOs want to rank for them!).
These were removed from the analysis, as there’s no doubt in my mind, having spent over 20 years in the SEO field, that Google has demonstrated on many occasions that all sites play by the same rules when it comes to ranking organically on Google (except for Google’s own sites of course).
Famous brands getting penalised (BMW and J C Penney being two of the most infamous) is one such example of the level-playing field Google says it maintains, with big advertisers being hit-hard with Google ranking penalties for allegedly participating in link buying schemes.
You categorically cannot buy your way to the top of organic search results by paying Google to favour your website!
Where Google has been proven to show bias or favouritism, it has been to its own web properties for its own commercial advantage. I go on to explore this in more detail shortly.
So, having removed these phrases, what remained was a list of 103 questions that related to some kind of bias or lack of quality in the search results. I classified each question as follows:
General Bias – a phrase indicating that the results are biased but not indicating biased towards any particular entity.
Left-Leaning – a phrase indicating that the user believes that Google is biased in favour of right-wing websites to the detriment of left-wing websites.
Right-Leaning – a phrase indicating that the user believes that Google is biased in favour of left-wing websites to the detriment of right-wing websites.
Financial Bias – a phrase indicating that the user believes that Google shows bias to companies who advertise with it or otherwise pay Google for favouritism.
Quality – a phrase indicating that Google’s search results are of poor quality or have deteriorated.
Paranoia – a phrase indicating that the user believes there are greater powers controlling Google search. (I have tried my best not to be judgmental here).
48% of phrases were users questioning the poor quality of Google search. The irony, of course, being that they are on Google asking the question! Let’s give them the benefit of the doubt and let’s assume they also ask these questions on other search engines like Bing and Duck Duck Go.
Interestingly, only 1 query was left-wing in nature, with 27% of all queries coming from a right-wing perspective accusing Google of bias against right-wing websites and agendas. I appreciate this is a tiny sample, but it is indicative that 9 or so in every 10 unanswered questions of a political nature about Google Search are framed in a way that suggest they are from right-wing leaning users.
Financial Bias
Does Google intentionally keep commercial sites out of search results to force them to spend money on Google ads its biggest revenue source?
Does ad size help ranking on Google?
General Bias
Why is Google search biased or omitting results like in China?
Why is Google search biased or omitting results like in China
Why does Google search obfuscate?
All of the news organizations I wish to block are the only results returned via Google search is Google going to force this type of information on everyone?
Why is it so hard to get unbiased news from a Google search?
What do search engines like Google Yahoo have to gain by allowing bias on their sites what is the end game?
Why is Google filtering my searches they do not include unbiased uncensored websites?
Why does certain search engines block websites and keep their open platform status?
Why does Google search obfuscate?
Why are Google search results so politically biased?
Why is the Google search so misleading and biased when it comes to politics in America?
How to unlock Google search to allow to get certain websites?
Why do my Google search results bring up hostile and transphobic images?
Why are all Google searches paid-for sponsors?
Why is Google search very biased politically?
Why does Google intentionally manipulate public opinion with weighted search results?
Why is Google actively trying to destroy the greatest country in the world?
If lying is illegal on the Internet Why does Google allow it?
Is Google the most biased corrupt organization in the world?
When it illegal to deny a person the same right to so research on Google as other human beings?
Why has the algorithm changed for Google search and Bing search during the Trump administration?
Paranoia
How does the FBI let people control your Google searches?
Why are Google search results so manipulative?
Why do the powerful control Google searches?
How far back can the government track my Google search?
Why are there Chinese hacking my Google searches?
Quality
Can Google search engine operators get any dumber?
Why is Google search so terrible what happened to the once great Google?
Why has the Google search team become incompetent?
Why has Google search gotten lazy over the rank past decade?
Why does Google search suck so much that I have to ask a question instead of just putting in search terms and finding the pages that include those words?
Why does Google search results act like it has autism 24/7?
Why is Google search always giving false info?
How did Google search go from being the best on the Internet to annually getting worse and more bloated with malware and bad adverts?
Why are thousands of fraudulent websites appearing in Google search results?
Why are Google searches so outdated and useless?
Why are Google search results so wrong in 2020?
Why does a Google search for a person’s picture in a news story bring up so many wrong results?
Why doesn’t Google search show useful results anymore like the way it used to a long time ago?
Why are Google search results less accurate than they used to be?
Why do Google search results return opinion articles from the Atlantic instead of actual usable articles?
Why do Google search results suck more every day?
How to get more accurate search results or a better free search site?
Google search results seem to have changed the quality of the results is generally poor now is it possible to access the old Google search results engine
Why are Google search results suddenly so poor was there an algorithm update recently?
Why is the Google search engine so poor at returning meaningful results?
Why is Google search engine rapidly decreasing in quality?
Why is Google search more useless every day?
Why are my search results so much better when using any search engine other than Google?
Why are my Google search results nothing like what I type into the search bar?
Why does Google search suck so badly and only produce crappy pop culture results?
Why is Google search so worthless I will use duckduckgo?
Why does Google permit thousands of low quality spam sites to be indexed in Google search results?
Why did Google search start returning garbage results around 2006?
Why doesn’t Google search work well anymore the way it did twenty years ago?
Why do Google search results seem to get worse and worse less and less relevant every year?
Why does Google search results suck now as compared to years ago when they actually posted the results you asked on?
Google search results are always useless
Why does Google search keep getting worse while other search engines keep getting better?
Why does Google Search give so many results for everything that I am not searching for?
How come Google search results don’t relate to what I am looking for Guess I will use duckduckgo?
Why do I get so many weird results when doing a Google search?
Why does Google search keep returning garbage results?
Why is it every time I search for an article on a specific website with this search engine that I get a full page of nonsense?
The Google search algorithm has gone down in quality so much the last few years
Why does Google search suck so badly when it comes to Barbie?
When did Google search start to suck?
Google search function sucks time to switch to someone else
Why has Google search become the most useless search engine on Earth?
Why is it so hard to find what I’m looking for on Google search?
Why does Google search suck so bad anymore?
Why did Google searches get so bad and completely irrelevant?
How stupid will the Google search engine became before they stop defending the new algorithm?
When did Google search become useless?
Why does Google search increasingly return irrelevant results Trump?
Right-leaning Questions
Why don’t Google search results ever show negative results for Democrats or liberals?
Why does Google only show liberal websites and not conservative websites when doing political search?
How can I remove the liberal lies from my Google search results?
Why do Google searches for info overwhelmingly provide information from left wing sources?
Why does Google search produce results that are all left leaning?
Why does the Conservative Treehouse not show up in a Google search?
Why is Google search so biased and left?
Why do I only get left wing media when I search Google?
Why do all Google search results lean left and ignore conservative links?
Why are my Google searches only coming up with democratic news sites?
Why do all of Google searches lead to liberal articles what happened to websites?
Why is it that all Google searches are from liberal sources?
Why is my Google searches hiding the truth about Biden?
Why do only liberal news outlets appear in Google search results?
Why are all the political Google search results left wing biased?
Why is Google unfairly and possibly illegal suppressing conservative websites and articles in their search engine?
Why is Google search engine not allowing conservative sights and always switching to liberal sights when requesting conservative information?
Why Does Google censor conservative websites in their search engine?
Why does my Google search not show conservative news outlets?
Why does Google search show biased results that promote leftist issues?
Why is it that only liberal publications pop up in Google searches?
Why does Google limit search results to liberal websites?
How can I have all Biden content removed from my Google search?
Why is Google so liberal when questions are asked on a conservative nature Google search engine does not address them?
Why is Google search biased for Democrats?
Why do Google search results have a liberal bias?
Why do only leftist opinions show up in Google searches?
Why is Google blacklisting and down ranking conservative sites?
When did these accusations of bias and censorship against Google start?
Looking at Google Trends data from September 2004 to present, show that queries for ‘Google censorship’ far outweighed queries for ‘Google bias’ until August 2012. At this point, queries for ‘Google bias’ started to predominate and have continued since to outstrip queries around censorship.
‘Google censorship’ queries peaked in January 2006, April 2010 and again in January 2012. The first date coincides with Google controversially setting up a censored version of google.cn in accordance with the Chinese government’s rules on what people can access and do online, and in March 2010, Google announced that it was diverting its mainland China users to an uncensored version of its site in Hong Kong which might explain the peak in April 2020 (source: BBC News: ‘Timeline: China and Net Censorship). I could not find a rational explanation for the subsequent peak in January 2012.
Queries for ‘Google bias’ have continued to climb and peaked in September 2020 at 5x the level of queries related to censorship. I think it is reasonable to assume that this peak reflects the build-up to the contentious presidential election between Biden and Trump in November 2020.
Checking Google’s AdWords service to look at individual search terms in the US shows that there’s only a few thousand people a month searching specifically for ‘Google bias’ and ‘Google censorship’. However, when you take a broader range of keywords on this theme you can see that there was a peak of around 2 million searches a month around the time of the US presidential elections in October and November 2020.
Is there any evidence of Google displaying bias?
Well, Yes. However, not necessarily in showing political bias in terms of the organic search results it presents to users. But Google has been found guilty on a number of occasions of showing bias for its own products and services to protect its dominant market position in search.
After around a 7-year investigation, in 2017 the EU Commission fined Google €2.42 billion for abusing its dominance as a search engine by giving an illegal advantage to its own comparison-shopping service.
In 2018, The EU commission fined Google again, this time to the tune of €4.34 billion for “illegal practices regarding Android mobile devices to strengthen the dominance of Google’s search engine”. These practices included the tying of the Google Search app with the Play Store in its agreements and subsequently tying Google Chrome with the Play Store and the Google Search app to effectively restrict competition.
In March 2019, the EU Commission fined Google again, this time for its actions related to Ad Sense. It levied a fine of €1.49 billion for breaching EU antitrust rules by abusing its market dominance by imposing a number of restrictive clauses in contracts with third-party websites which prevented Google’s rivals from placing their search adverts on these websites.
As recently as October 2020, the US Department of Justice has filed its own competition case against Google LLC accusing it of “unlawfully maintaining monopolies in the markets for general search services, search advertising, and general search text advertising in the United States through anti-competitive and exclusionary practices”.
So, could a company that has a track record of showing bias for its own products and services to the detriment of its competitors, also show bias in its ranking algorithms to favour websites that are aligned with its organisational goals and values to the detriment of websites that it finds itself at odds with?
Looking for evidence of general bias or political bias in Google’s SERPs
Google publishes its Search Quality Raters’ Guidelines; a rule book for a remote workforce that is tasked with assessing and providing feedback on the quality of its search results. I’ve reviewed this guide looking for specific advice on how to handle, avoid or flag any kind of bias or censorship, but this is not evident. However, sections 7.3 to 7.5 instruct its quality raters to give the lowest ratings to “Pages that Potentially Spread Hate”, “Potentially Harmful Pages” and “Pages that Potentially Misinform Users”.
And this is where it gets interesting. Even well-reasoned or scholarly articles can fall into the conspiracy theory category if a Search Quality Rater determines that a page might potentially misinform users. So, could a network of left-leaning search quality raters independently or collaboratively influence the search results we all see?
Whilst, there is no evidence at all in the guidelines of specific advice to favour Democrat-leaning websites over Republican-leaning websites, Google does point out in section 2.3 that political news is a ‘Your Money or Your Life’ (“YMYL”) topic and as such it requires the highest level of scrutiny as such types of pages or topics could potentially impact a person’s future happiness, health, financial stability, or safety.
It’s worth pointing out at this juncture, that Google goes to great lengths to point out that Raters’ ratings do not impact search rankings directly. The important point to note here, is that they provide feedback to the teams designing and improving the algorithms and so indirectly their work impacts the search results we ultimately see.
Search Quality Raters are hired worldwide through companies like Lionbridge that provide independent flexible workforce solutions to enterprises like Google. You can easily find a list of current and ex-Search Quality Raters yourself by searching on sites like LinkedIn, and if you do, you will see that they come from all walks of life.
There is no smoking gun here indicating that a grand conspiracy of left-wing Search Quality Raters are causing indirect bias in Google’s SERPs.
What does the Authoritas data show?
Authoritas has a proprietary database of over 125 million commonly searched phrases in multiple languages in major international markets. Every month, we check all these keywords and see who ranks in the first 100 results in the local version of Google in that market.
This is a large data set and sample size, but it cannot possibly cover every niche across the billions of search terms that Google sees each day. However, by comparing the prominence of different websites within this database, we can build a reasonably compelling perspective of how well a site is performing in Google search, at a scale and accuracy level that only machines can guarantee.
We’ve analysed and compared the performance of US websites in Google to look for evidence or signals that Google may be biased towards certain types of websites.
To do this as objectively as possible, we sourced a list of major media sites that have a leaning towards one side of the US political spectrum or the other from the sites listed on Wikipedia on these pages: Conservative Websites vs Left-wing Websites.
We then spot checked these lists using the Media Bias Fact Check website, which checks for political bias in over 3,500 websites.
We then took the Top 40 websites from each list that had the greatest prominence in Google, according to a set of generally accepted SEO metrics that Authoritas use and are widely used in the SEO industry. These metrics include, the number of ranking keywords and pages, the consumer demand for each keyword and a click-thru rate formula which is applied to estimate the organic search visibility for each keyword and page.
The unanswered questions above have an overwhelming number of users asking whether the Google search results are anti-conservative. In fact, our data analysis shows that on the contrary the conservative leaning websites actually outperform websites that show a bias towards the left side of the political debate.
When you add the total scores together – you can see that the top 40 conservative leaning websites have approximately 2.7x the visibility of the top 40 left-leaning websites.
Any evidence to support bias in favour of Left-leaning websites?
In the chart below, you can see a number of measures of the performance of each website relative to the others in organic search results in Google. The larger the score, the more prominent a site is in Google’s search engine results pages.
As you can see from the large line chart below, overall left-wing leaning websites seem to be relatively stable in the search results with no abnormal SEO Visibility deviations on a month-on-month basis.
(I cannot show all 80 websites at once in one graph, as it will be illegible. But you can always check yourself by performing a search on the Authoritas SEO Visibility tool.
So, when it comes to answering questions such as, “Why has the algorithm changed for Google search and Bing search during the Trump administration?”. All I can say is that whilst the major search engines will have updated their algorithms many times during Donald Trump’s tenure as President, there is no evidence to support any argument that shows that either left-wing or right-wing websites have been unfairly impacted.
Any evidence to support bias in favour of Right-leaning websites?
We amassed a large number of questions that implied that right-leaning websites are being demoted in Google SERPs for the benefit of more liberal sites. How do we answer those questions?
Well, when you look at the questions detail you can see that although they are phrased differently, the intent of the user is clear, in that he/she is implying that there is bias against showing conservative websites, links and opinions in favour of left-wing websites, links and opinions. Thus, it is very easy to group together a set of closely related questions by user intent and answer them all in one go.
Set 1 – Questions wanting to know why the user is only seeing liberal websites
“Why does Google only show liberal websites and not conservative websites when doing political search? Why do Google searches for info overwhelmingly provide information from left wing sources? Why does Google search produce results that are all left leaning? Why is Google search so biased and left? Why do I only get left wing media when I search Google? Why do all Google search results lean left and ignore conservative links? Why are my Google searches only coming up with democratic news sites? Why do all of Google searches lead to liberal articles what happened to websites? Why don’t Google search results ever show negative results for Democrats or liberals? Why is it that only liberal publications pop up in Google searches? Why does Google limit search results to liberal websites? Why is it that all Google searches are from liberal sources? Why do only liberal news outlets appear in Google search results?”
Answer: Google does not appear to be doing this at all. Our analysis of the top 40 conservative and liberal websites demonstrate that in fact conservative websites are performing well in comparison with the major liberal sites.
Set 2 – Questions wanting to know why conservative websites are being suppressed
“Why is Google unfairly and possibly illegal suppressing conservative websites and articles in their search engine? Why is Google search engine not allowing conservative sights and always switching to liberal sights when requesting conservative information? Why Does Google censor conservative websites in their search engine? Why does my Google search not show conservative news outlets? Why is Google blacklisting and down ranking conservative sites?”
Answer: Google does not appear to be doing this at all. In recent months you can see that many conservative websites like DrudgeReport and Breitbart have grown in organic (non-paid) visibility in Google.
Set 3 – Questions where the results are biased towards liberals in some way:
“Why do Google search results have a liberal bias? Why is Google search biased for Democrats?” Why does Google search show biased results that promote leftist issues? Why are all the political Google search results left wing biased? Why do only leftist opinions show up in Google searches?”
Answer: The data we’ve analysed does not paint a picture of political bias in favour of liberal websites, Democrat websites or leftist news outlets. Many of the right-wing websites in the chart below, which perform well in Google search, do not have content showing leftist opinions – quite the opposite in fact.
Here you can see a graph showing SEO Visibility lines for major conservative leaning media sites. I have grayed out Breitbart, only because it dwarves the other sites, but as you can see for yourself (you can search for any website for free from the Authoritas home page) the majority of sites are showing static visibility levels in Google or modest growth in the last quarter of 2020.
It is evident that Google is extremely commercially focused. Introducing any element of bias into its results does not make commercial sense – especially considering the polarisation of views in US politics across the US population. Upsetting close to 50% of your potential audience by barring or limiting exposure to left- or right-wing websites just does not make commercial sense at all.
Any evidence to support the apparent paranoia some people have about who is actually controlling their Google Search experience?
Here are just a few examples of the kind of unanswered questions that Google surfaces in its QuestionHub tool:
“How does the FBI let people control your Google searches?”
Answer: There’s no reported evidence suggesting the FBI has any influence on Google search (or any other search engine for that matter).
“Why are Google search results so manipulative?”
Answer: There’s no evidence to suggest they are manipulating your organic results in a targeted way at least. Obviously, Google’s algorithms and your personal search settings, history and location do to a certain effect have an impact on the SERPs you see.
“Why do the powerful control Google searches?”
Answer: If the powerful referred to by this user is the board of Google Inc or Alphabet Inc (Google’s parent company), then the answer is because they have control of the company and the power to direct it.
Summary – There’s insufficient evidence to show bias by Google in its organic ranking algorithms
There’s been a huge amount of content written about filter bubbles and echo chambers and I don’t aim to regurgitate that here. Although, suffice to say that the data I have looked at certainly shows that there’s a sizable minority of users who certainly don’t feel they are in the correct filter bubble!
The data we have access to does not really support a theory that Google is biased in favour of liberal websites to the detriment of more conservative leaning websites.
There are large, powerful sites ranking well in organic search on both sides of the argument. Perhaps, there’s a greater number of websites, media publications and bloggers with Democratic leanings? Perhaps left-leaning sites are more prolific publishers of content? Perhaps left-wing websites attract more links?
But we’ll have to leave those questions unanswered until the next piece of research – unless of course someone else finds them on Google’s QuestionHub and answers them in the meantime! 😉
Please have your say in the comments. Do you have any examples of Google showing bias in its organic search results?
[COMMENTS CLOSED]
