SGE Research Study
How will Google's planned integration of AI powered Search Generative Results into the SERP impact SEO?
In December 2023, we ran 1,000 commercial terms through a handful of SGE-enabled Google accounts using a desktop device in the US to assess the impact of Google’s Search Generative Experience on organic results and evaluate the likely impact for SEOs everywhere.
TL;DR:
This is arguably going to be the biggest change in Search in the past decade.
If Google decides to rollout the current incarnation of its Search Generative Results then SEOs will see the traffic and value from top organic rankings dissipate.
If your SEO agency is not talking to you about this already, then they should be!
Research Findings - (Highlights or Lowlights – depending upon your perspective)
• Google displays a Search Generative element for 86.8% of all search queries.
• 65.9% of the time a small generate button appears but 34.1% of the time a pre-populated SGE element appears with a ‘Show More’ link.
• When a user clicks on the SGE button or the ‘Show More’ link, then the No.1 organic listing shifts down the page by 1,562 pixels and 1,630 pixels respectively. This is ~1.5 times the height of the viewport and therefore it’s inconceivable that this won’t impact click-thru rates and organic traffic.
• On average there are 10.2 links in the SGE content post-click from 4 unique domains.
• The largest generative content introduced 37 new visible URLs above the organic rankings!
• 93.8% of generative links (in this dataset at least) came from sources outside the top-ranking organic domains. With only 4.5% of generative URLs directly matching a page 1 organic URL and only 1.6% showing a different URL from the same organic ranking domain.
SGE Rankings Data
I’ve shared some of the before and after SGE SERP and ranking data that we’ve produced as part of this research. This collection of spreadsheets shows you the full SERP data, rankings for Wikipedia and summary stats of the kind of domains that look like they will perform well in SGE.
SGE Ranking Data - Spreadsheets:
1K Keywords Researched - This is the file of keywords we researched. 10 categories, 100 head terms in each. Total Monthly Search Volume for all keywords ~140Mn searches a month.
Top Performing Domains in SGE - Lists all the domains that ranked in the SERPs for our keyword set by how many URLs they have ranked for every key non-paid SERP Feature.
Example Wikipedia Rankings - shows pre-click and post-click SGE rankings.
SGE Impact on No.1 Organic Results - This takes every top ranking organic URL across the whole dataset and looks at how it moves when a user clicks on the SGE Trigger.
SGE Impact on Featured Snippets - This shows the Featured Snippets in the SERPs before and after clicking on SGE.
Universal SERPs (Non-Paid Only) - This shows the non-paid ranking URLs on the first page of the SERPs before and after clicking on SGE.
I’ve also shared an automated slide deck that can be produced on demand for your keywords. If you believe having this kind of data would be useful for the keywords you care about then join our SGE-enabled rank tracker waitlist.
Sign-up before January 31st to an Authoritas platform package and you’re guaranteed to be the first wave of SEOs to get a full in-depth impact analysis for your site(s).
We’re ready to automate this analysis for you, as soon as Google rolls SGE out to non-logged in users.
SGE Ranking Data - Powerpoint Slides:
Impact of SGE on Organic Ranking Features and Positions - This presentation contains all the images in the blog post. Hopefully, you'll find it handy is you have to do a presentation on the likely impact of SGE and want to justify action and budget sooner rather than later.
Powerpoint Embed:
Research Study Results & Analysis
How often are SGE results triggered?
(Part 1) #Keywords with SGE results
If you are wondering whether SGE results will affect your website then the answer is 'Yes'. In our study, 86.83% of all keywords we checked had an SGE result.
Total number of rows: 995
Rows with SGE content: 864 (86.83%)
(Part 2) Percentage split by SGE Trigger Type
The majority of the search results feature some form of SGE content, with the "generate_button" type being more prevalent than the "show_more" type. The "generate_button" feature appears in more than half of all cases we examined (65.9%), while the "show_more" feature appears in about a third of the SGE cases (34.1%).
The 'Show More' type of SGE result is what I'm calling an 'SGE Teaser' as it already contains some SGE content, whereas the 'Generate Button' is just a call to action to get AI results. Clearly, it is very likely that these different SGE types will have differing levels of user engagement and as a result the impact of each on the organic rankings and universal SERP features that follow will be different.
Our study simulated real users searching but not clicking, so it's beyond the scope of this study to understand the magnitude of the difference, but hopefully the Google Search Console team will be kind to us all and eventually provide us with some useful data in Search Console. In my opinion, for keywords where the SGE Teaser appears, brands are likely to see a steeper decline in organic visibility. Of course, if you are looking to target prominent placement for your brand within the generative results, then you may give more weight to keywords with this type of SGE result.
How is the No.1 Organic Result impacted by SGE?
There is little doubt, that with SGE, the first organic search result is pushed further down the page, requiring more scrolling to reach it. The exact amount of scrolling varies depending on the type of SGE feature that's triggered.
We have no data yet from Google Search Console or clickstream studies to rely on, but experience tells us that as you go further and further down the page you get fewer and fewer clicks, so drop of a page or more is going to have a huge adverse effect on SEO traffic and as a result on the volume of conversions.
The first thing I wanted to understand was how the introduction of Google's SGE (Search Engine Generative Results) affects the position of the first organic search result on a desktop screen.
AVG Vertical Drop (Pixels) - This measures how far down the page the No.1 organic result has moved due to SGE. In simple terms, it's like measuring the distance in a straight vertical line from the original position to the new position, counted in pixels.
AVG Vertical Drop (Percentage of Desktop Viewport) - This is similar to the above, but instead of measuring in pixels, it measures the drop as a percentage of your entire screen's height. Think of it as how much of your screen you'd need to scroll down to find the first organic result.
(Part 1) AVG Vertical Drop (Pixels) of the #1 Organic Result
Here's an example of a random sample of keywords from the research data.
Remember, we're using a desktop browser (1920 x 1080) so the visible viewport is about 940 px high after considering the vertical space taken by your browser's header and footer.
So, you can see that the majority of keywords are already below-the-fold of the page (i.e., Not visible on initial page load), before the SGE element is expanded.
As soon as the SGE button or link is clicked to expand, then almost every No.1 organic position result falls below-the-fold of the page.
The vertical drop in pixels was calculated for each keyword in the study and averaged.
Irrespective of the type of SGE result that appears, the number 1 organic result drops on average around 1,500 pixels or ~1.5 times your visible viewport on desktop.
(We plan to conduct this analysis again for mobile browsers once the search generative experience has been fully rolled out to all users on all devices).
(Part 2) AVG Vertical Drop (Percentage of Desktop Viewport) of the #1 Organic Result by SGE Sub-Type
Our results are broken down by SGE type, although the impact is broadly similar once a user has clicked on the SGE result to expand it.
No SGE: When SGE is not used, there's no change. The No.1 organic result stays in its original place.
SGE Sub Type - Generate Button: Vertical Drop (Pixels): On average, the No.1 organic result moves down by about 1,630.9 pixels. Imagine scrolling down quite a bit on your screen to find the first usual search result.
SGE Sub Type - Generate Button: Vertical Drop (%): This represents about 151% of your screen height. This means you'd scroll past your entire screen length, and then some, to find the first organic result.
SGE Sub Type - Show More: Vertical Drop (Pixels): Here, the drop is around 1,562.1 pixels. It's slightly less than with the "generate_button", but still a significant scroll down.
SGE Sub Type - Show More: Vertical Drop (%): This drop is about 144.6% of your screen height. Similar to the generate_button, but you scroll ever so slightly less.
How much SERP real-estate does SGE Post-click occupy?
(Part 1) AVG Vertical Height (Pixels)
Both types of SGE content occupy a significant portion of the screen space on a desktop.
After clicking, the SGE content area expands to sizes larger than the screen's visible area, meaning you'd have to scroll down quite a bit to see the entire content.
(Part 2) AVG Vertical Drop %
The "show_more" content generally takes up slightly more space than the "generate_button" content.
Generate Button:
• Average Vertical Height (Pixels): 1,764.0
• Average Height Ratio to Viewport: 1.6
• This means that the "generate_button" SGE content, on average, takes up a space of 1764 pixels in height. This height is 1.6 times the height of the visible area of the web page. In simpler terms, this type of content takes up more than the full screen of the average desktop display, requiring scrolling to view all of it.
Show More:
• Average Vertical Height (Pixels): 1,841.9
• Average Height Ratio to Viewport: 1.7
• For the "show_more" SGE content, the average height is slightly larger, around 1841.9 pixels. The height ratio to the viewport is 1.7, indicating that this content also extends beyond the full screen height of a desktop, necessitating even more scrolling compared to the "generate_button" content.
How many websites and unique links appear in SGE Post-click?
(Part 1) AVG Number of Links and Unique Domains
SGE content on Google's SERPs features a mix of links, with an average of 10 links per SGE instance, coming from around 4 different websites.
General Overview (All Rows with Generative results)
• Average Total Links: 10.2
o On average, each instance (row) of SGE content contains about 10 links.
• Average Unique Links: 4.4
o Out of these links, on average, there are about 4 unique links per instance. This means there is some repetition of the same links across the SGE content.
• Average Number of Unique Domains: 4.0
o On average, these links come from 4 different websites (domains).
• Average Number of Links per Domain: 2.4
o This suggests that, on average, each domain has about 2 to 3 links pointing to it within the same SGE content.
(Part 2) AVG Number of Total Links by SGE Trigger
The "generate_button" type tends to have a higher concentration of links but fewer unique domains compared to the "show_more" type, which features a broader range of unique links and domains. This diversity suggests that SGE content provides a range of sources, but there is some repetition of the same links across different instances.
(Part 3) AVG Number of Unique Domains by SGE Trigger
Breakdown by SGE Sub-Type
At this stage, I'm not sure we can draw very many actionable insights from the differences between the types of SGE results, but for completeness here are the stats from our study.
Generate Button
• Average Total Links: 12.2
o SGE content with the "generate_button" has, on average, more links (about 12) compared to the overall average.
• Average Unique Links: 4.9
o There are about 5 unique links on average, suggesting some repetition of links.
• Average Domains: 4.5
o Links come from an average of 4.5 different domains.
• Average Links Per Domain: 2.9
o Indicates a slightly higher link concentration per domain compared to the overall average.
Show More
• Average Total Links: 10.7
o Slightly above the overall average, with about 11 links in SGE content triggered by "show_more".
• Average Unique Links: 5.4
o Contains, on average, more unique links than the "generate_button" type.
• Average Domains: 5.0
o Links in this type come from an average of 5 different domains, which is higher than in the "generate_button" type.
• Average Links Per Domain: 2.3
o This is close to the overall average, indicating a balanced distribution of links across different domains.
How are Organic Rankings used in Generative Links?
(Part 1) Generative URL matches by Organic Position
Trying to work out the right way to look at this and analyse it took me way too long. I wanted to understand which sources Google used to find the generative URLs displayed and whether they were taken from the URLs or domains of the top-ranking sites on page 1 of the SERP and if so, whether having a higher organic ranking benefitted your chances of ranking in generative results.
There’s two ways of doing this, either:
a) Looking at each generative URL in turn and seeing if it exactly matches the 1st organic URL, 2nd organic URL, etc. If not, then see if it matches the domain for the 1st organic URL and so on. Finally, if it doesn’t match then record ‘no match’ against each position.
Or;
b) Look at each organic URL in order and compare it with each generative URL in order, again recording all exact matches, then checking for domain matches (other pages from the same site) and finally recording ‘no matches’ where other URLs were appearing.
For a myriad of reasons, that I won’t get into here, it makes quite a difference to the results if you start with organic and compare with generative or the other way round. I chose ‘B’ as I felt it better reflected the raw data I had eyeballed.
• Total No Matches: 89,553 - A significant majority (about 93.8%) of the generative URLs do not match any URL in the top page 1 organic search results.
• Total Exact Matches: 4,327 - About 4.5% of the generative URLs exactly match a URL in the organic search results.
• Total Domain Matches: 1,560 - Around 1.6% of the generative URLs match the domain of a URL in the organic search results.
(Part 2) Generative URL matches by Organic Position (Exc No Matches)
In simple terms, when comparing the URLs that Google's SGE content generates against the URLs that appear in regular (organic) search results, there's a relatively low overlap. Only about 4.5% of the time does a URL in the generative content exactly match a URL in the organic results. An even smaller percentage (1.6%) matches at the domain level, meaning the generative content is from the same website as an organic result but not the exact same page. The vast majority of the time (93.8%), the generative content introduces URLs that are completely different from those in the organic search results.
This suggests that Google's SGE often provides unique or additional content not found in the top organic results.
How are Featured Snippets impacted by SGE?
(Part 1) AVG Vertical Drop (Pixels)
115 SERPs had a Featured Snippet – so the good news is that the Featured Snippet is still appearing in SGE SERP results…. Well, for now at least!
The bad news is that it is pushed a long way down the page by the expanded SGE content. The average vertical drop of the featured snippets with generative content is 1478.7 pixels. On average, the featured snippets are displaced downwards by about 136.9% of the viewport height. This means that the displacement is more than the entire height of the visible screen area, indicating that users would have to scroll significantly to view the featured snippet that has been pushed down by the generative content. This will affect how users interact with and view these featured snippets, as they now appear much lower on the page.
(Part 2) AVG Vertical Drop (Percentage of Desktop Viewport)
The really bad news? Well, I just cannot see Featured Snippets lasting much longer in their current form.
After, all the SGE content has it covered now and in any event Google has been experimenting with ‘super’ Featured Snippets which seem to combine data from multiple sources into new formats without an AI generated result.
I think the days of SEOs optimising for Featured Snippets are over.
Impact of SGE on a Domain's Universal Rankings by SERP Feature Type - Example: en.wikipedia.org
Organic - SGE Pre-click vs Post-click
I’ve analysed the SGE impact on every domain in the dataset.
Obviously, there are thousands, so I cannot publish them all here and, in any event, if you want this kind of SGE analysis then you need to run it for keywords that are relevant to your website.
So, I’ve picked Wikipedia as an example. The following charts show how its Universal Rankings (excluding paid and hidden results) change for different SERP features before and after a user clicks to expand the SGE content.
Key to these SGE ranking charts:
'_ui' - this refers to the Universal Initial rankings, e.g. SGE pre-click.
'_u' - this refers to the Universal rankings post SGE click.
Get this Google Search Generative SERP analysis for your website.
SGE Impact on Wikipedia’s Organic Visibility
• Wikipedia has 343 keywords with organic ranking positions in our database.
• Average Rank Change: -9.7 - Organic search positions dropped by 9.7 positions on average when SGE was clicked.
• Average Vertical Drop: 1,507.9 pixels (139.6% of the Viewport). Organic results are pushed down significantly, by around 1,500 pixels, more than the height of the screen.
SGE Impact on Wikipedia’s Visibility for People Also Ask results
• Wikipedia has 174 PAA rankings.
• Average Rank Change: -8.5 - The "People Also Ask" section drops by 8.5 positions on average.
• Average Vertical Drop: 1418.8 pixels (131.4%). The PAA section is moved down the SERP by an average of 1,418.8 pixels, more than the full screen height.
SGE Impact on Wikipedia’s Visibility for Generative results
Just to explain this a little more. Sometimes, URLs are present in the Generative results prior to clicking. Then when a user clicks on the SGE button or link, many more Generative URLs appear.
As you can see below, Wikipedia is big beneficiary of SGE results as it is cited multiple times in post-click SGE results.
• Count: 39 instances (358 unique instances).
• Average Rank Change: +0.2.
• There's a slight increase (0.2 positions) in the rank, which is almost negligible.
• Average Vertical Drop: -40.6 pixels (-3.8%).
• The generative content actually moves up slightly in the SERP layout.
Impact on Wikipedia’s Visibility for Image results
• Count: 26 instances.
• Average Rank Change: -9.4.
• The rank of image results drops by 9.4 positions on average.
• Average Vertical Drop: 590.0 pixels (54.6%).
• Image results are displaced downward by 590.0 pixels, over half the height of the screen.
SGE Impact on Wikipedia’s Organic Visibility for Featured Snippets
• Count: 13 instances.
• Average Rank Change: -8.8.
• The rank of featured snippets, on average, drops by 8.8 positions.
• Average Vertical Drop: 1388.6 pixels (128.6%).
• Featured snippets are moved down the page by an average of 1388.6 pixels, which is more than the full height of the screen (128.6%).
Summary Stats - All Domains
We analysed every domain that appeared in the SERPs for each keyword to understand which brands were the most prominent in organic and whether this prominence was mirrored in the new generative results.
The following charts show the Top 20 Domains based simply on how many URLs each had ranking for each SERP Feature.
Top 20 Organic Domains (by #URLs)
Top 20 Generative Domains (by #URLs)
Top 20 Featured Snippet Domains (by #URLs)
Whilst some domains perform well across all key non-paid SERP features (for the purposes of this study we consider these to be: organic, generative, featured snippets and people also ask), a few domains that are performing well organically hardly appear at all in the generative links, and a number of new domains that are not performing well organically at the moment have been rewarded with multiple generative links.
The final two combination charts look at how the Top 20 Generative Domains and the Top 20 Organic Domains perform respectively across other key non-paid SERP features.
SEO Performance of the Top 20 Generative Domains (by #URLs) across key Organic SERP Features
Hopefully, this chart will give SEOs cause for hope, as even if a domain is not performing well today in organic, featured snippets or people also ask results, it still can feature often in the new generative results and capture new users from search (without paying).
SEO Performance of the Top 20 Organic Domains (by #URLs) across key Organic SERP Features
This chart shows that many top organic sites will perform reasonably in the new generative results - but not all will.
What it doesn't show is that there are many brands that are not prominent today in the organic results that will appear above these brands' organic results.
This picture will be different for every market. So, by all means talk to us about getting this perspective for the markets and keywords you care about.
Methodology
Data was collected in December 2023 over a 2-day period. We sent 1,000 keywords and only analysed successful results. We used a Chrome browser from a logged in Google account holder in New York using a PC running Windows 11.
The browser’s viewport was set to 1920 x 1080 pixels. We calculate the vertical drop in pixels as a percentage of the browser’s viewport to give you an indication of the magnitude of a drop. e.g. If an element drops 1.5 times the viewport, then it’s tantamount to a page and half scroll down the page. This describes the process we followed to capture the data.
- We used our SERPs API and made a request to Google.com and first captured the initial state of the SERP to see whether an SGE Trigger (SGE Button or 'Show More' link is present). This is what we call the SERP pre-click, and you will see this represented as 'Universal Initial' or 'UI' in the charts and tables presented in the study. So, a ‘UI rank’, would be the rank of this URL before a user clicks on the SGE Trigger to generate expanded generative results.
- If we found an SGE Trigger, we then clicked on it to fully expand the generative results. We then captured the state of all elements on the page so that we could compare pre-click and post-click rankings and positions.
This is what we call the SERP post-click, and you will see this represented as 'Universal' or 'U' in the study. So, a ‘U rank’, would be the rank of this URL after a user clicks on the SGE Trigger to generate expanded generative results.
Interpreting the data
Understanding rankings - We calculate the x, y position of every object on the page pre-click and post-click. We know humans tend to read the SERP results in an F-pattern, so the top ranking is the one with an x, y position closest to the top left of the page (for left-to-right reading languages).
We can count rankings by SERP Feature Type, so you can see if you rank for the first item in organic, people also ask, generative results and so on.
We also count the URLs' Universal Rank on the page, so this is a URL's rank including all SERP types. Generally, unless specifically indicated otherwise, we exclude any paid adverts, paid shopping or irrelevant Google SERP features where you cannot rank a URL from your site (e.g. Travel Finder). We also only include 'Visible' results, since there are sometimes hundreds of hidden links in generative results and carousels.
N/B: When comparing how many places a URL has dropped after SGE clicking, we join the pre and post-click SERPs together and create a new 'Index', this index shows the position of the URL on the page amongst all non-paid items. You can use just this column to see how many places a result has dropped.
Potential Shortcomings & Further Research
Viewport calculations:
We appreciate the fact that the visible viewport is less than 1,080 pixels due to the browser header and footer, and consequently could have made these vertical drop calculations even more accurate. However, during our study we noticed that Google continues to experiment with design and markup changes and the current calculation is good enough for the task at hand today. If we changed this to 940 pixels it would only make the data look even scarier!
Different types of keywords:
We’ve only analysed a sample of pre-click and post-click SERP data for 1,000 keywords. Once this is rolled-out globally we will be able to undertake larger studies.
These are commercial keywords and are biased to ecommerce products and common services consumers purchase across 10 categories:
1. Automotive
2. eCommerce & Retail
3. Entertainment
4. Fashion & Apparel
5. Finance
6. Food & Beverage
7. Health & Wellness
8. Legal
9. Technology & Electronics
10. Travel & Hospitality
In further studies, we would want to assess the impact across different types of keywords by user intent by comparing SERP data for Navigation, Informational, Research and Transactional keywords. A separate study into the impact on Local SEO also merits attention. We also need to study the rate of change of the generative results. This will obviously vary considerably for ‘news’ keywords. But it will be interesting to compare this and see whether there is any evidence of seasonality with ecommerce products.
I’m also open to suggestions? Just DM me on LinkedIn.
Impact of personalisation:
We have not considered the impact of personalisation on these results. The queries were made at random across a small number of accounts, but no websites were clicked on (just the SGE button), so we don’t anticipate that we would have unduly influenced the AI generated results, but we cannot know for sure.
Rate of change of Generative results:
We will only be able to do this once Google has launched SGE to non-logged in users globally and we can setup a systematic process for capturing the before and after click state of the new SGE SERPs.
SEO Performance:
The SEO industry has for years relied on click-thru rate curves by ranking positions and keyword search volumes to create different models for calculating SEO Visibility. Every SEO tool has its own methodology for measuring SEO performance.
Obviously, this SGE analysis won't necessarily correlate with SEO performance today as SGE is not rolled-out to non-logged-in users, is not available in much of Europe and is limited to Google Chrome and Google App users on mobile.
Even, when SGE is launched, this is only going to give you a top-level view of which domains are performing well in your market, as we have not weighted each keyword by its monthly search volume, nor estimated visits using our industry standard CTR% model. Clearly these old SEO CTR% models are no longer going to hold true in an SGE world, since the SERP is so much more complex and there's the before and after click effect to consider now too.
So, we have kept the top performing domain analysis simple for now and no doubt this will be an area for further research and learning.
But if you want a rough rule of thumb indicator of the possible impact of SGE on organic visibility, then consider this very rough back of the envelope calculation:
SGE Impact on Organic Click-thru rates and Organic Traffic
SGE will shift the #1 organic result down the page by ~1,500 pixels once an SGE element is expanded.
Let's say that the average organic result is 150 pixels high (this is conservative as it can be 250 pixels with image thumbnails) and there's a 25-pixel gap between organic listings.
Then a drop of 1,500 pixels is roughly the equivalent of your No.1 Organic listing dropping to (at least) 10th place.
Our 2015 (admittedly very outdated) click-thru study showed a drop in click-thru rates from 21.4% to 2.7% for non-branded terms on desktop devices.
So, the 10th organic listing was getting roughly 8 times fewer clicks than the number 1 result for the same keyword.
It's impossible to say for sure today, but if user behaviour and click patterns are in any way similar, then we could see drops of this magnitude in our existing organic traffic when SGE is clicked by users.
One thing is clear. There's no point waiting for the worst to happen. Now is the time to start working on how to feature prominently in the generative results!
The first step starts with analysing your market and we can of course help you with this.
Sign-up to the SGE Rank Tracking waitlist.
Final Thoughts
In simple terms, the presence of generative content in Google's search results affects the positioning and visibility of traditional results like featured snippets, organic links, and images.
Once a user clicks on the ‘generate AI results’, an entire new section appears and pushes everything down the page. This has to have a noticeable impact on how users interact with and view search results, with some standard results being displaced considerably.
So, SGE is inevitable. Google will launch it early in 2024, when it is confident that users like it and that it doesn’t compromise their quarterly financials.
The uninitiated will claim once more that ‘SEO is dead‘ and that Google is killing publishers.
But the SEOs that have been around the block a few times know that any change brings opportunity as well as loss.
Learning what it takes to get prominent citations and references in generative links, is just a new optimisation game we’ll be playing this year.
Across my two decades in the search space, I have to say that the pace of change in 2023 was greater than ever.
But, if you think 2023 was fast paced, hold onto your hats because 2024 is going to be even faster!
