Google SGE: The Future of Search is Coming (and yes, it has AI)

The way people access information on the internet is changing. The meteoric rise of LLM technology through Chat-GPT has raised fundamental questions about what’s possible when it comes to interfacing with massive data sets.

Overview
The way people access information on the internet is changing. The meteoric rise of LLM technology through Chat-GPT has raised fundamental questions about what’s possible when it comes to interfacing with massive data sets.

Over the last 20 years, that question has been best answered by one player – Google. Google’s search engine connected users to their enormous index by allowing them to simply search. While Google’s algorithm has dominated the search engine space, it always had drawbacks. Users need to browse the results one-by-one, and visit them one at a time to manually evaluate the information.

LLM-powered technologies like Chat-GPT provide a direct answer to user queries, and even allow users to influence the format, quantity, and depth of the answer. Users can ask questions in their own words, and receive tailored responses that are clear and easy to understand.

SGE (Search Generative Experience) is Google’s answer to this challenge. Launched in May 2023, SGE aims to harness Google’s massive index of web data, and implement GPT-style responses within search results. So how does it work, and how might it impact SERP data collection? Let’s dive right in to get a better understanding of Google SGE.

Unlike the familiar list of links and summaries available on Google’s search results, SGE employs artificial intelligence to generate answers directly from a user’s search query. This AI-driven method combines information from multiple sources, and is positioned at the top of Google’s search results page.

How to use Google AI Search

Imagine searching for “compare shoes for running long distances”. Instead of sifting through numerous articles and product listings, Google AI search presents a succinct summary that highlights top recommendations, key features, and even user reviews, all pulled from various authoritative sources. This snapshot not only answers your initial query, but also suggests follow-up questions and related topics, creating a seamless, conversational experience.

What makes SGE even more user-friendly is a dynamic answer format (or as Google calls it – vertical experiences). This means that the result layout will be based on the search topic and intent, to provide a more informative and contextual experience.

There are currently a number of supported layouts:

1. Lists
2. Reviews
3. Instructions – step by step guides for recipes, repairs and more
4. Shopping- product views from Google Shopping
5. Local search (”restaurants near me”)
6. Comparisons
7. Code
8. Image generation

Under the hood

At the core of Google’s Search Generative Experience (SGE) are several advanced LLMs. The Multitask Unified Model (MUM) is a key player, trained in 75 languages, helping to find and connect information across the world. It’s especially good at picking out related topics in videos, making the results richer and more connected.

Then there’s PaLM 2, which understands multiple languages, solves complex problems, and reads code. It’s trained on a wide range of texts, including scientific content and coding languages, making SGE more versatile and accurate.

Speeding up the whole experience is Gemini, Google’s latest innovation, which makes Google generative ai not only faster but also capable of handling text, images, and audio more efficiently. This reduces waiting times by 40% and makes searches smoother and more responsive.

Challenges

Unlike traditional search algorithms that primarily rank results based on SEO factors like keywords, backlinks, and site authority, SGE employs a more complex, multifaceted approach. Approximately 45% of SGE results align with the top 10 organic search rankings, indicating that while there is some overlap with traditional SEO metrics, SGE also considers other factors for ranking. Furthermore, when looking at the top 3 results, there’s only a mere 16% overlap.

SGE’s unique ranking factors include the depth and breadth of content, with a preference for longer, detail-rich, and well-researched information. This is evident in its emphasis on EEAT (Experience, Expertise, Authoritativeness, and Trustworthiness), ensuring that the content not only matches the query but is also reliable and authoritative.

One other challenge that’s inherent to all AI models is providing false, inaccurate, or misleading information in the form of misinterpretation, bias or hallucinations- generating unrealistic or misrepresented answers. Minimizing these limitations is extremely important, especially in cases pertaining to health, financial and legal advice. Today, a relatively small number of YMYL queries return an AI response, and Google includes warnings on those that do.

What it means for the future of search

The introduction of SGE is changing the search landscape completely. No longer will the user need to navigate away from the SERP – they’ll get all the information they need immediately and in context. The addition of followup queries emphasizes this notion, as any questions left unanswered initially will be answered as part of this seamless journey, making it even less likely the user will leave the search interface.

If you’ve been experimenting with SGE since its debut, you’ve probably noticed frequent changes and a lot of variability as Google ran extensive A/B testing during the first months. Earlier iterations usually showed the AI snippet in full for almost all searches that triggered an AI response, taking up as much as an entire screen by default, pushing organic and paid results down and away from view. Later versions show either a “generate” button, or a smaller, expandable snippet that’s less intrusive.

In the last weeks, it looks like things are starting to stabilize, as changes are reducing and there’s more consistency in results and the overall experience with SGE.

SGE is still very much an experiment (with no official release date yet). Our testing has shown an increased likelihood of getting an SGE response. Today, more than 80% of the queries we run will produce some type of generative response (either an opt-in or expandable snippet), compared to about 43% in our earlier tests. Some search verticals, like shopping, now provide an AI response for over 92% of searches. Another big step forward is stability and consistency. Just a few months ago, a single search query might have given different response types at different times, and would sometimes not give an AI response at all. Taken together, these factors point toward increasing maturity for SGE as Google expands it’s AI capabilities.

Capturing SGE Responses with Nimble

With the advent of new features such as SGE, companies and data providers will have to work hard to keep up with SERP evolution. Nimble Labs experts have been thoroughly testing and analyzing SGE from the very beginning. We’ve developed sophisticated solutions, utilizing AI methodologies, to be able to handle SGE’s dynamic responses- retrieving, analyzing, parsing and interacting with any response type, making sure to stay up to date with ongoing updates, and adapting to feedback from our enterprise customers that have access to Nimble Labs.

To demonstrate these capabilities, we ran the aforementioned example through our system. Using the Nimble AI browser, Not only was the full page data retrieved, Nimble’s AI models also understood the full SGE response structure and element types, and provided a fully structured result, ready to be used.

Next Steps

While there’s no telling when Google SGE will be officially released, Google has made it very clear that they’re heavily invested in introducing Generative AI into every aspect of their products and services.

With the fierce competition from companies like Microsoft and OpenAI, this trend is likely to continue and accelerate.

We’re keeping an eye on SGE as new developments arise to ensure our customers will always enjoy the ability to collect accurate, complete SERP data.

‍