What Is Google-Extended, and What Does Blocking It Change?

Google-Extended controls: Gemini training and grounding

Google-Extended controls two uses of content that Google has already crawled. One is training future generations of Gemini models. The other is grounding, which Google defines as providing content from the Search index to a model at prompt time to improve factuality and relevancy. Both uses sit outside Google Search.

The wording comes from Google’s Google common crawlers page, last updated 15 October 2026. It names the products affected.

• Training: Gemini models that power Gemini Apps and the Vertex AI API for Gemini.
• Grounding: Gemini Apps, and Grounding with Google Search on Vertex AI.

So a block reaches into Google’s consumer assistant and its cloud platform. Developers who ground Gemini on Vertex AI may then not receive your pages.

The token’s scope has grown over time. Google’s Search Central changelog shows it launched for Bard and Vertex AI, was renamed for Gemini Apps, and was clarified again in April 2025.

Read that history as a reason to recheck. The products named today are the ones Google lists now, not a permanent set.

Google-Extended limits: Search, AI Overviews and AI Mode stay unaffected

Google-Extended does not decide whether your pages appear in Google Search, and Google states plainly that it is not used as a ranking signal. AI Overviews and AI Mode are part of Search, so the token leaves them alone. This is the point most published advice gets wrong.

Google’s page on AI features and your website is direct about it. Robots.txt directives for Googlebot are the control for how sites are crawled for Search.

To limit what Search shows, including in AI features, the same page points to nosnippet, data-nosnippet, max-snippet and noindex.

It then mentions Google-Extended only for limiting “AI training and grounding in some of Google’s other systems”. Search is not one of them.

So blocking the token to escape AI Overviews does nothing for that goal. Your pages can still be summarised and linked there.

The reverse fear is also unfounded. A block will not lower your rankings, because Google says the token plays no part in them.

Google-Extended has no user agent string: why

Google-Extended has no HTTP user agent because no separate crawler exists. Google’s documentation says crawling happens with existing Google user agent strings, and the robots.txt token is used “in a control capacity”. Googlebot fetches the page once, and the token decides what Google may do with it later.

That has a practical effect for anyone reading logs. You will never see “Google-Extended” in an access log, however long you search.

A block therefore does not reduce crawling. Googlebot still visits at the same rate, because Search still needs the page.

Compare Google-CloudVertexBot, listed on the same page. It does appear in requests, crawls only when a site owner asks to build Vertex AI Agents, and has no effect on Search.

Digital Magma uses this difference when clients ask why a block “isn’t working”. The logs look identical before and after, and that is expected.

Google-Extended in robots.txt: how to set it

Google-Extended takes two lines to block completely and slightly more to block part of a site. Add the group to the robots.txt file at the root of each host. Google’s own example keeps one archive page available while closing the rest of the archive folder.

# Opt out of Gemini training and grounding
User-agent: Google-Extended
Disallow: /

# Google's example: partial opt-out
user-agent: Google-Extended
allow: /archive/1Q84
disallow: /archive/

Leave your Googlebot group untouched. Changing it would affect Search, which is a separate decision.

Digital Magma checks for this slip in every robots.txt audit. A well-meant AI block placed in the Googlebot group closes Search as well, which is the opposite of the intent.

Google’s robots.txt specification says Google generally caches the file for up to 24 hours. Expect at least that delay before a change is read.

The same specification applies a 500 KiB size limit. Content beyond it is ignored, so keep long AI blocklists tidy.

Testing is limited, since no crawler identifies as Google-Extended. Confirm the group is valid in Search Console’s robots.txt report, then trust the documentation.

Google-Extended vs Googlebot vs nosnippet: choosing the right control

Google-Extended, Googlebot rules and snippet controls each answer a different question, and picking the wrong one causes most of the damage we see. Start with the goal you actually have, then use the control Google documents for it. The table follows Google’s own pages.

Google’s robots meta tag page says nosnippet applies to AI Overviews and AI Mode and prevents use as “a direct input” for both.

Notice the trade-off. Snippet controls shrink your regular search listings too, which costs clicks.

Digital Magma rarely recommends nosnippet across a whole site for that reason. A targeted data-nosnippet on one sensitive passage usually meets the real need with far less loss.

Google-Extended for a business: why it matters

Google-Extended matters to a business mainly as a licensing and brand decision, not a traffic one. Blocking it keeps your pages out of Gemini training and Gemini grounding while leaving Search traffic alone. Allowing it lets Gemini Apps draw on your content when people ask about your field.

Publishers with paid archives often block it. Their content is the product, and Gemini answers built on it could replace visits.

Service businesses usually gain from allowing it. A firm that wants Gemini to describe its offer accurately should let Gemini see that offer.

Digital Magma allows Google-Extended on its own site, in a named group with the same exclusions as every other crawler.

The worst outcome is the confused one. A site blocks the token to hide from AI Overviews, stays in AI Overviews, and loses Gemini visibility for nothing.

For how Gemini itself uses the web, see our Gemini entry and the one on grounding.

Google-Extended in our client work

Google-Extended comes up in Digital Magma’s client work whenever someone wants control over AI Overviews. For a Canadian welding equipment manufacturer that appears in AI Overviews and ChatGPT, blocking the token would not change AI Overviews at all. It would only cut Gemini off.

That client’s visibility rests on Search: 40+ keywords held in Google’s top 10. AI Overviews draw from that same index.

For a Toronto ENT clinic, the concern is accuracy. Patients asking Gemini about a specialist should get correct details, which favours allowing access.

For an app development company in six countries, one robots.txt per host sets the token consistently across every market.

Google-Extended: frequently asked questions

Google-Extended questions site owners ask Digital Magma.

Google-Extended: related terms, sources and author

Google-Extended relates to these Digital Magma entries:

• Gemini: Google’s assistant and model family
• AI Overviews: Google’s AI summaries in Search
• Grounding: feeding live sources to a model
• Robots.txt for AI bots: rules for each AI token
• AI crawler: the bots behind AI products
• Should you block or allow AI crawlers?

Sources: Google’s common crawlers page, AI features and your website, the robots meta tag page, the robots.txt specification and the Search Central changelog. All checked on 26 October 2026.