Gemini and AI Mode Fetch as Google and Google-Lens; 397 of 534 Fetcher Ranges Lack rDNS

Ras Al Khaimah, United Arab Emirates, 5 November 2026.

Gemini and Google's AI Mode are fetching web pages under two user agent tokens that do not appear anywhere in Google's crawler documentation. In a test published on 3 November, technical SEO consultant Dave Smart of Tame the Bots asked both products to fetch a page on his site and found Gemini's request in his logs with the bare token Google, and AI Mode's with Google-Lens. We checked Google's five crawler reference pages and its three published IP files for user-triggered fetchers to see what site owners can use to recognise this traffic today, and where the documented verification method breaks down.

The short answer: the tokens are undocumented, the IP ranges and reverse DNS masks that cover them are documented, and one of Google's three IP files is much harder to verify by reverse DNS than the other two.

What changed

Google's list of user-triggered fetchers names nine requestors, from Feedfetcher to Google-Agent, and says the list is not complete: it covers only the requestors most likely to show up in log files. It describes these fetchers as initiated by users inside a Google product and notes that "these fetchers generally ignore robots.txt rules". The page was last updated on 19 August 2026.

According to the Tame the Bots write-up, the Gemini and AI Mode requests fetched only the main HTML document, with no images, scripts or stylesheets, so the page was not rendered. Reverse DNS on the requests the author checked pointed to google-proxy-***-***-***-***.google.com hostnames, and some, but not all, of the addresses sat inside Google's user-triggered-fetchers-google.json file. The author also reported that one client's firewall had flagged the short Google user agent as suspicious, and advised against blocking either token.

Google has not commented on the tokens. Nothing on its crawler pages says which user agent Gemini or AI Mode uses when a person asks either product to read a URL.

Our check of Google's documentation

On 5 November we downloaded five pages of Google's crawling documentation and searched each for the strings "Lens", "Gemini" and "AI Mode". None of the five mentions Google-Lens, and none lists a fetcher identified by the single word Google.

Google documentation page	Last updated	Mentions Google-Lens	Gemini references
User-triggered fetchers	19 Aug 2026	No	Gemini Notebook fetcher only
Common crawlers	30 Oct 2026	No	Google-Extended (training and grounding)
Special-case crawlers	17 Sep 2026	No	None
Crawler overview	12 Jun 2026	No	None
Verify requests from Google	20 Mar 2026	No	None

The only Gemini product with a documented fetcher is Gemini Notebook, which identifies itself as Google-GeminiNotebook (it replaced Google-NotebookLM, which Google says was supported until August 2026). The Google-Agent token covers agents that browse and act for users, not a chat answer that reads one page. The Google-Extended token on the common crawlers page is a robots.txt control for training and grounding, not a user agent that appears in logs.

Our check of the IP files

Google publishes three IP files for user-triggered traffic: user-triggered-fetchers.json, user-triggered-fetchers-google.json and user-triggered-agents.json. All three carried a creation time of 4 November 2026 when we downloaded them. Between them they list 798 IPv4 prefixes: 534 in the first file, 252 in the second and 12 in the third. Every IPv4 prefix in the first two files is a /27, a block of 32 addresses.

Google's documentation says the reverse DNS of these fetchers matches either ***-***-***-***.gae.googleusercontent.com or google-proxy-***-***-***-***.google.com, depending on whether the fetcher is Google owned or user owned. We looked up the PTR record for the first host in every IPv4 prefix across the three files.

The results split cleanly. All 252 prefixes in the Google file and all 12 in the agents file resolved to google-proxy hostnames. In the general file, 397 of 534 prefixes (74%) returned no PTR record at all, 95 (18%) resolved to gae.googleusercontent.com, and 42 (8%) resolved to bc.googleusercontent.com, a Google Cloud hostname that matches neither documented mask.

One detail is worth flagging, and it is our reading rather than anything Google has said. Google's sentence pairs Google-owned fetchers with the gae mask and user-owned fetchers with google-proxy. In our lookups, the file whose name points to Google's own fetchers resolved only to google-proxy. Google does not state which file holds which kind of fetcher, so the safest approach is to accept either documented mask and confirm the IP against the files.

What this means for SEO teams

If you have a rule that only lets through requests whose user agent contains a known token such as Googlebot or Google-Agent, Gemini and AI Mode page reads may be landing in your blocked or challenged bucket. Because these fetchers generally ignore robots.txt, a robots.txt group for Google-Lens is unlikely to do anything, and nothing Google has published says whether Google-Extended affects these fetches. We have not tested that, so treat it as unknown.

Here is a practical way to handle the traffic:

1. Search your logs for the exact tokens. Look for user agents equal to Google or containing Google-Lens. A substring search for Google also returns every other Google crawler, so match the whole user agent string exactly.
2. Verify by IP first, not by reverse DNS. Load all three user-triggered JSON files and check whether the requesting address falls inside a listed prefix. In the general file, three quarters of prefixes have no PTR record, so a reverse DNS check alone would wrongly flag genuine Google traffic from those ranges.
3. Use reverse DNS as a second check where it exists. For addresses in the Google and agents files, a forward-confirmed google-proxy-*.google.com hostname is a strong signal. Google's verification guide describes the forward-confirmation step.
4. Refresh the files on a schedule. All three were regenerated on 4 November. Cached copies go stale, which is how legitimate fetches end up blocked.
5. Do not treat the user agent as proof. Google's own page warns that the string can be spoofed, and a token as generic as Google is an easy one to fake.

Our earlier work on this beat covers the same verification problem from other angles: our analysis of Google's 2,181 crawler IP prefixes, our test of Google IPs with no reverse DNS and fake Googlebots, and our guide to spotting Google-Agent in server logs.

Context

The finding fits a pattern of Google's AI products reaching the open web through user-triggered requests rather than through Googlebot. Gemini Notebook got its own documented token, and Google-Agent has its own token, documentation page and IP file, plus an experimental Web Bot Auth identity at https://agent.bot.goog listed on the fetchers page. Gemini chat and AI Mode, by Smart's account, do neither: one sends a single generic word and the other borrows the name of Google's visual search product. The Tame the Bots post itself notes that Google-Lens may be used for more than AI Mode.

The same write-up compares the behaviour to user-triggered agents such as OpenAI's ChatGPT-User. The difference is that OpenAI's token carries the product name, so it can be matched without guesswork.

Methodology

On 5 November 2026 we downloaded five Google crawling documentation pages (linked in the table above), stripped the HTML and searched the text for "Lens", "Gemini" and "AI Mode". The same day we downloaded Google's three user-triggered IP files, counted IPv4 and IPv6 prefixes and prefix lengths, and checked the files for overlap (none of the IPv4 prefixes in the general file appear in the Google file). For each of the 798 IPv4 prefixes we requested the PTR record of the first host address (the network address plus one) through a public DNS-over-HTTPS resolver, retrying once on error, and grouped the answers by hostname pattern. We did not test IPv6 prefixes, did not check every address in each block, and did not trigger Gemini or AI Mode fetches ourselves; the user agent observations come from the Tame the Bots test.

Conclusion

Gemini and AI Mode appear to read pages under two tokens Google has not documented, so any allowlist built from Google's crawler pages will not recognise them by name. Google's IP files and DNS masks are the documented way to confirm the traffic, but the general fetcher file is mostly unverifiable by reverse DNS, which makes the IP list the check that matters.

Until Google adds these tokens to its fetcher list, the safest course is to verify by IP range, avoid blocking on the user agent alone, and keep an eye on whether Google documents a named fetcher for its AI products.

Questions and answers

What user agent does Gemini use when it fetches a page?

In a test published on 3 November 2026 by Dave Smart of Tame the Bots, a page fetch requested in Gemini appeared in server logs with the bare user agent token Google. Google has not documented this token on its crawler pages.

What user agent does Google AI Mode use to fetch pages?

The same test logged an AI Mode fetch with the token Google-Lens. None of the five Google crawling documentation pages we checked on 5 November 2026 mentions Google-Lens.

Do Gemini and AI Mode fetches follow robots.txt?

Google has not said. Its user-triggered fetchers page says these fetchers generally ignore robots.txt rules because a user requested the fetch, but it does not list the Gemini or AI Mode tokens. We have not tested whether robots.txt or Google-Extended affects them.

How can I verify that a Gemini or AI Mode request really came from Google?

Check the requesting IP against Google's three user-triggered IP files (user-triggered-fetchers.json, user-triggered-fetchers-google.json and user-triggered-agents.json), then use forward-confirmed reverse DNS as a second check where a record exists. In our lookups, 397 of the 534 IPv4 prefixes in user-triggered-fetchers.json had no PTR record, so reverse DNS alone is not enough.

###