Pichai Says Agents Will Use Pages and Protocols; 6 of 699 Top Sites Publish Agent Files

Ras Al Khaimah, United Arab Emirates, 25 October 2026.

Google CEO Sundar Pichai says AI agents will reach websites in two ways: by browsing pages the way a person does, and by calling sites directly through agent protocols. He made the point in an interview on Alex Heath's Sources podcast, published on YouTube on 21 October, in a segment titled "How AI agents change the web". We checked what that means in practice: Google's own documentation already covers the browsing path, and a crawl of 1,000 top domains on 25 October found that only 6 of the 699 that answered publish a file for the protocol path.

What Pichai said

According to Search Engine Journal's account of the interview, Pichai said both approaches will develop side by side. He described browser-based computer use as natural because it mirrors what a user would do and keeps the user in the loop, and he described protocols as the more scalable route, though one that will take longer to mature. He named MCP (Model Context Protocol) and UCP (Universal Commerce Protocol) as examples with momentum and expected a messy period before standards settle.

Path 1: agents that browse your pages

Google already documents the agent that does this. Its list of user-triggered fetchers includes Google-Agent, which Google says is used by agents hosted on its infrastructure to navigate the web and perform actions on a user's request. Three details on that page matter for SEO and engineering teams:

• Robots.txt is not the control. Google says user-triggered fetchers generally ignore robots.txt rules, because a user asked for the fetch. Blocking Googlebot does not block Google-Agent.
• It has its own user agent string, with mobile and desktop versions that contain compatible; Google-Agent, so it can be separated from Googlebot in logs.
• It has its own IP list. Google-Agent uses the ranges in user-triggered-agents.json. When we downloaded it, the file carried a creation time of 23 October 2026 and listed 20 prefixes (12 IPv4, 8 IPv6). A copy at Google's older ipranges path still carried a 3 March timestamp and only 4 prefixes, so verification scripts pointed at the old location are out of date. We flagged the same split for crawler ranges in our analysis of Google's 2,181 crawler IP prefixes.

Google is also testing cryptographic identity for these agents. Its Web Bot Auth guide, last updated on 13 October, says a subset of Google-Agent requests are signed and identify as https://agent.bot.goog, with public keys published in a key directory. When we fetched that directory it returned five Ed25519 keys with a one-hour cache lifetime. Google stresses the protocol is experimental and that sites should keep IP and reverse DNS checks as the fallback, since not every request is signed.

Worked example: find Google-Agent in your logs

A first pass needs nothing more than the user agent token and Google's IP file:

grep "Google-Agent" access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head

Then check each IP against the prefixes in user-triggered-agents.json (a short Python script with the ipaddress module does this in a few lines). Requests that claim to be Google-Agent from outside those ranges are spoofed. If you run bot management through a CDN or WAF, Google says major bot-detection services, CDNs and WAFs support Web Bot Auth, so your provider may verify signatures for you. Cloudflare, for example, plans to score verified bots on robots.txt and 429 compliance, as we covered in our report on Cloudflare's verified bot trust bands.

Path 2: agents that call your site through a protocol

The protocol path depends on a site publishing a machine-readable file that tells an agent what it can do. Two public specifications define where those files live:

• UCP. Google introduced the Universal Commerce Protocol in a Google Developers Blog post dated 11 January 2026, written with Shopify, Etsy, Wayfair, Target and Walmart and endorsed by more than 20 other companies. Businesses publish a JSON profile at /.well-known/ucp that lists their commerce capabilities, endpoints and payment handlers. The UCP specification lets those services run over REST, MCP or A2A. Google says its reference implementation lets people buy from eligible businesses in AI Mode and the Gemini app. Google's agentic shopping push has continued since, most recently with Universal Cart's US launch.
• A2A. The Agent2Agent specification says clients can discover an agent through an Agent Card at /.well-known/agent-card.json, which describes the agent's skills, endpoints and authentication.

What our check found

We requested each of these files, plus the Web Bot Auth key directory and llms.txt for comparison, from the 1,000 most popular domains in the Tranco research list. Of the 699 domains that answered an HTTPS request, only six served a valid UCP profile or A2A agent card: forter.com, hp.com, ikea.com, wyzecam.com (which redirects to wyze.com), att.com and cloudflare.com, whose card sits at the older agent.json path. By contrast, 109 of the same 699 domains served an llms.txt file, a file Google says Search can ignore, as our llms.txt adoption study reported.

Every domain with at least one agent-facing file, by popularity rank:

Tranco rank	Domain	UCP profile	A2A agent card	Web Bot Auth keys
2	cloudflare.com	No	Yes (agent.json)	No
113	chatgpt.com	No	No	Yes
173	shopify.com	No	No	Yes
236	forter.com	Yes	Yes (both paths)	Yes
288	hp.com	Yes	No	No
355	checkpoint.com	No	No	Yes
430	substack.com	No	No	Yes
587	klaviyo.com	No	No	Yes
638	ikea.com	Yes	No	No
656	meta.com	No	No	Yes
832	att.com	No	Yes	No
914	wyzecam.com	Yes	No	No

Three patterns stand out:

• Key directories belong to agent operators, not destination sites. ChatGPT, Meta and Shopify publish Web Bot Auth key directories, which let other sites verify requests their bots sign. That is the identity layer of the browsing path, and it is separate from a site inviting agents in.
• UCP's co-developers do not all publish the profile on their main domains. walmart.com and target.com both loaded their homepages for us but returned 404 at /.well-known/ucp; etsy.com also returned 404. They may connect to Google's agents through other channels, but the public discovery file is not there.
• Shopify stores stand out. wyze.com's profile points back to a myshopify.com store. We also tested 10 well-known consumer brands ranked outside the top 1,000: allbirds.com, gymshark.com, fashionnova.com, brooklinen.com, skims.com, kyliecosmetics.com and glossier.com all served a UCP profile (version 2026-08-25) that references a myshopify.com store; bombas.com rate-limited us, and ruggable.com and heinz.com returned 404. Shopify's UCP page describes the protocol as co-developed with Google.

What this means for SEO teams

1. Separate agent traffic from crawler traffic. Add Google-Agent to log segmentation and analytics bot filters, and verify it against the current user-triggered-agents.json, not the legacy file. Decide whether your WAF challenges it, because a blocked agent cannot complete the task a user gave it.
2. Do not rely on robots.txt for agents. Google says its user-triggered fetchers generally ignore it. If you need to limit agent access, it has to happen at the server, CDN or authentication layer.
3. Make key pages work for software. A browsing agent needs what accessibility tools need: labelled forms and buttons, prices and stock in the HTML, and checkouts that do not depend on hover states or pop-ups.
4. Ecommerce sites on Shopify should check their profile. Request /.well-known/ucp on your own domain. If it returns JSON, confirm the listed capabilities and payment handlers match what you sell. If you run a headless storefront or a custom domain setup, check that the path is not being swallowed by your front end.
5. Non-commerce sites should watch, not rush. UCP is a commerce standard, and A2A agent cards make sense only if you run an agent others should call. Pichai himself described the protocol route as slower to arrive. For most content sites the browsing path is the one that matters now.

Methodology

On 25 October 2026, between 05:21 and 05:28 UTC, we requested six paths over HTTPS from each of the top 1,000 domains in Tranco list GQ32K: the homepage, /.well-known/ucp, /.well-known/agent-card.json, /.well-known/agent.json, /.well-known/http-message-signatures-directory and /llms.txt. Redirects were followed (up to five). Domains whose bare hostname failed were retried with a www prefix. 699 domains returned an HTTP response for the homepage; the rest (mostly CDN, ad and API hostnames) were excluded. A UCP file counted only if it returned status 200 with JSON containing a ucp object; an agent card only if it returned JSON with a name field; a key directory only if it returned the signatures-directory content type or JSON with keys; llms.txt only if it returned non-HTML text. Some sites block automated requests, so these counts are a floor. The Shopify brand sample was chosen by us, not drawn at random, and is not part of the 699-domain count. Google's IP file and Web Bot Auth key directory were downloaded on the same day.

Conclusion

Pichai's two paths are already visible on the web, but they are at very different stages. Google has documented its browsing agent, its IP ranges and an experimental signing scheme, so site owners can identify and manage that traffic today. The protocol path is still thin: outside Shopify-powered stores and a handful of large brands, almost no top site publishes a UCP profile or an A2A agent card.

For most SEO teams, the practical work this quarter is on the browsing side: recognising Google-Agent, verifying it correctly and making sure key pages and forms work for software acting on a user's behalf.

Questions and answers

What is Google-Agent?

Google-Agent is a user-triggered fetcher that Google says is used by agents hosted on its infrastructure to navigate the web and perform actions on a user's request. It has its own user agent strings and uses the IP ranges in Google's user-triggered-agents.json file.

Does robots.txt block Google-Agent?

Not reliably. Google's documentation says user-triggered fetchers generally ignore robots.txt rules because the fetch was requested by a user. Access has to be managed at the server, CDN or authentication layer.

Where does a site publish a UCP profile?

Under the Universal Commerce Protocol specification, businesses publish a JSON profile at /.well-known/ucp listing their commerce capabilities, endpoints and payment handlers. Google introduced UCP on 11 January 2026, developed with Shopify, Etsy, Wayfair, Target and Walmart.

How many top sites publish agent protocol files?

In our 25 October 2026 crawl of the top 1,000 Tranco domains, 699 answered, and 6 served a valid UCP profile or A2A agent card: forter.com, hp.com, ikea.com, wyzecam.com, att.com and cloudflare.com. By comparison, 109 served an llms.txt file.

Is Google's Web Bot Auth signing live for all requests?

No. Google describes it as experimental: only a subset of Google-Agent requests are signed, identifying as https://agent.bot.goog, and Google recommends keeping IP, reverse DNS and user agent checks as a fallback.

###