Google and OpenAI's API Recovered Only 28% of the Pages ChatGPT Cited, arXiv Study Finds

Ras Al Khaimah, United Arab Emirates, 19 October 2026.

Testing a brand's ChatGPT visibility through Google rankings or OpenAI's developer API misses most of the exact pages ChatGPT cites, according to a preprint posted to arXiv on 11 October 2026. In "Understanding ChatGPT Citations: Evidence from Production Web Search Across Model Generations", researchers from Xi'an Jiaotong University and the National University of Singapore report that Google Search and the OpenAI Responses API together recovered 28.3% of the URLs cited by GPT-5.3 Instant and 26.2% of those cited by GPT-5.5 Instant. At the hostname level, the same proxies recovered 50.3% and 48.7%.

The authors are Yunxuan Jiang, Shiqi Zhang, Anqi Xing, Xiangyu Chang and Xiaokui Xiao. Search Engine Journal reported on the paper on 18 October. The paper has not been peer reviewed.

Key findings

• Proxies find sites, not pages. Combined Google and API data matched roughly a quarter of cited URLs but about half of cited hostnames.
• Google alone is a weak proxy. Five pages of Google organic results covered 9.4% of GPT-5.3 Instant's cited URLs and 6.4% of GPT-5.5 Instant's. At the hostname level the figures were 26.1% and 22.8%.
• The API's own citations are the closest page-level match. Pages cited by the Responses API had the highest precision against ChatGPT's citations: 16.4% for GPT-5.3 and 23.3% for GPT-5.5.
• ChatGPT rewrites every query. In the observed GPT-5.3 Instant and GPT-5.5 Instant runs, no search used the user's exact wording. GPT-5.2 Instant did in 5.5% of runs.
• Chinese questions were often searched in English. 67.0% of GPT-5.3 Instant runs and 69.6% of GPT-5.5 Instant runs on Chinese questions issued English-only queries.
• Results churn between runs. 40.8% of GPT-5.3 Instant's candidate URLs and 44.5% of GPT-5.5 Instant's appeared in only one of 20 runs. Only 9.4% and 7.7% formed a stable core.

Proxy	GPT-5.3 Instant: cited URLs recovered	GPT-5.5 Instant: cited URLs recovered	GPT-5.3 Instant: cited hostnames recovered	GPT-5.5 Instant: cited hostnames recovered
Google organic results	9.4%	6.4%	26.1%	22.8%
Responses API candidates	22.9%	22.5%	38.3%	38.1%
Responses API citations	12.3%	13.0%	23.3%	23.9%
Google plus API candidates	28.3%	26.2%	50.3%	48.7%

How the study worked

The team sampled 80 questions from Search Arena, a crowd-sourced dataset of about 24,000 real interactions with search-enabled chatbots. The set was split evenly between Chinese and English and across four intents: factual lookup, information synthesis, analysis and recommendation.

Between 29 March and 29 July 2026, the researchers ran each question 20 times on GPT-5.3 Instant and GPT-5.5 Instant through signed-in ChatGPT web sessions, capturing the streamed data that shows each search query, the pages retrieved and the pages cited. GPT-5.2 Instant and GPT-5.4 Thinking were also observed, on fewer queries, because they were retired before collection finished.

For comparison, they pulled five pages of Google organic results per query through a commercial SERP data service (43.2 results per query on average) and queried OpenAI's Responses API with web search enabled. Each Responses API answer carried about 26 to 28 candidate pages and five to six citations. ChatGPT traces and both proxies were collected within the same three-day window for each query.

Retrieval depth differs by model

The Instant settings mostly searched once per answer. GPT-5.4 Thinking averaged 11.82 queries and 3.20 retrieval rounds per run, with 71.3% of runs using more than one round. GPT-5.5 Instant issued more than one query in 31.3% of runs, against 6.2% for GPT-5.3 Instant, while staying single-round in almost all cases.

GPT-5.5 Instant also pulled a wider pool: 132.2 distinct URLs per query across 20 runs, compared with 96.4 for GPT-5.3 Instant. The authors found that pages retrieved in more runs were more likely to be cited, and that a model combining external signals such as rank, API exposure frequency and snippet length predicted citations with AUC scores above 0.74 for both models. That prediction only works for pages the proxies capture in the first place.

Context

The paper adds to a growing body of work showing that AI answer engines do not simply cite the top organic results. Our earlier coverage of a large ChatGPT fan-out study found that ChatGPT searched the web in a minority of answers. This preprint looks only at answers where search ran, and shows how far that search diverges from what an SEO team can observe from outside.

The paper covers models up to GPT-5.5, so results for newer ChatGPT models may differ. The authors note that two collection sessions switched models partway through, and recommend logging which model produced each answer.

What this means for SEO teams

1. Report AI visibility at two levels. Track whether your domain is cited as well as which URL. The proxies matched sites about twice as often as pages.
2. Repeat prompts. With more than 40% of candidate pages appearing in only one of 20 runs, a single check per prompt is not a measurement.
3. Do not treat Google rank as a stand-in. Google results alone covered fewer than one in ten cited pages for either model.
4. Ask tracking vendors where their data comes from. API-based and app-based observations diverged in this study, so check which one a tool uses before comparing its numbers with what users see.
5. Write for rewritten and translated queries. ChatGPT never searched the exact question in recent Instant runs and often searched Chinese prompts in English, so pages should cover the underlying intent rather than one phrasing.

Conclusion

The study gives SEO teams a measured gap rather than an anecdote: two outside views of ChatGPT search, Google rankings and the OpenAI API, recovered only about a quarter of the pages ChatGPT actually cited, and about half of the sites. Until OpenAI exposes citation data directly, site-level tracking with repeated runs is the more reliable way to read AI visibility.

Questions and answers

What did the study measure?

It recorded which searches ChatGPT ran, which pages it retrieved and which it cited across four ChatGPT web settings from GPT-5.2 to GPT-5.5, then compared those citations with Google organic results and OpenAI Responses API output for the same 80 queries.

How well did Google results predict ChatGPT citations?

On their own, Google organic results covered 9.4% of the pages cited by GPT-5.3 Instant and 6.4% for GPT-5.5 Instant. At the site level the figures were 26.1% and 22.8%.

Did ChatGPT search with the user's exact question?

In the GPT-5.3 Instant and GPT-5.5 Instant runs the authors observed, none of the searches matched the original question exactly. GPT-5.2 Instant did so in 5.5% of runs.

Has the paper been peer reviewed?

No. It is an arXiv preprint (arXiv:2610.15382) submitted on 11 October 2026. arXiv does not peer review submissions.

###