A randomized field experiment finds Google’s AI Overviews reduce organic clicks to external websites by 38% on queries where they appear, while self-reported search satisfaction stays nearly unchanged when the summaries are removed.
The working paper by researchers at the Indian School of Business and Carnegie Mellon University was posted to SSRN this month. Authors Saharsh Agarwal and Ananya Sen describe it as the first randomized field experiment to test how AI Overviews affect user behavior in a real browsing environment.
How The Experiment Worked
Agarwal and Sen built a Chrome extension that randomly assigned 1,065 U.S. participants to one of three groups. People were recruited from Prolific and used Chrome on desktop. They also had to meet minimum browsing-history thresholds, so the sample reflects active desktop Chrome users rather than all Google users.
The control group saw Google Search normally. A “Hide AIO” group had the extension remove AI Overviews in real time. A third group was redirected to Google’s AI Mode for all searches. The study ran for two weeks per participant between January and February 2026.
Researchers pre-registered the experiment with the AEA RCT Registry before data collection. Over 95% of users in the Hide AIO group did not detect any changes during the study.
What The Researchers Found
AI Overviews appeared on 42% of queries, and removing them increased outbound clicks from 0.38 to 0.61 per search. They reduced outbound organic clicks by 38% on triggered queries, with zero-click search rising from 54% to 72%.
Effects were strongest when AI Overviews appeared at the top of the page, which occurred 85% of the time. Removing top-position AI Overviews nearly doubled outbound clicks, but lower ones had no effect.
Sponsored clicks and search frequency remained steady, indicating substitution between AI Overviews and organic visits.
The User Experience Finding
The endline survey used a 1-to-5 Likert scale to assess participants’ search experience. Responses from the control and Hide AIO groups were nearly identical across all measures, including satisfaction, information quality, and ease of finding information.
The researchers wrote that AI Overviews “divert traffic away from publishers without delivering measurable improvements in user experience.”
How AI Mode Compared
Participants directed to AI Mode had lower outbound click rates, higher zero-click rates, and lower satisfaction at endline compared to other groups.
The authors note that these results are exploratory, as higher attrition, some uninstalling of the extension, or finding workarounds may have influenced the outcomes.
Why This Matters
Independent measurements of the impact of AI Overviews on traffic have mostly been correlational. Pew Research found users click 8% of the time with AI Overviews, compared to 15% without. Ahrefs analyzed GSC data and reported a 58% drop in click-through rate for top-ranking pages when AI Overviews appeared.
This experiment adds a different approach by randomly assigning users to see AI Overviews or not, isolating the causal effect.
Google VP Liz Reid claims AI Overviews cut “bounce clicks,’ but provides no data backing the user-benefit side. The Agarwal and Sen paper tested a related question with a randomized design, finding no measurable change in satisfaction or ease of finding info.
Looking Ahead
The paper is a draft on SSRN and is not peer-reviewed. Authors will add more results, and we will provide an update if findings change.
