OpenAI’s automated crawl activity is estimated to have roughly tripled after the launch of GPT-5, according to a new analysis from Botify and guest author Chris Long.
In Botify’s dataset, OpenAI’s search crawler is now generating more log events than its training crawler. That’s a reversal from the period before GPT-5.
Long, co-founder of the SEO consultancy Nectiv, analyzed roughly 7 billion OpenAI-bot log events from Botify’s enterprise client dataset spanning November 2024 through March 2026.
What The Data Shows
Two of the three OpenAI user agents Botify measured saw activity spike around the GPT-5 launch.
OAI-SearchBot, which retrieves content when ChatGPT performs web searches, recorded about 3.5x more events after August 2025. That works out to roughly 2.2 billion additional events in Botify’s dataset.
GPTBot, which collects training data, recorded about 2.9x more events over the same period. That is another 1.8 billion events.
The third user agent, ChatGPT-User, moved in the opposite direction. Long reports a 28% drop in ChatGPT-User log events between December 2025 and March 2026. ChatGPT-User fires when a ChatGPT session fetches a page on behalf of a user, so the drop measures logged user-initiated fetches rather than ChatGPT usage overall.
Long offers two possible readings. One is that fewer sessions may be triggering real-time page fetches. The other, suggested by Botify’s team, is that OpenAI may be relying more on stored or indexed resources, reducing the need to fetch pages in real time. Long does not pick between them.
Search Bot Now Outpaces Training Bot
Before GPT-5, OAI-SearchBot and GPTBot ran at roughly even volumes in Botify’s dataset, with a ratio of about 0.95 search events per training event. After GPT-5, that ratio rose to about 1.14.
The pattern lines up with what Dan Petrovic wrote in August 2025 about GPT-5, arguing that OpenAI was sourcing more answers from live search than from trained memory. Botify’s data is consistent with that read.
Industry Breakdown
The post-GPT-5 search bot increases varied by industry. Healthcare sites saw about 740% more OAI-SearchBot activity after launch; Media and Publishing, 702%; and Marketplaces, Software, and Retail, 190-216%.
Travel sites had the smallest rise at 30%. The search and training balance also varies. Long reports a +256% OAI-SearchBot to GPTBot crawl difference for Media/Publishing, the largest gap. Software and Internet lean toward search, Healthcare and Retail favor training, with -50% and -33%. GPTBot is more active overall.
Botify and Long suggest OpenAI routes prompt types differently: news inquiries trigger live search, health and product queries rely on trained knowledge.
How OpenAI’s Crawl Compares To Google’s
Even after tripling, OpenAI’s crawl activity is much smaller than Google’s.
In Botify’s most recent 30-day window, Googlebot registered 18.2 billion events, compared with 887 million events from OpenAI’s crawlers combined. That puts OpenAI at about 4% of Google’s crawl volume.
A year earlier, the same comparison was 15 billion Google events to 207 million OpenAI events, or about 1.38%. The gap is closing, though Google’s crawl is still roughly 20 times larger in absolute terms.
Bingbot registered about 5.49 billion events in the most recent window, putting OpenAI at roughly 14% of Bing.
Methodology & Commercial Context
The dataset is Botify’s, covering enterprise clients in retail, ecommerce, technology, publishing, travel, and marketplaces. The analysis was conducted by Long as a guest author on Botify’s blog.
For transparency, Botify sells log file analysis and AI bot management software, and the post promotes a follow-up webinar and a product demo.
The dataset skews toward large enterprise websites rather than a representative cross-section of the web.
Why This Matters
In Botify’s dataset, OAI-SearchBot now generates more log events than GPTBot. Sites that block only GPTBot are not blocking the bot OpenAI says is used to surface websites in ChatGPT search answers.
Sites that block OAI-SearchBot may be excluding themselves from ChatGPT search answers.
How This Fits With Other Reports
Botify’s findings line up with patterns other vendors have reported. An Alli AI analysis covered earlier this month found OpenAI’s ChatGPT-User made 3.6x more requests than Googlebot in a smaller WordPress-heavy sample. A Hostinger analysis found OAI-SearchBot’s website coverage reaching 55% while GPTBot coverage fell. Akamai’s recent bot traffic report showed OpenAI leading AI bot traffic to publishing sites.
The reports suggest that AI training crawls and AI search crawls need to be measured separately, especially as OAI-SearchBot activity grows.
