Google’s expanded candidate set and the selection crisis

Google’s expanded candidate set signals a deeper shift in how search systems evaluate content. As AI systems process larger pools of information, visibility increasingly depends on verification, relationships, and trust signals instead of traditional keyword targeting alone.

That shift is pushing SEO beyond retrieval and ranking mechanics toward something closer to forensic architecture — systems designed to help machines verify and trust information at scale.

Search Engine Land recently published an article about Google’s expanded candidate set. Reading it, I felt a massive wave of relief and a shot of adrenaline. It confirmed that the rabbit hole I’ve been digging into for the last five years isn’t just a personal obsession. It’s exactly where the digital ecosystem is heading.

For over 30 years, I’ve worked to meet today’s requirements in ways that also serve tomorrow’s. That experience teaches you to recognize patterns early and make decisions that aren’t just tasks, but stepping stones toward where the industry is heading next.

The evolution: From library clerk to forensic investigator

To understand why the “selection crisis” is happening, you first have to distinguish between a crawler and an AI agent.

In the early days, Googlebot was a mechanical fetcher. It followed strict, rules-based logic: find a link, download the page, and index the words. It didn’t “think” about your content. It simply recorded it. It was a library clerk. 

The evolution toward intelligence

Over the last decade, that library clerk effectively went back to school, earned a PhD in linguistics, and became a forensic investigator:

• The thinking layer (2015): RankBrain allowed the system to infer intent for queries it had never seen before.
• The contextual shift (2019): BERT allowed the crawler to understand relationships between words, moving search beyond keywords and toward information gain (IG).
• The generative agent leap (2023–present): With Gemini and AI Overviews, the system now reads hundreds of pages simultaneously to synthesize a single, unique answer.

The OpenAI catalyst and the selection crisis

The arrival of ChatGPT in late 2022 accelerated the shift toward answer engines. Users stopped asking for recipes and started demanding meal plans.

This created what I call the “selection crisis.” Because an AI agent delivers a single, cohesive answer, it must select which facts to include and which to ignore. That leveled the playing field. A natural language interface allowed anyone to access high-quality information, regardless of their search literacy.

For those of us in the trenches, this validated that information gain and atomic facts are the only currencies that matter. If an AI system can summarize your 2,000-word page in two sentences, the other 1,980 words become context debt — unnecessary weight the machine will eventually ignore.

A 30-year journey toward information gain and atomic facts

This conclusion didn’t arrive through a “magic wand” moment. It came from 30 years of identifying zombie facts, or outdated and incorrect information masquerading as truth, along with extensive trial and error.

My path began in high-stakes industries: online pharmacies and regulated iGaming.

In these sectors, trust isn’t a buzzword. It’s the only way to stay in business. Back in 2018, I started digging into semantic triples and the knowledge graph. I realized the crawler didn’t just need to find us. It needed a logical map to understand us.

The commodity crisis

Later, while managing eight ecommerce sites selling identical products at identical prices, I ran into the commodity crisis. If everyone says the same thing, the answer engine has no logical reason to choose you. You must provide the atomic fact: the unique, verified piece of information only you can provide.

I spent a decade building tools to address the gaps I found:

• The E-E-A-T engine: A 500-point forensic audit system based on Google’s Search Quality Rater Guidelines.
• The atomic sandwich: A three-layer architecture (atomic fact, information gain, structural layer) that treats content like a technical blueprint.
• The forensic IG evaluator: A tool to measure whether your content actually adds something new to the conversation.

Eventually, the toolbelt became too heavy. The problems — context debt and the trust gap — required a more unified approach.

That led me to develop a framework designed to bridge high-level engineering and kitchen-table comprehension.

Building trust in the answer engine landscape

A recent forensic audit I conducted across 28 digital entities confirmed the selection crisis has reached the general web. As Search Engine Land reported, Google is now evaluating a much larger pool of pages for rankings.

In a field of hundreds, the machine is no longer asking who has the best keywords. It’s asking, “Who can I verify?” Rankings alone are no longer enough. You need to become a source AI systems can verify and trust.

To solve this, I use three pillars of forensic engineering:

• Pillar 1 – Cryptographic authority: In a deepfake economy, I use the JSON Web Signature (JWS) standard (RFC 7515) to sign an entity’s manifest. Think of it as a fast pass through the candidate set because it enables instant verification.
• Pillar 2 – The semantic graph: AI thinks in relationships, not paragraphs. Using W3C RDF-star standards, I export audits as structured knowledge graphs. This minimizes translation error when AI systems read your data.
• Pillar 3 – Regulatory alignment: I mapped the architecture to the EU AI Act (Regulation 2024/1689). This protects digital GDP against legislative shifts. If you want to be visible globally, you have to meet global requirements.

The answer engine changes what gets selected

The expansion of the candidate set shows how search engines are becoming answer engines. Visibility increasingly depends on whether AI systems can verify, connect, and trust the information associated with your entity.

That shift changes the job of SEO. It’s no longer just about retrieval and rankings. It’s increasingly about building systems that help machines understand relationships, validate information, and establish trust at scale.

The frameworks and standards required to support that shift already exist in the public domain. The challenge now is learning how to assemble them into a reliable foundation for visibility in AI-driven search.