That gap is not static. Promode Nedungadi, Chief Technology Officer, said the architectural and algorithmic trends driving AI are making the network problem harder, not easier. Techniques like mixture-of-experts models and the disaggregation of inference into separate prefill and decode stages all require more data movement.
“Every one of those requires more data to be moved around,” he said. “The amount of data being moved per token is growing.” The scale challenge also has more than one dimension. Perkins described three: scale-up, which refers to interconnecting GPUs within a single training domain; scale-out, which covers the broader cluster fabric; and what he called scale-across, an emerging requirement that standards bodies are beginning to address.
“We think that scale-across is quite interesting as well,” Perkins said.
Architecture: silicon, packaging, and software
A key differentiator for Eridu will come from silicon.
“There’s no doubt that we are developing our own silicon. We’re developing the most advanced silicon in the networking sector, bar none, period, and that’s absolutely necessary,” Perkins said. “You don’t get to an order-of-magnitude higher scale using off-the-shelf silicon.”
Eridu has a partnership with TSMC for process technology and advanced system integration. Perkins said TSMC sees the networking bottleneck as tied directly to its own business. The silicon approach is likely to benefit from chiplet-based architecture and advanced packaging.
“We believe you need to be on a different technology arc than what the mainstream technology is,” Omar Hassen, Chief Product Officer, told Network World. “In terms of things like advanced packaging, you’ve got to take advantage of everything you can from chiplet-based architecture, clean-sheet design, and advanced packaging. We believe we’re on the right technology arc that can take us beyond what the existing incumbents are doing.”
Fundamentally, Eridu’s approach is an attempt to break through the architectural ceiling facing incumbents to get an order-of-magnitude improvement.
“You have to start from a different place, otherwise you can hit the ceiling if you just use the same architecture as everybody else,” Hassen said.
Eridu at a glance:
- Founded: 2023
- Funding: $200 million.
- Investors: Socratic Partners, John Doerr, Hudson River Trading, Capricorn Investment Group, and Matter Venture Partners.
- Headquarters: Saratoga, California
- CEO: Drew Perkins
- What they do: AI networking
