Network and Fabrics

What it is

Three Networks Inside Every AI Cluster

• Scale-up network: joins GPUs inside a rack or pod so they share memory. Nvidia's NVLink, AWS's NeuronLink, Google's ICI and Huawei's UnifiedBus. An NVL72 rack holds about 5,000 copper cables, more than 2 miles of them.
• Back-end network: joins racks for training, at 800 Gbps per GPU today, on InfiniBand or AI-tuned Ethernet.
• Front-end network: ordinary data center Ethernet for storage, data ingest and serving users.
• Between sites: long-haul fiber. Microsoft laid about 120,000 miles of new US fiber in a year for its AI WAN; Lumen signed $5 billion of AI fiber deals.

Ethernet vs. InfiniBand

Ethernet Has Overtaken InfiniBand

• Share: Ethernet was about two-thirds of AI back-end switch sales in early 2026, and its sales more than doubled in a year (Dell'Oro). InfiniBand sales also tripled as older sites upgraded to Nvidia's 800G Quantum-X800.
• Vendors: in AI back-end Ethernet, Celestica ranked first, Nvidia second, Arista third and Cisco fourth. Cisco expects about $7.5 billion of AI infrastructure revenue in its fiscal 2027.
• Standards: the Ultra Ethernet Consortium published its 1.0 spec in June 2025; UALink published an open scale-up spec as an alternative to NVLink.
• Broadcom: its 102.4 Tbps Tomahawk 6 can join 128,000 accelerators in two switch layers; Jericho4 links sites 100 km or more apart.

Flagship

Nvidia Spectrum-X Ethernet

• Proof at scale: SpaceXSI's Colossus 1 joined 100,000 Hopper GPUs on Spectrum-X in Memphis, holding 95% data throughput against about 60% for standard Ethernet, by Nvidia's account.
• Adopters: Meta is building Spectrum-X into its own switches; Oracle uses it for Vera Rubin AI factories.
• Light instead of copper: Spectrum-X Photonics, with optics built into the switch package, entered full production in August 2026 for CoreWeave, Lambda, Oracle, Azure, IBM Cloud and Nebius. Nvidia says it uses 4x fewer lasers and cuts power per 1.6T port from 30 W to 9 W.
• Across sites: Spectrum-XGS extends the fabric between data centers.

Optics

Optics Ship in the Tens of Millions

• Volume: nearly 20 million 400G-and-faster optical modules shipped in Q1 2026 alone, and more than 10 million 1.6T modules are expected in 2026 (Cignal AI).
• China leads modules: InnoLight leads module revenue and units by a wide margin, followed by Eoptolink and Accelink. Huawei's CloudMatrix 384 uses 6,912 800G optical modules to join 384 chips.
• Lower power: linear pluggable optics drop the signal processor from the module; Arista says its 7060XE7 cuts interconnect power about 60%.
• Supply: laser supply runs about 30% short in 2026. See Bottleneck Atlas.

Fabrics

Fabrics and Switches

Related coverage

Related Coverage

• Cluster Layer
• Rack Layer
• Telecom and Content
• Microsoft Fairwater