1,024 GPU AI cluster network: GB300 NVL72

Fifteen racks for a target of 1,024, because 14 racks is 1,008. The largest of these, and the one where power planning dominates the design.

GPUs1,080
Racks of GPUs15
Compute switches27
Fabric tiers2
Total racks19
Total draw1,981 kW

The build

Platform	NVIDIA GB300 NVL72, 72 × GB300 (Blackwell Ultra) per rack
Compute fabric	infiniband, 72 rails at 800G
Leaf	18 × Q3400-RA (Quantum-X800)
Spine	9 × Q3400-RA (Quantum-X800)
Core	not required at this scale
Racks	15 compute + 4 network, 1 rack per rack, limited by fixed
Power	1,800 kW compute + 181 kW network = 1,981 kW

How this fabric was sized.

Each rail needs only 15 of a leaf's 72 downlinks, so 4 whole rails share each leaf. That is 18 leaf switches instead of 72, and every rail is still single-hop because no rail is split across leaves. The cost is fault domain: losing one leaf now costs 4 rails rather than one. Set rail packing off to keep strict one-rail-per-leaf isolation.

Asked for 1,024, built 1,080. An NVL72 rack is indivisible, so GPU counts round to multiples of 72. 15 racks is the smallest build that meets the target.

Fabrics

Compute

1,080 endpoints across 72 rails, 2 tiers, realised oversubscription 1:1 (non-blocking).

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	NVIDIA Q3400-RA (Quantum-X800)	18	72	72	2,376 / 2,592
spine	NVIDIA Q3400-RA (Quantum-X800)	9	144	0	1,296 / 1,296

Storage

302 endpoints across 1 rail, 2 tiers, realised oversubscription 1.91:1.

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	NVIDIA QM9700 (Quantum-2)	8	42	22	478 / 512
spine	NVIDIA QM9790 (Quantum-2, externally managed)	3	64	0	176 / 192

In-band management

270 endpoints across 1 rail, 2 tiers, realised oversubscription 3.67:1.

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	Cisco Nexus 93600CD-GX	13	22	6	348 / 364
spine	Cisco Nexus 93600CD-GX	3	28	0	78 / 84

Out-of-band management

370 endpoints across 1 rail, 2 tiers, realised oversubscription 1:1 (non-blocking).

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	Cisco Nexus 9348GC-FX3	8	48	4	370 / 384
spine	Cisco Nexus 93180YC-FX3	2	48	0	32 / 96

What the engine flagged

UNEVEN_STRIPING — 22 uplinks per leaf do not divide evenly across 3 spines. Link load will be slightly uneven; consider adjusting the switch count or oversubscription.

Bill of materials

Every line carries the calculation that produced its quantity. Prices are deliberately absent: the catalog ships without list prices, because inventing them would be worse than leaving them out.

Item	Qty	Why this quantity
Servers
GB300 NVL72 (72 x GB300 (Blackwell Ultra))	15	1,024 GPUs requested / 72 GPUs per node = 15 nodes (1,080 GPUs).
Network adapters
NVIDIA ConnectX-8 SuperNIC C8180 (XDR 800G) `900-9X81E-00EX-ST0`	1,080	15 nodes x 72 compute NICs per node (one per rail).
Compute fabric switches
NVIDIA Q3400-RA (Quantum-X800) `920-9B36F-00RX-8S0`	27	27 across Compute fabric (rail-optimized) leaf, Compute fabric (rail-optimized) spine.
NVIDIA QM9700 (Quantum-2) `MQM9700-NS2F`	8	8 across Storage fabric leaf.
NVIDIA QM9790 (Quantum-2, externally managed) `MQM9790-NS2F`	3	3 across Storage fabric spine.
Management switches
Cisco Nexus 93180YC-FX3 `N9K-C93180YC-FX3`	2	2 across Out-of-band management spine.
Cisco Nexus 9348GC-FX3 `N9K-C9348GC-FX3`	8	8 across Out-of-band management leaf.
Cisco Nexus 93600CD-GX `N9K-C93600CD-GX`	16	16 across In-band management leaf, In-band management spine.
Optics — transceivers
25G SFP28 SR `SFP-25G-SR-S`	66	32 x 25G OOB switch to OOB aggregation at 30 m. 30 m run needs discrete optics over structured fiber. 25G SFP28 SR at OOB switch uplink and 25G SFP28 SR at OOB aggregation, over LC duplex OM4 multi-mode patch.
800G OSFP 2xSR4 twin-port (NDR) `MMA4Z00-NS`	182	176 x 400G Storage leaf to Storage spine at 30 m. 30 m run needs discrete optics over structured fiber. 800G OSFP 2xSR4 twin-port (NDR) at Storage leaf and 800G OSFP 2xSR4 twin-port (NDR) at Storage spine, over MPO-12 APC OM4 multi-mode trunk. The Storage leaf module is twin-port, so it terminates 2 links and only 0.5 module is needed per link.
800G OSFP SR8 single-link (Spectrum-X)	2,670	1296 x 800G Leaf to Spine at 30 m. 30 m run needs discrete optics over structured fiber. 800G OSFP SR8 single-link (Spectrum-X) at Compute leaf and 800G OSFP SR8 single-link (Spectrum-X) at Compute spine, over MPO-12 APC OM4 multi-mode trunk.
Cables and assemblies
100G QSFP28 passive DAC	279	270 x 100G In-band NIC to In-band switch at 2 m. 2 m run is within the 2 m passive copper limit. Optics are integrated into the assembly, so no separate transceivers are needed.
800G OSFP passive DAC `MCP4Y10-N00A`	1,113	1080 x 800G Compute NIC to Leaf at 2 m. 2 m run is within the 2 m passive copper limit. Optics are integrated into the assembly, so no separate transceivers are needed.
800G OSFP to 2x400G OSFP splitter DAC `MCP7Y00-N001`	156	270 x 400G Storage NIC to Storage leaf at 2 m. 2 m run is within the 2 m passive copper limit. Optics are integrated into the assembly, so no separate transceivers are needed; one cage breaks out to 2 links.
Cat6a patch lead	382	370 x 1G BMC / device management port to OOB switch at 2 m. 1G over structured copper, 2 m within the 100 m limit for Cat6a patch lead.
Structured fibre
LC duplex OM4 multi-mode patch	33	32 x 25G OOB switch to OOB aggregation at 30 m. 30 m run needs discrete optics over structured fiber. 25G SFP28 SR at OOB switch uplink and 25G SFP28 SR at OOB aggregation, over LC duplex OM4 multi-mode patch.
MPO-12 APC OM4 multi-mode trunk	1,517	1296 x 800G Leaf to Spine at 30 m. 30 m run needs discrete optics over structured fiber. 800G OSFP SR8 single-link (Spectrum-X) at Compute leaf and 800G OSFP SR8 single-link (Spectrum-X) at Compute spine, over MPO-12 APC OM4 multi-mode trunk.
Racks
Rack	19	15 compute racks at 1 nodes each (fixed-limited) plus 4 network racks.
Power
Rack PDU	38	19 racks x 2 PDUs per rack.