256 GPU AI cluster network: GB300 NVL72

Four racks at 800G. Compare the switch count against the 32-node DGX build for the same GPU target, and against the GB200 racks on 400G.

GPUs288
Racks of GPUs4
Compute switches6
Fabric tiers2
Total racks5
Total draw524 kW

The build

Platform	NVIDIA GB300 NVL72, 72 × GB300 (Blackwell Ultra) per rack
Compute fabric	infiniband, 72 rails at 800G
Leaf	4 × Q3400-RA (Quantum-X800)
Spine	2 × Q3400-RA (Quantum-X800)
Core	not required at this scale
Racks	4 compute + 1 network, 1 rack per rack, limited by fixed
Power	480 kW compute + 44 kW network = 524 kW

How this fabric was sized.

Each rail needs only 4 of a leaf's 72 downlinks, so 18 whole rails share each leaf. That is 4 leaf switches instead of 72, and every rail is still single-hop because no rail is split across leaves. The cost is fault domain: losing one leaf now costs 18 rails rather than one. Set rail packing off to keep strict one-rail-per-leaf isolation.

Asked for 256, built 288. An NVL72 rack is indivisible, so GPU counts round to multiples of 72. 4 racks is the smallest build that meets the target.

Fabrics

Compute

288 endpoints across 72 rails, 2 tiers, realised oversubscription 1:1 (non-blocking).

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	NVIDIA Q3400-RA (Quantum-X800)	4	72	72	576 / 576
spine	NVIDIA Q3400-RA (Quantum-X800)	2	144	0	288 / 288

Storage

104 endpoints across 1 rail, 2 tiers, realised oversubscription 1.91:1.

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	NVIDIA QM9700 (Quantum-2)	3	42	22	170 / 192
spine	NVIDIA QM9790 (Quantum-2, externally managed)	2	64	0	66 / 128

In-band management

72 endpoints across 1 rail, 2 tiers, realised oversubscription 3.67:1.

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	Cisco Nexus 93600CD-GX	4	22	6	96 / 112
spine	Cisco Nexus 93600CD-GX	1	28	0	24 / 28

Out-of-band management

101 endpoints across 1 rail, 2 tiers, realised oversubscription 1:1 (non-blocking).

Tier	Switch	Count	Down/sw	Up/sw	Ports used
leaf	Cisco Nexus 9348GC-FX3	3	48	4	101 / 144
spine	Cisco Nexus 93180YC-FX3	2	48	0	12 / 96

Bill of materials

Every line carries the calculation that produced its quantity. Prices are deliberately absent: the catalog ships without list prices, because inventing them would be worse than leaving them out.

Item	Qty	Why this quantity
Servers
GB300 NVL72 (72 x GB300 (Blackwell Ultra))	4	256 GPUs requested / 72 GPUs per node = 4 nodes (288 GPUs).
Network adapters
NVIDIA ConnectX-8 SuperNIC C8180 (XDR 800G) `900-9X81E-00EX-ST0`	288	4 nodes x 72 compute NICs per node (one per rail).
Compute fabric switches
NVIDIA Q3400-RA (Quantum-X800) `920-9B36F-00RX-8S0`	6	6 across Compute fabric (rail-optimized) leaf, Compute fabric (rail-optimized) spine.
NVIDIA QM9700 (Quantum-2) `MQM9700-NS2F`	3	3 across Storage fabric leaf.
NVIDIA QM9790 (Quantum-2, externally managed) `MQM9790-NS2F`	2	2 across Storage fabric spine.
Management switches
Cisco Nexus 93180YC-FX3 `N9K-C93180YC-FX3`	2	2 across Out-of-band management spine.
Cisco Nexus 9348GC-FX3 `N9K-C9348GC-FX3`	3	3 across Out-of-band management leaf.
Cisco Nexus 93600CD-GX `N9K-C93600CD-GX`	5	5 across In-band management leaf, In-band management spine.
Optics — transceivers
25G SFP28 SR `SFP-25G-SR-S`	25	12 x 25G OOB switch to OOB aggregation at 30 m. 30 m run needs discrete optics over structured fiber. 25G SFP28 SR at OOB switch uplink and 25G SFP28 SR at OOB aggregation, over LC duplex OM4 multi-mode patch.
800G OSFP 2xSR4 twin-port (NDR) `MMA4Z00-NS`	68	66 x 400G Storage leaf to Storage spine at 30 m. 30 m run needs discrete optics over structured fiber. 800G OSFP 2xSR4 twin-port (NDR) at Storage leaf and 800G OSFP 2xSR4 twin-port (NDR) at Storage spine, over MPO-12 APC OM4 multi-mode trunk. The Storage leaf module is twin-port, so it terminates 2 links and only 0.5 module is needed per link.
800G OSFP SR8 single-link (Spectrum-X)	594	288 x 800G Leaf to Spine at 30 m. 30 m run needs discrete optics over structured fiber. 800G OSFP SR8 single-link (Spectrum-X) at Compute leaf and 800G OSFP SR8 single-link (Spectrum-X) at Compute spine, over MPO-12 APC OM4 multi-mode trunk.
Cables and assemblies
100G QSFP28 passive DAC	75	72 x 100G In-band NIC to In-band switch at 2 m. 2 m run is within the 2 m passive copper limit. Optics are integrated into the assembly, so no separate transceivers are needed.
800G OSFP passive DAC `MCP4Y10-N00A`	297	288 x 800G Compute NIC to Leaf at 2 m. 2 m run is within the 2 m passive copper limit. Optics are integrated into the assembly, so no separate transceivers are needed.
800G OSFP to 2x400G OSFP splitter DAC `MCP7Y00-N001`	54	72 x 400G Storage NIC to Storage leaf at 2 m. 2 m run is within the 2 m passive copper limit. Optics are integrated into the assembly, so no separate transceivers are needed; one cage breaks out to 2 links.
Cat6a patch lead	105	101 x 1G BMC / device management port to OOB switch at 2 m. 1G over structured copper, 2 m within the 100 m limit for Cat6a patch lead.
Structured fibre
LC duplex OM4 multi-mode patch	13	12 x 25G OOB switch to OOB aggregation at 30 m. 30 m run needs discrete optics over structured fiber. 25G SFP28 SR at OOB switch uplink and 25G SFP28 SR at OOB aggregation, over LC duplex OM4 multi-mode patch.
MPO-12 APC OM4 multi-mode trunk	365	288 x 800G Leaf to Spine at 30 m. 30 m run needs discrete optics over structured fiber. 800G OSFP SR8 single-link (Spectrum-X) at Compute leaf and 800G OSFP SR8 single-link (Spectrum-X) at Compute spine, over MPO-12 APC OM4 multi-mode trunk.
Racks
Rack	5	4 compute racks at 1 nodes each (fixed-limited) plus 1 network racks.
Power
Rack PDU	10	5 racks x 2 PDUs per rack.