We use cookies to improve performance, analyze traffic and personalize content. By continuing to use the site, you agree to the processing of data in accordance with our Privacy Policy and agree to the use of cookies
Standard AI Cluster Projec | ITPOD
Standard AI Cluster Project for Corporate Inference: 8 NVIDIA HGX B300 Nodes, InfiniBand XDR 800G Fabric Based on Quantum-X800 Q3400, and Optics Bill of Materials
22 September 2026
29 min
The most common request integrator partners bring to us at ITPOD in 2026 is corporate AI: a customer needs an in-house cluster for assistants, RAG, and coding helpers, without using the cloud and without leaking data outside. This request almost always sounds like a choice of compute nodes: which GPUs, how much memory. But nodes are only half the project. The other half is the compute fabric that connects the GPUs together: topology, switches, and optics. It is frequently calculated at the very last moment, and mistakes in it surface during installation.
Below is a standard project for a small AI cluster as we assemble it at ITPOD for our partners: 8 NVIDIA HGX B300 nodes, 64 Blackwell Ultra GPUs, an InfiniBand XDR 800G fabric based on NVIDIA Quantum-X800 Q3400-RA switches, an analysis of factory switch options, and a complete optics bill of materials by part numbers, with convergence checks that can be verified manually. Spoiler: the optics are identical in all scenarios, so the choice boils down to a single question—one Q3400 switch or two—and below, this is resolved using numbers rather than intuition.
In short - if you only need the answer:
Task: Productive inference of large MoE models (Kimi K3, DeepSeek-V4-Pro) on 8 NVIDIA HGX B300 nodes — 64 GPUs, 18.4 TB HBM, 64 ports of 800G.
Fabric: Single-tier InfiniBand XDR on NVIDIA Quantum-X800 Q3400-RA, without spines and without inter-switch links (ISLs). Any GPU reaches any other in a single hop.
Optics: Identical in all scenarios without ISLs: 96 transceivers (32 twin-port ITPOD-OSFP-FT-16DR8 per switch + 64 single ITPOD-OSFP-RHS-800DR4 on nodes) and 64 MTP-12 patch cords. The choice comes down to the number of switches.
Starting Scenario — Single Q3400: Zero slowdown, single subnet, 18-node ceiling; firmware update and chassis failure stop the entire fabric.
Recommended Scenario for 24/7 — Pair of Q3400 without ISLs (rail 4+4): Same optics, chassis failure is survived at half bandwidth, sequential firmware updates, 36-node ceiling; cost: two subnets and a 6.9% impact on all-to-all phase time.
Inference does not need ISLs: A 32-ISL Fat Tree adds 32 modules and 32 cords and lowers the pair's ceiling to 18 nodes. This is headroom for fine-tuning and any-to-any, not default insurance.
Key Project Numbers
Key Metric
Value
Cluster Nodes / GPUs / HBM
8 × HGX B300 / 64 × B300 (288 GB) / 18.4 TB
800G Ports from Nodes
64 (8 × ConnectX-8 per node, 6.4 Tbps per node)
Switch
NVIDIA Quantum-X800 Q3400-RA: 144 ports of 800G, 72 data OSFP connectors, 4U
Transceivers / Patch Cords (any scenario without ISLs)
96 / 64
All-to-all Slowdown: Single Q3400 / Pair without ISLs
0% / 6.9% (calculated using MIT Rail-only formula)
InfiniBand Subnets: Single Q3400 / Pair without ISLs
1 / 2
Node Ceiling without Spine: Single Q3400 / Pair without ISLs / Pair with ISLs
18 / 36 / 18
Thermal Dissipation of 1.6T Twin-Port Module
Up to 33.5 W; finned top for switch, flat top (RHS) for network card
The Task: Corporate Inference of MoE Models Kimi K3 and DeepSeek-V4-Pro on 64 HGX B300 GPUs
The problem conditions are typical for corporate AI in 2026. The cluster serves large open MoE models. Kimi K3 features 2.8 trillion parameters, with 16 out of 896 experts activated per token; weights take up about 1.4 TB in FP4. DeepSeek-V4-Pro has 1.6 trillion parameters, 49 billion active, a 1-million-token context, and about 1.6 TB of weights in FP8. The workload consists of corporate assistants, a RAG pipeline, and coding helpers. Training models of this scale runs on clusters of hundreds of nodes and is outside the scope of this task.
The absence of training does not mitigate bandwidth requirements. Inference of large MoE models introduces three sources of inter-node traffic: all-to-all Expert Parallelism on each MoE layer of every pass, KV-cache transfer between prefill and decode workers, and interactive SLAs where users experience the fabric's tail latencies as assistant lag. Therefore, each node retains all eight 800G ports—one NVIDIA ConnectX-8 network adapter per GPU, providing 6.4 Tbps per node.
Node selection in such projects happens quickly: large MoE models require eight GPUs with maximum HBM per node and one network adapter per GPU, after which memory and expert divisibility dictate the rest, following the math in Section 2. The fork where projects truly stall is how to assemble these 64 ports of 800G into a fabric: into a single switch or two, with or without inter-switch links (ISLs), and with what optics.
Six terms you need to know before moving forward:
MoE A Mixture-of-Experts model: instead of a single large network, it is a set of specialized ones, and for each token, the router turns on only a few of them. In Kimi K3, this is 16 experts out of 896.
Expert Parallelism Distributing experts across different accelerators. NVIDIA calls the cross-node variant WideEP: experts are distributed across the entire cluster, which frees up memory for the KV cache.
all-to-all An "every-to-every" exchange: at each layer, tokens fly out to their respective experts on other nodes and return back. This is the main source of inter-node traffic during MoE inference.
KV cache The working memory of a dialog: everything the model has already computed from the prompt and re-reads on each new token.
Rail A fabric "lane": ports with the same number across all cluster nodes. Ports of the same rail are wired into a single switch so that GPUs from different nodes communicate in a single hop. Covered in detail in Section 3.
Prefill and decode The two phases of request processing. Prefill computes the entire prompt in a single pass and builds the KV cache; decode generates the response one token at a time. These phases are often separated into different groups of accelerators, in which case the KV cache is transferred between them over the network.
Why 8 HGX B300 Nodes is the Working Size for Large MoE Inference
Eight nodes were not chosen as a round number. Three things converge at this size, and each can be verified with arithmetic.
Memory goes to KV cache, not weights. 64 accelerators with 288 GB each provide 18.4 TB of HBM. When experts are sharded across the entire cluster, the weights of Kimi K3 in FP4 take up 22 GB per accelerator, and DeepSeek-V4-Pro in FP8 takes 25 GB. The remaining 263 GB per accelerator is free for the KV cache and activations; specifically, KV capacity determines how many concurrent long-context sessions the cluster can serve. A full model replica per node would leave 0.9 TB per node for KV instead of 2.1 TB.
Experts and parallelism divide evenly. Kimi K3's 896 experts across 64 accelerators equal exactly 14 per accelerator. The number of accelerators is a power of two, so intra-node tensor parallelism of 8 and cluster-wide expert parallelism of 64 break down without fractions and without idle accelerators.
The fabric remains single-tier. 64 ports of 800G fit into a single switch; any accelerator reaches any other in a single hop, and a second tier of switching with its own optics and cost is completely unnecessary.
Key Metrics Summary
What We Are Calculating
Value
What This Implies
Cluster HBM
64 × 288 GB = 18.4 TB
Both target models run simultaneously
Kimi K3 FP4 Weights per Accelerator (Sharded)
1.4 TB ÷ 64 = 22 GB
263 GB is free for KV cache
Kimi K3 Experts per Accelerator
896 ÷ 64 = 14
Layout without remainders or idle time
Free for KV per Node: Sharded vs. Replicated
2.1 TB vs. 0.9 TB
More long-context sessions
800G Ports from Nodes
64
Fit into a single switch
How to Route 64 Ports of 800G Across Switches: One Q3400 or a Pair Without ISLs
There is only one fabric switch in this project—the NVIDIA Quantum-X800 Q3400-RA. The question is not the model, but how many switches to install and how to distribute the 64 node ports among them. There are two answers, and each has its own cost.
What the switch is. The Q3400-RA is a single ASIC, with 144 ports of 800G in a single switching plane and a 4U height. There are twice as many ports as OSFP connectors: each of the 72 data connectors carries a 1.6T twin-port module and serves two links. This is why the most common optics specification error is calculating modules by ports rather than connectors; it regularly makes it all the way to installation.
What a rail is. In an HGX node, each GPU has its own 800G ConnectX-8: eight GPUs mean eight ports facing outward. Ports with the same number across all nodes form a rail: rail 3 is the third port of all eight nodes. Ports of the same rail are wired into a single switch so that GPUs from different nodes can communicate in a single hop; this is NVIDIA's canonical rail-optimized layout.
Starting layout: all rails into a single switch. A single Q3400 takes all 64 ports—8 rails × 8 nodes out of 144. Any pair of GPU sees each other through a single ASIC, there is one subnet, and no intra-node forwarding is required. The only limitation is that there is only one switch, so a firmware update and a chassis failure stop the entire fabric.
Recommended layout: a pair of Q3400s without ISLs, rail 4+4. Rails 0–3 of all nodes go to the first switch, and rails 4–7 go to the second; there are no cables between the switches. The optics do not change by a single module, while the fabric survives a chassis failure and updates sequentially. The price paid is twofold: these are two InfiniBand subnets, and half of the off-node destinations (28 out of 56) require intra-node forwarding—data first travels via NVLink to the GPU whose port faces the correct switch, and only then leaves in that same single hop. In NCCL, the PXN mechanism handles this forwarding.
The cost of NVLink forwarding: 6.9% on all-to-all. Port bandwidth is not lost—each port carries the same 56 transmissions as in a non-blocking tree, with zero oversubscription. Only the transit time across NVLink is added, and NVLink 5 in one direction is nine times faster than an 800G port: 900 vs. 100 GB/s. According to the formula from the "Rail-only" paper (MIT, 2024), this adds 6.9% to the time of the all-to-all phase compared to zero on a single switch. This is a calculated estimate under uniform expert load rather than a direct measurement, and it applies to a single phase rather than the entire request; during a pilot, it is validated by two measurements—alltoall_perf from nccl-tests and tokens per second on the target model. It is against this cost—6.9% and two subnets—that the two scenarios are compared moving forward.
4. Starting Scenario: A Single Quantum-X800 Q3400 - Simple, Fast, with Maintenance Windows
If the project starts with the bare minimum, the fabric is assembled on a single Q3400: all eight rails of all nodes into a single ASIC, 64 ports out of 144. This is the simplest option both in procurement and operation: a single InfiniBand subnet, a single pair of OpenSM (master and standby), SHARP across the entire fabric, zero slowdown on all-to-all, and any GPU reaches any other in a single hop, without intra-node forwarding. The optics are the same as in any other scenario: 32 twin-port modules for the switch connectors, 64 single modules for nodes, and 64 patch cords. The 80 ports vacant at launch represent a paid-for ceiling of 18 nodes without re-cabling.
Scenario Comparison
Parameter
Single Q3400, All Rails into a Single ASIC
Pair of Q3400s Without ISLs, Rail 4+4
Optics Bill of Materials
96 modules and 64 cords
Same
InfiniBand Subnets
1
2
OpenSM Instances (Master + Standby)
1 + 1
2 + 2
SHARP
Across the entire fabric
Within a single switch
All-to-all Slowdown
None
6.9%
Chassis Failure
Fabric goes down
Half bandwidth remains
Firmware Update
Entire fabric, maintenance window needed
Sequentially, cluster keeps running
Node Ceiling
18
36
A single switch has a single, yet decisive limitation: there is only one. NVOS updates stop the entire fabric and inference along with it, and a chassis failure means downtime until replacement. Therefore, the starting scenario is suitable where regulations allow for planned windows: a pilot, an internal team, or an assistant without an availability SLA. The good news is that transitioning to a pair does not require a single new transceiver: a second Q3400 is purchased, and half of the twin-port modules along with the rail 4–7 cords move to it. However, this move involves re-cabling half the fabric on a running cluster, and if 24/7 operation is planned, it is cheaper to start with a pair right away.
5. Recommended Scenario for 24/7: A Pair of Q3400s Without ISLs - and When a Fat Tree Is Actually Needed
If inference serves a 24/7 service, the fabric is built on a pair of Q3400s. In terms of switching hardware, this is twice as expensive as the starting scenario, and this is our recommendation for production: only a pair survives a chassis failure and can be updated on the fly, with firmware applied sequentially while the cluster continues running at half bandwidth.
A pair is assembled in two ways, distinguished by a single design choice: whether or not there are inter-switch links (ISLs).
Without ISLs - the rail 4+4 layout from Section 3: rails 0–3 of all nodes go to the first Q3400, and rails 4–7 go to the second. The optics bill of materials does not change by a single module: the exact same 96 transceivers and 64 cords, with 16 twin-port modules per switch. The fabric pays the previously calculated 6.9% penalty during the all-to-all phase and operates across two subnets, but the growth ceiling is 36 nodes - each node dedicates only 4 links to a switch.
With ISLs - a classic Fat Tree: 32 inter-switch links, placing any pair of GPUs into a single subnet with guaranteed any-to-any connectivity. The slowdown disappears, SHARP works across the entire fabric, and adaptive routing gets 32 paths for load balancing. This headroom adds 32 twin-port modules and 32 cords to the baseline bill of materials—while cutting the node ceiling: half of each switch's ports are consumed by ISLs, limiting the fabric's growth to just 18 nodes.
Comparison of Pair Scenarios
Parameter
Pair of Q3400s Without ISLs
Pair of Q3400s with 32 ISLs (Fat Tree)
Optics Bill of Materials
96 modules and 64 cords
128 modules and 96 cords
InfiniBand Subnets
2
1
All-to-all Slowdown
6.9%
None
SHARP
Within a single switch
Across the entire fabric
Chassis Failure
Half bandwidth remains
Half bandwidth remains
Firmware Update
Sequentially, cluster keeps running
Sequentially, cluster keeps running
Node Ceiling
36
18
Do inference workloads need ISLs? No, and this can be calculated rather than assumed. The primary inter-node traffic of MoE inference—the expert all-to-all phase—either stays within a single switch in a single hop under a rail layout, or travels via NVLink to the GPU on the correct rail (PXN from Section 3) and then makes that exact same single hop. This traffic has no reason to travel between switches, and the price for the rail layout has already been accounted for: 6.9% added to the time of a single phase. ISLs are headroom for a different lifecycle of the cluster: fine-tuning, arbitrary workload layouts, or severe expert imbalance where any-to-any gives the scheduler more freedom. Our recommendation for a standard project is to build it without ISLs, and only add headroom if a specific future workload demands it—at which point those 32 modules and 32 cords are intentionally added to the bill of materials.
The pair's acceptance testing program includes two items: powering down one switch under live workload while measuring the drop in tokens per second and bringing the switch back up to verify that the topology recovers completely rather than partially, as collective libraries handle the loss of half their network cards in varying ways.
6. Who Manages the Fabric: OpenSM on HGX Nodes Instead of a Dedicated Server
In a standard eight-node project, the fabric is managed by OpenSM running directly on the HGX nodes—a dedicated server and UFM are not required. InfiniBand does not work without a subnet manager, and Quantum-X800 switches do not carry a built-in manager—it is always external: either UFM or OpenSM on a host with a port connected to the fabric. Network cards are already installed in the nodes, so a separate management server and its optics are not included in the specification. Management traffic travels across the fabric itself over the VL15 virtual lane and does not compete with inference for bandwidth. Two instances are sufficient for a single subnet—a master and a standby with a lower priority, necessarily on different nodes; the 4+4 layout duplicates this setup, providing a pair for each subnet, while partitions and routing are configured twice and compared using diff.
What OpenSM does not provide is telemetry, a web interface, and analytics. For eight nodes, this is acceptable, and the solution is reversible: the Q3400 features a separate fabric management connector for UFM that does not consume any fabric ports. As of September 2026, the validated platform configuration consists of NVOS 25.02.7002, ConnectX-8 firmware 40.48.1132, DOCA-OFED 3.3.0, OpenSM 5.26.1, and HPC-X 2.26.0; the mere fact that OpenSM is listed in the suite confirms that a host-based manager is a standard option for XDR rather than a custom workaround.
7. Cluster Service Networks: Storage, In-Band, and Out-of-Band
Aside from the compute fabric, a cluster following the NVIDIA SuperPOD logic contains three additional networks, and all four are physically separate—none share switches with another (RA-11339-001). A brief overview of their purposes:
Storage - node access to model weights, the RAG corpus, and KV-cache offloading. Ethernet with RDMA, typically 2 × 100G or 2 × 400G per node.
In-band management - node OS, provisioning, telemetry, and inference stack orchestration. Typically 25G per node.
Out-of-band - BMC and remote hardware management. A copper network, excluded from optics calculations.
These three service networks are variable in a standard project, which is normal: integrators have their own proven stacks, and customers have corporate standards for switches and monitoring, so service networks are assembled using whatever is standard in their infrastructure. There is no need to fix them in a standard project: compatible ITPOD modules are available for all popular switches—Cisco, Juniper, Arista, Huawei, and others—and across all common speeds: ITPOD-SFP-25GSR, ITPOD-QSFP-100SR4, ITPOD-QSFP-400DR4. Optics there are calculated using the classic rule "a link is two transceivers and one patch cord." They are not included in this article's bill of materials: this BOM is strictly for the compute fabric.
8. 800G Optics: How a Network Card Module (Flat Top RHS) Differs from a Switch Module (Finned Top)
At 800G, a transceiver ceases to be a consumable item simply plugged into a port. A twin-port 1.6T module dissipates up to 33.5 W-the level of a small processor-and cooling becomes part of the module's own structural design. The most interesting aspect here is that the compute fabric switch and the AI server network card solve this challenge in opposite ways. Therefore, a single OSFP standard exists in two thermal variants, and they cannot be swapped.
Finned top - for the switch. A ribbed heat sink is built directly onto the top cover of the module. The OSFP cages on the Quantum-X800 are ventilated straight through, the ribs sit directly in this airflow, and the module dissipates its own heat-switch cages lack clamp-on heat sinks.
Flat top in RHS configuration - for the network card. The module cover is flat, and heat is removed by a riding heat sink mounted over the network card's own interfaces. The ConnectX-8 in HGX nodes uses precisely these interfaces, which is why only the flat variant is used there.
How thermal specification mistakes happen. A ribbed module will not physically fit into a network card interface: height restrictions prevent it, and the error is caught at installation. Conversely, a flat module will easily slide into a switch, link up, and be left without proper heat dissipation: it has no ribs, and the switch has no heat sink. Such an error survives in the specification until the first heavy load, which is why the thermal specification is checked at the BOM stage rather than during installation. In ITPOD's nomenclature, this is embedded directly into the part number: the FT suffix denotes a finned top, while RHS denotes a flat top for a riding heat sink.
The connector rule. Counting links and counting modules are not the same thing, and at 800G this diverges by a factor of two. An OSFP connector on a switch carries a twin-port 1.6T module with two MPO-12 connectors and serves two 800G links; on the node side, a single 800G module goes into the ConnectX-8 port. Consequently, modules on the switch are counted by connectors, while on the nodes they are counted by ports. Copper DACs and active AOCs are excluded from the calculation: the fabric maintains a single optical standard to ensure a unified spare parts pool.
Optical Modules Reference Table
Part Number
Description
Location
Links per Module
ITPOD-OSFP-FT-16DR8
1.6T XDR OSFP224 2×DR4, two MPO-12, SMF 1310 nm, 500 m, finned top
Switch connectors
2
ITPOD-OSFP-RHS-800DR4
800G XDR OSFP224 DR4, MPO-12, SMF 1310 nm, 500 m, flat top (RHS)
ConnectX-8 ports in HGX nodes
1
ITPOD-MTP12-OS2-<N>M
MTP-MTP patch cord, 12 fibers, SMF OS2, type B
One cord per 800G link
—
This is the exact same optics used in our shipped clusters and storage systems; equivalence to NVIDIA modules has been verified against datasheets—links are provided at the end of the article. Additionally, there are two specification errors to watch out for alongside thermal configurations. First: single modules calculated for switch connectors instead of twin-port ones—you will "run out" of ports by a factor of two. Second: the previous-generation module ITPOD-OSFP-FT-800DR4 is an NDR module intended for Quantum-2 switches (QM9700) and is not used in a Quantum-X800 project.
9. How Many Transceivers and Patch Cords Are Needed for 8 HGX B300 Nodes: Part-Number Bills of Materials
An 8-node HGX B300 cluster requires 96 transceivers and 64 patch cords—in any scenario without ISLs. The calculation is straightforward because the fabric is single-tier: 64 GPU ↔ switch links, with 64 single modules on the node side, 64 ÷ 2 = 32 twin-port modules on the switching side, and 64 patch cords. Below are two bills of materials for the two scenarios from Sections 4 and 5.
Single Q3400 (All Rails into a Single ASIC)
Part Number
Role
Quantity
ITPOD-OSFP-FT-16DR8
1.6T twin-port, 32 connectors out of 72
32
ITPOD-OSFP-RHS-800DR4
800G, ConnectX-8 in HGX nodes
64
Total Modules
96
ITPOD-MTP12-OS2-<N>M
MTP-12 SMF patch cord, one per link
64
Pair of Q3400s Without ISLs (Rail 4+4 Across Switches)
Part Number
Role
Quantity (Total)
Quantity per Switch
ITPOD-OSFP-FT-16DR8
1.6T twin-port, 16 connectors out of 72 in each switch
32
16
ITPOD-OSFP-RHS-800DR4
800G, ConnectX-8 in HGX nodes
64
—
Total Modules
96
ITPOD-MTP12-OS2-<N>M
MTP-12 SMF patch cord, one per link
64
32
ISL Option for Fat Tree
+ ITPOD-OSFP-FT-16DR8 and + ITPOD-MTP12-OS2-<N>M
+32 and +32
+16
The final quantities for both scenarios match down to the individual item (96 modules and 64 cords) because the node-side count does not depend on the number of switches, and there are no inter-switch links in either setup. The only differences are the distribution of twin-port modules across the switches and the optional line item: the Fat Tree adds 32 modules and 32 cords.
Convergence checking is a habit that catches errors before installation:
Fat Tree checksum: 96 + 32 = 128 modules and 64 + 32 = 96 cords - checks out.
The cord length is encoded in the part number suffix: -3M, -5M, -10M. A single-tier fabric simplifies this as well: all 64 compute links converge into a single network rack, so the length variation is determined solely by the distance from the node rack to the network rack—the final breakdown is fixed after cable routing.
10. Power and Placement of the Q3400: What to Consider During Installation
The official specifications for the Q3400-RA are: 4U, 60 kg, eight power supplies, with a typical power consumption of 2,900 W using passive cables and up to 7,000 W maximum with active ones. The spread is more than twofold, which is why the rack cannot be planned based on a standard figure: optics are the main variable, with a twin-port 1.6T module consuming up to 33.5 W, adding up to 1.1 kW to the switch's base power consumption across 32 modules. For a pair of switches, power input and rack units must be calculated for two chassis in separate racks or at least on different power feeds—otherwise, the primary purpose of buying a pair is defeated.
XDR switches have a single airflow direction: air enters from the connector side and exhausts toward the power supplies, meaning the switch must be installed with its ports facing the cold aisle in the rack—this must be aligned with the node layout on the data center floor plan, not during installation. Additionally, front-panel discipline is crucial: dozens of MPO-12 cables per switch require patch panels and cable management trays planned in advance, with rail markings done during the installation phase, otherwise it will have to be sorted out during the first incident response.
11. Growth Ceiling: 18 Nodes on a Single Q3400 and 36 on a Pair Without a Second Tier
The ceiling arithmetic fits into a single line. A single Q3400: 144 ports ÷ 8 links per node = 18 nodes. A pair of Q3400s without ISLs: each node gives 4 links to each switch, 144 ÷ 4 = 36 nodes. A pair with ISLs: 18 nodes - half of the ports are occupied by inter-switch links, meaning the any-to-any headroom is paid for with a lower node ceiling. Expansion within these limits requires only optics and patch cords for the new nodes, without re-cabling or stopping the cluster.
Beyond a single-tier fabric begins a leaf-spine architecture based on the SuperPOD reference design—8 leaves per scalable unit of 72 nodes plus a spine layer, and compute optics double: every node link is duplicated by a leaf-spine link. For a standard corporate inference workload of 8–16 nodes, this scale is a future benchmark rather than a starting point.
12. Four Questions for the Customer Before Specification
Fabric adequacy is not an inherent property of a switch but rather the alignment of its characteristics with the workload profile. Therefore, before the bill of materials goes to procurement, the customer is asked four questions.
How many nodes will there be in two years? Up to 18 nodes, a single Q3400 is enough; up to 36, a pair without ISLs; beyond that, a leaf-spine architecture must be designed from the start.
Do regulations allow for planned maintenance windows with inference downtime? On a single switch, a firmware update stops the entire fabric, and a chassis failure even more so. If the cluster operates 24/7, a minimum of two switches is required.
Will the cluster outlive the initial inference task? Fine-tuning, arbitrary task layouts, and a single subnet with SHARP across the entire fabric are arguments in favor of ISLs and a Fat Tree, despite the additional optics and the 18-node ceiling.
How evenly does the model load the experts? The estimated 6.9% overhead of the 4+4 layout applies to uniform all-to-all. With hot experts, the bottleneck shifts to load imbalance, and any-to-any gives the scheduler more freedom- another vote for ISLs.
13. The Skeleton of a Standard AI Cluster Project for 8 Nodes
In a standard project, the fabric for 8 HGX B300 nodes is built as single-tier, without a spine and without inter-switch links, making the choice simpler than it appears in the catalog.
Optics do not dictate the switch choice. The bill of materials is identical across all scenarios without ISLs: 96 ITPOD modules and 64 patch cords. Savings and scalability belong in the "switches" row, not "transceivers."
For launch - a single Q3400. Zero slowdown, a single subnet, SHARP across the entire fabric, an 18-node ceiling, and the simplest operation at the cost of planned maintenance windows: firmware updates and chassis failures stop the entire fabric.
For 24/7 production - a pair of Q3400s without ISLs, which is our recommendation. The same optics, a chassis failure survived at half bandwidth, sequential firmware updates, and a 36-node ceiling. The price paid is two subnets and a 6.9% impact on the all-to-all phase - both of which are calculated figures rather than assumptions.
ISLs are a deliberate buffer, not default insurance. Inference does not need any-to-any; a Fat Tree adds 32 modules and 32 cords while cutting the pair's ceiling down to 18 nodes. Adding them makes sense for a stated future workload: fine-tuning, arbitrary tasks, or hot experts.
This is the skeleton of a standard corporate AI cluster project: nodes are chosen based on memory and expert divisibility, the number of switches is determined by operational regulations and growth ceiling, optics are calculated using the connector rule with correct thermal specifications and convergence checks performed before installation rather than after.
Frequently Asked Questions
How many transceivers are needed for an 8-node HGX B300 cluster with an 800G fabric?
96: 64 single 800G modules for the ConnectX-8 ports on the nodes and 32 twin-port 1.6T modules for the switch connectors, plus 64 MTP-12 patch cords. This number does not depend on whether there is a single Q3400 switch or a pair without ISLs.
Why are there half as many modules on the switch as there are ports?
Each Quantum-X800 OSFP connector carries a twin-port 1.6T module supporting two 800G links. The Q3400-RA has 144 ports and 72 data connectors. A specification calculated by ports will be off by a factor of two.
How many HGX B300 nodes can a single Quantum-X800 Q3400 support?
18: 144 ports ÷ 8 links per node. A pair of Q3400s without ISLs supports 36 nodes because each node dedicates only 4 links to a switch. A pair with 32 ISLs supports 18 nodes again.
Are inter-switch links (ISLs) necessary for MoE model inference?
No. The primary inter-node traffic—the expert all-to-all phase—occurs within a single hop inside a single switch under a rail layout, or after forwarding via NVLink within the node. ISLs are needed for fine-tuning, arbitrary task layouts, and severe expert imbalance.
What is the performance penalty for inference on a pair of switches without ISLs?
An estimated 6.9% added to the time of the all-to-all phase under uniform expert load (using the formula from the MIT "Rail-only" paper). Port bandwidth is not lost; only the transfer time across NVLink is added, which is nine times faster than an 800G port.
How does a finned top differ from a flat top (RHS), and can they be swapped?
A finned top features a heat sink on the cover and is installed in the ventilated chassis slots of a switch; a flat top is flat and transfers heat to a clamp-on heat sink mounted over the network card interfaces. A finned module will not physically fit into a network card, while a flat module placed in a switch will lack proper heat dissipation at 33.5 W. They cannot be swapped.
Are 800G NDR modules from Quantum-2 (QM9700) compatible with Quantum-X800?
No. XDR requires OSFP224 modules: ITPOD-OSFP-FT-16DR8 for the switch and ITPOD-OSFP-RHS-800DR4 for the nodes. The NDR module ITPOD-OSFP-FT-800DR4 is not used in this project.
Is UFM or a dedicated fabric management server required?
For eight nodes, no. Quantum-X800 switches do not feature an embedded subnet manager, but OpenSM runs natively on the HGX nodes: master and standby on different nodes, with a pair for each subnet. UFM can be added later via a separate fabric management connector without consuming any fabric ports.
Calculation Sources and Specifications
Every regulatory number in the article is drawn from official documentation rather than general assumptions about how AI clusters are built:
NVIDIA SuperPOD with DGX B300 Systems, Quantum-X800 InfiniBand and AC Power (RA-11339-001 V01 dated July 23, 2025) - per-node bandwidth and 1:1 blocking, the rail-optimized full-fat-tree principle, and four separate cluster networks.
NVIDIA XDR InfiniBand Switches Hardware User Manual (Q34xx family) - ports and connectors, dedicated fabric management port, power supplies, airflow, and power consumption.
Validated Configuration of the Quantum-X800 Platform - coordinated versions of NVOS, firmware, DOCA-OFED, OpenSM, and HPC-X; confirmation that running OpenSM on the host is the standard management option for XDR.
NVIDIA Optical Module Datasheets - parameters for the twin-port 1.6T OSFP 2×DR4 (finned top, up to 33.5 W) and single 800G OSFP DR4 (flat top, RHS); confirming equivalence to part numbers ITPOD-OSFP-FT-16DR8 and ITPOD-OSFP-RHS-800DR4.
"Rail-only: A Low-Cost High-Performance Network for Training LLMs with Trillion Parameters" (arXiv 2307.12169, final v5 revision dated September 15, 2024) - the slowdown formula for NVLink transfers and the published range of 8.2–11.2%. Intermediate revisions yielded numbers twice as low due to bidirectional bandwidth in the denominator; citing the final revision is recommended.
NVIDIA Blog on NCCL 2.12 - the PXN mechanism and the canonical definition of rail-optimized: NIC-0 of all nodes connected to one leaf, NIC-1 to the next.
DeepSeek V4 Technical Report and Kimi K3 Card - model parameters and Expert Parallelism requirements for inter-node bandwidth.
