The exponential growth of Generative AI, Large Language Models (LLMs), and complex High-Performance Computing (HPC) simulations require dozens, hundreds, or even thousands of GPU working in concert. The latest GPU technologies often steal the limelight but can put strain on a critical bottleneck in datacentres: the network. You can have thousands of the fastest GPUs on the planet, but if they can’t talk to each other fast enough, they sit idle waiting for data and not reaching their maximum throughput.
Enter Cornelis Networks and their CN5000 400Gbit Omni-Path technology—a scale-out fabric designed explicitly to eliminate bottlenecks and keep massive compute clusters fed.
We obtained samples of their Switch and HFA PCIe card for testing because of customer interest in this technology. We’ll be testing it ourselves and setting up environments for customers to verify it works in their application. Let’s jump into a breakdown of what makes the CN5000 family an exciting proposition for AI and HPC infrastructure.
From 100G to 400G: A Generational Leap
Omni-Path Architecture (OPA) was originally developed by Intel before spinning out into the independent entity Cornelis Networks. While previous generations pushed 100Gbps, respectable for the time, the demands of modern HPC and AI workloads require significantly more bandwidth.
The CN5000 generation jumped to 400Gbps of bandwidth per port using PCIe Gen 5. Cornelis have proved out their commitment to develop the existing technology beyond the original 100Gbps speeds, and have a technology roadmap, with CN6000 800G multi-protocol switches and SuperNICs on the horizon
Raw speed is not the complete story; the real magic is how Omni-Path handles traffic under extreme load scenarios.
The Hardware: Built for Modern Scale
The CN5000 isn’t just a switch; it’s an ecosystem consisting of Host Fabric Adaptors (SuperNICs), switches, and open-source software.
The Adapters (Host Fabric Adaptors)
Powered by an ASIC internally known as “Jackal River” (Some may remember the prior 100G omni-path being “Wolf River”), the CN5000 adapters sit inside the server and handle the heavy lifting. They utilize DMA (Direct Memory Access) engines to pull data directly into the card, bypassing the CPU to save processing overhead. This ultimately results in Remote DMA (RDMA) support for applications. They also move fabric management tasks to an integrated processor on the card, freeing up the host system to focus entirely on computation.
CN5000 400Gbit HFA PCIe card from our lab
When you see the card the first think to notice is the bright copper heatsink. Many manufacturers often have either various grades of silver or black. The bright copper and distinctive shaped heatsink always made the first generation omni-path instantly identifiable for me and suspect this will be the same going forward. The Heatsink is partially covered in a mylar air-shroud, presumably to funnel airflow. It features a single QSFP112 port and the QSFP cage has additional fins around it to help with heat dissipation. It comes in a standard PCIe formfactor. It’s a Half-Height Half-Length (HHHL) and can swap brackets between Full or Half Height.
CN5000 400Gbit HFA PCIe card from our lab with shroud removed
The Switches
Cornelis offers two main switch topologies to fit different cluster sizes:
CN5000 Edge Switch: A 1U switch delivering 48 ports of 400G connectivity (providing 38.4 Tbps of aggregate bandwidth).
CN5000 Director Class Switch: A behemoth, high-density chassis designed for hyperscale environments. It scales up to 576 ports of 400G, consolidating network tiers and reducing cabling complexity to keep latency incredibly low across thousands of nodes. Cornelis Networks CN5000 Overview: An Exciting Alternative - Boston Limited
Image Credit: Cornelis Website
Both switches are available in air-cooled and liquid-cooled configurations, acknowledging the immense heat generated by modern, dense data centres. Most deployments are likely to see the more traditional edge switch, however for larger deployments and those situations that benefit from non-blocking architecture, the director class system could remove necessity of a super spine. This would reduce cabling quantity, complexity, and reduce hops and thus latency.
Both Port-to- Fan and Fan-to-Port configurations are available depending on your rack and server configuration.
We’ve got one of the standard 1U 48-port CN5000 Switches in our lab. All silver utilitarian construction with the 48-ports taking center stage, with the outer 12 ports of each side oriented vertically, and the remaining 24 in 3 groups of 8 in the middle.
front of CN5000 Edge Switch
On the “back”, fan end of the switch, we’ve got a PSU on either side, two removable fan modules, and in the middle is the management port and single USB A. Worth noting the PSUs don’t use the standard C13/C14 connectors and instead have a proprietary locking connector, but the supplied cables lock satisfyingly and does then connect via the C13/C14 connection on a standard rack PDU.
Cornelis Networks CN5000 Overview: An Exciting Alternative - Boston Limited
Zero-Congestion Networking
Why not just use standard Ethernet? Ethernet was built for general-purpose networking and often struggles with the heavy “microbursts” of traffic inherent in AI workloads, leading to packet loss and retransmissions. Whilst there are mitigations, they can result in additional complication, or cost (such as in the case of over-provisioning bandwidth). Omni-Path takes a different approach.
- Credit-Based Flow Control: Unlike traditional ethernet networks that send data and hope it arrives, and any resulting lost packets need resending in a higher level of the network stack (e.g. TCP), CN5000 uses a credit system. A sender only transmits a packet if it knows the receiver has the buffer space (credits) to accept it. This completely eliminates buffer overruns and dropped packets at the hardware layer.
- Fine-Grained Adaptive Routing: The network continuously builds a real-time “heatmap” of switch buffer utilization via advanced telemetry. If one path gets congested, the fabric automatically routes traffic around the hotspot on a packet-by-packet basis.
- Link-Level Replay: If an error occurs on a link, the hardware catches it and retransmits it immediately at the link level, rather than waiting for an latency-inducing end-to-end software retransmission.
What This Means for Workloads
For AI training and inference, this means predictable, ultra-low latency data sharing and retrieval. Cornelis claims the CN5000 delivers up to six times faster collective communication compared to RoCE (RDMA over Converged Ethernet), drastically reducing the time it takes to train massive parameter LLMs.
For HPC applications like Computational Fluid Dynamics (CFD), climate modelling, and genomic research, the network’s high message rate (over 800 million packets per second) allows tightly coupled parallel simulations to complete up to 30% faster than comparable 400G NDR InfiniBand due to it’s packet handling ability. This isn’t a completely theoretical number either. Cornelis has a Wipepaper on Anysys Fluent, a fluid simulation software that can trigger million of small MPI messages. In these circumstance, message rates and latency become predominant factor, and not the raw bandwidth link.
For storage platforms, many support or even require RDMA in order to get the performance, which the CN5000 platform can deliver, we’re talking the likes of Weka, VDURA, BeeGFS. These amongst others are already technology partners and are supported.
Ultimately, the Cornelis CN5000 isn’t just moving data faster; it’s ensuring the most expensive parts of the data center—the GPUs and CPUs— are continually fed with data, and never have to wait for the information they need to do their jobs.
Compatibility: Bridging Infrastructure and Reducing Vendor Lock-In
It is designed for universal interoperability and being a standard PCIe device, it supports CPUs, GPUs, and accelerators from AMD, Intel, and NVIDIA. On the software side, it utilizes the open-source OpenFabrics (OFI/libfabric) stack rather than proprietary equivalents, making it easier to deploy in heterogeneous, multi-vendor environments.
Cornelis Omni-Path 400G Fabric is backward compatible with the previous Intel Omni-Path 100G fabric. There are also options to bridge both Omni-path and existing InfiniBand architecture through the use of LNet Routers. Both of these options allow you to leverage existing infrastructure without complete replacement or significant disruption.
Image Credit: https://wiki.lustre.org/LNet_Router_Config_Guide
A last option for IP-based networking exists over the Omni-Path networking, utilizing the IPoFabric and uses the ib_ipoib packages. This allows use of the Omni-Path fabric for applications that are older, may not support the software stack, and can prevent the need to run another parallel high-speed fabric. If the main use-case only supports and requires IP-based networking, a traditional Ethernet network is could be more appropriate
Support: Drivers and Documentation
Cornelis have all the documentation, firmware, drivers and technical advisories on their website. There’s no login, paywall, or customer entitlement numbers to enter just to quickly grab a copy of the driver which makes life simple designing, building and maintaining systems.
Image Credit: Screenshot July 2026 of Cornelis’s Website
Boston Labs: What’s next?
At Boston Labs we test equipment and put it through it’s paces to build our internal expertise and be sure we’re recommending the right solution. We have a dedicated Lab (Hence the name!) where we can test the latest and greatest technologies. That could be the latest 10U, 14KW GPU behemoth, a small 35W accelerator, a new software stack, or even network technologies from across vendors and platforms. We stress the hardware with a mix of synthetic and real-world loads to find it’s limits, strengths, weaknesses, and sometimes even the breaking point. Using this knowledge, we can create both reference and bespoke designs of both individual systems and clusters.
This facility enables real, physical, proof of concept (POCs) to be conducted on behalf of a customer before committing to a solution, or even offering a remote test-drive on the bare-metal, so customers can see for themselves the performance of features of a given solution.
We’re in the process of benchmarking and testing out the Cornelis CN5000 platform, at which point we’ll be making it available for general customer testing. If you are interested in more information, a test-drive, or possible POC, please reach out you account manager who will be more than happy in finding out if the Cornelis CN5000 platform meets you requirements.