Boston Labs unboxes UK's FIRST DGX H100
Targeting AI and Natural Language Processing, the DGX-H100 forms the foundation for the NVIDIA DGX SuperPOD, a true scaled up AI Supercomputer.
DGX H100 in stock
WHAT IS THE DGX-H100, EXACTLY?
The NVIDIA DGX-H100 is a high-performance computer (HPC) that is designed to accelerate the development and deployment of AI applications. Powered by eight NVIDIA H100 Tensor Core GPUs, the H100 can provide up to 6x the performance of previous generation DGX systems. Seriously snappy!
With the DGX-H100, NVIDIA bring together their latest hardware and infrastructure product line alongside NVIDIA’s proven software ecosystem and expertise to create an unrivalled, unified AI development and training solution.
Not only a hardware solution, DGX H100 platforms include NVIDIA Base Command™ and the NVIDIA AI Enterprise software suite, plus expert advice from NVIDIA DGXperts to help ensure you get the most out of your investment.
DGX platforms are optimised for a range of NVIDIA AI software solutions and rich ecosystem of third-party supported apps. Each can be deployed in minutes, via ready to go containers from NVIDIA GPU Cloud, enabling you to take advantage of the latest updates and developments in AI frameworks, without the time-consuming upkeep of self-install and maintain solutions.
PACKAGING AND CONTENT
The server was shipped in a typical NVIDIA cardboard packaging with a lift over box design. Contained in the box is the usual tool-less rail kit and accessories, which included power cables and screws. Also contained in the box is the striking and unique gold coloured NVIDIA front fascia.
Chassis
Contained in an 8U server chassis with a depth of 35.3inches and a weight of 130.45Kg which, to put into context, is more than a Grizzly bear!
The DGX-H100 is certainly eye catching and will need some consideration when planning rack layouts!
Fig. 3 Close-up view of chassis
The front of the chassis is a simple affair, containing a power button which doubles as a status LED, UID and fault indicator along the right hand.
Fig. 4 front view of chassis
Behind the bezel are the twelve dual fan modules, below which are the eight U.2 NVMes drives used as a data cache. Completing the front is the front console housing VGA and USB ports.
The DGX H100 chassis is comprised of several independent modules, each handling discrete functions in the system that integrate into the main chassis.
The GPU tray is found at the top rear of the system, with the motherboard tray underneath.
Fig. 5 DGX-H100 exploded chassis
At the very bottom of the system are the six hot swappable power supplies, which are bussed together, and active at the same time.
Fig. 6 Rear view with GPU tray removed
Fig. 7 Front view showing heatsinks
Fig. 8 Rear view of fans with GPU tray removed
The chassis holds everything together in a nice modular package. Neat!
What's inside?
To peek inside, we need remove the Motherboard and GPU trays, both of which can be accessed via the rear only and sit above the Power Supplies.
Motherboard Tray
The motherboard tray can be removed from the rear of the chassis, and contains a custom motherboard based on the Intel C622 chipset. This houses the Dual 56core Intel Xeon 8480C processors, running at a base clock speed of 2Ghz. Supporting a maximum 8 channels of DDR5 memory per CPU, this server is fully populated with 32 pieces of Micron 64GB DDR5-4800 DIMMS and supports PCIe 5.0 natively with a maximum of 80 PCI Express Lanes.
Fig. 9 Removal of motherboard tray
Shown below is a view of the motherboard tray from the top, with the IO ports on the left side and the connectors to the midplane on the right. The CPUs are in the middle section with the 32 memory sockets alongside.
To the right of the CPUs, we see the two DGX H100 networking modules, each containing four ConnectX-7 devices, each running at 400Gb/s, these connect to the midplane through the connectors shown on the right side of the tray diagram, whilst the ports for these cards are routed to the back of the system, as shown here, on the left-hand side of the diagram, via the light blue colored DensiLink cables.
Fig. 10 Densi Link Connectors for ConnectX-7
Completing the layout, on the left side of the tray are two PCIe riser cards that connect the standard PCIe form-factor version of the ConnectX-7 dual port card, and a carrier where the two M.2 drives for the OS are installed. Each DGX H100 also includes two NVIDIA BlueField-3 DPU (Data Processing Units) for intelligent, hardware-accelerated storage, security and network management functions.
Fig. 11 Top view of motherboard tray
GPU Tray
As the name would suggest, the GPU tray is home to the H100 Tensor Core GPUs. Eight of them to be exact, each with 80GB RAM per GPU for a total capacity of 640GB per DGX-H100 node.
Fig. 12 H100 GPUs
To the left of the GPUs are 3rd generation NVSwitches which enable 4th generation NVLink interconnectivity at 25GB/s per link, between the GPUs. This removes and overcomes the bottleneck of PCIe and provides 7x the bandwidth of PCIe Gen 5.
Fig. 13 NVIDIA H100 GPU on SXM5 module
Presented in the new SXM5 form factor, these 9th generation Hopper based H100 GPU’s form the core of the DGX-H100 system, with each system delivering 32 petaFLOPS of performance. The SXM5 board is also home to the HBM3 memory stacks, an industry first!
When combined with InfiniBand interconnectivity, and latest AI/HPC models, the H100 can deliver up to 30 times the performance of the previous generation A100.
DETAILED SPECS
For those interested in all the finer technical details the full specification, as provided by NVIDIA > https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise-dgx
CONCLUSION
The DGX-H100 is a brilliant AI system that is packed with the best combination of hardware and software. It is a powerful tool that can be used to accelerate a wide range of AI workloads. If you are looking for a high-performance AI system with best of breed software and expertise, the DGX-H100 should be top of the list.
At Boston Labs, we strive to empower our customers by providing the knowledge and resources needed to make informed decisions on the right hardware, software and overall solution for their specific needs.
If you’d like to test drive the DGX-H100 or learn more about the how Boston can help you, our team are standing by to work with you and bring your next IT project to reality.
Contact our knowledgeable sales and technical team today at [email protected] or call us on 01727 876100 and experience the capabilities of the NVIDIA DGX range.
Author: Hemant Mistry - Senior FAE at Boston Limited