During its GPU Technology Conference, Nvidia announced the world’s most powerful chip for AI-related computing called GB200, which powers the Blackwell B200 GPU. It is a successor to the H100 AI chip and offers significant improvements in performance and efficiency.
The new B200 GPU is capable of 20 petaflops of FP4 thanks to the 208 billion transistors inside the chip. In addition, the GB200 has 30 times the performance of H100 in LLM inference workloads while reducing energy consumption by 25 times. In the GPT-3 LLM benchmark, the GB200 is also seven times faster than the H100.
For example, training a model with 1.8 trillion parameters would require 8,000 Hopper GPUs and about 15 megawatts, while a set of 2,000 Blackwell GPUs could do it for just 4 megawatts.
To further improve efficiency, Nvidia designed a new network switch chip with 50 billion transistors that can handle 576 GPUs and let them communicate with each other at 1.8 TB/s of bi-directional bandwidth.
In this way, Nvidia addressed a problem with communication as before, where a system combining 16 GPUs would spend 60% of the time on communication and 40% of the time on computing.
Nvidia says it offers companies a complete solution. For example, the GB200 NVL72 allows for 36 CPUs and 72 GPUs in a single liquid-cooled rack. A DGX Superpod for DGX GB200, on the other hand, combines eight of those systems into one, making 288 CPUs and 576 GPUs with 240 TB of memory.
Companies such as Oracle, Amazon, Google, and Microsoft have already shared plans to integrate the NVL72 racks for their cloud services.
The GPU architecture used for the Blackwell B200 GPU is likely to be the foundation of the upcoming RTX 5000 series.
Source: GSMArena