NVIDIA Dominates MLPerf Training Benchmarks with Record-Breaking Performance

TLDR:

• NVIDIA achieved record-breaking performance in MLPerf Training benchmarks.
• The company tripled performance on large language models compared to last year.
• NVIDIA H200 GPU delivers 47% faster performance than H100 GPU.
• Broad partner ecosystem supports NVIDIA’s AI platform.

Great news for those working in Artificial Intelligence (AI)! NVIDIA has secured top marks in the latest MLPerf Training v4.0 benchmarks, showcasing exceptional performance and scalability of its AI platform.

Unprecedented Performance Gains

Compared to its record-setting submission last year, NVIDIA more than tripled the performance on the large language model (LLM) benchmark based on GPT-3 175B. This achievement was fueled by a combination of factors, including a larger scale (more than triple the GPUs used last year) and extensive software optimizations.

The powerful NVIDIA H200 Tensor Core GPU, boasting 141GB of memory and significant bandwidth improvements, also played a significant role. This new GPU extends the performance of its predecessor, the H100, by up to 47%.

Software Optimizations Drive Efficiency

Beyond hardware advancements, NVIDIA’s software stack optimizations played a crucial role. Submissions using a 512 H100 GPU configuration achieved a 27% performance boost compared to last year. This highlights the importance of ongoing software development in maximizing performance, even with existing hardware.

Remarkably, these optimizations resulted in nearly perfect scaling. As the number of GPUs increased from 3,584 to 11,616, the delivered performance mirrored this growth.

Excelling at Emerging AI Workloads

The growing need for customizing large language models is driving the importance of LLM fine-tuning. NVIDIA’s platform excelled in this area, completing the new LLM fine-tuning benchmark (based on Meta Llama 2 70B) in a record 1.5 minutes with 1,024 GPUs.

Furthermore, NVIDIA accelerated Stable Diffusion v2 training by up to 80% and showcased strong performance on the new graph neural network (GNN) test based on R-GAT. These results solidify NVIDIA’s position as a leader in accelerating various AI applications.

Learn more about these optimizations on the NVIDIA Technical Blog.

Broad Ecosystem Support

The extensive participation of ten NVIDIA partners, including ASUS, Dell Technologies, and Lenovo, further emphasizes the widespread adoption and trust in the NVIDIA AI platform across the industry.

The Future of AI Performance

MLCommons, the organization behind the MLPerf benchmarks, plays a vital role in fostering best practices in AI computing. By providing reliable comparisons and keeping pace with the rapid evolution of AI, MLCohttps://mlcommons.org/mmons empowers companies to make informed decisions.

Looking ahead, NVIDIA is poised to deliver even higher levels of AI performance with the upcoming NVIDIA Blackwell platform, designed for training and running massive generative AI models with trillions of parameters.