NVIDIA’s Post

Intel released the November 2023 update to its MLPerf Training 3.1 results and delivered a 103% performance increase compared to their projection of 90% back in June. There are only three accelerators that are currently submitting GPT-3 results on #MLPerf right now: Intel, NVIDIA and Google - making Intel's Gaudi 2 currently the only viable alternative to NVIDIA's GPUs (is that even the correct term anymore?) for MLPerf AI workloads. https://lnkd.in/ehs9sbzR

Nvidia rushes to deliver modified AI GPU chips to China customers, allegedly places 'Super Hot Run' priority order with TSMC | Tom's Hardware: Nvidia has urgently ordered AI and HPC GPUs from TSMC for its Chinese customers under the 'Super Hot Run' category, expected to be fulfilled in Q1 2024. These GPUs are designed to meet U.S. export regulations and cater specifically to the Chinese market. The exact details of the order and TSMC's role remain undisclosed. - Artificial Intelligence topics! #ai #artificialintelligence #intelligenzaartificiale

o BraneCell's custom, qubit-based, neural network, AI chips. o Potentially > $/chip revenue than Nvidia's recent GPU (https://lnkd.in/dRZwy2f7) o With minimum run of $ 1 Million non-recurring engineering. This is a high-profit, dual product business: service (chip tailoring) and hardware (chip fab). o TAM > $ 200 Billion/yr USD, Data Centers and Edge, by 2027. o ASIC market generally 1/10th, but BraneCell's superposition features expands the custom chip applicability (combining features of the ASIC+AI+QPU markets).

Interesting approach. Lamini Doubles down on AMD GPUs. If they get the performance they need, they can lead the way for meaningful competition for NVIDIA

Learn how to achieve top inference performance with the NVIDIA H100 Tensor Core GPU and NVIDIA TensorRT-LLM. They enable models like Llama 2 70B to execute using accelerated FP8 operations on H100 GPUs while maintaining inference accuracy. #DataCenter #AI #NVIDIADGX

PNY certifies NVIDIA H100 within DDN Labs, Demonstrating Real-Time GPU Processing without IO Bottlenecks Bordeaux, October 4th – PNY Technologies, a leading provider of #ai solutions & DDN Storage, a global leader in #Ai & #HPC storage, proudly announce the successful benchmarking of NVIDIA #H100 within DDN Labs, achieving World Premiere Performance. The H100 and DDN Storage #A3i showcased unparalleled ability to handle data-intensive workloads without any #GPU wait time for #IO. This solidifies the solution as a game-changer for organizations requiring seamless, uninterrupted GPU utilization, eliminating IO bottlenecks and maximizing computational efficiency. "We are thrilled to collaborate with DDN," said Laurent Chapoulaud, PNY Marketing Director EMAI. "This live demonstration clearly showcases that customers can rely on NVIDIA H100 & DDN to deliver real-time GPU processing, ensuring smooth and uninterrupted operations without any wait time for IO. It's best-in-class in GPU utilization and ease of deployment." “By achieving wire speed for IO, Businesses can now fully harness the power of GPUs, enabling faster data analytics, accelerate machine learning algorithms, and power other demanding workloads effortlessly.”, adds Laurent Thiers from DDN. As data-intensive workloads grow, organizations can confidently and elegantly expand their GPU and DDN infrastructure, knowing that the solution will seamlessly and linearly scale. Ask us the NVIDIA DDN References Architectures for the success of your Ai Projects.

To resolve the high failure rate of Nvidia H100 GPUs and HBM3 memory during Meta's large language model training, a multi-pronged approach is necessary: 1. Improve GPU and memory reliability: Nvidia should work closely with Meta to identify the root causes of the failures and implement design changes to enhance the durability and thermal management of the H100 GPUs and HBM3 memory. This may involve optimizing manufacturing processes, enhancing cooling solutions, and improving quality control measures. 2. Develop more robust diagnostic tools: Meta's team leveraged tools like PyTorch's NCCL flight recorder to quickly diagnose and resolve issues related to NVLink and RoCE failures. Expanding the capabilities of these diagnostic tools to provide even earlier detection and localization of GPU and memory faults can help minimize the impact of failures. 3. Implement advanced failure mitigation strategies: Techniques like straggler GPU identification and prioritization of problematic communications helped Meta maintain overall training efficiency. Exploring additional mitigation strategies, such as dynamic resource allocation and redundancy, can further reduce the impact of individual component failures. 4. Optimize power delivery and cooling: The large fluctuations in power consumption by the GPU cluster stressed Meta's data center infrastructure. Ensuring adequate power capacity and implementing advanced cooling solutions tailored to the H100's thermal profile can help maintain stable operating conditions and minimize failures. 5. Collaborate with Nvidia on firmware and driver optimizations: Optimizing the firmware and drivers for the H100 specifically for large-scale training workloads can help uncover and address potential issues that may contribute to the high failure rate. By addressing the hardware reliability, diagnostic capabilities, failure mitigation strategies, infrastructure support, and software optimizations, Meta and Nvidia can work together to significantly reduce the failure rate and maintain high availability for large-scale AI training workloads on the H100 platform. Citations: [1] Faulty Nvidia H100 GPUs and HBM3 memory caused half of failures ... https://lnkd.in/gxRv2hjP [2] Faulty Nvidia H100 GPUs and HBM3 memory caused half of failures ... https://lnkd.in/g3xPnEAV [3] NVIDIA H100 GPU Specs and Price for ML Training and Inference https://lnkd.in/gw6ku_Xa [4] NVIDIA at SC23: H200 Accelerator with HBM3e and Jupiter ... https://lnkd.in/ga9zjaT2 [5] NVIDIA H100 Tensor Core GPU https://lnkd.in/graMc9vB NVIDIA Meta

🚀 NVIDIA's H100 Tensor Core GPU, powered by the TensorRT-LLM, achieves top-notch inference performance. 🎉 The open-source TensorRT-LLM, optimized for the Hopper architecture, accelerates models like Llama 2 70B with impressive FP8 operations. 📊 In a recent comparison, the H100 outpaces AMD's MI300X chip by 2x when properly benchmarked. 🚅 The DGX H100 server's measured performance showcases its capability, processing over five Llama 2 70B inferences per second with fixed response time, making it a game-changer for AI applications. 🔗 https://lnkd.in/dSSXkNB6 🔗 https://lnkd.in/d63zhbta #ai #aicommunity #nvidiadgx #NVIDIA #performance #inference #llm #artificialintelligence OpenZeka Cordatus AI

#Nvidia new #AI chip 30 times faster than anything on the market 🤯 The AI arms race just got even faster. With over 200 billion transistors power the B200 "Blackwell" superchip will accelerate the speed, scale and scope of what AI can do. Also, power and potential are matched by efficiency. Putting this into perspective: ‘Training a 1.8 trillion parameter model would have previously taken 8,000 Hopper GPUs and 15 megawatts of power, Nvidia claims. Today, Nvidia’s CEO says 2,000 Blackwell GPUs can do it while consuming just four megawatts.’ Things just got hotter 🤔 #ArtificialIntelligence #Technology #Innovation Dr Andy Campbell Esther O'Callaghan OBE Sean Frew Theofilos Tzanidis Luke Judge Fraser Maitland Patrick Afchain Dr Emma Reid Kevin Thomson Dr Andy Hogg Thorsten Brønholt, PhD https://lnkd.in/ecYZbx9S