10/17/24, 6:49 PM Unleashing the Potential of Alternative Deep Learning Hardware - EE Times

en nach Tromso München nach Norwegen

JETZT BUCHEN €54 JETZT BUCHEN

Advertisement

 

DESIGNLINES | AI & BIG DATA DESIGNLINE

Unleashing the Potential of Alternative Deep Learning Hardware

OPINION
By Simran Khokha 07.11.2024 0
GPUs have become synonymous with deep learning (DL) and AI, powering numerous advancements in
technology. These highly parallel processors have enabled breakthroughs in image recognition, natural
language processing (NLP) tasks and more. However, as the demand for more efficient, scalable and
specialized hardware solutions grows, it is crucial to explore alternatives beyond GPUs.

Current state of GPUs

The GPU market is experiencing unprecedented growth. According to recent market analysis, the GPU
market is expected to grow significantly in the coming years. Dominated by a few key players, with Nvidia
leading the pack, GPUs have become the go-to hardware for rendering and DL tasks. Nvidia holds an
overwhelming 87% of the market share, with AMD trailing at 10% and other competitors sharing the remaining
3%.

This concentration highlights the limited diversity in GPU options, potentially stifling innovation and driving up
costs. GPUs excel at parallel processing, making them ideal for the matrix and vector computations standard
in DL. Despite their prowess, GPUs often leave some of their potential untapped. Specific applications might
not fully utilize GPUs’ massive parallelism, leading to inefficiencies. For instance, some DL tasks require
sequential processing or have irregular memory access patterns that do not align well with the GPU
architecture.

PARTNER CONTENT VIEW ALL 

ABLIC is advancing the creation of high-value analog-digital integration, aiming for a

leading position in niche semiconductor fields

By ABLIC 10.15.2024

Engineering Considerations for Connectors in Renewable Energy Applications

https://www.eetimes.com/unleashing-the-potential-of-alternative-deep-learning-hardware/ 1/5
10/17/24, 6:49 PM Unleashing the Potential of Alternative Deep Learning Hardware - EE Times

By EDAC 10.08.2024

GUC Announces Adoption of HBM3E IP by CSP Data Center

By Global Unichip Corp. 09.24.2024

Additionally, optimizing code to maximize GPU utilization can be complex and time-consuming, meaning that
not all software can leverage the full power of GPUs. These factors contribute to the underutilization of GPU
capabilities in specific scenarios. The energy consumption of GPUs is also a growing concern—particularly as
AI models become larger and more complex.

Beyond GPUs: Emerging alternatives

To address the limitations of GPUs, researchers and engineers are exploring various alternative hardware
solutions. These alternatives fall into three categories: general-purpose hardware, dedicated DL hardware
and neuromorphic hardware.

General-purpose hardware

While GPUs have been the cornerstone of DL, general-purpose hardware like CPUs continues to evolve. The
debate between CPU and GPU performance for DL is ongoing. Traditionally, CPUs were considered less
efficient for DL due to their lower parallel processing capabilities. However, advancements in CPU
architecture and specialized software optimizations are narrowing the gap.

The NeoCPU, a notable development, aims to improve DL performance on general-purpose processors.

NeoCPU integrates advanced instruction sets and optimization techniques to handle DL workloads efficiently.
Specifically, NeoCPU incorporates features like hardware accelerators for matrix operations and enhanced
memory bandwidth to manage the data-intensive nature of DL tasks better.

Tools like OpenVINO from Intel make deploying AI models on CPUs easier. OpenVINO accelerates DL
inference by optimizing models to leverage Intel hardware features, providing significant performance
improvements without needing specialized GPUs. Intel’s CPUs’ ubiquity and constant advancement prompt a
closer look at these versatile processors as viable alternatives for DL tasks. CPUs are widely available and
cost-effective, making them attractive for specific applications—especially when combined with advanced
optimization tools.

Dedicated DL hardware

DL computations specifically require dedicated hardware, which improves performance and energy efficiency.
The rapid expansion of DL applications has spurred many startups and established companies to develop
specialized DL hardware.

Google’s TPUs are among the most well-known dedicated DL hardware. Google designed TPUs to accelerate
matrix multiplications—a fundamental operation in neural networks. The TPU architecture includes the matrix
multiplier unit (MXU), unified buffer (UB) and activation unit (AU). A systolic array optimizes the MXU’s power
and area efficiency. Unlike GPUs, which use vectors and scalars, TPUs operate on matrices, providing
significant performance gains for DL tasks.

Amazon’s Inferentia chip is another prominent example. Designed for high-performance DL inference,
Inferentia incorporates four NeuroCores for tensor-level operations and features a large on-chip cache to
minimize primary memory access. This architecture makes it a powerful tool for large-scale AI applications in
cloud environments. Other notable dedicated hardware includes Graphcore’s IPUs and Intel’s Nervana neural
network processors (NNPs). These processors were specifically designed to improve the speed and efficiency
of DL workloads. For instance, the IPU optimizes its massively parallel architecture for graph-based
computations—a common feature in neural networks.

https://www.eetimes.com/unleashing-the-potential-of-alternative-deep-learning-hardware/ 2/5
10/17/24, 6:49 PM Unleashing the Potential of Alternative Deep Learning Hardware - EE Times

Artist’s rendition of an artificial neural network scheme. (Source: Adobe Stock)

Neuromorphic hardware

Neuromorphic hardware represents a frontier in computing that mimics the brain’s architecture. These chips
aim to replicate the structure and function of the human brain, offering unique advantages for AI applications.

IBM’s TrueNorth chip is a pioneering example of neuromorphic hardware. IBM designed TrueNorth with a
highly connected network of artificial neurons, mimicking the brain’s parallel processing capabilities. Each
neuron can simultaneously store and process data, which significantly differs from traditional von Neumann
architectures, where processing and memory are separate. This design allows for highly efficient data
processing and low power consumption.

Intel’s Loihi chip takes neuromorphic computing a step further by introducing learning capabilities similar to
those of the brain. Loihi uses a spike-timing-dependent plasticity (STDP) model, which adjusts synaptic
strength based on the timing of neuron spikes. This mechanism allows Loihi to learn and adapt in real time,
offering potential advantages for applications requiring continuous learning and adaptation.

Intel’s second-generation Loihi architecture includes spikes with integer magnitudes and
the ability to program different neuron models. (Source: Intel)

Applications and challenges

In AI, neuromorphic chips can help with rapid, lifelong learning—something traditional DL models struggle
with. For example, they can be used in robots to help them learn from their environment and adapt to new

https://www.eetimes.com/unleashing-the-potential-of-alternative-deep-learning-hardware/ 3/5
10/17/24, 6:49 PM Unleashing the Potential of Alternative Deep Learning Hardware - EE Times
situations quickly. In neurology, these chips can help scientists understand how different brain parts work
together to create thoughts, feelings and consciousness.

However, neuromorphic chips are still new and not widely used. There are several challenges to overcome
before they can be adopted on a large scale. These include the complexity of making these chips, as their
intricate design is complex to produce in large numbers. Integrating the software is also challenging, as
existing AI programs must be adjusted to work with neuromorphic chips. Scalability is another issue because
expanding neuromorphic systems to handle large-scale ap plications is challenging.

Despite these challenges, neuromorphic computing holds much promise. Its potential to revolutionize AI
and our understanding of the brain keeps driving research and development in this exciting field.

Sustainability and efficiency

Beyond performance, sustainability is becoming a crucial consideration in hardware development. The energy
consumption of AI systems is a growing concern, particularly as models become more complex. Achieving
green AI through energy-efficient DL is an increasingly important goal.

Neuromorphic computing, in particular, offers significant potential for reducing the energy consumption of AI
systems. By mimicking the brain’s energy-efficient processing methods, neuromorphic chips can perform
complex computations with minimal power. This alignment with broader environmental sustainability goals
makes neuromorphic hardware an attractive option for future AI development.

Dedicated DL hardware, such as TPUs and Inferentia, also offers improvements in energy efficiency.
Designed to perform DL computations with optimal power usage, these specialized processors reduce the
overall energy footprint of AI systems.

In the future, other devices will be more suitable for DL and AI. By exploring and investing in alternatives like
general-purpose CPUs, dedicated DL hardware, and neuromorphic chips, we can unlock new levels of
performance, efficiency, and sustainability. These innovations promise to advance AI’s capabilities, as well as
make AI more accessible and environmentally friendly. Embracing these emerging technologies is essential
as they have the potential to transform the digital landscape.

By adopting a modular and scalable approach to hardware development, similar to the strategies used in
autonomous vehicles, we can ensure that our AI systems are ready to meet the challenges of tomorrow. The
future of AI lies in embracing a diverse array of hardware solutions, each offering unique advantages that
together will drive the next wave of innovation in DL.

—Simran Khokha is a product manager at Infineon Technologies.

Suggested Reading

Register for FREE PowerUP Virtual Event

Silicon 100: Startups Worth Watching in 2024 - EE Times

Leveraging Safety Processor Expertise to Develop RISC-V Based Automotive

Implementations - EE Times Podcast

Apple R1 Opens New Avenues in Spatial Computing - EE Times

Silicon 100: Startups Worth Watching in 2024

Subscribe Today!

Get the latest electronics industry news delivered right to your inbox.
Sign up for our free eNewsletters.

Subscribe

https://www.eetimes.com/unleashing-the-potential-of-alternative-deep-learning-hardware/ 4/5
10/17/24, 6:49 PM Unleashing the Potential of Alternative Deep Learning Hardware - EE Times

RELATED TOPICS: AI AND BIG DATA

RELATED ARTICLES

 ABLIC is advancing the creation of high-value analog- Spin memristor mimics brain for energy efficiency in AI

digital integration, aiming for a leading position in niche


semiconductor fields
eetimes.eu edn.com

https://www.eetimes.com/unleashing-the-potential-of-alternative-deep-learning-hardware/ 5/5