Multicore Processors: Challenges, Opportunities,

Emerging Trends

Christian Märtin
Faculty of Computer Science
Augsburg University of Applied Sciences
Augsburg, Germany
[email protected]

Abstract— This paper undertakes a critical review of the cur- However, in a similar way as the necessary transition from
rent challenges in multicore processor evolution, underlying complex single core architectures with high operating frequen-
trends and design decisions for future multicore processor cies to multicore processors with moderate frequencies was
implementations. It is first shown, that for keeping up with caused by the exponentially growing thermal design power
Moore´s law during the last decade, the VLSI scaling rules for (TDP) of the complex single core processors for reaching
processor design had to be dramatically changed. In future mul- linear performance improvements [2], the ongoing multicore
ticore designs large quantities of dark silicon will be unavoidable evolution has again hit the power wall and will undergo
and chip architects will have to find new ways for balancing dramatic changes during the next several years [3]. As new
further performance gains, energy efficiency and software com-
analytical models and studies show [4], [5], power problems
plexity. The paper compares the various architectural alterna-
tives on the basis of specific analytical models for multicore
and the limited degree of inherent application parallelism will
systems. Examples of leading commercial multicore processors lead to rising percentages of dark or dim silicon in future mul-
and architectural research trends are given to underscore the ticore processors. This means that large parts of the chip have
dramatic changes lying ahead in computer architecture and to be switched off or operated at low frequencies all the time. It
multicore processor design. has to be studied, whether the effects of such pessimistic fore-
casts will affect embedded applications and system environ-
Keywords—multicore processor, Moore’s law, Post-Dennard ments in a milder way than the software in more conservative
scaling, multicore architectures, many-core architecture, multicore standard and high-performance computing environments.
performance models, dark silicon
In this paper we discuss the reasons for these developments
I. INTRODUCTION together with other future challenges for multicore processors.
We also examine possible solution approaches to some of the
More than 12 years after IBM started into the age of multi- topics. When discussing the performance of multicore systems,
core processors with the IBM Power4, the first commercial we must first have a look on adequate multicore performance
dual core processor chip, software and system developers as models that both consider the effects of Amdahl’s law on
well as end users of business, engineering and embedded appli- different multicore architectures and workloads, and on the
cations still take it for granted, that the performance gains consequences of these models with regard to multicore power
delivered by each new chip generation maintain a more than and energy requirements. We use the models also to introduce
linear improvement over the decade ahead. Moore´s law the different architectural classes for multicore processors.
appears still to be valid as demonstrated by Intel´s fast track
from 32 to 22nm mass production and towards its new 14nm The paper will therefore give an overview of the most
CMOS process with even smaller and at the same time more promising current architectures and predictable trends and will
energy efficient structures every two years [1]. finally point at some typical implementations of server, work-
station, and embedded multicore chips. Multicore processor
Very successfully and at the extreme end of the perfor- implementations in the same architectural class may vary sig-
mance spectrum, Moore´s law is also expressed by the indus- nificantly depending on the targeted application domain and
try´s multi-billion transistor multicore and many-core server the given power budget. As will be shown, the trend towards
chips and GPUs. Obviously the transistor raw material needed more heterogeneous and/or dynamic architectures and innova-
for integrating even more processor cores and larger caches tive design directions can mitigate several of the expected
onto future chips for all application areas and performance problems.
levels is still available.

embedded world Conference 2014

 II. MOORE´S LAW AND DARK SILICON ciency can only be improved by 40 % per generation. This
The major reason for the current situation and the upcom- dramatic effect, called dark silicon, already can be seen in
ing trend towards large areas of dark silicon are the new scaling current multicore processor generations and will heavily affect
rules for VLSI design. Dennard’s scaling rules [6] were per- future multicore and many-core processors.
ceived in 1974 and have held for more than 30 years until These considerations are mainly based on physical laws
around 2005. As is well known, power in CMOS chips can be applied to MOSFET transistor scaling and CMOS technology.
modeled as: However, the scaling effects on performance evolution are
confirmed by researchers from industry. With the advances in
transistor and process technology, new architectural
𝑃 = 𝑄𝑓𝐶𝑉 ! +  𝑉𝐼!"#$#%" (1) approaches, larger chip sizes, introduction of multicore proces-
sors and the partial use of the die area for huge caches micro-
𝑄 is the number of transistor devices, 𝑓 the operating fre- processor performance increased about 1000 times in the last
quency of the chip, 𝐶 the capacitance and 𝑉 the operating volt- two decades and 30 times every ten years [3]. But with the new
age. The leakage current 𝐼!"#$#%" could be neglected until 2005 voltage scaling constraints this trend cannot be maintained.
with device structures larger than 65nm. Without significant architectural breakthroughs Intel’s Shekar
With Dennard’s scaling rules the total chip power for a Borkar only predicts a 6 times performance increase for stand-
given area size stayed the same from process generation to ard multicore based microprocessor chips in the decade
process generation. At the same time, with a scaling factor between 2008 and 2018. Interestingly enough, this projection –
although directly derived from the company’s planned logic
𝑆 = 2  , feature size shrinked at a rate of 1/𝑆 (the scaling transistor budget, planned cache size, 2x increase in operating
ratio)1, transistor count doubled (Moore´s law) and the frequency and 3x increase by redesigned transistor devices –
frequency increased by 40 % [5] e very two years. With feature corresponds to the theoretical 40% increase per generation and
sizes below 65nm, these rules could no longer be sustained, over five generations in Post-Dennard scaling that amounts to
because of the exponential growth of the leakage current. To
𝑆 ! = 5.38.
lessen the leakage current, Intel when moving to 45nm,
introduced extremely efficient new Hafnium based gate Therefore, computer architects and system designers have
isolators for the Penryn processor. When moving to 22nm, to find effective strategic solutions for handling these major
Intel optimized the switching process by using new 3D FinFET technological challenges. Formulated as a question, we have to
transistors that are currently used in the Haswell processors and ask: Are there ways to increase performance by substantially
will also be scaled down to 14nm. more than 40% per generation, when novel architectures or
heterogeneous systems are applied that are extremely energy-
However, even these remarkable breakthroughs could not
efficient and/or use knowledge about the software structure of
revive the scaling of the operating voltage, because no further
the application load to make productive use of the dark silicon?
scaling of the threshold voltage is possible as long as the oper-
ating frequency is kept at the current already very low level. The rising amount of transistors used for on-chip L3 caches
Therefore operating voltage has remained at a constant value of was the first reaction of multicore chip makers to counter this
around 1 V for several processor chip generations. unavoidable situation. Larger caches need less than 1/10 of the
power of logic transistor blocks. But there are limits to the size
With Post-Dennard scaling, like with Dennard scaling the
of useful caches. Another reaction could be hierarchical on-
number of transistors grows with 𝑆 ! and the frequency with 𝑆
chip networks, so far not used on processor dies, that would
from generation to generation, i.e. the potential computing keep communication costs and power in many-core chips as
performance increases by 𝑆 ! or 2.8 between two process low as possible [3]. Another direction could be the 3D organi-
!
generations. Transistor capacitance also scales down to under zation of multicore processing chips with eDRAM dies, as Intel
!
both scaling regimes. However, as threshold and thus operating has demonstrated in [8].
voltage cannot be scaled any longer, it is no longer possible to
keep the power envelope constant from generation to genera- III. MODELING PERFORMANCE AND POWER OF MULTICORE
tion and simultaneously reach the potential performance ARCHITECTURES
improvements. Whereas with Dennard scaling power remains When Gene Amdahl wrote his famous article on how the
constant between generations, Post-Dennard Scaling leads to a amount of the serial software code to be executed influences
power increase of 𝑆 ! = 2 per generation for the same die area the overall performance of parallel computing systems, he
[7]. At the same time utilization of a chip’s computing re- could not have envisioned that a very fast evolution in the
!
sources decreases with a rate of ! per generation. fields of computer architecture and microelectronics would
!
lead to single chip multiprocessors with dozens and possibly
This means that at runtime large quantities of the transistors hundreds of processor cores on one chip. However, in his
on the chip have to be switched off completely, operated at paper, he made a visionary statement, valid until today, that
lower frequencies or organized in completely different and pinpoints the current situation: “the effort expended on
more energy efficient ways. For a given chip area energy effi- achieving high parallel processing rates is wasted unless it is
accompanied by achievements in sequential processing rates of
1 very nearly the same magnitude” [9], [10].
An error was removed. The original text was: “At the same
time feature size shrinked at a rate of 𝑆 = 1/ 2”
 Today we know, that following Pollack’s rule [3], single fixed application sizes, the achievable maximum speedup,
core performance can only be further improved with great whether multiprocessors or multicores are used, is always lim-
effort. When using an amount of 𝑟 transistor resources, scalar !
ited to 𝑆!"# =   . With a fixed application size, the models
!!!
performance rises to 𝑃𝑒𝑟𝑓   𝑟 =   𝑟. At the same time single
presented by Hill and Marty also reach their performance limits
core power 𝑃 (or the TDP) rises with 𝑃 =   𝑃𝑒𝑟𝑓 ! [2] with
relatively soon, and for workloads with 𝑓 < 0.99 a large num-
𝛼 = 1.75 or nearly the square of the targeted scalar perfor- ber of cores does not seem adequate.
mance. For the rest of this paper we assume that 𝑃 includes
dynamic as well as leakage power. Thus, there is a direct
relation between the TDP of a sequential core, the transistor
resources and the performance. Power can be expressed, as
shown in [11] by:

𝑃 =   𝑃𝑒𝑟𝑓 ! = ( 𝑟)∝ = 𝑟 ∝/! (2)

When Grochowski derived his equation from comparing

Intel processor generations from the i486 to the Pentium 4
Cedarmill (all of them normalized to 65nm technology), multi-
core evolution was slowly gathering momentum and the Post-
Dennard scaling effects only were in their initial phase. With Fig. 2. Asymmetric multicore processor consisting of a complex core with
current technology, leakage currents and not further down- 𝑃𝑒𝑟𝑓 = 4 and 12 BCEs.
scalable supply voltages would forbid the use of the entire die
area of mainstream microprocessor chips for a single core In the engineering and embedded system domains there are
processor. Therefore analytical multicore models for existing many standard and real time workloads with moderate data
or envisioned architectural alternatives have to consider the size, irregular control and data flow structures and with a lim-
implications of different workload characteristics, both, to the ited degree of exploitable parallelism. For these workload
performance, and the power requirements of these architec- characteristics, the easily understandable study of Hill and
tures. Marty with some extensions can lead to valuable insights of
how different multicore architectures may influence the
achievable performance.

Fig. 1. Symmetric multicore processor: each block is a basic core equivalent

(BCE) including L1 and L2 caches. L3 cache and onchip-network are not
modeled. The computing performance of a BCE is normalized to 1.

Fig. 3. Dynamic multicore processor: 16 BCEs or one large core

A. Standard Models
with  𝑃𝑒𝑟𝑓 =   16.
In 2008 Hill and Marty started this discussion with perfor-
mance models for three types of multicore architectures: sym- The idea behind the different models is that the given chip
metric (fig. 1), asymmetric (fig. 2), and dynamic (fig. 3) multi- area allows for the implementation of 𝑛 simple cores or so
core chips [12] and evaluated their speedup models for differ- called basic core equivalents (BCEs) with a given performance
ent workloads, characterized by the respective degree 𝑓 of code of 1. To construct the architectural alternatives, more complex
that can be parallelized as in Amdahl’s law, where the Speedup cores are modeled by consuming 𝑟 of the 𝑛 BCEs, leading to a
𝑆 achieved by parallel processing is defined as: performance 𝑝𝑒𝑟𝑓 𝑟 =   𝑟 per core, if we go along with
Pollack’s performance rule for complex cores.
!
𝑆!"#$!! (𝑓) =   ! (3) These considerations lead to the following speedup equa-
!!! ! tions for the three alternatives:
!

In this equation 𝑓 is the fraction of possible parallel work !

and 𝑛 is the number of available cores. As Amdahl’s law uses 𝑆!"# 𝑓, 𝑛, 𝑟 =   !!! !∙! (4)
 !  
!"#$(!) !"#$ ! ∙!

embedded world Conference 2014

 !
𝑆!"#$ (𝑓, 𝑛, 𝑟) =   !!! ! (5)
 !  
!"#$(!) !"#$ ! !!!!  

!
𝑆!"#  (𝑓, 𝑛, 𝑟) =   !!! ! (6)
 !  
!"#$ ! !

Note, that for the asymmetric case in equation (5), the par-
allel work is done together by the large core and the 𝑛 − 𝑟
small cores. If either the large core, or the small cores would be
running, to keep the available power budget in balance, the
equation would change to:

!
𝑆!"#$ (𝑓, 𝑛, 𝑟) =   !!! ! (7)
 !  
!"#$(!) !!!  

Fig. 5. Dynamic multicore with 4  cores and frequency scaling using the
In addition to achievable performance, power (TDP) and power budget of 8 BCEs. Currently one core is at full core TDP, two cores are
energy consumption are important indicators for the feasibility at ½ core TDP. One core is switched off.
and appropriateness of multi- and many-core-architectures with
symmetric and asymmetric structures. Woo and Lee [13] have
extended the work of Hill and Marty towards modeling the C C C C
average power envelope, when workloads that can be modeled
                       
with Amdahl’s law are executed on multicore processors. The
equations for the symmetric case are given here: A A
1      
B B
𝑊 =   !! !!! ! !!!
! (8)
!!!  !  
! C C
D D E E
In this equation 𝑊 is the average power consumption, 𝑘 is            
           
the fraction of power that one core consumes in idle state, and
!"#$%#&'()"
the power of a core in active state is 1. For we get Fig. 6. Heterogeneous multicore with a large core, four BCEs, two acceler-
!"#$%
ators or co-processors of type A, B, D, E, each. Each co-processor/accelerator
uses the same transistor budget as a BCE.
!"#$ !
= (9) B. Models for Heterogeneous Multicores
! !! !!! ! !!!
Heterogeneous architectures on first glance look similar to
However, if we have a look at the evolution of commercial asymmetric multicores. However, in addition to a conventional
multicore processors, in addition to the already discussed complex core, they introduce unconventional or U-cores [11]
architectural alternatives, we meanwhile can see new architec- that represent custom logic, GPU resources, or FPGAs. Such
tural variants for asymmetric systems (fig. 4), dynamic systems cores can be interesting for specific application types with
(fig. 5) and heterogeneous multicore systems (fig. 6). SIMD parallelism, GPU-like multithreaded parallelism, or
specific parallel algorithms that have been mapped to custom
or FPGA logic. To model such an unconventional core, Chung
et al. suggest to take the same transistor budget for a U-core as
for a simple BCE core. The U-core then executes a specific
parallel application section with a relative performance of 𝜇,
while consuming a relative power of 𝜑 compared to a BCE
with 𝜇 =  𝜑 = 1. If 𝜇   > 1 for a specific workload, the U-core
works as an accelerator. With 𝜑 < 1 the U-core consumes less
power and can help to make better use of the dark silicon. The
resulting speedup equation by Chung et al. can directly be
derived from equation (7):

!
𝑆!!"!#$%!&!$'( (𝑓, 𝑛, 𝑟, 𝜇) =   !!! ! (10)
 !  
!"#$(!) !(!!!)  

Fig. 4. Asymmetric multicore with 2 complex cores and 8 BCEs

 This equation is very similar to equation (7) that gives the Borkar are applied, the performance will only increase by 34%
speedup model for asymmetric multicore systems. In their with a power reduction of 74%. The model is evaluated using
evaluation, Woo and Lee [13] show that such asymmetric mul- the 12 real benchmarks of the PARSEC suite for parallel per-
ticores are much more appropriate for power scaling than formance evaluation.
symmetric architectures, either with complex or simple cores.
Going a step further, it follows that heterogeneous multicores
will even scale better, if 𝜇   > 1 and 𝜑 < 1 or at least 𝜑 = 1. The results of the study show that the entire future 8nm
As prototypical results in current research towards a better multicore chip will only reach a 3.7× (conservative scaling) to
utilization of the dark silicon in logic chips demonstrate, future 7.9× speedup (ITRS scaling). These numbers are the geometric
architectures might profit mostly in performance and energy mean of the 12 PARSEC benchmarks executed on the model.
consumption, if not only one, but many additional coprocessors The study also shows that 32 to 35 cores at 8nm will be enough
or accelerator cores are added to one or a small number of large to reach 90% of the ideal speedup. Even though the bench-
cores [14]. marks contain relatively much exploitable parallelism, there is
If, in addition to a conservative complex core, there are dif- not enough parallelism to utilize more cores, even with an
ferent accelerator/co-processor cores on an heterogeneous chip, unlimited power budget. For more realistic standard applica-
they will typically be active on different parts of the entire tions, the speedup would be limited by the available power
application workload. Let us assume that the entire die area can envelope, rather than by exploitable parallelism. The power
gap would lead to large fractions of dark silicon on the multi-
hold resources for 𝑛 = 𝑟 +   𝑟! + 𝑟! + ⋯ + 𝑟! BCEs with 𝑟
core chip. The executable model by Esmaeilzadeh et al. can be
denoting the resources for the complex core and 𝑟! giving the
executed for different load specifications and hardware set-
resources for each specific coprocessor core. We can now
tings. The authors have made it available at a URL at the
derive a more flexible speedup equation that mirrors the real
University of Wisconsin [5].
application structure and that can easily be used to model ap-
plication-specific power and energy consumption: Although this model was devised very carefully and on the
basis of available facts and plausible future trends, it cannot
! predict future breakthroughs that might influence such
𝑆!"#$%!"# =   !!! ! (11) hardware parameters that are currently treated like immovable
 !   !!!! !
!"#$(!) !! ∙!! objects. An example is the slow DRAM access time, which is
responsible for the memory wall and has existed for decades.
The 𝑓!  are exploitable fractions of the application workload, In Esmaeilzadeh’s model it is kept constant through to the 8nm
𝑓 =   𝑓! +   𝑓! + ⋯ + 𝑓! with 𝑓   ≤ 1 and 𝜇! is the relative process, whereas off-chip memory bandwidth is believed to
performance of a coprocessor, when using 𝑟! BCE resources. If scale linearly. Currently memory makers like Samsung and
𝜇! = 1, the coprocessor can be interpreted as a cluster of 𝑟! HMC are working on disruptive 3D-DRAM technologies that
simple cores working in parallel. will organize DRAM chips vertically and speedup access time
and bandwidth per chip by a factor of ten, without needing new
C. Other Studies
DRAM cells [17]. Such and other technology revolutions
In [15] it is shown, that there will be no limits to multicore might affect all of the discussed models and could lead to more
performance scaling, if the application size grows with the optimistic multicore evolution scenarios.
number of available cores, as was stated first for classic multi-
processors by Gustafson [16]. Sun and Chen show that IV. MULTICORE EVOLUTION
Gustafson’s law also holds for multicore architectures and that In order to be able to give meaningful answers to the chal-
the multicore speedup scales as a linear function of 𝑛, if the lenging questions asked in the earlier chapters, we will now
workload is also growing with the system size. They show, that examine the current state and the ongoing evolution of multi-
even, if continuously growing parts of the workload will reside core processor architectures.
in slow DRAM memory, unlimited speedup scaling is still
possible, as long as the DRAM access time is a constant. Note, Industry has successfully maintained Moore´s law for
that without scaling up the workload of the LINPACK bench- nearly 50 years. In the last decade commercial processor chips
mark, the multi-million-core supercomputers presented in the have arrived at a transistor count of several billion transistor
Top 500 list twice every year, would not make sense at all. devices. However, depending on the application domain and
context, there exist various architectural approaches in current
A recent and much more pessimistic study by Esmaeilza- and planned multicore and many-core processors. In the fol-
deh et al. [4] constructs a very fine grained performance model lowing we will examine some important state-of-the art multi-
for future multicore processors. The model includes separate core processors as examples for the classes defined in the last
partial models for Post-Dennard device scaling, microarchi- chapter: symmetric, asymmetric, dynamic and heterogeneous
tecture core scaling, and the complete multicore. The model’s multicore processors.
starting point is the 45nm i960 Intel Nehalem quadcore proces-
sor that is mapped to all future process generations until a ficti- A. Symmetric Multicore Processors
tious 8nm process. Following optimistic ITRS scaling, the During the last three technology generations (45nm to
downscaling of a core will result in a 3.9× core performance 22nm) the number of on-chip cores has not changed dramati-
improvement and an 88% reduction in core power consump- cally for mainstream and high-end server systems by Intel,
tion. However, if the more conservative projections by Shekar IBM, Fujitsu, Oracle, and AMD. Again core microarchitecture

embedded world Conference 2014

performance and energy efficiency were improved and larger Multicore processors manufactured in Intel’s new 14nm
last-level caches were implemented. Much effort by all con- process are codenamed Broadwell. Early Broadwell chips will
tenders is put into the memory system bandwidth optimization. arrive in the form of desktop processors in late 2014. For 2015
Fast buffer caches are inserted between the processor cores and Intel is planning a Broadwell-based Xeon EP chip with 18
the memory controllers. cores.
In the embedded sector we see a differentiation to very AMD Kaveri with Steamroller microarchitecture. AMD’s
small and energy-efficient multicore chips on the one hand, and latest desktop processor Kaveri is manufactured in 28nm tech-
high-performance multicore implementations with mainstream- nology and packs 2.41 billion transistors on its die area of
like micro-architectural features. With its new 22nm Atom 245mm2. At 4 GHz its TDP varies between 45W and 85W
multicore processors, in 2013, Intel has aggressively chal- depending on the graphics load. The dual core processor is
lenged the ARM-dominated market for tablet, and smartphone targeted at the midrange desktop and gaming market and is the
chips. A different picture can be seen in the area of symmetric first AMD chip to implement the new Steamroller microarchi-
many-core processors for embedded systems, servers, and tecture [19]. A huge GPU (Graphics Core Next, GCN) with
high-performance computing. Up to 72 cores are or will be 512 shaders grouped in eight shader cores occupies 47% of the
integrated in many-core products by Intel and Tilera. Imple- die area. Compute cores and GPU equally share up to 32 GB of
mentations of research prototypes integrate even more simple main memory. Each dual-threaded compute core is organized
cores onto the dies. Some of the research prototypes also in two integer modules and one FPU. Each integer module is
demonstrate, how more efficient memory architectures could fed by a decoder with up to four micro-instructions per cycle.
be constructed. In the following only a few examples of current The two decoders also feed the FPU. Each integer module
designs can be given. offers four independent arithmetic pipelines working concur-
rently. The Steamroller microarchitecture will also be used in
Intel Haswell and Broadwell. Intel’s current microarchi-
future eight or more core high-end server chips codenamed
tecture for mainstream desktop and server processors is called Berlin which will include five HyperTransport-links per chip
Haswell [18]. It is manufactured using 22nm process technol-
for even better NUMA support.
ogy and 3D FinFET transistors. The die size of the standard
quad core Core i7 desktop chip including 8 MB of shared L3 IBM Power8. In September 2013 IBM has announced its
cache, a complex graphics accelerator, fast video and audio new 12 core Power8 high-end server chip that will be manu-
transcoding, and 3D support is 177m2 and integrates 1.4 billion factured in 22nm and enter the market in 2014 [20]. The chip
transistors. Compared to its Sandy Bridge predecessors, the will offer a 4 GHz operating frequency. The cores use a modi-
Haswell quad core chips offer a lower TDP (84 W) and an fied version of the Power7+ microarchitecture. A remarkable
increased maximum turbo frequency of up to 3.9 GHz. Even step forward is the support of eight simultaneous threads. Each
higher frequencies are planned for 2014. The new Turbo Boost core has a 512 KB L2 cache. All cores share a 96 MB L3 on-
2.0 technology uses thermal monitoring technologies and chip cache that is implemented in space-efficient eDRAM
allows a higher maximum package temperature than older Core technology. At 4 GHz single thread performance was improved
i7 chips. An eight core desktop version is planned for 2014. by 60% compared to the Power7+ core.
The Haswell microarchitecture again uses simultaneous Each Power8 chip has eight high-speed memory channels,
multithreading (Hyperthreading), allows to issue up to eight each of which is connected to a Centaur memory buffering chip
decoded instructions per cycle, and offers larger internal that puts 16 MB L4 cache of eDRAM between the memory
buffers for all stages of the superscalar pipeline compared to its channel coming from the processor chip and the DDR3-1600
predecessors. For instance, the reorder buffer (out-of-order DRAM controller residing on the Centaur chip. Each Power8
window) now holds up to 192 instructions (Sandy Bridge: 168, chip can offer a sustained memory bandwidth of up to 230
Nehalem 128). The AVX2 instruction set extension offers 256- GB/s with a peak bandwidth of 410 GB/s. In Power8 servers
bit wide registers for enhanced SIMD parallelism. The IBM will support transactional memory that accesses shared
instruction set was also extended by fast multiply-add, integer memory speculatively without locks and semaphores. If other
numeric, and encryption instructions. The microarchitecture processes have written on the same memory location the trans-
has again improved its branch prediction accuracy. action has to be made undone and repeated. For typical data-
base loads, transactional memory will lead to massive perfor-
The cores on the die are connected to each other and to the
mance boosts.
slices of the L3-cache and the graphics engine by a fast ring
interconnect. Haswell also supports a large off-die L4 cache Oracle Sparc T5. The Sparc T5 was announced in 2012. It
that is implemented in eDRAM technology and can optimize is clocked at 3.6 GHz, manufactured in 28nm technology and
graphics and gaming performance. Most of Intel’s current integrates 16 cores that are interconnected by a crossbar switch
Xeon server chips are still manufactured using the 32nm pro- to the eight banks of the shared 8MB L3 cache [21]. Core
cess that allows to integrate up to ten cores. However, the count per chip was doubled compared to the T4 predecessor.
Xeons already offer two memory controllers, four DRAM Core speed was enhanced by 20 percent. Each core can manage
channels (i.e. around 100 GB/s of memory bandwidth per up to eight threads dynamically (round robin). The T5 offers
chip), and four Quick-Path-Interconnect-Links per chip. instruction set extensions for data encryption. Four coherence
Haswell-based Xeon server chips in 22nm technology will be units are positioned between the L3 cache and memory con-
available in 2014 with up to 14 cores and cache sizes of up to trollers to handle cache-misses and DMA to the I/O subsystem.
45 MB. Up to eight chips can be interconnected using a glueless one-
hop interconnect that guarantees coherence between all eight cache and a completely new 512-bit vector-processing unit
sockets and up to 128 cores. with a 3-address vector instruction ISA extension. Each core
supports four dynamic (round robin) threads. The cores and
Fujitsu Sparc64 X. This chip was introduced in 2012 and
caches are ring interconnected. The ring interface also accesses
contains 16 Sparc64 X processor cores. The chip is produced in eight GDDR5 memory controllers with two memory channels
28nm technology with an enormous die size of 540mm2 and a
each. Each chip can access 16 GB of GDDR5 memory and
transistor count of 2.950 billion. The operating frequency is 3 offers a maximum memory bandwidth of 352 GB/s. All cores
GHz. The cores contain new hardware for a decimal and a
are equipped with Turbo Boost 1.0 technology with a
cipher engine. Parts of these sections contain accelerator hard- maximum turbo frequency of 1.333 GHz. The maximum TDP
ware for specific target functions on UNIX servers. This hard-
of the package is 300 W. In contrast to GPGPU accelerator
ware is called SWoC and improves performance for specific chips, like NVIDIA’s Fermi [24], which need the CUDA
decimal arithmetic and encryption functionality by a factor of
programming environment and cannot be seen as general
up to 104 leading to a 120× better performance/Watt ratio [22]. purpose computers, the Intel chips can be programmed with
ARM Cortex-A processors. This series of (multicore) pro- standard languages like C++ and C# and fully support Intel and
cessors offers a broad range of IP solutions from three-issue Open Source development environments like OpenMP, Intel
superscalar high-performance systems (e.g. ARM Cortex-A15) TBB and Cilk.
for mobile computing, digital home, servers and wireless infra- For 2015 Intel has announced the 14nm MIC Knight’s
structures down to the extremely energy-efficient A7 and A5 Landing with the same TDP of 300W, but up to 72 cores based
cores [23]. The Cortex-A chips use advanced microarchitec- on the 2-issue superscalar Atom Silvermont embedded plat-
tures with relatively long pipelines (up to 13 stages) and form architecture that offers many Ivy-Bridge-like microarchi-
efficient branch prediction hardware. For multicore support the tecture features, 24 KB data, 32 KB instruction and 1 MB L2-
processors are equipped with a coherent bus interface and spe- caches. These new MICs will able to work as standalone sys-
cific hardware for multicore debug and trace purposes. The tems, but will be connected with other MICs or system compo-
Cortex-A15 integrates 32KB L1 data and instruction caches nents over fast QPI channels. The system is targeted at a
and supports an L2-cache size of up to 4MB. maximum performance of 3 TFLOPS, i.e. 10  𝐺𝐹𝐿𝑂𝑃𝑆/𝑊.
The 64-bit ARM V8 architecture was announced in 2011.
Tilera TILE-Gx. Tilera is an MIT spin-off that has spe-
Meanwhile several 64-bit implementations are available. 64-bit cialized on the development of symmetric many-core proces-
ARM multicore processors like the ARM Cortex-A53 are tar-
sors originally derived from the 16-core MIT RAW processor.
geted at the high-end market for heterogeneous platforms in- The largest available chip, the TIL-Gx8072 [25] is imple-
cluding FPGA functionality and for the server market. AMD’s
mented in 40nm technology and includes 72 efficient, 2D-
most powerful core is the Cortex-A57 with a deep three-issue mesh-connected 64 RISC cores with 32 KB L1-data and in-
out-of-order superscalar pipeline and eight dispatch-ports (like
struction caches and 256 KB L2 caches. In addition the chip
Intel’s Haswell) and an out-of-order instruction-window with offers 18 MB of coherent L3 cache. The 2D-mesh connection
size 128. AMD is planning a micro-server equipped with 16 of
network consists of five independent concurrent networks.
these ARM Cortex-A57 cores. Switching occurs at a speed of one hop from core to core
Intel Atom. Intel’s new 64-bit Atom processors are manu- within one cycle. Operating frequency of the cores is 1.0 GHz.
factured in 22nm and are targeted both at the market for high- The full aggregate communication bandwidth surpasses 100
end smart phones and tablets [18]. The Silvermont architecture Tbps. In addition the chip includes four DDR3 memory con-
directly attacks the fastest ARM Cortex-A processors. Each trollers allowing up to 1 TB memory, six high-performance
core comes with a dual-issue out-of-order superscalar microar- transaction ports for chip-to-chip or FPGA interconnect, spe-
chitecture, an on-chip GPU and a very low TDP of 1.5 W. For cial hardware for packet-switched communication and acceler-
laptops, smart phones, and tablets the Z3770D quad-core im- ators for encryption and security functions.
plementation with 1.5 GHz operating frequency and an on-chip
B. Techniques for Asymmetric and Dynamic Multicore
GPU is already available. For embedded or large scale, low
energy servers, Intel offers Atoms C2750, C2550, and C2350 Processing
codenamed Avoton with eight, four, and two cores, and a TDP Many of the processors discussed above, already offer
of 20W, 14 W, and 6 W, respectively. Their maximum turbo techniques for dynamic processing or support asymmetric
frequency is 2.6 GHz. The C2750 supports memory sizes of up designs. Current research activities of the leading manufactur-
to 64 GB and with two memory channels reaches a maximum ers will lead to even more efficient design solutions.
memory bandwidth of 25.6 GB/s. Current asymmetric multicore designs built around the
Intel Many Integrated Cores (MIC) processors. Starting ARM architecture for more flexible and even more energy-
with a 32 core prototype in 2012 [24], Intel has entered the efficient embedded and fine-tuned system behavior, use a
many-core market segment in 2013 with the first implementa- combination of one or two complex ARM cores (Cortex-A15
tion of the Xeon PHI multicore processor, the 22nm Knight’s or Cortex-A57) and up to four extremely energy-efficient cores
Corner. This multicore chip operates as an accelerator co- (Cortex-A17 or Cortex-A53). The cores can be integrated on an
processor in Xeon-based workstations, servers, and supercom- SoC in a so-called big.LITTLE configuration together with
puters. Communication to the host uses a PCI express 2.0 inter- additional heterogeneous cores, e.g. for signal processing. They
face. The chip offers up to 61 64-bit x86 cores that are based communicate via the cache-coherent CoreLink CCI-400 inter-
on the Pentium platform. Each core comes with 512 KB of L2- connect. To the software they appear as an homogeneous mul-

embedded world Conference 2014

ticore processor. In addition big.LITTLE systems also show requirements. The cores come in banks of four cores. Each
dynamic behavior, as the big cores, as well as the little cores bank shares 2 MB of L2 cache. The chip has three integrated
offer different frequency (and voltage) levels and can be com- DDR3 memory controllers with a 1.5 MB prefetch-cache.
pletely switched off, if not needed [23]. For high-end configu- What makes the chip a typical heterogeneous multicore archi-
rations system developers can also couple several Cortex-A57 tecture is its Data Path Acceleration Hardware (DPAA) that
and A53 cores. contains an I/O manager and a frame manager that parses
headers of incoming packages and assigns the packages to
The idea of dynamically changing the performance level of
buffers. The queue manager assigns the packages to cores or to
multicore chips as modeled by Hill and Marty was imple- specific hardware that include security accelerators (SEC), a
mented in second-generation multicores in the form of dynamic
pattern matching engine (PME), and a data compression en-
voltage and frequency scaling (DVFS). Intel’s Turbo Boost 1.0 gine.
allows the balancing of the core frequencies of the available
cores against the available thread parallelism. Cores can com- Altera Stratix 10. Altera is known for its high-end FPGA
pletely be switched off and the frequency of the remaining solutions [29]. The recently announced Stratix 10 SoC will be
threads can be raised, as long as the maximum TDP and fre- fabricated by Intel and uses the new 14nm process using
quency limit is respected. FinFET transistors. Intel will integrate four ARM Cortex-A53
cores on the chip. Together with integrated FPU accelerators
Today, the operating voltage level of transistors in proces- the multicore hardware will enable the design of very flexible
sors for most market segments is already very low and only
applications with high-performance, as the FPGA will operate
2.5× the threshold level of the transistors. DVFS downward at 1 GHz, and with a low power budget.
scaling would result in a rapid drop of frequency and
performance [7] and is an option only for applications with V. CONCLUSION
exactly known performance envelopes. Nevertheless Near-
Threshold Voltage (NTV) logic is currently actively As VLSI scaling puts new challenges on the agenda of chip
researched. Intel has developed an NTV x86 processor [26], makers, computer architects and processor designers have to
[27] that shows a remarkable DVFS flexibility and can be seen find innovative solutions for future multicore processors that
as a prototypical core for future extremely energy-efficient make the best and most efficient use of the abundant transistor
multicore systems. budget offered by Moore’s law that will be valid for another
decade. In this paper, we have clarified the reasons of the Post-
Currently Intel’s mainstream processors use Turbo Boost Dennard scaling regime and discussed the consequences for
2.0 for software hot spot and single thread acceleration. This future multicore designs.
technique temporarily boosts core performance by frequency
scaling for several seconds until the processor reaches its For the four typical architectural classes: symmetric,
nominal temperature and as long as the thermal capacitance of asymmetric, dynamic, and heterogeneous multicore processors,
the chip’s heat sink is not overstrained. After this, the processor we have compared and extended some straightforward perfor-
falls back to a relatively low power level. Similar techniques mance and power models. Although various results from re-
are applied by the four ARM Cortex-A15 cores used in search and industrial studies predict that huge parts of future
big.LITTLE configurations. With future phase-change materi- multicore chips will be dark or dim all the time, it can already
als, an even higher temporary performance leverage, also be seen in contemporary commercial designs, how these areas
called computational sprinting, will be possible [7]. can be used for making intelligent use of the limited power
budget with heterogeneous coprocessors, accelerators, larger
Intel’s Haswell chips offer fine-grained power management caches, faster memory buffers, and improved communication
by so-called fully integrated voltage regulators on the chip that networks.
allow the separate voltage and frequency control of the cores,
the integrated GPU, the L3 cache and the system interface. In the future, the success of many-core architectures will
IBM’s Power8 offers separate voltage and frequency control first and foremost depend on the exact knowledge of specific
for each core. workload properties as well as easy-to-use software environ-
ments and software developing tools that will support pro-
C. Heterogeneous Multicore Processors grammers to exploit the explicit and implicit regular and
Many multicore products are offered as IP cores that can be heterogeneous parallelism of the application workloads.
used as building blocks for designing complex custom or
VI. REFERENCES
FPGA-based heterogeneous multicore systems. ARM, Texas
Instruments, MIPS, Freescale, Altera, Xilinx and other vendors
offer solutions for various target markets that include mobile [1] Bohr, M., Mistry, K., “Intel’s revolutionary 22 nm Transistor Tech-
nology,” Intel Corporation, May 2011
IT, automotive, manufacturing, and other areas. In the follow-
ing, out of a very rich landscape, we only give two examples of [2] Grochowski, E., ”Energy per Instruction Trends in Intel Micro-
processors,” Technology@Intel Magazine, March 2006
typical heterogeneous designs.
[3] Borkar, S., Chien, A.A.,”The Future of Microprocessors,” Communi-
Freescale QorlQ T Series. The T4240 is a 12-core proces- cations of the ACM, May 2011, Vol. 54, No. 5, pp. 67-77
sor based on Power Architecture e6500 64-bit dual-threaded [4] Esmaeilzadeh, H. et al., “Power Challenges May End the Multicore
cores with a clockspeed of up to 1.8 GHz [28]. The chip sup- Era,” Communications of the ACM, February 2013, Vol. 56, No. 2, pp.
93-102
ports industrial embedded applications, datacenter loads, and
routing applications with high-performance and low power
[5] Esmaeilzadeh et al., “Executable Dark Silicon Performance Model”: [16] Gustafson, J.L, “Reevaluating Amdahl’s Law,” Communications of the
http://research.cs.wisc.edu/vertical/DarkSilicon, last access Jan. 20, 2014 ACM, Vol. 31, No. 5, 1988, pp. 532-533
[6] Dennard, R.H. et al., “Design of Ion-Implanted MOSFET’s with Very [17] Courtland, R., “Chipmakers Push Memory Into the Third Dimension,”
Small Physical Dimensions,” IEEE J. Solid-State Circuits, vol. SC-9, http://spectrum.ieee.org, last access: January 20, 2014
1974, pp. 256-268 [18] http://www.intel.com
[7] Taylor, M.B., “A Landscape of the New Dark Silicon Design Regime,” [19] http://www.amd.com
IEEE Micro, September/October 2013, pp. 8-19
[20] Prickett Morgan, T., “Power8 Processor Packs a Twelve-Core Punch-
[8] Borkar, S., “3D Integration for Energy Efficient System Design,” And Then Some,” The Four Hundred, http://www.itjungle.com/
DAC’11, June 5-10, San Diego, California, USA tfh/thf090913-printer01.html, last access Jan. 20, 2013
[9] Amdahl, G.M., “Validity of the Single Processor Approach to Achieving [21] Feehrer, J. et al., “The Oracle Sparc T5 16-Core Processor Scales to
Large Scale Computing Capabilities,” Proc. AFIPS Conf., AFIPS Press, Eight Sockets,” IEEE Micro, March/April 2013, pp. 48-57
1967, pp. 483-485
[22] Yoshida, T. et al., “Sparc64 X: Fujitsu’s New-Generation 16-Core
[10] Getov, V., “A Few Notes on Amdahl’s Law,” IEEE Computer, Processor for Unix Servers,” IEEE Micro, November/December 2013,
December 2013, p. 45 pp. 16-24
[11] Chung, E.S., et al., “Single-Chip Heterogeneous Computing: Does the [23] http://www.arm.com
Future Include Custom Logic, FPGAs, and GPGPUs?” Proc. 43rd
[24] Heinecke, A., Klemm, M., Bungartz, H.-J., “From GPGPU to Many-
Annual IEEE/ACM International Symposium on Microarchitecture,
Core: Nvidia Fermi and Intel Many Integrated Core Architecture,” IEEE
2010, pp. 225-236
Computing in Science & Engineering, March/April 2012, pp. 78-83
[12] Hill, M.D., Marty, M., “Amdahl’s Law in the Multicore Era,” IEEE
Computer, July 2008, pp. 33-38 [25] http://www.tilera.com
[13] Woo, D.H., Lee, H.H., “Extending Amdahl’s Law for Energy-Efficient [26] Ruhl, G.,”IA-32 Processor with a Wide-Voltage-Operating_Range in 32
nm CMOS,” IEEE Micro, March/April 2013, pp. 28-36
Computing in the Many-Core Era,” IEEE Computer, December 2008,
pp. 24-31 [27] Kaul, H. et al.,”Near-Threshold Voltage (NTV) Design – Opportunities
[14] Venkatesh, G. et al., “Conservation Cores: Reducing the Energy of and Challenges”, Design Automation Conference ’49, June 3-7, 2012,
San Francisco
Mature Computations,” Proc. 15th Architectural Support for
Programming Languages and Operating Systems Conf., ACM, 2010, pp. [28] http://www.freescale.com
205-218 [29] http://www.altera.com
[15] Sun, X.-H., Chen, Y., ”Reevaluating Amdahl’s law in the multicore
era,” J. Parallel Distrib. Comput. (2009), doi: 10.1016/j.jpdc2009.05.002

embedded world Conference 2014