Chapter 1

Fundamentals of Quantitative Design and Analysis

CSC 520

Dr. George Clark

Spring 2024
* Material based on text by Hennessy & Patterson
Computer Technology

• Performance improvements:
- Improvements in semiconductor technology
• Feature size, clock speed
- Improvements in computer architectures
• Enabled by HLL compilers, UNIX
• Led to RISC architectures

• Have ridden the wave of Instruction Level Parallelism (ILP)

• Implicit - hidden to programmer
• Advances in hardware architecture
- Pipelining
- Multi-issue (superscalar, VLIW)
- Cache design
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 2
Single Processor Performance
Move to multi-processor

RISC

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 3

Shift to multi-proc/core
• Multiprocessors imminent in 1970s, ‘80s, ‘90s, …
• “… today’s processors … are nearing an impasse as technologies approach the speed of light..”
David Mitchell, The Transputer: The Time Is Now (1989)

• Transputer was premature

Custom multiprocessors strove to lead uniprocessors
Procrastination rewarded: 2X seq. perf. / 1.5 years

• “We are dedicating all of our future product development to multicore designs. … This is a sea of
change in computing”
Paul Otellini, President, Intel (2004)

• Difference is all microprocessor companies switch to multiprocessors (AMD, Intel, IBM, Sun; all
new Apples 2 CPUs)
Procrastination penalized: 2X sequential perf. / 5 yrs
Biggest programming challenge: 1 to 2 CPUs

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 4

Current Trends in Architecture

• Cannot continue to leverage Instruction-Level parallelism (ILP)

- Single processor performance improvement ended in 2003
- Why
• Brick wall (ILP, power, memory time)
• Ahmdal’s Law, Moore’s Law, Dennard scaling

• New models for performance:

- Data-level parallelism (DLP)
- Thread-level parallelism (ThLP)
- Request-level parallelism (RLP)

- These require explicit restructuring of the application

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 5
Classes of Computers

• Personal Mobile Device (PMD)

- e.g. smart phones, tablet computers
- Emphasis on energy efficiency and real-time
• Desktop Computing
- Emphasis on price-performance
• Servers
- Emphasis on availability, scalability, throughput
• Clusters / Warehouse Scale Computers
- Used for “Software as a Service (SaaS)”
- Emphasis on availability and price-performance
- Sub-class: Supercomputers, emphasis: floating-point performance and fast internal networks
• IoT / Embedded Computers
- Emphasis: price
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 6
Parallelism

• Classes of parallelism in applications:

- Data-Level Parallelism (DLP)
- Task-Level Parallelism (TaLP)

• Classes of architectural parallelism:

- Instruction-Level Parallelism (ILP)
- Vector architectures/Graphic Processor Units (GPUs)
- Thread-Level Parallelism (ThLP)
- Request-Level Parallelism (RLP)
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 7
Flynn’s Taxonomy - 1966

• Hardware classification based on parallelism in instruction & data streams

• Single instruction stream, single data stream (SISD)
• Single instruction stream, multiple data streams (SIMD)
- Vector architectures
- Multimedia extensions
- Graphics processor units
• Multiple instruction streams, single data stream (MISD)
- No commercial implementation
- Some very tailored fault tolerant systems exist
• Multiple instruction streams, multiple data streams (MIMD)
- Tightly-coupled MIMD (ThLP)
- Loosely-coupled MIMD (RLP, clusters)
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 8
Defining Computer Architecture

• “Old” view of computer architecture:

- Instruction Set Architecture (ISA) design
- i.e., decisions regarding:
• registers, memory addressing, addressing modes, instruction operands, available operations,
control flow instructions, instruction encoding

- Other aspects were considered implementation

• “Real” computer architecture:
- Specific requirements of the target machine
- Design to maximize performance within constraints: cost, power, and availability
- Includes ISA, microarchitecture, hardware
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 9
A Simple Implementation Scheme
Datapath With Control

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 10

Instruction Set Architecture

• Class of ISA
- General-purpose registers
- Register-memory vs load-store
• RISC-V registers Register Name Use Saver
- 32 g.p., 32 f.p. x9 s1 saved callee
Register Name Use Saver x10-x17 a0-a7 arguments caller
x0 zero constant 0 n/a x18-x27 s2-s11 saved callee
x1 ra return addr caller x28-x31 t3-t6 temporaries caller
x2 sp stack ptr callee f0-f7 ft0-ft7 FP temps caller
x3 gp gbl ptr f8-f9 fs0-fs1 FP saved callee
x4 tp thread ptr f10-f17 fa0-fa7 FP arguments callee
x5-x7 t0-t2 temporaries caller
f18-f27 fs2-fs21 FP saved callee
saved/
x8 s0/fp callee f28-f31 ft8-ft11 FP temps caller
frame ptr

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 11

Instruction Set Architecture

• Memory addressing
- RISC-V: byte addressed, aligned accesses are faster
• Addressing modes
- RISC-V: Register, immediate, displacement (base+offset)
- Other examples: autoincrement, indexed, PC-relative
• Types and size of operands
- RISC-V: 8-bit, 32-bit, 64-bit

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 12

Instruction Set Architecture

• Operations
- RISC-V: data transfer, arithmetic, logical, control, floating point
- See Fig. 1.5 in text
• Control flow instructions
- Use content of registers (RISC-V) vs. status bits (x86, ARMv7, ARMv8)
- Return address in register (RISC-V, ARMv7, ARMv8) vs. on stack (x86)
• Encoding
- Fixed (RISC-V, ARMv7/v8 except compact instruction set) vs. variable length
(x86)
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 13
RISC-V Registers

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 14

RISC-V Instructions (subset)

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 15

The MIPS Instruction Set

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 16

RISC-V Floating Point

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 17

RISC-V Instruction Formats

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 18

MIPS/ARM Instruction Encoding

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 19

Moore’s Law: 2X transistors / “year”

• “Cramming More Components onto Integrated Circuits”

- Gordon Moore, Electronics, 1965
• # of transistors / cost-effective integrated circuit double every N months (12 ≤ N ≤ 24)

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 20

Trends in Technology (~20 years)

• Integrated circuit technology (Moore’s Law)

- Transistor density: 35%/year
- Die size: 10-20%/year
- Integration overall: 40-55%/year

• DRAM capacity: 25-40%/year (slowing)

- 8 Gb (2014), 16 Gb (2019), possibly no 32 Gb

• Flash capacity: 50-60%/year

- 8-10X cheaper/bit than DRAM

• Magnetic disk technology: 40%/year --> 5%/year

- Density increases may no longer be possible (platters/disk)
- 8-10X cheaper/bit than Flash
- 200-300X cheaper/bit than DRAM
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 21
Bandwidth and Latency (~20 years)

• Bandwidth (throughput)
- Total work done in a given time
- 32,000-40,000X improvement for processors
- 300-1200X improvement for memory and disks
• Latency or response time
- Time between start and completion of an event
- 50-90X improvement for processors
- 6-8X improvement for memory and disks
• Interesting Observation
- Bandwidth hurts latency
- Latency helps bandwidth
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 22
Latency Lags Bandwidth (last ~20 years)
CPU high,
Memory low • Bandwidth improves by more
(“Memory Wall”) than square of Latency
- Moore’s law
- Distance
- OS overhead

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 23

Transistors and Wires

• Feature size
- Minimum size of transistor or wire in
x or y dimension

- 10 microns in 1971 to .032 microns (aka 32nm) in 2011

- Transistor performance scales linearly
• Wire delay does not improve with feature size!
- Integration density scales quadratically
• 22nm and 14nm on in newest i7’s
(7nm on horizon)

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 24

Power and Energy

• Problem: Get power in, get power out

• Three concerns for systems designers
1. Max power that a processor requires
2. Thermal Design Power (TDP)
• aka sustained power consumption
• Used as target for power supply and cooling system
• Lower than peak power (1.5X higher), higher than average power consumption
• Clock rate can be reduced dynamically to limit power consumption

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 26

Power and Energy

3. Energy and power efficiency

• Power is energy per unit time, 1 watt = 1 joule per second
• Energy per task is often a better measurement than power
- tied to a task and time for task
• energy for a workload = avg power times the execution time for the workload
- compare energy consumption for executing a task
• power measurement useful as a constraint

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 27

Dynamic Energy and Power

• For CMOS chips, traditional dominant energy consumption has been in switching
transistors, called dynamic power
Power dynamic ∝ 1/2 × Capacitive load × Voltage 2 × Frequency Switched

• For mobile devices, energy better metric

Energy dynamic ∝ Capacitive load × Voltage 2

• For a fixed task, slowing clock rate (frequency switched) reduces power, but not energy
• Capacitive load a function of number of transistors connected to output and
technology, which determines capacitance of wires and transistors
• Dropping voltage helps both, so went from 5V to 1V
• To save dynamic power, most CPUs now turn off clock of inactive modules (e.g. Fl. Pt.
Unit)

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 28

Example of quantifying power

• Suppose 15% reduction in voltage results in a 15% reduction in

frequency. What is impact on energy & dynamic power?
- Energy
𝐄𝐧𝐞𝐫𝐠𝐲𝒏𝒆𝒘 (𝑽𝒐𝒍𝒕𝒂𝒈𝒆 𝒙 𝟎 . 𝟖𝟓) 𝟐
𝟐
= = 𝟎 . 𝟖𝟓 = 𝟎 . 𝟕𝟐
𝐄𝐧𝐞𝐫𝐠𝐲𝒐𝒍𝒅 𝑽𝒐𝒍𝒕𝒂𝒈𝒆 𝟐

- Power
Powernew (FrequencySwitched × 0.85)
= 0.72 = 0.61
Powerold FrequencySwitched

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 29

Clock rate growth – why stalled?

• Intel 80386 consumed

~2W

• 3.3 GHz Intel Core i7

consumes 130 W

• Heat must be
dissipated from 1.5 x
1.5 cm chip

• This is the limit of what

can be cooled by air

Figure 1.11 Growth in clock rate of microprocessors in Figure 1.1

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 30

Reducing Power

• Techniques for reducing

power:
- Do nothing well
- Dynamic Voltage-Frequency
Scaling (DVFS)

- Low power state for DRAM,

disks

- Overclocking, turning off cores

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 31

Static Power

• Because leakage current flows even when a transistor is off, now

static power important too
Power static ∝ Current static × Voltage
• Leakage current increases in processors with smaller transistor sizes
• Increasing the number of transistors increases power even if they are
turned off
• In 2006, goal for leakage is <25% of total power consumption; high
performance designs at <40%
• Very low power systems even gate voltage to inactive modules to
control loss due to leakage
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 32
Define and quantify dependability (1/3)

• How decide when a system is operating properly?

• Infrastructure providers now offer Service Level Agreements (SLA) to
guarantee that their networking or power service would be dependable
- could pay customer a penalty

• Systems alternate between 2 states of service with respect to an SLA:

1. Service accomplishment, where the service is delivered as specified in SLA
2. Service interruption, where the delivered service is different from the SLA
• Failure = transition from state 1 to state 2
• Restoration = transition from state 2 to state 1
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 33
Define and quantify dependability (2/3)

• Module reliability = measure of continuous service accomplishment (or time to

failure).
2 metrics
1. Mean Time To Failure (MTTF) measures Reliability
2. Failure Rate = 1/MTTF, the rate of failures
• Traditionally reported as failures per billion hours of operation or Failures In Time (FIT)

• Mean Time To Repair (MTTR) measures Service Interruption

- Mean Time Between Failures (MTBF) = MTTF+MTTR
• Module availability measures service accomplishment as alternation between the
2 states of accomplishment and interruption (number between 0 and 1, e.g. 0.9)
- Module availability = MTTF / ( MTTF + MTTR)

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 34

Example calculating reliability

• If modules have exponentially distributed lifetimes (age of module

does not affect probability of failure), overall failure rate is the sum
of failure rates of the modules
• Calculate FIT and MTTF for 10 disks (1M hour MTTF per disk), 1
disk controller (0.5M hour MTTF), and 1 power supply (0.2M hour
MTTF):
FailureRate =

MTTF =

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 35

Example calculating reliability

• If modules have exponentially distributed lifetimes (age of module

does not affect probability of failure), overall failure rate is the sum
of failure rates of the modules
• Calculate FIT and MTTF for 10 disks (1M hour MTTF per disk), 1
disk controller (0.5M hour MTTF), and 1 power supply (0.2M hour
MTTF): FailureRate = 10 × (1/1, 000, 000) + 1/ 500, 000 + 1/ 200, 000
= (10 + 2 + 5) /1, 000, 000
= 17 /1, 000, 000
= 17, 000 FIT
MTTF = 1, 000, 000, 000 /17, 000
≈ 59, 000hours

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 36

Performance & Speedup

• Performance is in units of things per sec

- bigger is better
• If we are primarily concerned with response time

- " X is n times faster than Y" means

- Example: time taken to run a program

• 10s on system A, 15s on system B
• Execution TimeB / Execution TimeA = 15s / 10s = 1.5
• So system A is 1.5 times faster than system B
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 37
Measuring Execution Time

• Elapsed time
- Total response time, including all aspects
• Processing, I/O, OS overhead, idle time
- Determines system performance
• CPU time
- Time spent processing a given job
• Discounts I/O time, other jobs’ shares

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 38

CPU Clocking

• Operation of digital hardware governed by a constant-rate clock

• Clock period: duration of a clock cycle

- e.g., 250ps = 0.25ns = 250×10-12 s
• Clock frequency (rate): cycles per second
- e.g., 4.0GHz = 4000MHz = 4.0×109 Hz
• Clock rate is the inverse of the clock period
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 39
CPU Time

• Performance improved by
- Reducing number of clock cycles
- Increasing clock rate
- Hardware designer must often trade off clock rate against cycle count

CPU Time = CPU Clock Cycles × Clock Cycle Time

CPU Clock Cycles
=
Clock Rate

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 40

CPU Time Example

• Computer A: 2GHz clock, 10s CPU time

• Designing Computer B
- Aim for 6s CPU time
- Can do faster clock, but causes 1.2 × clock cycles
• How fast must Computer B clock be? Clock CyclesB 1.2 × Clock Cycles A
Clock RateB = =
CPU Time B 6s
Clock Cycles A = CPU Time A × Clock Rate A
9
= 10s × 2GHz = 20 × 10
9 9
1.2 × 20 × 10 24 × 10
Clock RateB = = = 4GHz
6s 6s

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 41

Instruction Count and CPI

Clock Cycles = Instruction Count × Cycles per Instruction

CPU Time = Instruction Count × CPI × Clock Cycle Time
Instruction Count × CPI
=
Clock Rate
• Instruction Count for a program
- Determined by program, ISA and compiler
• Average cycles per instruction
- Determined by CPU hardware
- If different instructions have different CPI
• Average CPI affected by instruction mix
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 42
CPI Example

• Computer A: Cycle Time = 250ps, CPI = 2.0

• Computer B: Cycle Time = 500ps, CPI = 1.2
• Same ISA
• Which is faster, and by how much?

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 43

Cycles Per Instruction

• “Average Cycles per Instruction”

(CPUtime(sec) ⋅ ClockRate(Cycles per sec))
CPI = =
Cycles
IC IC
n
CPUtime(sec) = CycleTime(sec per cycle) ⋅ ∑ CPI i ⋅ ICi
i =1

• “Instruction Frequency”
n
ICi
CPI = ∑ CPI i ⋅ Fi where Fi =
i =1 IC

• Invest Resources where time is spent!

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 44
Example: Calculating CPI

Base Machine (Reg / Reg)

Op Freq CPIi F· CPIi (% Time)

ALU 50% 1 .5 (33%)

Load 20% 2 .4 (27%)

Store 10% 2 .2 (13%)

Branch 20% 2 .4 (27%)

Typical Mix 1.5

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 45

Focus on the Common Case

• Common sense guides computer design

- Since its engineering, common sense is valuable
• In making a design trade-off, favor the frequent case over the infrequent case
- E.g., Instruction fetch and decode unit used more frequently than multiplier, so optimize it 1st
- E.g., If database server has 50 disks / processor, storage dependability dominates system
dependability, so optimize it 1st

• Frequent case is often simpler and can be done faster than the infrequent case
- E.g., overflow is rare when adding 2 numbers, so improve performance by optimizing more
common case of no overflow

- May slow down overflow, but overall performance improved by optimizing for the normal case
• What is frequent case and how much performance improved by making case faster
=> Amdahl’s Law
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 46
 Amdahl’s Law

⎡ Fraction enhanced ⎤
ExTimenew = ExTimeold × ⎢(1 − Fraction enhanced )+ ⎥
⎣ Speedupenhanced ⎦

ExTimeold 1
Speedupoverall = =
ExTimenew Fraction enhanced
(1 − Fractionenhanced ) +
Speedupenhanced

Best you could ever hope to do:

1
Speedupmaximum =
(1 - Fractionenhanced )

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 47

Amdahl’s Law example

• New CPU 10X faster

• I/O bound server, so 60% time waiting for I/O
1
Speedup overall =
Fraction enhanced
(1 − Fraction enhanced )+
Speedup enhanced
1 1
= = = 1.56
0.4 0.64
(1 − 0.4)+
10

• Apparently, its human nature to be attracted by 10X faster, vs.

keeping in perspective its just 1.6X faster
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 48
Measuring Performance

• Typical performance metrics:

- Response time
- Throughput
• Speedup of X relative to Y
- Execution timeY / Execution timeX
• Execution time
- Wall clock time: includes all system overheads
- CPU time: only computation time
• Benchmarks
- Kernels (e.g. matrix multiply)
- Toy programs (e.g. sorting)
- Synthetic benchmarks (e.g. Dhrystone)
- Benchmark suites (e.g. SPEC06fp, TPC-C)
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 49
Performance: What to measure

• Usually rely on benchmarks vs. real workloads

• To increase predictability, collections of benchmark applications-- benchmark suites -- are popular
• SPECCPU: popular desktop benchmark suite
- CPU only, split between integer and floating point programs
- SPEC2017: has 10 integer & 13 flt-point pgms
- SPECSFS (NFS file server) and SPECWeb (WebServer) added as server benchmarks
• Transaction Processing Council measures server performance and cost-performance for
databases
- TPC-C Complex query for Online Transaction Processing
- TPC-H models ad hoc decision support
- TPC-W a transactional web benchmark
- TPC-App application server and web services benchmark

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 50

How Summarize Suite Performance (1/3)

• Arithmetic average of execution time of all programs?

- But they vary by 4X in speed, so some would be more important than others
in arithmetic average

• Could add a weights per program, but how pick weight?

- Different companies want different weights for their products
• SPECRatio: Normalize execution times to reference computer,
yielding a ratio proportional to performance =

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 51

How Summarize Suite Performance (2/3)
• If program SPECRatio on Computer A is 1.25 times bigger than Computer B,
then

ExecutionTimereference
SPECRatio A ExecutionTime A
1.25 = =
SPECRatioB ExecutionTimereference
ExecutionTimeB
ExecutionTimeB Performance A
= =
ExecutionTime A PerformanceB

• Note that when comparing 2 computers as a ratio, execution times on the

reference computer drop out, so choice of reference computer is irrelevant

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 52

How Summarize Suite Performance (3/3)

• Since ratios, proper mean is geometric mean

(SPECRatio unitless, so arithmetic mean meaningless)

n
GeometricMean = n ∏ SPECRatioi
µg i =1

• 2 points make geometric mean of ratios attractive to summarize

performance:
1. Geometric mean of the ratios is the same as the ratio of the geometric means
2. Ratio of geometric means
= Geometric mean of performance ratios
choice of reference computer is irrelevant!
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 53
Ratio of means – Means of ratios – Hugh??

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis

51 54
Manufacturing ICs

• Yield: proportion of working dies per wafer

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 55

Integrated Circuit Cost

• Integrated circuit (video)

-
• Bose-Einstein formula:
- Defects per unit area = 0.016-0.057 defects per square cm (2010)
- N = process-complexity factor = 11.5-15.5 (40 nm, 2010)

-
1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 56
Summary

• Architectural Trends

• Performance Metrics

• Geometric Means

• IC cost

1/24/2024 Chapter 1 Fundamentals of Quantitative Design and Analysis 57