CMP3010: Computer Architecture

L01: Computer Abstractions and Technology

Dina Tantawy
Computer Engineering Department
Cairo University
 Agenda
• Introduction
• Eight great ideas in computer architecture
• Performance
• Real stuff: Benchmarking the Intel core i7
• Fallacies and pitfalls
 Introduction
• This course is all about how computers work

• But what do we mean by a computer?

– Different types: desktop, servers, embedded devices

– Different uses: automobiles, graphics, finance,…

– Different manufacturers: Intel, Apple, IBM, Microsoft, Sun…

– Different underlying technologies and different costs!

 Introduction
– How does the hardware execute a program?
– What is the interface between the software and the hardware
– What determines the performance of a program
– How can we improve the performance?
– What great ideas did computer architects come up with that lay the foundation of
modern computing?
Growth of Computer Performance

Intel cancelled high performance

uniprocessor, joined IBM and Sun
for multiple processors
 Eight great ideas in computer architecture
1. Design for Moore’s Low
– The one constant for computer designers is
rapid change
– Computer architects must anticipate where the
technology will be when the design finishes
rather than design for where it starts.
 Eight great ideas in computer architecture
2. Use abstraction to simplify design
– A major productivity technique for hardware
and software is to use abstraction to
represent the design at different levels of
representation.
– Lower level details are hidden to offer a
simpler model at higher levels.
 Eight great ideas in computer architecture
3. Make the common case fast
– Making the common case fast will tend to enhance the
performance better than optimizing the rare case
– The common case is often simpler than the rare case
and is often easier to enhance.
 Eight great ideas in computer architecture
4. Performance via parallelism

• Instruction-Level Parallelism (ILP): pipelining, superscalar,

speculation…etc
• Data-Level Parallelism (DLP): vector architecture, GPU…etc
• Thread-Level Parallelism (TLP): multiprocessor,
multicomputer…etc
• Request-Level Parallelism (RLP): Clusters, warehouse scale
computers…etc
 Eight great ideas in computer architecture
5. Performance via pipelining

6. Performance via prediction

– In some cases it can be faster on average to guess and
start working rather than wait until you know for sure.
 Eight great ideas in computer architecture
7. Hierarchy of memories
– Used to provide the contradicting demands of
programmers of wanting fast, large, and cheap memory
– The fastest, smallest and most expensive memory at the
top of the hierarchy and the slowest, largest and
cheapest at the bottom.

8. Dependability via redundancy

– we make systems dependable by including redundant
components that can take over when a failure occurs and
to help detect failure.
 How our program Works
• The hardware in a computer can only execute extremely simple
low-level instructions.
• To go from a complex application to the simple instructions
involves several layers of software that interpret or translate
high level operations into simple computer
instructions.(Abstraction)
 Abstractions

• Abstraction helps us deal with complexity

– Hide lower-level detail
• Instruction set architecture (ISA)
– The hardware/software interface
– Standardizes instructions, machine language and
the native commands executed by the processor.
• Implementation
– The details underlying and interface
Abstraction
• Abstraction helps us deal with complexity
– Hide lower-level detail
• Delving into the depths reveals more
information
• Instruction set architecture (ISA)
– The hardware/software interface
– Standardizes instructions, machine language
and the native commands executed by the
processor.
• Implementation
– The details underlying and interface
Instruction Set Architecture (ISA)

software

instruction set

hardware
 Instruction Set Architecture (ISA)
• The instruction set provides commands to the processor, to tell it
what it needs to do.
• The instruction set consists of addressing modes, instructions,
native data types, registers, memory architecture, interrupt,
exception handling, and external I/O.
 Instruction Set Architecture (ISA)
• Modern instruction set architectures:
– IA-32, PowerPC, MIPS, SPARC, ARM, RISC-V and others
 Computer Components
• The five classic components of a computer are input, output,
memory, datapath, and control, with the last two sometimes
combined and called the processor.
Computer Components
 Inside the Processor - Datapath
• Datapath is the hardware that performs all the required operations.
• The datapath is the "brawn" of a processor, since it implements the
fetch-decode-execute cycle.
 Inside the Processor - Control
• Control is the hardware that tells the datapath what to do, in terms of
switching, operation selection, data movement between ALU
components.
• The Control is the “Brain” of the processor.
Performance
 Defining Performance
• Which airplane has the best performance?

Boeing 777 Boeing 777

Boeing 747 Boeing 747

BAC/Sud BAC/Sud
Concorde Concorde
Douglas Douglas DC-
DC-8-50 8-50

0 100 200 300 400 500 0 2000 4000 6000 8000 10000

Passenger Capacity Cruising Range (miles)

Boeing 777 Boeing 777

Boeing 747 Boeing 747

BAC/Sud BAC/Sud
Concorde Concorde
Douglas Douglas DC-
DC-8-50 8-50

0 500 1000 1500 0 100000 200000 300000 400000

Cruising Speed (mph) Passengers x mph

 Performance
• Measure, Report, and Summarize
• Make intelligent choices
• See through the marketing hype
• Key to understanding underlying organizational motivation
Computer Performance: TIME, TIME, TIME
• Response Time (execution time, latency)
— How long does it take for my job to run?
— How long does it take to execute a job?
— How long must I wait for the database query?

• Throughput (bandwidth)
— How many jobs can the machine run at once?
— What is the average execution rate?
— How much work is getting done?

• If we upgrade a machine with a new processor what do we increase?

• If we add a new machine to the lab what do we increase?

 Execution Time
• Elapsed Time
– counts everything (disk and memory accesses, I/O , etc.)
– a useful number, but often not good for comparison purposes
• CPU time
– doesn't count I/O or time spent running other programs
– can be broken up into system time, and user time

• Our focus: user CPU time

– time spent executing the lines of code that are "in" our program
 Book's Definition of Performance
• For some program running on machine X,

PerformanceX = 1 / Execution timeX

• "X is n times faster than Y"

PerformanceX / PerformanceY = n
 Relative Performance
• Computer A runs a program in 10s and
• computer B runs it in 15s
• how much faster is A than B?
 Relative Performance
• If Computer A runs a program in 10s and computer B runs it in 15s,
how much faster is A than B?
– Performance A/ Performance B = Execution Time B/ Execution Time A
= 15/10 = 1.5

A is 1.5 times faster B

 Clock Cycles
• Instead of reporting execution time in seconds,
we often use cycles
seconds cycles seconds
= 
program program cycle

Clock period

Clock (cycles)

Data transfer
and computation
Update state

◼ Clock period: duration of a clock cycle

◼ e.g., 250ps = 0.25ns = 250×10–12s
◼ Clock frequency (rate): cycles per second
◼ e.g., 4.0GHz = 4000MHz = 4.0×109Hz
 Machine Clock Rate
• Clock rate (clock cycles per second in MHz or
GHz) is inverse of clock cycle time (clock
period)
CC = 1 / CR
one clock period

10 nsec clock cycle => 100 MHz clock rate

5 nsec clock cycle => 200 MHz clock rate
2 nsec clock cycle => 500 MHz clock rate
1 nsec (10-9) clock cycle => 1 GHz (109) clock rate
500 psec clock cycle => 2 GHz clock rate
250 psec clock cycle => 4 GHz clock rate
200 psec clock cycle => 5 GHz clock rate
 Performance
• What Impact Performance?
 Performance
• What Impact Performance?
 How to Improve Performance

seconds cycles seconds

= 
program program cycle

• Performance improved by
– Reducing number of clock cycles
– Increasing clock rate
– Hardware designer must often trade off clock
rate against cycle count
 How many cycles are required for a program?

• Could assume that number of cycles equals number of instructions

2nd instruction
3rd instruction
1st instruction

4th
5th
6th
...
time

This assumption is incorrect,

different instructions take different amounts of time on different machines.

Why? hint: remember that these are machine instructions, not lines of C code
 Different numbers of cycles for different instructions

time

• Floating point operations take longer than integer ones

• Accessing memory takes more time than accessing

registers

• Important point: changing the cycle time often changes the

number of cycles required for various instructions (more later)
 Example
• Our favorite program runs in 10 seconds on computer A, which has a 4 GHz. clock.
We are trying to help a computer designer build a new machine B, that will run this
program in 6 seconds. The designer can use new (or perhaps more expensive)
technology to substantially increase the clock rate, but has informed us that this
increase will affect the rest of the CPU design, causing machine B to require 1.2 times
as many clock cycles as machine A for the same program. What clock rate should we
tell the designer to target?"
 Example
seconds cycles seconds
• Our favorite program runs in 10 seconds on = 
computer A, which has a 4 GHz. clock. We program program cycle
are trying to help a computer designer build a
new machine B, that will run this program in
6 seconds. The designer can use new (or
perhaps more expensive) technology to
substantially increase the clock rate, but has
informed us that this increase will affect the
rest of the CPU design, causing machine B to
require 1.2 times as many clock cycles as
machine A for the same program. What
clock rate should we tell the designer to
target?"
 Terminologies
• Cycle time (seconds per cycle) CT seconds cycles seconds
= 
program program cycle
• Clock rate (cycles per second) CR
𝑐𝑦𝑐𝑙𝑒𝑠 𝑐𝑦𝑐𝑙𝑒𝑠
• CPI (cycles per instruction) = 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛 𝐶𝑜𝑢𝑛𝑡 ×
𝑃𝑟𝑜𝑔𝑟𝑎𝑚 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛
• IC instruction Count

𝑠𝑒𝑐𝑜𝑛𝑑𝑠 𝑐𝑦𝑐𝑙𝑒𝑠 𝑠𝑒𝑐𝑜𝑛𝑑𝑠

= 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛 𝐶𝑜𝑢𝑛𝑡 × ×
𝑃𝑟𝑜𝑔𝑟𝑎𝑚 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛 𝑐𝑦𝑐𝑙𝑒

Execution time = IC x CPI x CT

Execution time = IC x CPI / CR
 Instruction Count and CPI
Clock Cycles = Instruction Count  Cycles per Instruction
CPU Time = Instruction Count  CPI  Clock Cycle Time
Instruction Count  CPI
=
Clock Rate
• Instruction Count for a program
– Determined by program, ISA and compiler
• Average cycles per instruction
– Determined by CPU hardware
– If different instructions have different CPI
• Average CPI affected by instruction mix
Back to the Same Formula, CPI (Cycles/Instruction)
seconds cycles seconds
= 
program program cycle
 CPI Example
• Suppose we have two implementations of the same instruction set
architecture (ISA).

For some program,

Machine A has a clock cycle time of 250 ps and a CPI of 2.0

Machine B has a clock cycle time of 500 ps and a CPI of 1.2

What machine is faster for this program, and by how much?

 CPI Example
• Suppose we have two implementations of the same
instruction set seconds cycles seconds
architecture (ISA). = 
program program cycle
For some program,

Machine A has a clock cycle time of 250 ps and a CPI of

2.0
Machine B has a clock cycle time of 500 ps and a CPI of
1.2

What machine is faster for this program, and by how

much?
 Let’s Complicate Things A Little bit…

A compiler designer is trying to decide between Which sequence will be faster? How much?
two code sequences for a particular machine.
Based on the hardware implementation, there are
three different classes of instructions: Class A,
Class B, and Class C, and they require one, two,
and three cycles (respectively).

What is the CPI for each sequence?

The first code sequence has 5 instructions: 2 of A, 1

of B, and 2 of C

The second sequence has 6 instructions: 4 of A, 1 of

B, and 1 of C.
 Let’s Complicate Things A Little bit…

A compiler designer is trying to decide Which sequence will be faster? How much?
between two code sequences for a
particular machine. Based on the
hardware implementation, there are
three different classes of instructions: Clock cycles = σ 𝐶𝑃𝐼𝑖 × 𝐼𝐶𝑖
Class A, Class B, and Class C, and they
require one, two, and three cycles
(respectively).

What is the CPI for each sequence?

The first code sequence has 5

instructions: 2 of A, 1 of B, and 2 of C

The second sequence has 6 instructions:

4 of A, 1 of B, and 1 of C.
 A Simple Example
Op Freq CPIi Freq x CPIi
ALU 50% 1 .5 .5 .5 .25

Load 20% 5 1.0 .4 1.0 1.0

Store 10% 3 .3 .3 .3 .3

.4 .4 .2 .4
Branch 20% 2

= 2.2 1.6 2.0 1.95

• How much faster would the machine be if a better data cache

reduced the average load time to 2 cycles?
CPU time new = 1.6 x IC x CC so 2.2/1.6 means 37.5% faster

• How does this compare with using branch prediction to shave a

cycle off the branch time?
CPU time new = 2.0 x IC x CC so 2.2/2.0 means 10% faster

• What if two ALU instructions could be executed at once?

CPU time new = 1.95 x IC x CC so 2.2/1.95 means 12.8% faster
 Determinates of CPU Performance
CPU time = Instruction_count x CPI x clock_cycle

Instruction_ CPI clock_cycle

count
Algorithm

Programming
language
Compiler

ISA

Core
organization
Technology
 Determinates of CPU Performance
CPU time = Instruction_count x CPI x clock_cycle

Instruction_ CPI clock_cycle

count
Algorithm
X X

Programming X X
language
Compiler X X

ISA
X X X

Core
X X
organization
Technology
X
Component Analysis
 Performance Summary
The BIG Picture

Instructio ns Clock cycles Seconds

CPU Time =  
Program Instructio n Clock cycle

• Performance depends on
– Algorithm: affects IC, possibly CPI
– Programming language: affects IC, CPI
– Compiler: affects IC, CPI
– Instruction set architecture: affects IC, CPI, Tc
 Pitfall 1
• Can we use subset of the performance equation as a performance
measure ?? !!!

Instructio ns Clock cycles Seconds

CPU Time =  
Program Instructio n Clock cycle
 What is MIPS?

• Instruction execution rate => higher is better

• Issues:
– Can not compare processors with different
instruction sets
– Varies between programs on the same processor
(a computer can’t have single MIPS rating)
MIPS Example
MIPS Example
 Pitfall 2
• Expecting the improvement of one aspect of a computer to increase
overall performance by an amount proportional to the size of the
improvement.
 Amdahl's Law
• The performance enhancement of an improvement is limited by
how much the improved feature is used. In other words: Don’t
expect an enhancement proportional to how much you enhanced
something.
 Amdahl's Law
• Example:
"Suppose a program runs in 100 seconds on a machine, with multiply
operations responsible for 80 seconds of this time. How much do we have to
improve the speed of multiplication if we want the program to run 5 times faster?"
 Benchmarking the intel core i7
• Performance best determined by running a real application
– Use programs typical of expected workload
– Or, typical of expected class of applications
e.g., compilers/editors, scientific applications, graphics, etc.
• Small benchmarks
– nice for architects and designers
– easy to standardize
• SPEC (System Performance Evaluation Cooperative)
– companies have agreed on a set of real program and inputs
– valuable indicator of performance (and compiler technology)
 SPEC CPU Benchmark
• Programs used to measure performance
– Supposedly typical of actual workload
• Standard Performance Evaluation Corp (SPEC)
– Develops benchmarks for CPU, I/O, Web, …

• SPEC CPU2006
– Elapsed time to execute a selection of programs
• Negligible I/O, so focuses on CPU performance
– Normalize relative to reference machine
– Summarize as geometric mean of performance ratios
• CINT2006 (integer) and CFP2006 (floating-point)

n
n
 Execution time ratio
i=1
i
CINT2006 for Intel Core i7 920
 Recap
• Introduction
• Eight great ideas in computer architecture
• Performance
• Real stuff: Benchmarking the Intel core i7
• Fallacies and pitfalls