Computer Architecture and

Organization
Lecture 01: Introduction
Dr. Samr Ali
Abu Dhabi University
Fall 2024

Dr. Huma Zia and Dr. Samr Ali 1

 Outline

• Welcome and Course Overview

• Why Computer Architecture Matters
• Evolution of Computing
• The Building Blocks: Components of a Computer
• Instruction Execution Cycle
• The Road Ahead: Topics to Explore

Dr. Huma Zia and Dr. Samr Ali 2

 Course Outcomes

CLO1: Demonstrate an understanding of computers design and architecture.

CLO2: Apply pipelining, processing, and memory principles to formulate and solve computer
design problems.

CLO3: Design a VHDL basic computer architecture systems incorporating engineering constraints
and standards.

CLO4: Communicate effectively by developing a quality project report and a presentation.

CLO5: Develop and conduct experimentation on VHDL memory, VHDL ALUs, and VHDL control
logic. Analyze and interpret collected data, and draw engineering conclusions.
Dr. Huma Zia and Dr. Samr Ali 3
Why Computer
Architecture
Matters

Dr. Huma Zia and Dr. Samr Ali 4

Question:
What Is This?

Dr. Huma Zia and Dr. Samr Ali 5

Answer: Masterpiece of A Famous Architect

Dr. Huma Zia and Dr. Samr Ali 6

 A Key Question
• How Was Wright Able To Design Fallingwater?
• Can have many guesses
• (Ultra) hard work, perseverance, dedication (over decades)
• Experience of decades
• Creativity
• Out-of-the-box thinking
• Principled design
• A good understanding of past designs
• Good judgment and intuition
• Strong combination of skills (math, architecture, art, …)
• …

• (You will be exposed to and hopefully develop/enhance many of these skills

in this course)

Dr. Huma Zia and Dr. Samr Ali 7

 A Quote from The Architect Himself

“architecture […] based upon principle, and not upon precedent”

Dr. Huma Zia and Dr. Samr Ali 8

Dr. Huma Zia and Dr. Samr Ali 9
 Major High-Level Goals of This
Course
• Understand the principles
• Understand the precedents

• Based on such understanding:

• Enable you to evaluate tradeoffs of different designs
and ideas
• Enable you to develop principled designs
• Enable you to develop novel, out-of-the-box designs

• The focus is on:

• Principles, precedents, and how to use them for new
designs

Dr. Huma Zia and Dr. Samr Ali 10

Role of the (Computer)
Architect

• from Yale Patt’s lecture notes

Dr. Huma Zia and Dr. Samr Ali 11

What is Computer
Architecture?
• The science and art of designing, selecting,
and interconnecting hardware components
and designing the hardware/software
interface to create a computing system that
meets functional, performance, energy
consumption, cost, and other specific goals.

Dr. Huma Zia and Dr. Samr Ali 12

Why Study Computer Architecture?
• Enable better systems: make computers faster, cheaper, smaller,
more reliable, …
• By exploiting advances and changes in underlying
technology/circuits
• Enable new applications
• Life-like 3D visualization 20 years ago?
• Virtual reality?
• Personalized genomics? Personalized medicine?
• Enable better solutions to problems
• Software innovation is built into trends and changes in
computer architecture
• > 50% performance improvement per year has enabled
this innovation
• Understand why computers work the way they do

Dr. Huma Zia and Dr. Samr Ali 13

Role of The (Computer) Architect
• Look backward (to the past)
• Understand tradeoffs and designs, upsides/downsides, past workloads.
Analyze and evaluate the past.
• Look forward (to the future)
• Be the dreamer and create new designs. Listen to dreamers.
• Push the state of the art. Evaluate new design choices.
• Look up (towards problems in the computing stack)
• Understand important problems and their nature.
• Develop architectures and ideas to solve important problems.
• Look down (towards device/circuit technology)
• Understand the capabilities of the underlying technology.
• Predict and adapt to the future of technology (you are designing for N years
ahead). Enable the future technology.
Dr. Huma Zia and Dr. Samr Ali 14
Levels of Transformation
“The purpose of computing is insight” (Richard Hamming)
We gain and generate insight by solving problems
How do we ensure problems are solved by electrons?

Problem
Algorithm
Program/Language
Runtime System
(VM, OS, MM)
ISA (Architecture)
Microarchitecture
Logic
Circuits
Electrons

Dr. Huma Zia and Dr. Samr Ali 15

The Power of Abstraction
• Levels of transformation create abstractions
• Abstraction: A higher level only needs to know about the interface to the lower level, not
how the lower level is implemented
• E.g., high-level language programmer does not really need to know what the ISA is and
how a computer executes instructions

• Abstraction improves productivity

• No need to worry about decisions made in underlying levels
• E.g., programming in Java vs. C vs. assembly vs. binary vs. by specifying control signals of
each transistor every cycle

• Then, why would you want to know what goes on underneath or

above?
Dr. Huma Zia and Dr. Samr Ali 16
 Crossing the Abstraction
Layers
• As long as everything goes well, not knowing what happens in the
underlying level (or above) is not a problem.

• What if
• The program you wrote is running slow?
• The program you wrote does not run correctly?
• The program you wrote consumes too much energy?

• What if
• The hardware you designed is too hard to program?
• The hardware you designed is too slow because it does not
provide the right primitives to the software?

• What if
• You want to design a much more efficient and higher
performance system?

Dr. Huma Zia and Dr. Samr Ali 17

Crossing the Abstraction
Layers
• Two key goals of this course are

• to understand how a processor works underneath

the software layer and how decisions made in
hardware affect the software/programmer

• to enable you to be comfortable in making design

and optimization decisions that cross the
boundaries of different layers and system
components

Dr. Huma Zia and Dr. Samr Ali 18

Abstraction Layers

Dr. Huma Zia and Dr. Samr Ali 19

Architecture is engineering design under
constraints

Dr. Huma Zia and Dr. Samr Ali 20

What Will You Learn?

• Fundamental principles and tradeoffs in designing the

hardware/software interface and major components of a modern
programmable microprocessor
• Focus on state-of-the-art (and some recent research and trends)
• Trade-offs and how to make them

• How to design, implement, and evaluate a functional modern processor

• Semester-long lab assignments
• A combination of RTL implementation and higher-level simulation
• Focus is functionality first (some on “how to do even better”)

• How to dig out information, think critically and broadly

• How to work even harder!

Dr. Huma Zia and Dr. Samr Ali 21

Course Goals

Dr. Huma Zia and Dr. Samr Ali 22

 • Required background: Digital Circuits, Programming,
Assembly language

• Learn the material thoroughly

• attend lectures, do the readings, do the homeworks
What Do I • Do the work & work hard
• Ask questions, take notes, participate
Expect • Perform the assigned readings
• Come to class on time
From You? • Start early – do not procrastinate
• If you want feedback, come to office hours

• Remember “Chance favors the prepared mind.”

(Pasteur)

Dr. Huma Zia and Dr. Samr Ali 23

 • Absolutely no form of cheating will be tolerated

A Note on • You are all adults and we will treat you so

Cheating
• See syllabus, ADU Policy, and ECBE Academic
and Integrity Policy
• Linked from syllabus
Academic
• Cheating → Failing grade (no exceptions)
Dishonesty • And, perhaps more

Dr. Huma Zia and Dr. Samr Ali 24

 Evolution of
Computing

Dr. Huma Zia and Dr. Samr Ali 25

Computers Then
Dr. Huma Zia and Dr. Samr Ali 26
Computing devices now..
Dr. Huma Zia and Dr. Samr Ali 27
 What do
computers
actually do?

Dr. Huma Zia and Dr. Samr Ali 28

Charles Babbage • Difference Engine, 1823: a special-purpose computer
(1791-1871) – evaluated polynomial functions by method of successive differences
(requiring only additions)
– The Difference Engine was an early attempt at creating a
mechanical calculator, and it operated using a system of gears and
rotating components.

• Analytical Engine, 1833: a general-purpose computer

– programmed by punch cards, “assembly language” included
loops and branches
– 1000 word data store, punch card I/O
– unfortunately never completed (would have been
10x30 meters,

Dr. Huma Zia and Dr. Samr Ali 29

[images from Wikipedia]
 100 Years of Technology Advances
• Mechanical, 1800s
– gears, chains, pulleys, and steam power
– punch cards!!
• Electromechanical, early 1900s
– switches, relays, acoustic delay line memory
– e.g. Harvard/IBM Mark 1, Aiken 1939~1944, 50ft long, 5ton, 750K parts, 3~6 sec
per addition
Used ideas from Analytical Engine
• Electrical, mid 1900s and on
– plugboards, vacuum tubes, cathode ray tube memory
– and later DRUM, core, transistors and so on . . . . .
Changing demands and economics?
Dr. Huma Zia and Dr. Samr Ali 30
Electronic Numerical Integrator and Computer
(ENIAC)

• Programmed by 3000 switches in the

function table and plug-cables (later
became stored program for faster
program loading)

Dr. Huma Zia and Dr. Samr Ali 31

Electronic
Discrete Variable
Automatic
Computer
(EDVAC)

Dr. Huma Zia and Dr. Samr Ali 32

 Commercialization in the 50s
• UNIVAC (Universal Automatic Computer, 1951) the first commercial computer
• contract price $400K, actual cost ~$1M, sold 48 copies
• IBM 701 (1952) “leased” 19 units, $12K per month
• IBM 650 (1953) sold ~2000 units at $200K ~ 400K
• IBM System/360, 1964 Redefined Industry!!
• a family of binary compatible computers (previously, IBM had 4
incompatible lines)
• 19 combinations of varying speed and memory capacity from $200K ~ $2M
• ISA still alive today in z/Architecture mainframes

Dr. Huma Zia and Dr. Samr Ali 33

Dr. Huma Zia and Dr. Samr Ali 34
Cheaper or Faster in
60s and 70s
• Minicomputers
• DEC PDP-8, 1965, $20K, size of large refrigerators
• less powerful than “mainframes”, 10x cheaper
• departmental computers, timesharing---PDP-11 and VAXs
enjoyed extreme popularity in the 70s and 80s
• Supercomputers
• performance at all cost!! (ECL, liquid-cool, hand-built)
• biggest customers: national security, nuclear weapons, cryptography,
(also aerospace, petroleum, automotive, pharmaceutical, sciences)
• see Seymour Cray (1925~1996) on Wikipedia

What happened to these computer lines?

Dr. Huma Zia and Dr. Samr Ali 35
 Early Examples

[images from Wikipedia]

DEC PDP 8, 1963 Xerox Alto, 1973
an early mini an early “PC” with
mouse and GUI
Dr. Huma Zia and Dr. Samr Ali 36
Cray
3,
1993

90KW: liquid cooled by “Fluorinert”

$30,000,000
15 GFLOPS (1 sec on Cray3 67 years ENIAC) [images from Wikipedia]

Dr. Huma Zia and Dr. Samr Ali 37

 The “Killer Micros” from 70s and on

• Intel 4004, first single chip CPU

– 4-bit processor for calculator
– 2,300 transistors
– 16-pin DIP package
– 740kHz (eight clock cycles per
CPU cycle of 10.8 msec)
– ~100K OPs per second

download the actual schematic

from www.4004.com
[from Molecular Expressions]

Dr. Huma Zia and Dr. Samr Ali 38

Intel Itanium
• 64-bit processor
(Montecito) 2004
• 1.7 billion transistors
• 1.7 GHz, issue up to 8
instructions per cycle
• 26 MByte of cache!!

In ~30 years, about

[from Best Servers of 2004, 100,000 fold growth
Microprocessor Report, January 2005.] in transistor count and
performance!

Dr. Huma Zia and Dr. Samr Ali 39

The Era of
Moore’s
Law

[http://www.intel.com/research/silicon/mooreslaw.htm]
Original article at Cramming More Components Onto Integrated Circuits | IEEE
Journals & Magazine | IEEE Xplore
Moore's law: past, present and future | IEEE Journals &
Magazine | IEEE Xplore
Dr. Huma Zia and Dr. Samr Ali 40
Computing’s Brave New World
Microsoft Catapult
[MICRO 2016,
Caulfield, et al.]

Google TPU
[Hotchips, 2017,
Jeff Dean]

CALCM, ©2023
Dr. Huma Zia and Dr. Samr Ali 43
 Computing’s Brave New World

[Cerebras Wafer-Scale AI Processor]

[Selene AI Supercomputer]

Dr. Huma Zia and Dr. Samr Ali 44

Why the Disparity in Slowdowns?
CORE
matlab1 gcc 2
CORE Multi-Core
Chip

L2 L2
CACHE CACHE
unfairness
INTERCONNECT
Shared DRAM
DRAM MEMORY CONTROLLER Memory System

DRAM DRAM DRAM DRAM

Bank 0 Bank 1 Bank 2 Bank 3

Dr. Huma Zia and Dr. Samr Ali 47

DRAM Bank Operation
Access Address:
(Row 0, Column 0) Columns
(Row 0, Column 1)
(Row 0, Column 85)

Row decoder
(Row 1, Column 0)

Rows
Row address 0
1

Row 01
Row
Empty Row Buffer CONFLICT
HIT !

Column address 0
1
85 Column mux

Data

Dr. Huma Zia and Dr. Samr Ali 48

DRAM Controllers
• A row-conflict memory access takes significantly longer
than a row-hit access

• Current controllers take advantage of the row buffer

• Commonly used scheduling policy (FR-FCFS) [Rixner 2000]*

(1) Row-hit first: Service row-hit memory accesses first
(2) Oldest-first: Then service older accesses first

• This scheduling policy aims to maximize DRAM

throughput
*Rixner et al., “Memory Access Scheduling,” ISCA 2000.
*Zuravleff and Robinson, “Controller for a synchronous DRAM …,” US Patent 5,630,096, May 1997.

Dr. Huma Zia and Dr. Samr Ali 49

The Problem
• Multiple applications share the DRAM controller
• DRAM controllers designed to maximize DRAM data
throughput

• DRAM scheduling policies are unfair to some

applications
• Row-hit first: unfairly prioritizes apps with high row buffer locality
• Threads that keep on accessing the same row
• Oldest-first: unfairly prioritizes memory-intensive applications

• DRAM controller vulnerable to denial of service attacks

• Can write programs to exploit unfairness

Dr. Huma Zia and Dr. Samr Ali 50

A Memory Performance Hog
// initialize large array A // initialize large array A

for (j=0; j<N; j++) { for (j=0; j<N; j++) {

index = j*linesize; streaming index = rand(); random
A[index] = B[index]; A[index] = B[index];
… …
} }

STREAM RANDOM
- Sequential memory access - Random memory access
- Very high row buffer locality (96% hit rate) - Very low row buffer locality (3% hit rate)
- Memory intensive - Similarly memory intensive

Moscibroda and Mutlu, “Memory Performance Attacks,” USENIX Security 2007.

Dr. Huma Zia and Dr. Samr Ali 51

What Does the Memory Hog Do?

Row decoder
T0: Row 0
T0:
T1: Row 05
T1:
T0:Row
Row111
0
T1:
T0:Row
Row16
0
Memory Request Buffer Row
Row 00 Row Buffer

Row size: 8KB, cache blockColumn mux

size: 64B
T0: STREAM
125
T1: (8KB/64B)
RANDOM requests of T0 serviced
Data before T1

Moscibroda and Mutlu, “Memory Performance Attacks,” USENIX Security 2007.

Dr. Huma Zia and Dr. Samr Ali 52

DRAM in the System
Multi-Core
Chip

L2 CACHE 1
L2 CACHE 0
SHARED L3 CACHE

DRAM INTERFACE

DRAM BANKS
CORE 0 CORE 1

DRAM MEMORY
CONTROLLER

L2 CACHE 2

L2 CACHE 3
CORE 2 CORE 3

*Die photo credit: AMD Barcelona

Dr. Huma Zia and Dr. Samr Ali 55
Another Example
• DRAM Refresh

Dr. Huma Zia and Dr. Samr Ali 56

A DRAM Cell
wordline (row enable)
bitline

bitline

bitline

bitline
◼ A DRAM cell consists of a capacitor and an access transistor
◼ It stores data in terms of charge in the capacitor
◼ A DRAM chip consists of (10s of 1000s of) rows of such cells

Dr. Huma Zia and Dr. Samr Ali 57

DRAM Refresh
◼DRAM capacitor charge leaks over time

◼The memory controller needs to refresh each row

periodically to restore charge
❑Activate each row every N ms
❑Typical N = 64 ms

◼Downsides of refresh
-- Energy consumption: Each refresh consumes energy
-- Performance degradation: DRAM rank/bank unavailable while
refreshed
-- QoS/predictability impact: (Long) pause times during refresh
-- Refresh rate limits DRAM capacity scaling
Dr. Huma Zia and Dr. Samr Ali 58
The Earliest Instruction Sets Burks, Goldstein &
von Neumann ~1946

Dr. Huma Zia and Dr. Samr Ali 60

Programming: Single Accumulator Machine

Dr. Huma Zia and Dr. Samr Ali 61

… but, first …
• Let’s understand the fundamentals…

• You can change the world only if you understand it well enough…
• Especially the past and present dominant paradigms
• And, their advantages and shortcomings – tradeoffs
• And, what remains fundamental across generations
• And, what techniques you can use and develop to solve problems

Dr. Huma Zia and Dr. Samr Ali 62

Fundamental Concepts

Dr. Huma Zia and Dr. Samr Ali 63

 • We will cover all three components

Processing

control Memory
What is A (sequencing) (program I/O
and data)
Computer? datapath

Dr. Huma Zia and Dr. Samr Ali 64

“Classic” Computer Organization
CPU CPU
ALU RF ALU RF

cache cache

Memory “Bus”

I/O I/O “Bus”

Main Memory Bridge
(DRAM) Kbd & Net-
Disk
Disk Video
Disk
Disk Mouse work

Dr. Huma Zia and Dr. Samr Ali 65

The Von Neumann Model/Architecture
• Also called stored program computer (instructions in
memory). Two key properties:

• Stored program
• Instructions stored in a linear memory array
• Memory is unified between instructions and data
• The interpretation of a stored value depends on the control signals
When is a value interpreted as an instruction?

• Sequential instruction processing

• One instruction processed (fetched, executed, and completed) at a time
• Program counter (instruction pointer) identifies the current instr.
• Program counter is advanced sequentially except for control transfer
instructions

Dr. Huma Zia and Dr. Samr Ali 66

 Memory
• Register • ROM (Read-Only Memory)
• Description: Smallest, fastest type of memory located in the CPU. • Description: Non-volatile memory that can't typically be written to by the
• Use Cases: Storing data that the CPU is currently processing. user.
• Use Cases: Firmware storage, such as BIOS/UEFI.
• Cache (L1, L2, L3)
• Description: Fast, on- or near-CPU memory. Varies in size and speed. L1 is • Hard Drive/SSD
the smallest/fastest, L3 is the largest/slowest. • Description: Non-volatile storage devices. SSDs are faster than HDDs.
• Use Cases: Temporarily storing frequently accessed data for quick HDDs use magnetic storage, SSDs use flash memory.
retrieval by the CPU. • Use Cases: Long-term data storage, including OS, applications, and user
files.
• Main Memory (RAM)
• Description: Volatile memory used for running applications and OS. Types • Virtual Memory
include DRAM, SDRAM. • Description: Uses a section of HDD/SSD to emulate RAM.
• Use Cases: Storing running applications, system processes, and • Use Cases: Expanding effective RAM; managing and isolating application
temporary data. processes.
• Static RAM (SRAM) • Flash Memory
• Description: Fast, stable RAM made of flip-flops. Doesn't need refreshing. • Description: Non-volatile memory that can be electrically erased and
• Use Cases: CPU caches, certain memory buffers. reprogrammed.
• Use Cases: USB drives, SD cards, SSDs, mobile devices.
• Dynamic RAM (DRAM)
• Description: Slower, cheaper RAM that needs refreshing. • Optical Memory
• Use Cases: Main system memory in most computers. • Description: Uses lasers to read/write data. Non-volatile.
• Use Cases: CDs, DVDs, Blu-ray discs.

Dr. Huma Zia and Dr. Samr Ali 67

DRAM Vs SRAM
• Basic Operation:
• DRAM: In DRAM, each bit of data is stored as a charge in a capacitor. Because these charges tend to leak, they need to be refreshed periodically, typically thousands of
times per second, which is why it's called "dynamic."
• SRAM: In SRAM, each bit is stored using flip-flop circuits (made of transistors). There's no need for periodic refreshing, which makes it "static."
• Speed:
• DRAM: Generally slower than SRAM. The need to refresh the capacitors introduces extra latency.
• SRAM: Faster than DRAM. Since there's no need for refreshing, access times are quicker.
• Complexity and Size:
• DRAM: Each cell in DRAM typically consists of one transistor and one capacitor, making it simpler and allowing for higher densities.
• SRAM: Each SRAM cell requires multiple transistors (typically six). This makes SRAM cells larger and less dense than DRAM cells.
• Cost:
• DRAM: Cheaper per bit than SRAM because of its simpler design and higher density. This is why DRAM is used for main system memory, where large quantities are
required.
• SRAM: More expensive per bit than DRAM because it uses more transistors for each memory cell.
• Power Consumption:
• DRAM: Consumes less power when active compared to SRAM. However, it continuously consumes power due to the need for periodic refreshing.
• SRAM: Generally consumes more power when active, but in low-power or standby modes, it can be more power-efficient since it doesn't need refreshing
• Use Cases:
• DRAM: Typically used for the main memory in computers and other devices because of its lower cost per bit and reasonable speed.
• SRAM: Due to its speed, SRAM is often used for cache memory in CPUs, where fast access times are crucial and the memory size requirements are smaller than main
memory.
• Volatility:
• Both DRAM and SRAM are volatile memory types, meaning they lose their content when the power is turned off. However, the term "volatile" is more often associated
with DRAM due to its need for periodic refreshing even when power is continuously applied.

Dr. Huma Zia and Dr. Samr Ali 68

The Von Neumann Model (of a Computer)
MEMORY
Mem Addr Reg

Mem Data Reg

PROCESSING UNIT
INPUT OUTPUT
ALU TEMP

CONTROL UNIT

IP Inst Register

Dr. Huma Zia and Dr. Samr Ali 69

The Dataflow Model (of a Computer)
• Von Neumann model: An instruction is fetched and
executed in control flow order
• As specified by the instruction pointer
• Sequential unless explicit control flow instruction

• Dataflow model: An instruction is fetched and executed

in data flow order
• i.e., when its operands are ready
• i.e., there is no instruction pointer
• Instruction ordering specified by data flow dependence
• Each instruction specifies “who” should receive the result
• An instruction can “fire” whenever all operands are received
• Potentially many instructions can execute at the same time
• Inherently more parallel

Dr. Huma Zia and Dr. Samr Ali 71

Von Neumann vs Dataflow
◼Consider a Von Neumann program
❑What is the significance of the program order?
❑What is the significance of the storage locations?
a b
v <= a + b;
w <= b * 2;
x <= v - w + *2
y <= v + w
z <= x * y
- +
Sequential
*
Dataflow

z
◼Which model is more natural to you as a programmer?
Dr. Huma Zia and Dr. Samr Ali 72
More on Data Flow
• In a data flow machine, a program consists of data flow
nodes
• A data flow node fires (fetched and executed) when all it
inputs are ready
• i.e. when all inputs have tokens

• Data flow node and its ISA representation

Dr. Huma Zia and Dr. Samr Ali 73

Data Flow Nodes

Dr. Huma Zia and Dr. Samr Ali 74

An Example Data Flow Program

OUT

Dr. Huma Zia and Dr. Samr Ali 75

ISA-level ◼ Do we need an instruction pointer in the ISA?
❑Yes: Control-driven, sequential execution
Tradeoff: ◼ An instruction is executed when the IP
points to it
Instruction ◼ IP automatically changes sequentially
(except for control flow instructions)
Pointer ❑No: Data-driven, parallel execution
◼ An instruction is executed when all its
operand values are available (data flow)

◼ Tradeoffs: MANY high-level ones

❑Ease of programming (for average
programmers)?
❑Ease of compilation?
❑Performance: Extraction of parallelism?
❑Hardware complexity?

Dr. Huma Zia and Dr. Samr Ali 76

ISA vs. Microarchitecture Level Tradeoff
• A similar tradeoff (control vs. data-driven execution) can
be made at the microarchitecture level

• ISA: Specifies how the programmer sees instructions to be

executed
• Programmer sees a sequential, control-flow execution order vs.
• Programmer sees a data-flow execution order

• Microarchitecture: How the underlying implementation

actually executes instructions
• Microarchitecture can execute instructions in any order as long
as it obeys the semantics specified by the ISA when making the
instruction results visible to software
• Programmer should see the order specified by the ISA

Dr. Huma Zia and Dr. Samr Ali 77