Unit 2: Concepts about Algorithms

․Course contents:
 Computational complexity
 NP-completeness
 Algorithmic Paradigms
․Readings
 Chapters 3, 4, and 5

Unit 2 1
Chang, Huang, Li, Lin, Liu

Now You Need Algorithms…

․To put devices/interconnects together into VLSI chips
․Fundamental questions: How do you do it smartly?
․Definition of algorithm in a board sense:
 A step-by-step procedure for solving a problem
․Examples:
 Cooking a dish
 Making a phone call
 Sorting a hand of cards
․Definition for a computational problem:
 A well-defined computational procedure that takes some
values as input and produces desired values as output

Unit 2 2
Chang, Huang, Li, Lin, Liu
 On Algorithms
․Algorithm: A well-defined procedure for transforming
some input to a desired output.
․Major concerns:
 Correctness: Does it halt? Is it correct? Is it stable?
 Efficiency: Time complexity? Space complexity?
 Worst case? Average case? (Best case?)
․Better algorithms?
 How: Faster algorithms? Algorithms with less space
requirement?
 Optimality: Prove that an algorithm is best
possible/optimal? Establish a lower bound?
․Applications?
 Everywhere in computing!

Unit 2 3
Chang, Huang, Li, Lin, Liu

O: Upper Bounding Function

․Def: f(n)= O(g(n)) if  c >0 and n0 > 0 such that 0  f(n)
 cg(n) for all n  n0.
 Examples: 2n2 + 3n = O(n2), 2n2 = O(n3), 3n lg n = O(n2)
․Intuition: f(n) “ ” g(n) when we ignore constant
multiples and small values of n.

Unit 2 4
Chang, Huang, Li, Lin, Liu
 Big-O Notation
․How to show O (Big-Oh) relationships?
 f(n) = O(g(n)) iff limn   f ( n ) = c for some c  0.
g (n)
․“An algorithm has worst-case running time O(f(n))”:
there is a constant c s.t. for every n big enough, every
execution on an input of size n takes at most cf(n)
time.

Unit 2 5
Chang, Huang, Li, Lin, Liu

Infinity

Unit 2 6
Chang, Huang, Li, Lin, Liu
 Big-Oh Examples

․ Def: f(n)= O(g(n)) if  c > 0 and n0 > 0 such that

0  f(n)  cg(n) for all n  n0.

1. 3n2 + n = O(n2)? Yes

2. 3n2 + n = O(n)? No
3. 3n2 + n = O(n3)? Yes

3n2 + n  cn2?

Take c = 4, n0 = 1

f (n)
f(n) = O(g(n)) implies that limn   = c for some c  0,
if the limit exists. g(n)
Unit 2 7
Chang, Huang, Li, Lin, Liu

Computational Complexity
․Computational complexity: an abstract measure of the
time and space necessary to execute an algorithm as
function of its “input size”.
․Scalability with respect to input size is important
 How does the run time of an algorithm change when the input
size doubles?
 Function of input size n
 Examples: n +3n, 2n, nlogn, ...
2

 Generally, large input sizes are of interest

 n > 1,000 or even n > 1,000,000

․Time complexity is expressed in elementary

computational steps (e.g., an addition, multiplication,
pointer indirection).
․Space Complexity is expressed in memory locations
(e.g. bits, bytes, words).
Unit 2 8
Chang, Huang, Li, Lin, Liu
 Asymptotic Functions
․Polynomial-time complexity: O(nk), where n is the input
size and k is a constant.
․Example polynomial functions:
 999: constant
 lg n: logarithmic
 n : sublinear
 n: linear
 n lg n: loglinear
 n2: quadratic
 n3: cubic
․Example non-polynomial functions
 2n, 3n: exponential
 n!: factorial
Unit 2 9
Chang, Huang, Li, Lin, Liu

Running-time Comparison
․Assume 1000 MIPS (Yr: 200x), 1 instruction /operation

Unit 2 10
Chang, Huang, Li, Lin, Liu
 Optimization Problems
․Problem: a general class, e.g., “the shortest-path problem
for directed acyclic graphs.”
․Instance: a specific case of a problem, e.g., “the shortest-
path problem in a specific graph, between two given
vertices.”
․Optimization problems: those finding a legal configuration
such that its cost is minimum (or maximum).
 MST: Given a graph G=(V, E), find the cost of a minimum
spanning tree of G.
․An instance I = (F, c) where
F is the set of feasible solutions, and


 c is a cost function, assigning a cost value to each feasible

solution c : F  R
 The solution of the optimization problem is the feasible solution
with optimal (minimal/maximal) cost
․ c.f., Optimal solutions/costs, optimal (exact) algorithms (Attn:
optimal  exact in the theoretic computer science community).
Unit 2 11
Chang, Huang, Li, Lin, Liu

The Traveling Salesman Problem (TSP)

․TSP: Given a set of cities and that distance between
each pair of cities, find the distance of a “minimum tour”
starts and ends at a given city and visits every city
exactly once.

Unit 2 12
Chang, Huang, Li, Lin, Liu
 Decision Problem
․Decision problems: problem that can only be
answered with “yes” or “no”
 MST: Given a graph G=(V, E) and a bound K, is there a
spanning tree with a cost at most K?
 TSP: Given a set of cities, distance between each pair of cities,
and a bound B, is there a route that starts and ends at a given
city, visits every city exactly once, and has total distance at
most B?
․ A decision problem , has instances: I = (F, c, k)
 The set of of instances for which the answer is “yes” is given
by Y.
 A subtask of a decision problem is solution checking: given f 
F, checking whether the cost is less than k.
․Could apply binary search on decision problems to
obtain solutions to optimization problems.
․NP-completeness is associated with decision problems.

Unit 2 13
Chang, Huang, Li, Lin, Liu

The Circuit-Satisfiability Problem (Circuit-SAT)

․The Circuit-Satisfiability Problem (Circuit-SAT):
 Instance: A combinational circuit C composed of AND, OR,
and NOT gates.
 Question: Is there an assignment of Boolean values to the
inputs that makes the output of C to be 1?
․A circuit is satisfiable if there exists a a set of Boolean
input values that makes the output of the circuit to be 1.
 Circuit (a) is satisfiable since <x1, x2, x3> = <1, 1, 0> makes the
output to be 1.

Unit 2 14
Chang, Huang, Li, Lin, Liu
 Tractability
․Problems are classified into “easier” and “harder”
categories
 Class P: a polynomial time (in size of input) algorithm is known
for the problem (hence, it is a tractable problem)
 Class NP(non-deterministic polynomial time): a solution is
verifiable in polynomial time
 P  NP. Is P = NP? (Find out and become famous!)
 Practically, for a problem in NP but not in P: polynomial solution
not found yet (probably does not exist)
 exact (optimal) solution can be found in exponential time
 NP-completeness, NP-hardness, etc.
 Most CAD problems are NP-complete, NP-hard, or worse
 Be happy with a “reasonably good” solution

Unit 2 15
Chang, Huang, Li, Lin, Liu

Algorithmic Paradigms
․ Exhaustive search: Search the entire solution space
․ Branch and bound: Search with pruning
․ Greedy: Pick a locally optimal solution at each step
․ Dynamic programming: if subproblems are not independent
․ Divide-and-conquer (a.k.a. hierarchical): Divide a problem into
subproblems (small and similar), solve subproblems, and then
combine the solutions of subproblems
․ Multilevel: Bottom-up coarsening followed by top-down
uncoarsening
․ Mathematical programming: Solve an objective function under
constraints
․ Local search: Move from solution to solution in the search space
until a solution deemed optimal is found or a time bound is elapsed
․ Probabilistic: Make some choices randomly (or pseudo-randomly)
․ Reduction: Transform into a known and optimally solved problem
Unit 2 16
Chang, Huang, Li, Lin, Liu
 Algorithm Types
․Algorithms usually used for P problems
 Exhaustive search
 Divide-and-conquer (a.k.a. hierarchical)
 Dynamic programming
 Greedy
 Mathematical programming
 Branch and bound
․Algorithms usually used for NP (but not P) problems (strategy:
not seeking “optimal solution”, but a “good” one)
 Approximation
 Pseudo-polynomial time: polynomial form, but NOT to input size
 Restriction: restrict the problem to a special case that is in P
 Exhaustive search/branch and bound
 Local search: simulated annealing, genetic algorithm, ant colony
 Heuristics: greedy, … etc.
Unit 2 17
Chang, Huang, Li, Lin, Liu

Spanning Tree v.s. Steiner Tree

․ Manhattan distance: If two points (nodes) are located at
coordinates (x1, y1) and (x2, y2), the Manhattan distance between
them is given by d12 = |x1-x2| + |y1-y2|.
․ Rectilinear spanning tree: a spanning tree that connects its nodes
using Manhattan paths (Fig. (b) below).
․ Steiner tree: a tree that connects its nodes, and additional points
(Steiner points) are permitted to used for the connections.
․ The minimum rectilinear spanning tree problem is in P, while the
minimum rectilinear Steiner tree (Fig. (c)) problem is NP-complete.
 The spanning tree algorithm can be an approximation for the Steiner
tree problem (at most 50% away from the optimum).
Steiner
points

Unit 2 18
Chang, Huang, Li, Lin, Liu
 Exhaustive Search v.s. Branch and Bound

․ TSP example

Backtracking/exhaustive search

Branch and bound

Unit 2 19
Chang, Huang, Li, Lin, Liu

Divide-and-Conquer
․Divide and conquer:
 Divide: Recursively break down a problem into two or more
sub-problems of the same (or related) type
 Conquer: Until these become simple enough to be solved
directly
 Combine: The solutions to the sub-problems are then
combined to give a solution to the original problem
․Correctness: proved by mathematical induction
․Complexity: determined by solving recurrence relations

Unit 2 20
Chang, Huang, Li, Lin, Liu
 Example: Fibonacci Sequence
․Recurrence relation: Fn= Fn-1+ Fn-2, F0 = 0, F1 = 1
 e.g., 0, 1, 1, 2, 3, 5, 8, …
․Direct implementation:
 Recursion!

Unit 2 21
Chang, Huang, Li, Lin, Liu

What’s Wrong?

․What if we call fib(5)?

 fib(5)
 fib(4) + fib(3)
 (fib(3) + fib(2)) + (fib(2) + fib(1))
 ((fib(2) + fib(1)) + (fib(1) + fib(0))) + ((fib(1) + fib(0)) + fib(1))
 (((fib(1) + fib(0)) + fib(1))+(fib(1) + fib(0)))+((fib(1) + fib(0))+fib(1))
 A call tree that calls the function on the same value many
different times
 fib(2) was calculated three times from scratch
 Impractical for large n
Unit 2 22
Chang, Huang, Li, Lin, Liu
 Dynamic Programming: Memoization
․Store the values in a table
 Check the table before a recursive call
 Top-down!
 The control flow is almost the same as the original one

Unit 2 23
Chang, Huang, Li, Lin, Liu

Dynamic Programming: Bottom-up?

․Store the values in a table
 Bottom-up
 Compute the values for small problems first
 Much like induction

Unit 2 24
Chang, Huang, Li, Lin, Liu
 Mathematical Induction
․The first domino falls.
․If a domino falls, so will the next domino.
․All dominoes will fall!

Unit 2 25
Chang, Huang, Li, Lin, Liu

Abstraction
․All about abstraction!
 Graph: objects and their relationships
 Vertices: objects
 Edges: relationships!
․Examples:

Unit 2 26
Chang, Huang, Li, Lin, Liu
 Graphs
․Definition: A graph G = (V, E) consists two sets V and E
 V(G): a finite nonempty set of vertices (nodes)
 E(G): a set of edges (links, arcs)
․Undirected graph: undirected edges  (u, v) == (v, u)
․Directed graph: directed edges  <u, v> != <v, u>
․Weighted graph: weight w: E → R
 Sometimes, vertices also have weights

Unit 2 27
Chang, Huang, Li, Lin, Liu

Representation: Adjacency Matrix

․The adjacency matrix a of G is an VxV matrix where
 a[i][j] = 1 iff (u, v)  E(G) (or <u, v>  E(G))
 a[i][j] = 0, otherwise
․Degree?
 Undirected: row sum or column sum
 Directed: in-degree = column sum; out-degree = row sum
․Space: O(V2)
 Suitable for dense graph
 How to save space if undirected?
 What if sparse graph?

Unit 2 28
Chang, Huang, Li, Lin, Liu
 Representation: Adjacency List
․The adjacency list is an array of V chains, one for each
vertex, represents vertices adjacent from it
․Space: O(V+E)
 Good for sparse graph

Unit 2 29
Chang, Huang, Li, Lin, Liu

Edge/Vertex Weights
․Edge weights
 Usually represent the “cost” of an edge
 Examples:
 The delay of a wire in a circuit
 Distance between two cities
 Width of a data bus
 Representation
 Adjacency matrix: instead of 0/1, keep weight
 Adjacency list: keep the weight in an additional field of the linked
list item
․Vertex weights
 Usually used to enforce some “capacity” constraint
 Examples:
 The size of gates in a circuit
 The delay of operations in a “data dependency graph”

Unit 2 30
Chang, Huang, Li, Lin, Liu
 Graph Traversal
․Purpose: to visit all vertices
․Algorithms
 Depth-first search
 Breadth-first search
 Topological sort
 Applicable for DAGs
․Example:

Unit 2 31
Chang, Huang, Li, Lin, Liu

Depth-First-Search (DFS)
․Time complexity:
 Adjacency list: O(V+E)
 Adjacency matrix: O(V2)

Unit 2 32
Chang, Huang, Li, Lin, Liu
 Breadth-First-Search (BFS)
․Time complexity:
 Push each vertex into queue once
 Adjacency list: O(V+E)
 Adjacency matrix: O(V2)

Unit 2 33
Chang, Huang, Li, Lin, Liu

Topological Sort
․Handle precedence constraints
․Time complexity: O(V+E)
 Use a queue/stack to keep “free” vertices
 1--4: parse graph once and push free vertices into queue/stack
 5--9: pop free vertices from queue/stack one at a time
 Output it and remove out-going edges
 Push new free vertices into queue/stack

Unit 2 34
Chang, Huang, Li, Lin, Liu
 Example: Topological Sorting

Unit 2 35
Chang, Huang, Li, Lin, Liu

Application: Maze Routing

․Problem: find a connection from A to B

․Quite useful for two pin nets, rip-up-and-reroute, etc.

Unit 2 36
Chang, Huang, Li, Lin, Liu
 BFS: Lee’s Algorithm

Unit 2 37
Chang, Huang, Li, Lin, Liu

DFS: Line Search Algorithms

․Reduce memory requirement
 Send out a line and use it to guide the search
 No guarantees on being able to find a route

Unit 2 38
Chang, Huang, Li, Lin, Liu
 Classifications of Popular EDA Algorithms

Unit 2 39
Chang, Huang, Li, Lin, Liu

Framework Evolution
․Billions of transistors may be fabricated in a single chip
for nanometer technology
․Need frameworks for very large-scale designs
․Framework evolution for EDA tools:
 Flat  Hierarchical  Multilevel

Unit 2 40
Chang, Huang, Li, Lin, Liu
 Flat Framework
․Process the circuit components in the whole chip
․Limitation: Good for small-scale designs, but hard to
handle larger problems directly

Unit 2 41
Chang, Huang, Li, Lin, Liu

Hierarchical Framework
․The hierarchical approach recursively divides a circuit
region into a set of subregions and solve those
subproblems independently
․Good for scalability for large-scale design, but lack the
global information for the interaction among subregions

Unit 2 42
Chang, Huang, Li, Lin, Liu
 Multilevel Framework
․Bottom-up coarsening (clustering)
 Iteratively groups a set of circuit components & collects global
information
․Top-down uncoarsening (declustering)
 Iteratively ungroups clustered components & refines the
solution
․Good for scalability and quality trade-off

Unit 2 43
Chang, Huang, Li, Lin, Liu

CAD Related Conferences/Journals

․ Important Conferences:
 ACM/IEEE Design Automation Conference (DAC)
 IEEE/ACM Int'l Conference on Computer-Aided Design (ICCAD)
 IEEE Int’l Test Conference (ITC)
 ACM Int'l Symposium on Physical Design (ISPD)
 ACM/IEEE Asia and South Pacific Design Automation Conf. (ASP-DAC)
 ACM/IEEE Design, Automation, and Test in Europe (DATE)
 IEEE Int'l Conference on Computer Design (ICCD)
 IEEE Custom Integrated Circuits Conference (CICC)
 IEEE Int'l Symposium on Circuits and Systems (ISCAS)
 Others: VLSI Design/CAD Symposium/Taiwan
․ Important Journals:
 IEEE Transactions on Computer-Aided Design (TCAD)
 ACM Transactions on Design Automation of Electronic Systems
(TODAES)
 IEEE Transactions on VLSI Systems (TVLSI)
 IEEE Transactions on Computers (TC)
 IEE Proceedings – Circuits, Devices and Systems
 IEE Proceedings – Digital Systems
 INTEGRATION: The VLSI Journal
Unit 2 44
Chang, Huang, Li, Lin, Liu