CSE408

Fundamentals of Algorithms

Lecture #1

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
What is an algorithm?

An algorithm is a sequence of unambiguous instructions

for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of time.

problem

algorithm

input “computer” output

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Notion of algorithm and problem

problem

algorithm

input “computer” output

(or instance)

algorithmic solution
(different from a conventional solution)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Some Well-known Computational Problems

 Sorting
 Searching
 Shortest paths in a graph
 Minimum spanning tree
 Primality testing
 Traveling salesman problem
 Knapsack problem
 Chess
 Towers of Hanoi
 Program termination

Some of these problems don’t have efficient algorithms,

or algorithms at all!
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Basic Issues Related to Algorithms

 How to design algorithms

 How to express algorithms

 Proving correctness

 Efficiency (or complexity) analysis

 Optimality

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Algorithm design strategies

 Brute force  Greedy approach

 Divide and conquer  Dynamic programming

 Decrease and conquer

 Backtracking and branch-and-bound
 Transform and conquer
 Space and time tradeoffs

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Analysis of Algorithms

 How good is the algorithm?

• Correctness
• Time efficiency
• Space efficiency

 Does there exist a better algorithm?

• Lower bounds
• Optimality

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
What is an algorithm?

 Recipe, process, method, technique, procedure,

routine,… with the following requirements:
1. Finiteness
 terminates after a finite number of steps
2. Definiteness
 rigorously and unambiguously specified
3. Clearly specified input
 valid inputs are clearly specified
4. Clearly specified/expected output
 can be proved to produce the correct output given a valid input
5. Effectiveness
 steps are sufficiently simple and basic

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Why study algorithms?

 Theoretical importance

• the core of computer science

 Practical importance

• A practitioner’s toolkit of known algorithms

• Framework for designing and analyzing algorithms for new problems

Example: Google’s PageRank Technology

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Euclid’s Algorithm

Problem: Find gcd(m,n), the greatest common divisor of two

nonnegative, not both zero integers m and n

Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?

Euclid’s algorithm is based on repeated application of equality

gcd(m,n) = gcd(n, m mod n)
until the second number becomes 0, which makes the problem
trivial.

Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Two descriptions of Euclid’s algorithm

Step 1 If n = 0, return m and stop; otherwise go to Step 2

Step 2 Divide m by n and assign the value of the remainder to r
Step 3 Assign the value of n to m and the value of r to n. Go to
Step 1.

while n ≠ 0 do
r ← m mod n
m← n
n←r
return m

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Other methods for gcd(m,n) [cont.]

Middle-school procedure
Step 1 Find the prime factorization of m
Step 2 Find the prime factorization of n
Step 3 Find all the common prime factors
Step 4 Compute the product of all the common prime factors
and return it as gcd(m,n)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Fundamentals of Algorithmic Problem Solving

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Two main issues related to algorithms

 How to design algorithms

 How to analyze algorithm efficiency

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Algorithm design techniques/strategies

 Brute force  Greedy approach

 Divide and conquer  Dynamic programming

 Decrease and conquer  Iterative improvement

 Transform and conquer  Backtracking

 Space and time tradeoffs  Branch and bound

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Analysis of algorithms

 How good is the algorithm?

• time efficiency
• space efficiency
• Correctness

 Does there exist a better algorithm?

• lower bounds
• optimality

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Important problem types

 sorting

 searching

 string processing

 graph problems

 combinatorial problems

 geometric problems
 numerical problems

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Sorting (I)

 Rearrange the items of a given list in ascending

order.
• Input: A sequence of n numbers <a1, a2, …, an>
• Output: A reordering <a´1, a´2, …, a´n> of the input
sequence such that a´1≤ a´2 ≤ … ≤ a´n.
 Why sorting?
• Help searching
• Algorithms often use sorting as a key subroutine.
 Sorting key
• A specially chosen piece of information used to guide
sorting. E.g., sort student records by names.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Sorting (II)
 Examples of sorting algorithms
• Selection sort
• Bubble sort
• Insertion sort
• Merge sort
• Heap sort …
 Evaluate sorting algorithm complexity: the number of key
comparisons.
 Two properties
• Stability: A sorting algorithm is called stable if it preserves the relative
order of any two equal elements in its input.
• In place : A sorting algorithm is in place if it does not require extra
memory, except, possibly for a few memory units.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Selection Sort

Algorithm SelectionSort(A[0..n-1])
//The algorithm sorts a given array by selection sort
//Input: An array A[0..n-1] of orderable elements
//Output: Array A[0..n-1] sorted in ascending order
for i  0 to n – 2 do
min  i
for j  i + 1 to n – 1 do
if A[j] < A[min]
min  j
swap A[i] and A[min]

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Searching
 Find a given value, called a search key, in a given set.
 Examples of searching algorithms
• Sequential search
• Binary search …
Input: sorted array a_i < … < a_j and key x;
m (i+j)/2;
while i < j and x != a_m do
if x < a_m then j  m-1
else i  m+1;
if x = a_m then output a_m;

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
String Processing

 A string is a sequence of characters from an alphabet.

 Text strings: letters, numbers, and special characters.
 String matching: searching for a given word/pattern in a text.

Examples:
(i) searching for a word or phrase on WWW or in a
Word document
(ii) searching for a short read in the reference genomic
sequence

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Graph problems

 A graph can be thought of as a collection of points called

vertices, some of which are connected by line segments called
edges.
 Graph-traversal algorithms (how can one reach all the points
in a network?)
 Shortest-path algorithms (what is the best route between two
cities?)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Combinatorial Problems

 The traveling salesman problem (TSP) is the problem of

finding the shortest tour through n cities that visits every city
exactly once.
 The graph-coloring problem seeks to assign the smallest
number of colors to the vertices of a graph so that no two
adjacent vertices are the same color.
 The traveling salesman problem and the graph coloring
problem are examples of combinatorial problems.
 These are problems that ask, explicitly or implicitly, to find a
combinatorial object—such as a permutation, a combination, or
a subset—that satisfies certain constraints

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Geometric Problems

 Geometric algorithms deal with geometric objects such as

points, lines, and polygons.
 The closest-pair problem is self-explanatory: given n points in
the plane, find the closest pair among them.
 The convex-hull problem asks to find the smallest convex
polygon that would include all the points of a given set.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Numerical problems

 Numerical problems, another large special area of applications,

are problems that involve mathematical objects of continuous
nature: solving equations and systems of equations, computing
definite integrals, evaluating functions, and so on.

 The majority of such mathematical problems can be solved

only approximately.

 Another principal difficulty stems from the fact that such

problems typically require manipulating real numbers, which
can be represented in a computer only approximately.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1