Data Structures and Algorithms

Analysis of Algorithms 1
 Course contents
• Analysis of Algorithms: worst case time and
space complexity
• Recursive & iterative algorithms
• Sorting algorithms: merge sort, quick sort, …..
• Searching algorithms: binary and linear;
• Data Structures: stack, queue, linked list,
tree,, heap, and hash; 
• Additional topics: data compression, string
matching

Analysis of Algorithms 2
Why Data Structure and Algorithms?

• You will be able to evaluate the quality of a

program (Analysis of Algorithms: Running time and
memory space )
• You will be able to write fast programs
• You will be able to solve new problems
• learn how to solve problems
• dealing with abstractions
• be more precise

Analysis of Algorithms 3
Chapter 1

Fundamental ideas
of
data structure
and
algorithm
 Programs and programming
• What is a program?
– A set of instructions working with data
designed to accomplish a specific task
• What is programming
– The art and skill of writing programs

5
 Introduction to Programming
• Programming is to solve problems using computers
– How to do it at all ?
– How to do it robustly ?
– How to do it effectively ?
• Programming consists of two steps:
– Algorithmic design (the architects)
– Coding (the construction workers)
• Programming requires:
– A programming language (C/C++/C#) to express your ideas
– A set of tools to design, edit, and debug your code
– A compiler to translate your programs into machine code
– A machine to run the executable code

6
 Good Programs
• There are a number of facets to good
programs: they must
– run correctly
– run efficiently
– be easy to read and understand
– be easy to debug and
– be easy to modify

• better running times will generally be

obtained from use of the most appropriate
data structures and algorithms

7
 Definition
Data Structures: A systematic way of organizing and
accessing data.
--No single data structure works well for ALL purposes.

Input Algorithm Output

An algorithm is a step-by-step procedure for

solving a problem in a finite amount of time.
 Algorithm Descriptions
• Nature languages: Chinese, English, etc.
• Pseudo-code: codes very close to computer languages, e.g.,
C programming language.
• Programs: C programs, C++ programs, Java programs.

Goal:
• Allow a well-trained programmer to be able to implement.
• Allow an expert to be able to analyze the running time.

Analysis of Algorithms 9
 An Example of an Algorithm

Algorithm sorting(X, n)
Input array X of n integers
Output array X sorted in a non-decreasing order

for i  0 to n  1 do
for j  i+1 to n do
if (X[i]>X[j]) then
{ s=X[i];
X[i]=X[j];
X[j]=s;
}
return X

Analysis of Algorithms 10
 Analysis of Algorithms

• Estimate the running time

• Estimate the memory space required.

Depends on the input size.

Analysis of Algorithms 11
 Running Time
• Most algorithms transform best case
average case
input objects into output worst case
objects. 120

• The running time of an 100

algorithm typically grows with

Running Time
80
the input size.
60
• Average case time is often
40
difficult to determine.
• We focus on the worst case 20

running time. 0
1000 2000 3000 4000
– Easier to analyze
Input Size
– Crucial to applications such as
games, finance and robotics

Analysis of Algorithms 12
 Experimental Studies
• Write a program 9000

implementing the algorithm 8000

• Run the program with inputs 7000

of varying size and 6000

Time (ms)
composition 5000
• Use a method like 4000
System.currentTimeMillis() to 3000
get an accurate measure of
2000
the actual running time
1000
• Plot the results
0
0 50 100
Input Size

Analysis of Algorithms 13
 Limitations of Experiments

• It is necessary to implement the algorithm,

which may be difficult
• Results may not be indicative of the running
time on other inputs not included in the
experiment.
• In order to compare two algorithms, the
same hardware and software environments
must be used

Analysis of Algorithms 14
 Theoretical Analysis
• Uses a high-level description of the
algorithm instead of an implementation
• Characterizes running time as a function of
the input size, n.
• Takes into account all possible inputs
• Allows us to evaluate the speed of an
algorithm independent of the
hardware/software environment
Analysis of Algorithms 15
 Pseudocode
Example: find max element
• High-level description of of an array
an algorithm
• More structured than Algorithm arrayMax(A, n)
English prose Input array A of n integers
Output maximum element of A
• Less detailed than a
program currentMax  A[0]
for i  1 to n  1 do
• Preferred notation for if A[i]  currentMax then
describing algorithms currentMax  A[i]
• Hides program design return currentMax
issues

Analysis of Algorithms 16
 Pseudocode Details

• Control flow • Expressions

– if … then … [else …] ¬ Assignment
– while … do … (like  in Java)
– repeat … until … = Equality testing
– for … do … (like  in Java)
– Indentation replaces braces n2 Superscripts and other
mathematical formatting
• Method declaration allowed
Algorithm method (arg [, arg…])
Input …
Output …

Analysis of Algorithms 17
 Primitive Operations
• Basic computations performed
• Examples:
by an algorithm – Assigning a value to
• Identifiable in pseudocode a variable
• Largely independent from the – Indexing into an
array
programming language – Calling a method
• Exact definition not important – Returning from a
• Assumed to take a constant method
amount of time in the RAM
model

Analysis of Algorithms 18
 Counting Primitive Operations
• By inspecting the pseudocode, we can determine the maximum
number of primitive operations executed by an algorithm, as a
function of the input size

Algorithm arrayMax(A, n)
# operations
currentMax  A[0] 2
for (i =1; i<n; i++) 2n
(i=1 once, i<n n times, i++ (n-1)
times)
if A[i]  currentMax then 2(n  1)
currentMax  A[i] 2(n  1)
return currentMax 1
Total 6n 1
Analysis of Algorithms 19
 Estimating Running Time
• Algorithm arrayMax executes 6n  1 primitive
operations in the worst case.
Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive operation
• Let T(n) be worst-case time of arrayMax.
Then
a (6n  1)  T(n)  b(6n  1)
• Hence, the running time T(n) is bounded by
Analysis of Algorithms 20
two linear functions
 Growth Rate of Running Time

• Changing the hardware/ software

environment
– Affects T(n) by a constant factor, but
– Does not alter the growth rate of T(n)
• The linear growth rate of the running time
T(n) is an intrinsic property of algorithm
arrayMax

Analysis of Algorithms 21
 n logn n nlogn n2 n3 2n

4 2 4 8 16 64 16
8 3 8 24 64 512 256

16 4 16 64 256 4,096 65,536

32 5 32 160 1,024 32,768 4,294,967,296

64 6 64 384 4,094 262,144 1.84 * 1019

128 7 128 896 16,384 2,097,152 3.40 * 1038

256 8 256 2,048 65,536 16,777,216 1.15 * 1077

512 9 512 4,608 262,144 134,217,728 1.34 * 10154

1024 10 1,024 10,240 1,048,576 1,073,741,824 1.79 * 10308

The Growth Rate of the Six Popular functions

Analysis of Algorithms 22