Analysis of Algorithm

Text book
Introduction to Algorithms, 2009, Third Edition
by
Thomas H. Cormen, Charles E. Liserson, Ronald L. Rivest, Clif Ord
Stein

2
References
Chapter 1
Introduction to Algorithms, 2009, Third Edition
by
Thomas H. Cormen
Charles E. Liserson
Ronald L. Rivest
Clifford Stein
http://www.mathsolutions.com/documents/978-1-935099-01-
7_CH4.pdf
https://en.wikipedia.org/
These slides contain material from the above resources.
3
Algorithms
 An algorithm is any well-defined computational procedure
that takes some value, or set of values, as input and produces
some value, or set of values, as output.

 An algorithm is a sequence of computational steps that

transform the input into the output.

 An algorithm can also be view as a tool for solving a well-

specified computational problem.

4
Computational procedure
 A computational procedure is a series of steps or actions
what we use when operating on numbers.

For example, when we add, subtract, multiply, and divide we

use some sort of procedure. Some procedures are algorithms.

5
Sorting
 Sorting is a fundamental operation in computer science

 Input: A sequence of n numbers 〈a1, a2, …, an〉.

 Output: A permutation (reordering) a1’, a2’ , . . . , an’ of the

input sequence such that a1’ ≤ a2’ ≤ ・ ・ ・ ≤ an’

 The number that we wish to sort are also known as the keys.

6
Sorting
Example:

 Input: 〈22, 1, 0, 10, 23, 12 〉

 Output: 〈0, 1, 10, 12, 22, 23 〉

 Input sequence is called an instance of the sorting problem

7
What kinds of problems are solved by
algorithms?
Practical applications of algorithms are ubiquitous, for example:
 Human Genome Project.

 The Internet enables people all around the world to quickly

access retrieve large amounts of information.

 E-commerce (it depends on the privacy of personal

information such as credit card numbers, passwords, and
bank statements).

 Manufacturing and other commercial enterprises often need

to allocate scarce resources in the most beneficial way.

8
Which is the best algorithm?
Which algorithm is best for a given application depends on:

 The number of items to be sorted.

 The extent to which the items are already somewhat sorted.

 Possible restrictions on the item values.

 The architecture of the computer.

 The kind of storage devices to be used: main memory, disks,

or even tapes.
9
Correct algorithm
 An algorithm is said to be correct if, for every input instance,
it halts with the correct output.

 An incorrect algorithm might not halt at all on some input

instances, or it might halt with an incorrect answer.

 Incorrect algorithms can sometimes be useful, if we can

control their error rate.

10
Algorithms vs. Data Structure
 Algorithm: method for solving a problem.

 Data structure: method to store information.

11
Why study algorithms?
 “ For me, great algorithms are the poetry of computation. Just
like verse, they can be terse, allusive, dense, and even
mysterious. But once unlocked, they cast a brilliant new light
on some aspect of computing”.
— Francis Sullivan

 “ An algorithm must be seen to be believed”.

— Donald Knuth

12
Why study algorithms?

13
Insertion sort vs. Merge sort [1]
 Insertion sort, takes time roughly equal to c1n2 or c1n.n to
sort n items, where c1 is a constant that does not depend on
n.

 Merge sort, takes time roughly equal to c2n lgn or c2n.lgn,

where lgn stands for log2n and c2 is another constant that
also does not depend on n.

 Insertion sort typically has a smaller constant factor than

merge sort, so that c1 < c2.

 Insertion sort has a factor of n in its running time, merge sort

has a factor of lgn, which is much smaller.

14
Insertion sort vs. Merge sort [2]
For example:
When n = 1000
log1000 = 6.9078 (approximately)

When n = 1,000,000 (one million or 106)

log106 = 13.8155 (approximately)

15
Analysis of algorithms
The theoretical study of computer-program performance and
resource usage.

16
Million vs. Billion
 A million (10 lacs or 10 lakhs) is 106, or 1,000,000.

 A crore equal to ten million (10,000,000; in scientific notation:

107).

 A billion (100 crore or 1 Arab) is one thousand million, or

1,000,000,000 (109).

17
Example: Insertion sort vs. Merge sort
[1]
 A faster computer, named computer A running insertion sort.

 A slower computer, named computer B running merge sort.

 They each must sort an array of 1 million (i.e., 106) numbers.

 Suppose that computer A executes 1 billion (i.e., 109)

instructions per second.

 Also suppose computer B executes only 10 million (i.e., 107)

instructions per second.
18
Example: Insertion sort vs. Merge sort
[2]
 It means computer A is 100 times faster than computer B in
raw computing power.

 Suppose that the world’s craftiest programmer codes

insertion sort in machine language for computer A, and the
resulting code requires 2n2 instructions to sort n numbers.

 Suppose further that just an average programmer

implements merge sort, using a high-level language with an
inefficient compiler, with the resulting code taking 50nlgn
instructions

19
Example: Insertion sort vs. Merge sort
[3]
To sort 1 million numbers, computer A take
𝟐𝐧𝟐 2. 106 2 instuctions
= = 2000 seconds
1 billion 𝐢𝐧𝐬𝐭𝐮𝐜𝐭𝐢𝐨𝐧𝐬/𝐬𝐞𝐜𝐨𝐧𝐝𝐬 109 instuctions/seconds

To sort 1 million numbers, computer B take

𝟓𝟎.𝐧.𝐥𝐠𝐧 50.106 lg106 instuctions
= ≈ 100 seconds
10 million 𝐢𝐧𝐬𝐭𝐮𝐜𝐭𝐢𝐨𝐧𝐬/𝐬𝐞𝐜𝐨𝐧𝐝𝐬 107 instuctions/seconds

⇒ computer B runs 20 times faster than computer A.

20
Suggested readings
 1.1 Algorithms
 1.2 Algorithms as a technology

21