Analysis of Algorithm

Lecture 1

Dr Munawwar Iqbal (Assistant Professor)

Institute of Informaiton Technology
[email protected]
 Acknowledgement
 This lecture note has been summarized from
lecture note on Data Structure and Algorithm,
Design and Analysis of Computer Algorithm all
over the world.
 However, I’d like to thank all professors who
create such a good work on those lecture notes.
Without those lectures, this slide can’t be finished.

Analysis of Algorithm Sunday, July 14, 2024 2

 Course Information
 Class Timings

According to Timetable
 Grading policy

Midterm: 25; Quizes: 15; Assignments+Presentatin: 10 Final: 50

Analysis of Algorithm Sunday, July 14, 2024 3

 More Information

Textbook

Introduction to Algorithms 2nd ,Cormen,
Leiserson, Rivest and Stein, The MIT Press, 2004.
o
Data Structures and Algorithm Analysis

Others

Introduction to Design & Analysis Computer Algorithm 3rd,
Sara Baase, Allen Van Gelder, Adison-Wesley, 2000.

Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice
Hall, 2004.

Introduction to The Design and Analysis of Algorithms 2 nd
Edition, Anany Levitin, Adison-Wesley, 2007.

Analysis of Algorithm Sunday, July 14, 2024 4

 More Information

Others
1.R. Neapolitan, Foundations of Algorithms, 5/e, Jones and Bartlett Learning,
2014.
2.Robert Sedgewick, Kevin Wayne, Algorithms, 4/e, Addison-Wesley Professional,

2011.
M H Alsuwaiyel, Algorithms: Design Techniques and Analysis, World
Scientific Pub Co Inc, Year: 2021.

Analysis of Algorithm Sunday, July 14, 2024 5

 Course Objectives
 This course introduces students to the analysis
and design of computer algorithms. Upon
completion of this course, students will be able to
do the following:

Analyze the asymptotic performance of algorithms.

Demonstrate a familiarity with major algorithms and data
structures.

Apply important algorithmic design paradigms and
methods of analysis.

Synthesize efficient algorithms in common engineering
design situations.

Analysis of Algorithm Sunday, July 14, 2024 6

 Objective of course
In this course we will cover the following topics:
 Understand foundations of algorithms & design and analysis various variants
algorithms
Accuracy
Efficiency
Comparing efficiencies

 Make use of skills to understand mathematical notations in algorithms and

their simple mathematical proofs

 Gain familiarity with a number of classical problems that occur frequently in

real-world applications

Analysis of Algorithm Sunday, July 14, 2024 7

 COURSE
COURSEDETAILS
DETAILS

 Preliminaries : different types of algorithms, analyzing methodologies,

notations, proof techniques and limits.
 Asymptotic Notation : different notations and their examples, standard
notations and their common functions.
 Analysis of Algorithms : analyzing control structures, using barometer
instruction, amortization, and different examples for analysis and solving
recurrences.
 Structures: use of arrays, stacks, queues, records, pointers, lists, graphs,
trees, hash tables, heaps and binomial heaps.
 Searching/Sorting Algorithms : Various searching and sorting algorithms and
their comparisons.

Analysis of Algorithm Sunday, July 14, 2024 8

Advanced Algo Analysis
COURSE
COURSEDETAILS
DETAILS

Analysis of Algorithm Sunday, July 14, 2024 9

 What is Algorithm?

Algorithm

is any well-defined computational procedure that
takes some value, or set of values, as input and
produces some value, or set of values, as
output.

is thus a sequence of computational steps that
transform the input into the output.

is a tool for solving a well - specified
computational problem.

Any special method of solving a certain kind of p
roblem (Webster Dictionary)
Analysis of Algorithm Sunday, July 14, 2024 10
 Algorithm

It is the science that lets designer's study

and evaluate the effect of algorithms
based on various factors so that the best
algorithm is selected to meet a particular
task in given circumstances. It is also the
science that tells how to design a new
algorithm for a particular job.

Analysis of Algorithm Sunday, July 14, 2024 11

 What is a program?

A program is the expression of an algorithm i

n a programming language
a set of instructions which the computer will f
ollow to solve a problem

Analysis of Algorithm Sunday, July 14, 2024 12

 Introduction

Why need algorithm analysis ?


writing a working program is not good enough

The program may be inefficient!

If the program is run on a large data set, then
the running time becomes an issue

Analysis of Algorithm Sunday, July 14, 2024 13

 Types of Algorithms

Analysis of Algorithm Sunday, July 14, 2024 14

 PROBABILISITIC ALGORITHM

•In this algorithm, chosen values are used

in such a way that the probability of chosen
each value is known and controlled.

e.g. Randomize Quick Sort

Analysis of Algorithm Sunday, July 14, 2024 15

 HEURISTIC ALGORITHM

This type of algorithm is based largely on

optimism and often with minimal theoretical
support. Here error can not be controlled but
may be estimated how large it is.

Analysis of Algorithm Sunday, July 14, 2024 16

 APPROXIMATE ALGORITHM

In this algorithm, answer is obtained that is

as précised as required in decimal
notation. In other words it specifies the
error we are willing to accept.

For example, two figures accuracy or 8

figures or whatever is required.

Analysis of Algorithm Sunday, July 14, 2024 17

 Where We're Going
 Learn general approaches to algorithm design

Divide and conquer

Greedy method

Dynamic Programming

Basic Search and Traversal Technique

Graph Theory

Linear Programming

Approximation Algorithm

NP Problem

Analysis of Algorithm Sunday, July 14, 2024 18

 Some Application

Study problems these techniques can be ap

plied to

sorting

data retrieval

network routing

Games

etc

Analysis of Algorithm Sunday, July 14, 2024 19

 What do we analyze about them?

 Correctness
Does the input/output relation match algorithm r


equirement?
 Amount of work done (aka complexity)

Basic operations to do task finite amount of time
 Amount of space used

Memory used

Analysis of Algorithm Sunday, July 14, 2024 20

 Strengthening the Informal Definition

Important Features:

Finiteness.[algo should end in finite amount of
steps]

Definiteness.[each instruction should be clear]

Input.[valid input clearly specified ]

Output.[single/multiple valid output]

Effectiveness.[ steps are sufficiently simple and
basic]

Analysis of Algorithm Sunday, July 14, 2024 21

 RAM model

 has one processor

 executes one instructi
on at a time
 each instruction takes
"unit time“
 has fixed-size operan
ds, and
 has fixed size storage
(RAM and disk).

Analysis of Algorithm Sunday, July 14, 2024 22

 Few Classical Examples

Classical Multiplication Algorithms

English

American

A la russe

Divide and Conquer

Analysis of Algorithm Sunday, July 14, 2024 23

 Classic Multiplication

Analysis of Algorithm Sunday, July 14, 2024 24

 Multiplication

Analysis of Algorithm Sunday, July 14, 2024 25

 Multiplication

Analysis of Algorithm Sunday, July 14, 2024 26

 The Selection Problem

Which algorithm is better?

Analysis of Algorithm Sunday, July 14, 2024 27

 Which algorithm is better?

The algorithms are

correct, but which is
the best?
 Measure the running time
(number of operations
needed).
 Measure the amount of
memory used.
 Note that the running
time of the algorithms
increase as the size of
the input increases.

Analysis of Algorithm Sunday, July 14, 2024 28

 Running Time
 Most algorithms transform best case
input objects into output average case
worst case
objects. 120
 The running time of an 100
algorithm typically grows

Running Time
80
with the input size.
 Average case time is often 60

difficult to determine. 40

 We focus on the worst case 20

running time. 0
1000 2000 3000 4000

Easier to analyze
Input Size

Crucial to applications such as
games, finance and robotics

Analysis of Algorithm Sunday, July 14, 2024 29

 Experimental Studies
 Write a program 9000

implementing the algorithm 8000

 Run the program with 7000

inputs of varying size and 6000

Time (ms)
composition 5000
 Use a method like 4000
System.currentTimeMillis() to 3000
get an accurate measure 2000
of the actual running time
1000
 Plot the results
0
0 50 100
Input Size

Analysis of Algorithm Sunday, July 14, 2024 30

 Limitations of Experiments
It is necessary to implement the
algorithm, which may be difficult
Results may not be indicative of the
running time on other inputs not included
in the experiment.
In order to compare two algorithms, the
same hardware and software
environments must be used

Analysis of Algorithm Sunday, July 14, 2024 31

 Theoretical Analysis
Uses a high-level description of the
algorithm instead of an implementation
Characterizes running time as a
function of the input size, n.
Takes into account all possible inputs
Allows us to evaluate the speed of an
algorithm independent of the
hardware/software environment

Analysis of Algorithm Sunday, July 14, 2024 32

 Counting Primitive Operations (§3.4)
 By inspecting the pseudo code, we can determine the
maximum number of primitive/basic operations executed by
an algorithm, as a function of the input size

Algorithm arrayMax(A, n)
currentMax  A[0]
for (i =1; i<n; i++) n
(i=1 once, i<n n times, i++ (n-1)
times:post increment)
if A[i]  currentMax then (n  1)
currentMax  A[i] (n  1)
return currentMax
Total 3n

Analysis of Algorithm Sunday, July 14, 2024 33

 Estimating Running Time

 Algorithm arrayMax executes 6n  1 primitive

operations in the worst case.
Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive operation
 Let T(n) be worst-case time of arrayMax. Then
a (6n  1)  T(n)  b(6n  1 )
 Hence, the running time T(n) is bounded by two
linear functions

Analysis of Algorithm Sunday, July 14, 2024 34

 Growth Rate of Running Time
Changing the hardware/ software
environment

Affects T(n) by a constant factor, but

Does not alter the growth rate of T(n)
The linear growth rate of the running
time T(n) is an intrinsic/basic property
of algorithm arrayMax

Analysis of Algorithm Sunday, July 14, 2024 35

 Function of Growth rate

Analysis of Algorithm Sunday, July 14, 2024 36

  FURTHER DISCUSSION

Next Class …………….

Analysis of Algorithm Sunday, July 14, 2024 37