Muhammad Nawaz Shareef University of

Agriculture, Multan

CS-408
Design and Analysis of Algorithm

Instructor : Dr. Abdul Razzaq

Email: [email protected]

Advanced Algorithms Analysis

Lecture 01
Welcome to
CS-408
Introduction to Design and
Analysis of Algorithm

Please be on time
(better: before time)
 Learning Goals for Today
• Course objectives and outcomes

To become familiar with the concept of algorithms:

– What they are?

– What is their use?
– What do they consist of?
– What are the techniques used for representing them?
• In particular, we will look at the building blocks that are used
in all algorithms
• We will also discuss the pseudo code and flowcharts for
particular problems
• In addition, we will outline the pros and cons of those two
 Course objective
Major objective of this course is:
• Explain what is meant by “best”, “expected”, and “worst” case behavior of an
algorithm
• Determine informally the time and space complexity of simple algorithms
• Use big O, Omega, Theta notation formally to give asymptotic upper bounds on
time and space complexity of algorithms
• Use of the strategies (brute-force, greedy, divide-and conquer, and dynamic
programming) to solve an appropriate problem
• Solve problems using graph algorithms, including single source and all-pairs
shortest paths, and at least one minimum spanning tree algorithm
 Course Contents
The following major topics will be covered in this
semester:
Introduction to algorithms
• Role of algorithms in computing,
• Analysis on nature of input and size of input
• Asymptotic notations; Big-O, Big Ω, Big Θ, little-o, little-ω,
• Sorting Algorithm analysis, loop
• Recursion and recurrence relations;
• Algorithm Design Techniques,
• Brute Force Approach, Divide-and-conquer approach;
• Merge, Quick Sort, Greedy approach; Dynamic programming;
• Elements of Dynamic Programming,
• Graph algorithms, shortest paths, sparse graphs , String matching
 Solving Problems (1)
When faced with a problem:
1. We first clearly define the problem
2. Think of possible solutions
3. Select the one that we think is the best
under the prevailing circumstances
4. And then apply that solution
5. If the solution woks as desired, fine; else
we go back to step 2
6
 Solving Problems (2)
• It is quite common to first solve a problem
for a particular case
• Then for another
• And, possibly another
• And watch for patterns and trends that
emerge
• And to use the knowledge form those
patterns and trends in coming up with a
general solution
7
 Solving Problems (3)
• It helps if we have experienced that problem or
similar ones before

• Generally, there are many ways of solving a

given problem; the best problem-solvers
come-up with the most appropriate solution
more often than not!

• The process that can be used to solve a

problem is termed as the “algorithm” 8
 al.go.rithm
steps
sequence
Sequence of steps
that can be taken to solve a given problem

9
 Examples
• Addition
• Conversion from decimal to binary
• The process of boiling an egg
• The process of mailing a letter
• Sorting
• Searching

10
 Algorithm Applications
A large variety of problems in computer science, mathematics and other disciplines
depend on the use of algorithms for their solutions. The broad categories of
applications types are:
• •Searching Algorithms(Linear and non-linear)
• •Sorting Algorithms(Elementary and Advanced)
• •Strings Processing( Pattern matching, Parsing, Compression, Cryptography)
• •Optimization Algorithms(Shortest routes, minimum cost)
• •Geometric Algorithms( Triangulation, Convex Hull)
• •Image Processing( Compression, Matching, Conversion)
• •Data Mining Algorithms( Clustering, Cleansing, Rules mining)
• •Mathematical Algorithms(Random number generator, matrix operations, FFT, etc)
• Problem Domains

11
 Algorithm (Better Definition)
1st Definition:
Sequence of steps that can be taken
to solve a problem

Better Definition:
A precise sequence of a limited
number of unambiguous, executable
steps that terminates in the form of a
solution

12
 What is Algorithm?
• A computer algorithm is a detailed step-by-step method
for solving a problem by using a computer.

• An Algorithm is well defined computational procedure that

takes some value, or set of values, as input and produces
some value, or set of values as output.
• More generally, an Algorithm is any well defined
computational procedure that takes collection of elements
as input and produces a collection of elements as output.

Input output
Algorithm
 Three Requirements:
1. Sequence is:
a. Precise
b. Consists of a limited number of steps

2. Each step is:

a. Unambiguous
b. Executable

3. The sequence of steps terminates in

the form of a solution 14
 Why Algorithms are Useful?
• Once we find an algorithm for solving a
problem, we do not need to re-discover it the
next time we are faced with that problem

• Once an algorithm is known, the task of

solving the problem reduces to following
(almost blindly and without thinking) the
instructions precisely

• All the knowledge required for solving the

problem is present in the algorithm 15
 Analysis of Algorithms
• Analysis in the context of algorithms is concerned with
predicting the resources that requires:
– Computational time
– Memory
– Logic functions

• However, Time – generally measured in terms of the

number of steps required to execute an algorithm - is
the resource of most interest

• By analyzing several candidate algorithms, the most

efficient one(s) can be identified
16
 Important Designing Techniques
• Brute Force
– Straightforward, naive approach
– Mostly expensive
• Divide-and-Conquer
– Divide into smaller sub-problems
• Iterative Improvement
– Improve one change at a time
• Decrease-and-Conquer
– Decrease instance size
• Transform-and-Conquer
– Modify problem first and then solve it
• Space and Time Tradeoffs
– Use more space now to save time later
 Heuristic
Not guaranteed to be optimal or perfect, but sufficient for the immediate goals .
• A procedure that usually, but not always, works
or that gives nearly the right answer

• Some problems, such as the traveling

salesman problem, take far too long to compute
an exact, optimal solution. A few good
heuristics have been devised that are fast and
find a near-optimal solution more often than not
• Is a heuristic, an algorithm? Yes? No? Why?
18
The Traveling Salesman Problem
A salesman needs A possible sequence for n = 6
to visit each of the n
cities one after the 3
other and wants to 5
finish the trip where 1
it was started
2

Determine the 4
sequence of cities
such that the
traveling distance is 6
minimized 19
 The Brute Force Strategy
• A strategy in which all possible combinations are
examined and the best among them is selected

• What is the problem with this approach?

A: Doesn’t scale well with the size of the
problem

• How many possible city sequences for n=6? For

n=60? For n=600?
20
 Algorithm Representation
• We have said enough about algorithms – their
definition, their types, etc.
• But, how do we actually represent them?
• Generally, SW developers represent them in
one of three forms:
– Pseudo code
– Flowcharts
– Actual code

21
 Pseudo Code
• Language that is typically used for writing
algorithms

• Similar to a programming language, but not as

rigid

• The method of expression most suitable for a

given situation is used:
– At times, plain English
– At others, a programming language like syntax
22
 Flowchart
• A graphical representation of a process (e.g. an
algorithm), in which graphic objects are used to
indicate the steps & decisions that are taken as
the process moves along from start to finish

• Individual steps are represented by boxes and

other shapes on the flowchart, with arrows
between those shapes indicating the order in
which the steps are taken

23
 Start or stop

Process

Input or output
Flowchart
Elements Decision

Flow line
Connector

Off-page connector 24
 Sequences
A sequence of instructions that are executed
in the precise order they are written in:

statement block 1
statement block 1
statement block 2 statement block 2
statement block 3
statement block 3

25
 Conditionals
Select between alternate courses of action
depending upon the evaluation of a condition

If ( condition = true )
statement block 1 True False
Else condition
statement block 2
End if statement statement
block 1 block 2

26
 Loops
Loop through a set of statements as long as a
condition is true

Loop while ( condition = true )

statement block
End Loop

statement True
condition
block
False

27
We will now present the algorithm for a
problem whose solution is familiar to us

We will first go through the problem

statement and then present the algorithm
in three different formats:

1. Pseudo code
2. Flowchart
3. Actual code

28
 Problem Statement
Convert a decimal number into binary

29
Convert 75 to Binary
2 75 remainder
2 37 1
2 18 1
2 9 0
2 4 1
2 2 0
2 1 0
0 1

1001011 30
 Solution in Pseudo Code
1. Let the decimal number be an integer x, x > 0
2. Let the binary equivalent be an empty string y
3. Repeat while x > 0 {
Determine the quotient & remainder of x
÷2
y = CONCATENATE( remainder, y )
x = quotient
}
4. Print y
31
5. Stop
Q: Is this the only possible algorithm for
converting a decimal number into a binary
representation?

If not, then is this the best?

In terms of speed?
In terms of memory requirement?
In terms of ease of implementation?

You must ask these questions after

writing any algorithm!

32
Tips on Writing Good Pseudo Code
• Use indention for improved clarity
• Do not put “code” in pseudo code – make your
pseudo code language independent
• Don’t write pseudo code for yourself – write it
in an unambiguous fashion so that anyone with
a reasonable knowledge can understand and
implement it
• Be consistent
• Prefer formulas over English language
descriptions 33
Start Flowchart of Decimal
to Binary Conversion
Get x

Yes Find quotient y = CONC(remainder, x)

x>0 ? & remainder
of x ÷ 2
x = quotient
No

Print y
x is the decimal number
Stop y is the binary equivalent
34
1. Does the flowchart depict the “correct”
algorithm?
2. What do we mean by “correct”, or better
yet, what do we check for
“correctness”?
3. One way is to check the algorithm for a
variety of inputs
4. Does it perform satisfactorily for:
• x=0?
• negative numbers?
• numbers with fractional parts? 35
Decimal to Binary Conversion in C
language

36
 Another Example: Sorting
Sort the following objects w.r.t. their heights

37
Expected Result

38
 Strategy
There are many strategies for solving this
problem. We demonstrate a simple one:

Repeat the following steps while the list is un-sorted:

Start with the first object in the list

Swap it with the one next to it if they are in the wrong order

Repeat the same with the next to the first object

Keep on repeating until you reach the last object in the list
39
Back to the Objects to be Sorted

40
Q: Is the list sorted?

A: No

41
Sorting: Step A1

42
 Sorting: Step A1

Swap? Yes

43
Sorting: Step A2

44
Sorting: Step A2

Swap? Yes

45
Sorting: Step A3

46
Sorting: Step A3

Swap? No

47
Sorting: After Step A7

48
Q: Is the list sorted?

A: No

49
Sorting: Step B1

50
 Sorting: Step B1

Swap? Yes

51
Sorting: Step B2

52
Sorting: Step B2

Swap? No

53
Sorting: After Step B7

54
Q: Is the list sorted?

A: No

55
Sorting: Step C1

56
 Sorting: Step C1

Swap? No

57
Sorting: After Step C7

58
Q: Is the list sorted?

A: Yes

59
STOP
60
 Pros and Cons of Flowcharts (1)
• The process of writing an algorithm in the form
of a flowchart is just too cumbersome(bulky)

• And then converting this graphical form into

code is not straight forward

• However, there is another kind of flowcharts –

called Structured Flowcharts – that may be
better suited for software developers

61
 Pros and Cons of Flowcharts (2)
• The good thing about flowcharts is that their
symbols are quite intuitive and almost
universally understood

• Their graphical nature makes the process of

explaining an algorithm to one’s peers quite
straightforward

62
Pros and Cons of Pseudo Code (1)
• Quite suitable for SW development as it is
closer in form to real code

• One can write the pseudo code, then use it as a

starting point or outline for writing real code

• Many developers write the pseudo code first

and then incrementally comment each line out
while converting that line into real code
63
Pros and Cons of Pseudo Code (2)
• Pseudo code can be constructed quite quickly
as compared with a flowchart

• Unlike flowcharts, no standard rules exist for

writing pseudo code

64
Difference between Algorithm, Flowchart
and Pseudo Code
• An algorithm is a sequence of instructions used
to solve a particular problem
• Flowchart and Pseudo code are tools to
document and represent the algorithm.
• In other words, an algorithm can be
represented using a flowchart or a pseudo
code.

65
 In Today’s Lecture, We …
Became familiar with the concept of algorithms:
– What they are?
– What is their use?
– What do they consist of?
– What are the techniques used for representing
them?
• In particular, we looked at the building blocks that
are used in all algorithms
• We also discussed the pseudo code and flowcharts
for particular problems
• In addition, we outlined the pros and cons of those
two techniques 66