ALGORITHMS

ANALYSIS AND DESIGN

Lecture 1
DR. Hossam Hawash
 LECTURE CONTENT

1. Algorithms (Definitions and Essentials)

2. Sorting (Insertion)
3. Loop invariants
4. Algorithm Analysis
5. Order of growth
6. Algorithm Design
7. Sorting (Merge)
8. Quiz

L1.2
ALGORITHMS (DEFINITIONS AND ESSENTIALS)

Algorithm is any well-defined computational procedure that takes set

of values, as input and produces some set of values, as output.
An algorithm is thus a sequence of computational steps that transform
the input into the output
We can also view an algorithm as a tool for solving a well-specified
computational problem

L1.3
ALGORITHMS (TYPES)

Algorithms

Sorting

Searching Graphs Strings

L1.4
THE PROBLEM OF SORTING
Input: sequence a1, a2, …, an of numbers.

Output: permutation a'1, a'2, …, a'n such

that a'1  a'2  …  a'n .

Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
L1.5
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6

L1.6
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6

L1.7
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6

L1.8
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6

L1.9
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

L1.10
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

L1.11
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

L1.12
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

L1.13
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

L1.14
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

L1.15
EXAMPLE OF INSERTION SORT
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done

L1.16
FLOWCHART
Flowcharts are graphical diagrams that show the
steps and logic of an algorithm.
They use symbols, arrows, and text to illustrate
the inputs, outputs, processes, decisions, and
loops of an algorithm.
Flowcharts are useful for visualizing the overall
structure and flow of an algorithm, and for
identifying potential errors or inefficiencies.

L1.17
PSEUDOCODE
Pseudocode is a textual representation of an algorithm that uses natural
language and some basic programming concepts.
It is not meant to be executed by a computer, but rather to describe the
logic and steps of an algorithm in a clear and concise way.
Pseudocode is useful for planning and testing an algorithm, and for
translating it into code later.

L1.18
 INSERTION SORT: PSEUDOCODE
INSERTION-SORT (A, n) ⊳ A[1 . . n]
for j ← 2 to n
do key ← A[ j]
i←j–1
while i > 0 and A[i] > key
do A[i+1] ← A[i]
i←i–1
A[i+1] = key
1 i j n
A:
key
sorted
L1.19
 CORRECTNESS VS EFFICIENCY
• A fully correct algorithm is when it’s
implementation can solve any instance of
the problem that the solution is proposed,
that include inconvenient and big inputs.
• An efficient algorithm is when it solves a
problem without taking long time or
consuming a great amount of memory.

L1.20
LOOP INVARIANT
A loop invariant is a condition that is necessarily true immediately
before and immediately after each iteration of a loop.
A good loop invariant should satisfy three properties:
• Initialization: The loop invariant must be true before the first execution of
the loop.
• Maintenance: If the invariant is true before an iteration of the loop, it should
be true also after the iteration.
• Termination: When the loop is terminated the invariant should tell us
something useful, something that helps us understand the algorithm.

L1.21
ALGORITHM ANALYSES
The time complexity of an algorithm quantifies the amount of time
taken by an algorithm to run as a function of the length of the input.
The space complexity of an algorithm quantifies the amount of
memory space taken by an algorithm to run as a function of the length
of the input.

L1.22
ALGORITHM ANALYSES
Worst-case: (usually)
• T(n) = maximum time of algorithm
on any input of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm
over all inputs of size n.
• Need assumption of statistical
distribution of inputs.
Best-case: (NEVER)
• Cheat with a slow algorithm that
works fast on some input.
L1.23
ALGORITHM ANALYSES

L1.24
 Q-NOTATION
DEF:
Q(g(n)) = { f (n) : there exist positive constants c1, c2, and
n0 such that 0  c1 g(n)  f (n)  c2 g(n)
for all n  n0 }
Basic manipulations:
• Drop low-order terms; ignore leading constants.
• Example: 3n3 + 90n2 – 5n + 6046 = Q(n3)

L1.25
ORDER OF GROWTH
The rate at which running time increases as a function of input is called
Rate (order) of Growth.
We write that insertion sort has a worst-case running time of
the𝑡𝑎 (𝑛2 )

L1.26
INSERTION SORT ANALYSIS
Worst case: Input reverse sorted.
n
T ( n) =  Q ( j ) = Q (n 2) [arithmetic series]
j =2
Average case: All permutations equally likely.
n
T ( n) =  Q( j / 2) = Q(n 2 )
j =2
Is insertion sort a fast sorting algorithm?
• Moderately so, for small n.
• Not at all, for large n.
L1.27
ALGORITHM DESIGN:MERGE SORT
MERGE-SORT A[1 . . n]
1. If n = 1, done.
2. Recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .
3. “Merge” the 2 sorted lists.

Key subroutine: MERGE

L1.28
MERGE SORT EXAMPLE
99 6 86 15 58 35 86 4 0
MERGE SORT EXAMPLE
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0
MERGE SORT EXAMPLE
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0
MERGE SORT EXAMPLE
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0
 MERGE SORT EXAMPLE
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

4 0
 MERGE SORT EXAMPLE

99 6 86 15 58 35 86 0 4

Merge 4 0
 MERGE SORT EXAMPLE

6 99 15 86 35 58 0

99 6 86 15 58 35 86 0 4

Merge
 MERGE SORT EXAMPLE

6 99 15 86 35 58 0 4

99 6 86 15 58 35 86 0 4

Merge
 MERGE SORT EXAMPLE

6 99 15 86 35 58 0 4 86

99 6 86 15 58 35 86 0 4

Merge
MERGE SORT EXAMPLE

6 0

6 99 15 86 35 58 0 4 86

Merge
MERGE SORT EXAMPLE

6 15 0 4

6 99 15 86 35 58 0 4 86

Merge
MERGE SORT EXAMPLE

6 15 86 99 0 4 35

6 99 15 86 35 58 0 4 86

Merge
MERGE SORT EXAMPLE

6 15 86 99 0 4 35 58 86

6 99 15 86 35 58 0 4 86

Merge
MERGE SORT EXAMPLE
0

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4 6

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4 6 15

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4 6 15 35

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4 6 15 35 58

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4 6 15 35 58 86

6 15 86 99 0 4 35 58 86

Merge
MERGE SORT EXAMPLE
0 4 6 15 35 58 86 86 99

6 15 86 99 0 4 35 58 86

Merge
ANALYZING MERGE SORT
T(n) MERGE-SORT A[1 . . n]
Q(1) 1. If n = 1, done.
2T(n/2) 2. Recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .
Q(n) 3. “Merge” the 2 sorted lists
Sloppiness: Should be T( n/2 ) + T( n/2 ) ,
but it turns out not to matter asymptotically.

L1.50
CONCLUSIONS
• Q(n lg n) grows more slowly than Q(n2).
• Therefore, merge sort asymptotically
beats insertion sort in the worst case.
• In practice, merge sort beats insertion
sort for n > 30 or so.

L1.51