Analysis and Design of

Algorithms
Lecture 01
Dr. Eman Monir
Faculty of Computers and Artificial Intelligence
Benha University

Spring 2024
 Basic Course Information

• Course code:
• Course name: Analysis and Design of Algorithms
• Level: 2nd Year / B.Sc.
• Course Credit: 3 credits
• Instructor: Dr. Eman Monir
• Teaching Assistant:
 Course Material

Book
T. H. Cormen, C. E. Leiserson, R. L.Rivest,
C. Stein. Introduction to Algorithms , 3rd
Edition, 2009.

Slides
- Should be available after the class

1/26
 Table of Contents

1. Foundations
- The Role of Algorithms in Computing
- Design and Analysis algorithms
- Growth of Functions
2. Sorting
- Heapsort
- Quicksort
- Sorting in Linear Time
3. Graph Algorithms
- Elementary Graph Algorithms
- Minimum Spanning Trees
4. Advanced Design and Analysis Techniques
- Dynamic Programming
- Greedy Algorithms
5. Selected Topics
- Linear Programming
- String Matching

3/24
 Mathematical background needed to understand algorithms

• A. Summations C. Counting and Probability

- Summation formulas and properties - Counting
- Bounding summations - Probability
• B. Sets, Etc. - Discrete random variables
- Sets - The geometric and binomial distributions
- Relations D. Matrices
- Functions - Matrices and matrix operations
- Graphs - Basic matrix properties
- Trees

4/24
Ch1: Foundations

5/24
 Algorithms

❑ An algorithm is a set of computational steps for solving a well-specified

computational problem.
❑ Properties of an algorithm
➢ Input: an algorithm has input values from a specified set
➢ Output: from each set of input values an algorithm produces output values from a specified
set.
The output values are the solution to the problem
➢ Definiteness: the steps of an algorithm must be defined precisely
➢ Correctness: an algorithm should produce the correct output values for each set of input
values
➢ Finiteness: an algorithm should produce the desired output after a finite number of steps for
input in the set
➢ Effectiveness: it must be possible to perform each step of an algorithm exactly and in a
finite amount of time
➢ Generality: the algorithm should be applicable for all problems of the desired form not just
for a particular set of input values

6/24
 What is an algorithm?
An algorithm is a sequence of unambiguous instructions
for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of
time.
problem

algorithm

input “computer” output

1-9
 Algorithm

• An algorithm is a sequence of unambiguous

instructions for solving a problem, i.e., for obtaining a
required output for any legitimate input in a finite
amount of time.

• Can be represented various forms

• Unambiguity/clearness
• Effectiveness
• Finiteness/termination
• Correctness
1-10
 What kinds of problems are solved by algorithms?

• Basic numerical algorithms

- arithmetic problems like finding the sum, product or quotient of two numbers
• Cryptographic algorithms
- digitally sign and encrypt your data when it’s sent over the internet

• Sorting and searching algorithms

- allow programmers to arrange data in memory in many different ways so that
specific pieces of data can later be retrieved efficiently when they are needed
• Signal processing algorithms
- compress audio and video data very efficiently and allow us to store a whole 3-
hour UHD movie on a single plastic disc, or stream it live over the internet

7/24
 What kinds of problems are solved by algorithms?

• Graph algorithms
- find an efficient path through some kind of network — be it a computer network, a
road network

• Computational geometry algorithms

- important in games and are also heavily used in CAD software

• Computer vision algorithms

- allow software to recognize features in images
• Computer graphics algorithms
- Video games and movies are the most spectacular uses

8/24
Algorithms Task

9/24
 Algorithms Task (cont.)

❑ Analysis: predict the cost of an algorithm in terms of resources

and performance

❑ Design: creating an efficient algorithm to solve a problem in an

efficient way using minimum time and space.

10/
 Time Complexity & Space Complexity

• Time Complexity is a function describing the amount of

time required to run an algorithm in terms of the size of the
input.

• Space Complexity is a function describing the amount of

memory an algorithm takes in terms of the size of the input.

11/
24
 Some Well-known Computational Problems
• Sorting
• Searching
• Shortest paths in a graph
• Minimum spanning tree
• Primality testing
• Traveling salesman problem
• Knapsack problem
• Chess
• Towers of Hanoi
• Program termination
Some of these problems don’t have efficient algorithms, or
algorithms at all!
1-7
 Basic Issues Related to Algorithms
• How to design algorithms

• How to express algorithms

• Proving correctness

• Efficiency or (complexity) analysis

– Theoretical analysis

– Empirical analysis

• Optimality

1-17
 Analysis of Algorithms
• How good is the algorithm?
– Correctness
– Time efficiency
– Space efficiency

• Does there exist a better algorithm?

– Lower bounds
– Optimality

1-18
Algorithm Design Strategies

 Greedy approach
 Dynamic programming

 Backtracking and branch-and-bound

 Space and time tradeoffs

1-19
 What is an Algorithm?

• Recipe, process, method, technique, procedure,

routine,… with the following requirements:
1. Finiteness
 terminates after a finite number of steps
2. Definiteness
 rigorously and unambiguously specified
3. Clearly specified input
 valid inputs are clearly specified
4. Clearly specified/expected output
 can be proved to produce the correct output given a valid input
5. Effectiveness
 steps are sufficiently simple and basic

1-20
 Comparing between two Algorithms

Algorithm A o Model of computation :

Hardware, Operating
Exec_Time = 10msec System, Processor, etc
RAM = 100KB

Algorithm B
o If I change the model of
computation, the time
Exec_Time = 2msec
and space may change
RAM = 10KB

12/
 Sorting Problem
❑ Sort a sequence of numbers into non-decreasing order

Example:
Input:
8 2 4 9 3 6
Output:

2 3 4 6 8 9
- Input sequence is called an instance of the sorting problem.
13/
 Sorting Problem (cont)

❑ Efficiency
o Different algorithms devised to solve the same problem often differ
dramatically in their efficiency.
o These differences can be much more significant than differences due
to hardware and software
o In this chapter, we will see two algorithms for sorting problem
o The first, known as insertion sort, takes time roughly equal to 𝒄𝟏𝒏𝟐 to sort 𝑛 items,
where 𝒄𝟏 is a constant that does not depend on 𝒏.
o The second, merge sort, takes time roughly equal to 𝒄𝟐𝒏𝒍𝒐𝒈𝟐n, where 𝒄𝟐 is
another constant that also does not depend on 𝒏.

14/
 Insertion Sort

• Solves the sorting problem

• An efficient algorithm for sorting a small
number of elements
• Works the way many people sort a hand of playing
cards
• Start with an empty left hand and the cards face
down on the table
• Then remove one card at a time from the table and
insert it into the correct position in the left hand
• To find the correct position for a card, we compare
it with each of the cards already in the hand, from right to left

15/
Insertion Sort (cont.)
keys: The numbers
that we wish to
sort
Example of Insertion Sort

17/
 Analysis of Algorithms

❑ Analysis of Algorithms is the determination of the amount of time,

storage and/or other resources necessary to execute them.
❑ Analyzing algorithms is called Asymptotic Analysis
❑ We shall assume a generic one processor, random-access machine (RAM)
model of computation as our implementation technology and understand that
our algorithms will be implemented as computer programs.
❑ In the RAM model, instructions are executed one after another
❑ The running time depends on the input: an already sorted sequence is easier
to sort and short sequences are easier to sort than long ones.

18/
 Analysis of Algorithms

• We can have three cases to analyze an algorithm

1. Worst-case
• The case that causes maximum number of operations to be executed
• 𝑇(𝑛) = upper bound on running time of an algorithm on any input of
size 𝑛.
Example: Search for number 8

2 3 5 4 1 7 6

19/
 Analysis of Algorithms

2. Average-case: (sometimes)
• We take all possible inputs and calculate computing time for all of the inputs
• 𝑇(𝑛) =expected time of algorithm over all inputs of size 𝑛

3. Best-case
• Calculate lower bound on running time of an algorithm.
• The case that causes minimum number of operations to be executed.
Example: Search for number 2

2 3 5 4 1 7 6
Analysis of Insertion Sort

o We presenting the INSERTION-SORT

procedure with the time “cost” of each
statement and the number of times each
statement is executed.

o Let 𝒕𝒋 denote the number of times the

while loop test in line 5 is executed for
that value of 𝒋.

o Comments are not executable

statements, and so they take no time.

21/
Analysis of Insertion Sort

time number of times 22/24

 Analysis of Insertion Sort

❑ Running time of insertion sort

❑ In insertion sort, the best case occurs if the array is already sorted.

23/24
 Analysis of Insertion Sort

❑ Best Case
❑ If 𝐴 is sorted: 𝑂(𝑛) comparisons

❑ Worst Case
❑ If 𝐴 is reversed sorted: 𝑂(𝑛2) comparisons

❑ Average Case
❑ If A is randomly sorted: 𝑂(𝑛2) comparisons

24/24
 Example of computational problem: sorting

• Statement of problem:
– Input: A sequence of n numbers <a1, a2, …, an>

– Output: A reordering of the input sequence <a´ 1, a´2, …,

a´n> so that a´i ≤ a´ j whenever i < j

• Instance: The sequence <5, 3, 2, 8, 3>

• Algorithms:
– Selection sort
– Insertion sort
– Merge sort

1-34
 Selection Sort
• Input: array a[1],…,a[n]

• Output: array a sorted in non-decreasing order

• Algorithm:

for i=1 to n
swap a[i] with smallest of a[i],…,a[n]

• Is this unambiguous? Effective?

1-35
 Euclid’s Algorithm
Problem: Find gcd(m,n), the greatest common divisor of
two nonnegative, not both zero integers m and n

Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?

Euclid’s algorithm is based on repeated application of

equality
gcd(m,n) = gcd(n, m mod n)
until the second number becomes 0, which makes the
problem
trivial.

Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

1-36
 Two Descriptions of Euclid’s algorithm
Finding gcd(m,n)
Step 1 If n = 0, return m and stop;
otherwise go to Step 2
Step 2 Divide m by n and assign the value of the remainder
to r
Step 3 Assign the value of n to m and the value of r to n.
Go to Step 1. gcd( n, r) , r = m mod n

Pseudo code:
while n ≠ 0 do
r ← m mod n r= m % n;
m← n m = n;
n←r
return m
1-37
 other Description for computing gcd(m,n)

Consecutive integer checking algorithm

Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder is 0, go to Step 3;
otherwise, go to Step 4
Step 3 Divide n by t. If the remainder is 0, return t and
stop;
otherwise, go to Step 4
Step 4 Decrease t by 1 and go to Step 2

Is this slower than Euclid’s algorithm?

How much slower?
O(n), if n <= m , vs O(log n)
1-15
 Other methods for gcd(m,n)
• Middle-school procedure gcd (15,3) =
• Step 1 Find the prime factorization of m
Step 2 Find the prime factorization of n
Step 3 Find all the common prime factors
• Step 4 Compute
the product of all the common
prime factors and return it as gcd(m,n)
Is this an algorithm? 15= 5x3x1 , 3=3x1
How efficient is it? Time complexity: O(sqrt(n))
1-39
 Sieve of Eratosthenes
Input: Integer n ≥ 2 if n=8
Output: List of primes less than or equal to n: 2,3,5,7

for p ← 2 to n do A[p] ← p

for p ← 2 to n do
if A[p]  0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j←j+p

Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20
Time complexity: O(n)
1-40
Thank You