DESIGN AND ANALYSIS OF

ALGORITHMS
Course Code : AR212
Credit Hours :3
Prerequisite : AR211

Dr.Laith Baniata 1
Instructor Information

Name Dr. Laith Baniata

Office No. --
Tel (Ext) None
E-mail [email protected]
Office Hours Sun, Tue, Thu 12:30-13:30, Mon Wed 13:00-14:30
Class Times Building Day Start Time End Time Room No.
Engineering Sun -Tue, 12:30 13:20 004
Thu
AI Mon Wed, 13:00 14:15 Technical
simulation
lab

Dr.Laith Baniata 2
This course is to design efficient computer algorithms, prove their
correctness, and analyze their running times. it includes mathematical
analysis of algorithms (summations and recurrences), advanced data
structures (balanced search trees), algorithm design techniques (divide-
and-conquer, dynamic programming, and greedy algorithms), graph
algorithms (breadth-first and depth-first search, minimum spanning
trees, shortest paths).
Course Title: DESIGN AND ANALYSIS OF ALGORITHMS
Credit Hour(3)

[Pre-req. AR211
Dr.Laith Baniata 3
Textbook: Introduction to Algorithms, Thomas Cormen, Charls Leiserson & Ronald Rivest. 3rd Edition,
MIT Press, 2009.

Image of the textbook Cover

Dr.Laith Baniata 4
COURSE OBJECTIVES

The main learning objectives of the course are to:

1. Learn about the use of several design techniques and the importance of studying the complexity of
a particular algorithm
2. Computational complexity analysis of algorithms and worst-case algorithm runtime analysis using
asymptotic analysis and mathematical analysis of algorithms (summations and recursion)
3. Writing and analyzing recursive relationships for recursive algorithms
4. Describe the divide and conquer approach as a principle for algorithm design
5. Learn about searching and sorting algorithms
6. Ability to design, analyze and validate graph algorithms and shortest path algorithms
7. Ability to design and analyze algorithms that follow the approach of greedy algorithms
8. Understand dynamic programming algorithms and optimization algorithms and know when the
algorithm design situation calls for it.
9. Learn about advanced data structures

Dr.Laith Baniata 5
COURSE SYLLABUS

Week Course Topic

Week 1 Introduction to Algorithms, The Efficiency of Algorithms, Attributes of Algorithms. A Choice of
Algorithms. Classifications of algorithms: by implementation, by design, by field of study and by
complexity.
Week 2 Mathematical Background (summations and recurrences)
Week 3 Measuring Efficiency, Measuring the asymptotic growth of functions. The basic techniques for
manipulating bounded summations, searching techniques (linear search, binary search).
Week 4 Techniques for problem-solving: Divide and Conquer, the General Method, Recurrence Equations,
Solving Recurrence Equations (Master and iteration Methods).
Week 5 and 6 Sorting techniques based on Divide and Conquer: Merge Sort.
Quick Sort, Strassen's Matrix Multiplications.
Week 7 Other sorting techniques: Insertion sort, Selection Sort, Heap Sort
Week 8 Midterm Exam
Week 9 Graph Algorithms, Formal Definition of Graphs. Graphs Types.
Graph Terminologies, Strongly Connected Graph, Representations of Graphs.
Week 10 Graph Traversal, Breadth-First Search Algorithm, Depth-First Search Algorithm. Topological Sort
Algorithm.
Week 11 Greedy Algorithms, Minimum spanning trees
Week 12 Shortest paths algorithms
Week 13 Dynamic Programming
Week 14 Optimization Algorithms
Week 15 Advanced data structures (balanced search trees)
Week 16 Final Exam Dr.Laith Baniata 6
Week 1

Dr.Laith Baniata 7
 Chapter 1
Introduction to Algorithms
Dr.Laith Baniata
Autonomous Systems Dept.
Design and Analysis of algorithms Course

Dr.Laith Baniata 8
Outline
1. Computational problems
2. Algorithms
3. Problem-solving Process
4. Components of an algorithm
5. Algorithm’s Representations
6. Algorithm Efficiency
7. Algorithms Classifications

Dr.Laith Baniata 9
Computational problems

• A computational problem specifies an input-output

relationship
• What does the input look like?
• What should the output be for each input?

• Example:
• Input: an integer number n
• Output: Is the number prime?

• Example:
• Input: A list of names of people
• Output: The same list sorted alphabetically
Dr.Laith Baniata 101
0
 Algorithms

• An Algorithm is a series of steps that can be followed to complete a

specific tasks.
• It is not the same thing as a computer program which is an
implementation of an algorithm! We use algorithms as part of the
planning stage, so that before we can write the program, we have to
work out the steps needed to solve a given problem.
• A tool for solving a well-specified computational problem

Input Algorithm
Dr.Laith Baniata
Output 1
11
1
Correct and incorrect algorithms

• Algorithms must be:

• Correct: For each input produce an appropriate output
• Efficient: run as quickly as possible, and use as little memory as possible
• Algorithm is correct if, for every input instance, it ends with the correct output. We
say that a correct algorithm solves the given computational problem.
• An incorrect algorithm might not end at all on some input instances, or it might end
with an answer other than the desired one.
• We shall be concerned only with correct algorithms.

Dr.Laith Baniata 12
Problems and Algorithms

• We need to solve a computational problem

• “Convert a weight in pounds to Kg”

• An algorithm specifies how to solve it, e.g.:

• 1. Read weight-in-pounds
• 2. Calculate weight-in-Kg = weight-in-pounds * 0.455
• 3. Print weight-in-Kg

• A computer program is a computer-executable description of an algorithm

Dr.Laith Baniata 13
The Problem-solving Process

Analysis
Problem
specification
Design

Algorithm
Implementation
Program

Compilation
Executable
Dr.Laith Baniata
Dr. Aryaf Al-Adwan (solution) 141
4
Practical Examples

• Internet and Networks

• The need to access large amount of information with the shortest time.
• Problems of finding the best routs for the data to travel.
• Algorithms for searching this large amount of data to quickly find the pages on
which particular information resides.

• Electronic Commerce
• The ability of keeping the information (credit card numbers, passwords, bank
statements) private, safe, and secure.
• Algorithms involves encryption/decryption techniques.

Dr.Laith Baniata 15
Hard problems

• We can identify the Efficiency of an algorithm from its speed (how long does the
algorithm take to produce the result).
• Some problems have unknown efficient solution.
• These problems are called NP-complete problems.
• If we can show that the problem is NP-complete, we can spend our time developing
an efficient algorithm that gives a good, but not the best possible solution.

Dr.Laith Baniata 16
Components of an Algorithm

• Variables and values

• Instructions
• Sequences
• A series of instructions
• Procedures
• A named sequence of instructions
• we also use the following words to refer to a “Procedure” :
• Sub-routine
• Module
• Function

Dr.Laith Baniata 17
Components of an Algorithm Cont.

• Selections
• An instruction that decides which of two possible sequences is
executed
• The decision is based on true/false condition
• Repetitions
• Also known as iteration or loop
• Documentation
• Records what the algorithm does

Dr.Laith Baniata 18
Algorithm’s Representations

How to represent an algorithm?

1. Give a description in your own language, e.g. English, Spanish, …

2. Pseudo code

3. Graphical (Flow Charts)

Dr.Laith Baniata 19
 Pseudocode

• Pseudocode is a form of Structured English for writing an algorithm.

• It uses programming-style constructs, but is not written in an actual
programming language.
• You do not need to worry about the detailed syntax or be precise about
how the code will complete a particular task.
• Writing in pseudocode helps you concentrate on the logic (Process) and
efficiency of your algorithm before you have to start thinking about the
actual code you will be using.

Dr.Laith Baniata 20
 Flow Charts

• Flowcharts can be used to represent algorithms visually, they use

diagrams which use particular symbols to show the flow of data,
processing and input/output that takes place within a program or task.
• The next image below shows that standard flowchart symbols that we
use:

Dr.Laith Baniata 21
Flow Chart’s Symbols

Dr.Laith Baniata 22
Flow Chart and
Pseudocode
Example

Dr.Laith Baniata 23
Algorithm Efficiency

• Consider two sort algorithms

• Insertion sort
• takes c1n2 to sort n items
• where c1 is a constant that does not depends on n
• it takes time roughly proportional to n2
• Merge Sort
• takes c2 n lg(n) to sort n items
• where c2 is also a constant that does not depends on n
• lg(n) stands for log2 (n)
• it takes time roughly proportional to n lg(n)

Dr.Laith Baniata 24
Algorithm Efficiency Cont.

• Consider now:
• A faster computer A running insertion sort against
• A slower computer B running merge sort
• Both must sort an array of one million numbers
• Suppose
• Computer A execute one billion (109) instructions per second
• Computer B execute ten million (107) instructions per second
• So computer A is 100 times faster than computer B
• Assume that
• c1 = 2 and c2 = 50

Dr.Laith Baniata 25
Algorithm Efficiency Cont.
• To sort one million numbers
• Computer A takes
2 . (106)2 instructions
109 instructions/second
= 2000 seconds
• Computer B takes
50 . 106 . lg(106) instructions
107 instructions/second
 100 seconds

• By using algorithm whose running time grows more slowly, Computer B runs 20 times faster than
Computer A
• For ten million numbers
• Insertion sort takes  2.3 days
• Merge sort takes  20 minutes

Dr.Laith Baniata 26
Classifications of Algorithms

Dr.Laith Baniata 272

7
 Algorithms Classifications

• There are various ways to classify algorithms, each with its own merits:
1) By implementation
2) By design paradigm
3) By field of study
4) By complexity

Dr.Laith Baniata
282
8
 By Implementation

• One way to classify algorithms is by the way of implementation:

1. Iterative or Recursive: iterative algorithms use loops to solve a given problem
while recursive algorithm invoke itself repeatedly until a certain termination condition
satisfied such as tower of Hanoi algorithm.
2. Serial or parallel: Serial algorithms execute one instruction at a time in a
sequential fashion on a serial computer. Parallel algorithms take advantage of
computer architectures where several processors can work on a problem at the
same time, whereas distributed algorithms utilize multiple machines connected with
a computer network. Parallel algorithms divide the problem into more symmetrical or
asymmetrical subproblems and collect the results back together.
3. Exact or approximate: exact algorithms give an exact solution to a given problem,
approximate algorithms give a solution that is close to the exact solution such as the
TSP problem.

Dr.Laith Baniata 29
 By design paradigm

• Another way of classifying algorithms is by their design methodology or

paradigm:
1. Brute-force or exhaustive search: is to try every possible solution to see which is
the correct. This method is good only for small input size or when the speed is not
an issue.
2. Divide and conquer: Divides the problem into subproblems, solve each one into a
sub-solution and then combine the sub solutions together to produce the solution to
the main problem. Usually implemented using recursion. Such as merge sort.
3. Dynamic Programming: avoids recomputing solutions that have already been
computed by storing keeping the founded solutions and storing them. Such as the
Fibonacci series.
4. Greedy Method: choses the current optimal solution at each step in an attempting to find
the overall optimal solution such as prims algorithm.

Dr.Laith Baniata 30
 By Field of Study

• Every field of science has its own problems and needs efficient algorithms.
1. Search Algorithms
2.Sorting Algorithms
3. Graph Algorithms
4.String Algorithms
5.Machine learning Algorithms
6. Cryptography Algorithms
7.And more

Dr.Laith Baniata 31
 By complexity

• Algorithms can be classified by the amount of time they need to complete

compared to their input size:
1. Constant time: if the time needed by the algorithm is the same, regardless of the
input size. E.g. an access to an array element.
2. Logarithmic time: if the time is a logarithmic function of the input size. E.g. binary
search algorithm.
3. Linear time: if the time is proportional to the input size. E.g. the traverse of a list.
4. Polynomial time: if the time is a power of the input size. E.g. the bubble sort
algorithm has quadratic time complexity.
5. Exponential time: if the time is an exponential function of the input size. E.g.
Brute-force search.

Dr.Laith Baniata 32
Dr.Laith Baniata 33