CH01

Introduction to Algorithms
Eng. Jibril Hassan (Msc IT)

Design and Analysis of Algorithms - Chapter 1 1

 Objectives
• After studying this chapter you should
able to;
• Define “Algorithm” and it’s historical
Perspective
• Explain the role of Algorithm in
computing
• Describe the fundamentals of algorithms
• List the important types of algorithms
• Describe the fundamental data structures
Design and Analysis of Algorithms - Chapter 1 2
 Algorithm?
• Levitin defines lgorithm as given below;
• An algorithm is a sequence of unambiguous
instructions for solving a problem, i.e., for
obtaining a required output for any
legitimate input in a finite amount of time.
• We use algorithms to solve problems in a
systematic way. It is difficult to give a single
specific definition for an algorithm.
• Different fields of development and study
use algorithms to solve their own problems.

Design and Analysis of Algorithms - Chapter 1 3

 Notion of algorithm

problem

algorithm

input “computer” output

Algorithmic solution
Design and Analysis of Algorithms - Chapter 1 4
 Historical Perspective
• Muhammad ibn Musa al-Khwarizmi –
9th century mathematician (c. 780 – c.
850)
• Around 820 CE he was appointed as
the astronomer and head of the library
of the House of Wisdom(‫ة بيت‬EEE‫مكتب‬
‫ )الحكمة‬in Baghdad in Abbasid Caliphs.
• (c. 813–833 CE) presented the first
systematic solution of linear and
quadratic equations and introduced
the methods of "reduction" and he is • It was translated into
the father or founder of algebra.
Latin as Algoritmi de
• On the Calculation with Hindu numero Indorum. Al-
Numerals written about 820, was
Khwārizmī, rendered as
principally responsible for spreading
the Hindu–Arabic numeral system (Latin) Algoritmi, led to
throughout the Middle East and the term "algorithm".
Europe.

Design and Analysis of Algorithms - Chapter 1 5

Muhammad ibn Musa al-
Khwarizmi

Design and Analysis of Algorithms - Chapter 1 6

 Muhammad ibn Musa al-
Khwarizmi
• In the 12th century, Latin translations of his
textbook on arithmetic which codified the
various arabic numerals, introduced the
decimal positional number system to the
Western world.

Design and Analysis of Algorithms - Chapter 1 7

Historical
Perspective
A stamp issued
September 6,
1983 in the Soviet
Union,
commemorating
al-Khwārizmī's
(approximate)
1200th birthday.

Design and Analysis of Algorithms - Chapter 1 8

Historical
Perspective

Statue of Al-
Khwārizmī in
his birth town
Khiva,
Uzbekistan.

Design and Analysis of Algorithms - Chapter 1 9

Historical
Perspective
Statue of al-
Khwarizmi in front
of the Faculty of
Mathematics of
Amirkabir
University of
Technology in
Tehran,IRAN

Design and Analysis of Algorithms - Chapter 1 10

Definitions of algorithm
•An algorithm is defined as a set of well defined
instructions used to accomplish a particular task.
•It is considered as the cornerstone of good
programming.
•The efficiency of algorithms depends upon
speed, size and resource consumption.
•Different algorithms may finish the same
process with a different set of instructions in
more or less time, space, or effort than others.
•There is no fixed definition for an algorithm.
•It varies according to the area of use.

Design and Analysis of Algorithms - Chapter 1 11

 Various definitions of
algorithms are given below
• It is the exact set of instructions describing the
order of actions to achieve the result of the
decision problem in finite time – (Old
interpretation).
• Algorithm is a finite set of rules that defines the
sequence of operations to solve a specific set of
goals and has five important features: finiteness,
definiteness, input, output, efficiency" – (D.
Knuth).
• "Algorithms are all systems of calculations
performed on strictly defined rules, which, after a
number of steps obviously leads to the solution of
the problem" – (A. Kolmogorov).
Design and Analysis of Algorithms - Chapter 1 12
 Various definitions of
algorithms are given below
•"Algorithm is the exact prescription, defining
the computing process, going from the variable
input data to the desired result" – (A. Markov).
• "Algorithm is the exact requirement of the
performance in a specific order of certain
operations, leading to the solution of all
problems of this type" – (Philosophical
dictionary, ed. M. Rosenthal)

Design and Analysis of Algorithms - Chapter 1 13

 Properties of Algorithm
• An algorithm may have zero or more inputs externally and
it should produce one or more output. Also an algorithm
must terminate after a finite number of steps. Properties of
Algorithm include;
• Correctness - (It should provide correct and
accurateoutput for all legitimate input).
• Definiteness - (Each instruction should be clear, precise
and unambiguous. Input and output must be well defined.)
• Finiteness - (when we trace out the instructions of an
algorithm, it has to terminate after a finite number of
steps, for all clases).
• Effectiveness - ( Every instruction must be very basic so
that it can be carried out in priciple).
• Generality - ( Algorithms need to have general
instructions which can be applied in any case).

Design and Analysis of Algorithms - Chapter 1 14

Algorithm, program and
psuedocode
• An algorithm is a finite set of unambiguous
statement to solve a problem in finite
amount of time. It can be natural language
expressions designed for any common man.
• A program can be called as an algorithm
implemented using the required data
structures. It is the expression of an
algorithm in a programming language with
all the language specific codes.
• Procedure, function and subroutine are
synonyms for a program.

Design and Analysis of Algorithms - Chapter 1 15

Algorithm, program and
psuedocode
• A Pseudocode is a compact, informal
environment-independent description of a
computer programming algorithm.
• It uses the structural conversions of a
programming language, but only humans can
read it and not machines.
• Pseudocode omits details that are not
essential for human understanding of the
algorithm, such as variable declarations,
system-specific code and subroutines.
• There is no specific standard for pseudocode
syntax.
Design and Analysis of Algorithms - Chapter 1 16
 Use of algorithms in
computing
• We use algorithms to perform various logical and
mathematical operations such as sorting,
searching, merging, logic programming, matrix
processing, parsing, predicting statistics, solving
basic geometric problems, displaying graphics and
performing common mathematical calculations.
• Sorting algorithms arrange the data elements
in a specific order. It can be in ascending or
descending order. Internal sorting is a type of
sorting which arranges data internally in the
memory of the computer. Algorithms become more
sophisticated when the number of elements to be
sorted is more.
Design and Analysis of Algorithms - Chapter 1 17
 Use of algorithms in
computing
• Searching is the process where a particular element
is looked up in a collection of elements. The two basic
searching algorithms are linear search and binary
search. Linear search looks up a sequence of data
objects one by one. Binary search is a sophisticated
search and is faster than linear search
• We use merging algorithms to combine sets of
elements to form a single set according to some
criteria. Programming uses merge sort which is a
basic algorithm.
• Algorithms that manipulate strings include those that
arrange text data into lines and paragraphs or search
for occurrences of a given pattern in a document.

Design and Analysis of Algorithms - Chapter 1 18

 Use of algorithms in
computing
•Science and engineering field have a
wide range of applications that use
algorithms.
•Basic algorithms for mathematical
computation include those for generating
random numbers, performing operations
on matrices, solving simultaneous
equations, and numerical integration.

Design and Analysis of Algorithms - Chapter 1 19

 Role of Algorithm in
Computing
• We believe that a computer can do
anything and everything that we imagine.
But the truth is that the computers work on
algorithms written by humans themselves!
We need to write more software algorithms
due to the increasing complexity of
computers.
• Algorithms are available for many different
applications and highly advanced systems
such as artificial intelligence. These
algorithms may become common in the
future.
Design and Analysis of Algorithms - Chapter 1 20
 Fundamentals of
Algorithm
• Asymptotic notations
• Asymptotic complexity expresses the
main component of cost of the
algorithm.
• Big-O notation – Big-O is the method
of expressing the upper bound of the
running time of an algorithm. It is a
measure of the longest amount of time
it could possibly take for the algorithm
to complete.

Design and Analysis of Algorithms - Chapter 1 21

 Fundamentals of
Algorithm
• Big-Omega notation – It describes the best
that can happen for a given data size. This has
the same definition as Big-O, except that "f(n)
≥ cg(n)", this makes g(n) a lower bound
function, instead of an upper bound function.
• Theta notation – For non-negative functions,
f(n) and g(n), f(n) is theta of g(n) if and only if
f(n) = O(g(n)) and f(n) = Ω(g(n)). We denote
this as "f(n) = Θ(g(n))". This implies that the
function, f(n) is bounded both from the top
and bottom by the same function, g(n).

Design and Analysis of Algorithms - Chapter 1 22

 Fundamentals of
Algorithm
• Little-O notation – For non-negative
functions, f(n) and g(n), f(n) is little o of g(n)
if and only if f(n) = O(g(n)), but f(n) ≠
Θ(g(n)). We denote this as "f(n) = o(g(n))".
This represents a loose bounding version of
Big O. g(n) bounds from the top, but it does
not bound the bottom.
• Little Omega notation – For non-negative
functions, f(n) and g(n), f(n) is little omega
of g(n) if and only if f(n) = Ω(g(n)), but f(n)
≠ Θ(g(n)). We denote this "f(n) = ω(g(n))"

Design and Analysis of Algorithms - Chapter 1 23

Asymptotic notations

Design and Analysis of Algorithms - Chapter 1 24

 Complexity
• Space complexity
– Space complexity is defined as the
amount of memory that an algorithm
needs. A good algorithm minimizes the
memory needed.
• Time complexity
– It is a function that describes the time
of execution of an algorithm based on
it’s input parameters. In complex
algorithms it is difficult to find the
exact time taken.
Design and Analysis of Algorithms - Chapter 1 25
 Types of Algorithms
• Simple Recursive • Greedy approach

• Brute force • Dynamic programming

• Divide and conquer

• Backtracking and Branch
and bound
• Decrease and conquer

• Space and time tradeoffs

• Transform and
conquer

Design and Analysis of Algorithms - Chapter 1 26

 Types of Algorithms
• Simple Recursive: A simple recursive algorithm
solves the basic problems using recursion. It converts
the solution to a simpler sub problem to arrive at the
correct solution.
• Backtracking: are based on a depth-first recursive
search.
• Divide and conquer: we use divide and conquer
algorithms when there are two or more recursive calls.
• Dynamic programmig algorithm stores the previous
results and uses them to find new ones.
• Greedy algorithms: an optimization problem is one in
which you want to find, not just a solution, but the best
solution. It works by making the decision that seems
most promising at any moment.

Design and Analysis of Algorithms - 27

Chapter 1
 Types of Algorithms
• Branch and bound Algorithms: we use
branch and bound algorithms generally for
optimization problems. We form a tree of sub
problems in branch and bound algorithm.
• Brute force algorithms: simply tries all
possibilities until a satisfactory solution is
found by optimizing and satisfying.
• Randomized Algorithms: or probabilistic
algorithm uses a random number at least
once during the computation to make a
decision, it employs a degree of randomness
as part of its logic.
Design and Analysis of Algorithms - 28
Chapter 1
 Analysis of Algorithms
• How good is the algorithm?
– Correctness
– Time efficiency
– Space efficiency

• Does there exist a better algorithm?

– Lower bounds
– Optimality

Design and Analysis of Algorithms - Chapter 1 29

 Why study algorithms?
• Theoretical importance

– the core of computer science

• Practical importance

– A practitioner’s toolkit of known algorithms

– Framework for designing and analyzing

algorithms for new problems

Design and Analysis of Algorithms - Chapter 1 30

 Fundamental Data
Structures
• Data structures are an essential part of
algorithms; we use data structures such
as priority queue, binary heaps, and
binomial heaps to build complex
algorithms. It’s classified into;
• Primitive and non-primitive data
structures

Design and Analysis of Algorithms - Chapter 1 31

 Classifications of Data
Structure
• Primitive data structures are the basic data structures
that can be used directly in machine instructions, eg.
Integer, character, Boolean, string, double, float.
• Non-primitive data structures are the data structures
that are derived from primitive data structures, eg.
– Array: it is a set of same data elements grouped
together.
– Stack: it is a last in first out(LIFO) data structure.
– Queue: it is a first in first out(FIFO) data structure.
– List: it is a sequence of data elements connected to
each otherwhere every element has a link field which
refers to the location of the next element.
– Tree: it is a hierarchical representation of elements as
nodes linked with each other.

Design and Analysis of Algorithms - Chapter 1 32

 Classifications of Data
Structure
• Homogeneous and heterogeneous data structures:
non primitive data structures can be homogenous or
heterogeneous. Homogeneous DS has all the elements of
same data type in it, eg. Array. Heterogeneous DS contains
different types of data elements in it, eg. A record.
• Static and dynamic data structures: the system
allocates memory for static data structure at the compile
time itself( design time allocation). For dynamic DS
memoryis allocated during run time( through functions
such as colloc(), malloc()).
• Linear and non-linear data structures: Linear DS such
as arrays and lists maintains a constant and direct
relationship between its elements. Non-linear DS such as
trees – don’t maintain a constant and direct relationship
b/w their elements

Design and Analysis of Algorithms - Chapter 1 33

 Terminal Questions
• What are the properties of an
algorithm?
• What are asymptotic notations?
• Explain greedy algorithm with a suitable
example.
• How are data structures classified?
• Compare the characteristics of different
types of data structures.

Design and Analysis of Algorithms - Chapter 1 34

 Self Assessment
Questions
• 1. The rules for performing arithmetic
using Arabic numerals were originally
known as Algorism
• 2. The efficiency of algorithms depends
upon Speed, size, and resources
consumption.
• 3. An algorithm is considered as the
cornerstone of Good programming .

Design and Analysis of Algorithms - Chapter 1 35

 Self Assessment
Questions
• 4. To analyze an algorithm is to
determine the amount of Resources
necessary to execute it.
• 5. Merging algorithms are used to
combine to sets of elements to form a
single set according to some criteria
• 6. Programs is the expression of an
algorithm in a programming language
with all the language specific codes.

Design and Analysis of Algorithms - Chapter 1 36

 Self Assessment
Questions
• 7. An algorithm that invokes itself within
the process is called Direct recursive.
• 8. Big –O is the method of expressing
the upper bound of an algorithm's
running time.
• 9. Space Complexity is defined as the
number of memory cells which an
algorithm needs.

Design and Analysis of Algorithms - Chapter 1 37

 Self Assessment
Questions
• 10. A simple Recursive algorithm
converts the solution to a simpler sub
problem to arrive at the correct solution.
• 11. A brute Force algorithm simply tries
all possibilities until a satisfactory
solution is found.
• 12. Back Tracking algorithms are based
on a depth-first recursive search.

Design and Analysis of Algorithms - Chapter 1 38

 Self Assessment
Questions
• 13. A Data Structure is a set of data
elements grouped together under one
name.
• 14. Homogeneous data structure has
all the elements of same data type in it.
• 15. For Static data structure, the
memory is allocated at the compile time
itself

Design and Analysis of Algorithms - Chapter 1 39

 Summary
• The rules performing arithmetic using arabic
numerals were originally known as Algorism.
• The efficiency of algorithms depends upon speed,
size and resource consumption.
• To analyze an algorithm is to determine the
amount of resources necessary to execute it.
• Merging algorithms are used to combine to sets of
elements to form a single set according to some
criteria.
• An algorithm that invokes itself within the process
is called Direct recursive.
• Big-O notation is the method of expressing the
upper bound of an algorithms running time.
Design and Analysis of Algorithms - Chapter 1 40
 Summary
• Space complexity is defined as the number of
memory cells which an algorithm needs.
• Simple Recursive algorithm converts the solution
to a simpler sub problem to arrive at the correct
solution.
• A Brute force Algorithm simply tires all possibilities
until a satisfactory solution is found.
• Backtracking algorithms are based on a depth-first
recursive search.
• A data structure is a set of data elements grouped
together under one name.
• Homogeneous data structure has all the elements
of same data types in it.
Design and Analysis of Algorithms - Chapter 1 41
 Summary
• For static data structure, the memory is
allocated at the compile time itself.

Design and Analysis of Algorithms - Chapter 1 42

END

Design and Analysis of Algorithms - 43

Chapter 1