Overview of Algorithm

Analysis
Ms. Andleeb Yousaf Khan
Spring 2020
Lecture 2
What is an algorithm?
• A high level description of a process.
• Based on some input, an algorithm describes a
method to generate output.
• An algorithm should solve some problem –
captured by the input/output relationship

2
 Example algorithm

• Problem: find the index of an element in an array

• Input: the array, A, and the element, x
• Output: the index in A of element x
function find(x,A)
i « 0
while i < size(A)
• Method if A[i] = x
return i
end if
i « i + 1
end while
end 3
Pseudocode
• Pseudocode allows you to describe an algorithm without
the syntax and semantics of a specific programming
language.
• Pseudocode can gloss over implementational details.
• Pseudocode can include plain English sentences or phrases
along with code.
• It can be convenient to think of the relationship between
pseudocode and code as similar to that between an
“outline” and a paper.
4
Why study algorithms?
• Efficiency is fundamental to successful software
engineering.
• Designing efficient algorithms is essential.
• Identifying inefficiencies is just as important.
• Studying algorithms will help you write efficient
software.
• Also, it will get you a job.
5
Why search and sort?
• Search is fundamental to many IO operations – find a file,
book a flight, find an article, Google.
• Structuring information can make retrieval more efficient.
• Sorting is an easy to understand linear structuring of
information.
• Sorting and searching is a case study for the interaction
between structure and retrieval.
• Graphs and graph functions, hashing etc. are all more
complicated examples of this structure/retrieval relationship.
6
Review: Data Structures
• Three data structures that we will be relying on
heavily in this course.
• Arrays
• Trees
• Graphs

7
Review: Arrays
• Arrays are linear series of data.
• Operations we assume to be available:
• [i] – access the ith element of the array
• size – how many elements are in the array

i-2 i-1 i i+1 i+2 i+3 i+4 i+5

8
 Review: Trees
• Binary Trees contain a set of
Nodes. Each Node can have a
left or right child.
• Operations we assume to be
available:
• parent – get the parent node
• left – get the left child
• right – get the right child
9
Review: Graphs
• A Graph is defined by a set of Vertices or Nodes, and a set of Edges that
connect pairs of Vertices.
• Operations we assume to be available:
• Vertex::adjacent_vertices
• Vertex::out_edges
• Vertex::in_edges
• Edge::source_vertex
• Edge::dest_vertex

• Both Arrays and Trees can be

viewed as special cases of Graph
10
 Math we’ll use
• Exponents and Logarithms are used heavily. Log with no subscript is
taken to be log base 2. log xy =log x +log y
n
x =y x
log =log x - log y
n =log x y y
n n log x =log x n
å 1 =n n n
i(i +1)
i=1 å x =nx å i=
2
  • Summations  
i=1 i=1

• Inductive proofs (Appendex A)

  11

   
Example: Insertion Sort

• Sort an Array A = [5, 2, 4, 6, 1, 3]

• Take element j=2,move it to the right until A[1..2]
is correctly sorted.
• Take element j=3, move it to the left until A[1..3] is
sorted
• Continue up to j=n.

12
Insertion Sort

13
 Insertion Sort
function insertionSort(A)
for j « 2 to size(A) do
key « A[j]
i « j – 1
while i > 0 and A[i] > key do
A[i + 1] « A[i]
i « i – 1
end while
A[i + 1] « key
end for
end

14
 Correctness of Insertion Sort

• We show insertion sort is correct using a construct

called a Loop Invariant.
• Loop invariant is a condition that is necessarily
true immediately before and immediately after
each iteration of a loop. (Note that this says nothing
about its truth or falsity part way through an
iteration.)

15
Invariant = never changing.
• In computer science, an invariant is a condition
that can be relied upon to be true during execution
of a program, or during some portion of it. It is a
logical assertion that is held to always be true
during a certain phase of execution.

16
Correctness of Insertion Sort
• We show insertion sort is correct using a construct
called a Loop Invariant.
• Three properties of a Loop Invariant for an
algorithm to be considered correct.
1 Initialization – It is true at the start of the loop.
2 Maintenance – It is true at the end of the loop.
3 Termination – When the loop terminates, the Loop
Invariant should show that the algorithm is correct.
17
Insertion Sort Loop Invariant
• Loop Invariant: At the start of each for loop
iteration, A[1..j-1] contains the items initially in
A[1..j-1] but in sorted order.
function insertionSort(A)
for j « 2 to size(A) do
key « A[j]
i « j – 1
while i > 0 and A[i] > key do
A[i + 1] « A[i]
i « i – 1
end while
A[i + 1] « key
end for
end 18
 Insertion Sort Loop Invariant

• Loop Invariant: At the start of each for loop

iteration, A[1..j-1] contains the items initially in
A[1..j-1] but in sorted order.
• Initialization: When j=2, the subarray A[1..j-
1]=A[1..1]=A[1] only contains one element, so it is
sorted.

19
 Insertion Sort Loop Invariant

• Loop Invariant: At the start of each for loop iteration,

A[1..j-1] contains the items initially in A[1..j-1] but in
sorted order.
• Maintenance: The body of the for loop moves A[j-1]
down to A[1] one position to the right, until the correct
position for A[j] is found. At which point A[j] is inserted.
• Formally, we need to show that at the point where A[j] is
inserted A contains only elements less than or equal to A[j] to
the left, and only elements greater than A[j] to the right.

20
 Insertion Sort Loop Invariant

• Loop Invariant: At the start of each for loop

iteration, A[1..j-1] contains the items initially in
A[1..j-1] but in sorted order.
• Termination: The loop ends when j=n+1.
Therefore, A[1..n] contains the items initially in
A[1..n] but in sorted order. Thus, the entire array is
sorted.

21
 Insertion Sort – run time

function insertionSort(A) n
for j « 2 to size(A) do n- 1
key « A[j] n- 1
//insert A[j] into A[1…..j-1] n- 1
i « j – 1 å
n
j
j=2
while i > 0 and A[i] > key
  do n
å ( j - 1)
A[i + 1] « A[i]   j=2

å
n
( j - 1)
i « i – 1     j=2

end while  
A[i + 1] « key
 
n- 1
 
end for
22
end
 Insertion Sort
function insertionSort(A)
for j « 2 to size(A) do n
key « A[j] n- 1
i « j – 1 n- 1
while i > 0 and A[i] > key do n- 1
å
n
A[i + 1] « A[i] j=2
j
i « i – 1   å
n
( j - 1)
end while   å
j=2
n
( j - 1)
A[i + 1] « key     j=2

end for    
n- 1
end  

• Total Runtime =  
j +å ( j - 1) + å
n n n
n + n - 1+ n - 1+ n - 1+ å ( j - 1) + n - 1
j=2 j=2 j=2

23

 
 Insertion Sort
j +å ( j - 1) + å
n n n

• Total runtime= n + n - 1+ n - 1+ n - 1+ å ( j - 1) + n - 1
j=2 j=2 j=2

n(n +1) æn(n - 1) ö n n(n +1)

- 1+ 2ç ÷+ 5n - 4 å j= -1
2 è 2 ø j=2 2
 2
n +n n n(n - 1)
- 1+ n 2 - n + 5n - 4 å j=2
( j - 1) =
2
2  
3 2 9
n + n- 5
2 2  
Q (n 2 )

• Insertion sort has “quadratic runtime” or

 
24