CMPT 225

Lecture 9 – Simple Sorting Algorithms

1
 Learning Outcomes

 At the end of the next few lectures, a student will be able to:
 describe the behaviour of and implement simple sorting
algorithms:
selection sort
insertion sort
 describe the behaviour of and implement more efficient sorting
algorithms:
quick sort
merge sort
 analyze the best, worst, and average case running time (and
2 space) of these sorting algorithms
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Last Lecture

 We saw how to …
 Describe Queue
 Define public interface of Queue ADT
 Design and implement Queue ADT using various data
structures (CDTs)
 Compare and contrast these various implementations
using Big O notation
 Give examples of real-life applications (problems) where
we could use Queue to solve the problem
3  Solve problems using Queue ADT
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Today’s menu

 Quiz 1 this F Feb. 2

 Information about our Quiz 1 posted on our course web
site
Please read this info before Quiz 1 as it will tell you what to
study
 Quick Review of Searching algorithms
 Simple sorting
Selection sort
Insertion sort
 Analyze their best, worst, and average case running time
and space efficiency of these sorting algorithms
4

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Review – Searching Algorithms

1. One of the worst case scenarios of the linear search

algorithm would be: looking for target _____________ in
this array
5 7 2 1 8 9 11 3 4 15 6

a. 15
b. 5
c. 9
d. 12
e. None of the above
5

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Review – Searching Algorithms

2. The best case scenario of the binary search algorithm

would be: looking for target _____________ in the array
2 5 8 9 11 23 24 35 56 78 89

a. 2
b. 23
c. 89
d. 100
e. None of the above

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Review – Searching Algorithms

3. A worst case scenario of the binary search algorithm

would be: looking for target _____________ in the array
2 5 8 9 11 23 24 35 56 78 89

a. 2
b. 23
c. 89
d. 100
e. None of the above

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Review – Searching Algorithms

4. A worst case scenario of the binary search algorithm

would be: looking for target _____________ in the array
5 7 2 1 8 9 11 3 4 15 6

a. 15
b. 5
c. 9
d. 12
e. None of the above

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Review – Searching Algorithms
5. Modify the binary search algorithm below such that it
can quickly tell if target is not in the array, i.e., in O(1)?

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Review – Searching Algorithms
6. How can we tweak the linear search algorithm such that it …
… finds the first occurrence of the target in the array?
… finds the last occurrence of the target in the array?
… finds all occurrences of the target in the array?
7. What else can it return aside from the target itself?

if there is one
10

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Time/Space Efficiency

 Linear search
Time efficiency of worst case scenario: O(n)
Space efficiency: O(1)
 Binary search
Time efficiency of worst case scenario: O(log2 n)
Space efficiency: O(1)

11
 Binary Search vs Linear Search

 Advantages:
1. Binary search is much faster than linear search especially
when searching large data because it does not have to
look at every element (at every iteration, it ignores ½ of
data being searched).
 Disadvantages:
1. A bit more complicated to implement and test.
2. Data structure must be sorted.
Great if data is already sorted, but if this is not the case …

12 How much work does this sorting requires?

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Why Sorting?
 Definition: Process of placing elements in a particular sort
order based on the value of a/some search key(s).
 Ascending/descending sort order
 Why sorting?
 Easier to deal with sorted data: easier to search (e.g. binary
search)
 Common operation but time consuming
 What can be sorted?
 Internal data (data fits in memory)
 External data (data that must reside on secondary storage)
13
 How to sort?
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Selection Sort

14
 How Selection Sort works

 Initially, the array has n elements and the entire array is

considered unsorted
So, the actual
 Start with first element (at index 0) sorting is done
when we select
 Until the array is sorted an element.
1. Find (i.e., select) the smallest (or largest) element
in the unsorted section of array
This is done by comparing one element with all other elements
2. Swap it with the first element in the unsorted section of array
15 The sorted section of array has just grown by one element

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Demo - Let’s have a look at Selection Sort

16

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Space Efficiency
Analysis Selection Sort is an in place algorithm
 in-place: algorithm does not require additional space i.e., another
array(s) aside from the original array
 Selection sort starts with an unsorted array
 The start:
 As the array is being sorted, the unsorted section decreases and the
sorted section increases:

17  The end:
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Time Efficiency Analysis of Selection Sort

Unsorted Number of Number of

elements comparisons swapping
required to select
either the smallest or
the largest
n n-1 1
n-1 n-2 1
… … …
3 2 1
2 1 1
1 0 0
18
n(n-1)/2 n-1
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Time Efficiency Analysis of Selection Sort

Note that  In selection sort …

comparing n – 1 1. … makes n – 1 comparisons
times and Done in
iterating n – 1 2. … performs 1 swap (i.e., 3 assignments) sequence
time is the most
amount of work
-> max[ O(n – 1), O(1) ] = max[ O(n), O(1) ] = O(n)
selection sort
does when
sorting data  Then 1. and 2. are done in sequence n – 1 times
-> O(n – 1) * O(n) = O(n) * O(n) = O(n * n) = O(n2)

19

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Time Efficiency Analysis of Selection Sort
 Is O(n2) the time efficiency of the best, worst or average case
scenario?
 Would the way the data is organized affect the number of
operations selection sort perform (affect its time efficiency)?
 For example:
If the data was already sorted (in the desired sort order, e.g.,
ascending)?
If the data was sorted but in the other sort order (e.g.,
descending)?
If the data was unsorted?
20
Let’s check it out! ☺ https://www.toptal.com/developers/sorting-algorithms
 Summary – Selection Sort Algorithm

 The way the data is organized does not affect the

number of operations selection sort perform, i.e., does
not affect its time efficiency
 Time efficiency
Best case scenario: O(n2)
Worst case scenario: O(n2)
Average case scenario: O(n2)
 Space efficiency
in-place sorting algorithm => O(1)
21

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Insertion Sort

22
 How Insertion Sort works
 Array has n elements
 At the start, insertion sort considers the first element of the array to be
already sorted -> sorted section
 Starts with the second element (at index 1)
 Repeat until the array is sorted
1. Pick the first element of the unsorted section and store it in a temp
variable
2. Comparing it with each element of the sorted section (starting with
the last element and moving towards the first element), looking for
So, the where in this sorted section it should be placed (i.e., in its proper sort
actual sorting order)
is done when 3. Shift the elements in the sorted section up one position (starting with
we insert the last element of sorted section all the way down to the desired
an element.
23 place) to make space for this element (if needed)
4. Insert it in this newly vacated place in the sorted section
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Space Efficiency
Analysis
Insertion Sort is an in place algorithm
 in-place: algorithm does not require additional space i.e.,
another array(s) aside from the original array
 Insertion sort starts with a sorted section of 1 element:
 The start:
 As the array is being sorted, the unsorted section decreases
and the sorted section increases:

…
24
 The end:
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Demo - Let’s have a look at Insertion Sort

25

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Time Efficiency Analysis of Insertion Sort

Number of Worst case

Number of Best case
elements in Comparison elements
Shift in Comparison Shift
Sorted Sorted
section section
1 1 11 1 0
2 2 22 1 0
… … …… 1 0
n-1 n-1 n-1
n-1 1 0
n(n-1)/2 n(n-1)/2
1 n-1 0
26

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Time Efficiency Analysis of Insertion Sort
 Time efficiency of insertion sort is affected by the way data
is organized in the array to be sorted

 As we saw on the previous slide, the best case scenario

Requires a total of n - 1 comparisons
Requires no shifting

 Activity:
How should the array be organized in order to achieve
the best case scenario of the time efficiency of insertion
sort?
27
Give an example of such array:

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Time Efficiency Analysis of Insertion Sort
 As we saw on the slide (two slides ago), in the worst case scenario
 Every element in sorted section of array is compared with the
element currently being sorted
 Every element in sorted section of array has to be shifted to make
space for the element currently being sorted

 The outer loop runs n-1 times

 In the first iteration, one comparison and one shift is executed
… Nested
 In the last iteration, n-1 comparisons and n-1 shifts are executed
 For every element: (n-1) * (n-1 comparisons and n-1 shifts)
Nested: O((n-1) * max(O(n-1), O(n-1)))
= O((n) * max(O(n), O(n))
28 = O(n) * O(n)
= O(n2)
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 Time Efficiency Analysis of Insertion Sort

 Activity:
How should the array be organized in order to
achieve the worst case scenario of the time
efficiency of insertion sort?
Give an example of such array:

29
 Summary – Insertion Sort Algorithm

 Time efficiency
Best case scenario: O(n)
Worst case scenario: O(n2)
Average case scenario: O(n2)
 Space efficiency
in-place sorting algorithm => O(1)

30

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Stability and Sorting Algorithm

 The stability of a sorting algorithm is concerned with how

the algorithm treats equal (or duplicated) elements.
 Stable sorting algorithms preserve the relative order of
equal elements, while unstable sorting algorithms don’t.
In other words, stable sorting maintains the position of
two equals elements relative to each other.

31
Source: https://www.baeldung.com/cs/stable-sorting-algorithms
 Stability - Example from wiki
 Here is an example of stability of sorting algorithm using
playing cards. When the cards are sorted by rank using a
stable sorting algorithm, the two 5’s remain in the same order
(in relation to each other) in the sorted output as they were in
originally. When they are sorted using a non-stable sorting
algorithm, the 5’s may end up in the opposite order (in
relation to each other) in the sorted output:

32

Source: https://en.wikipedia.org/wiki/Sorting_algorithm
 When does stability matter?
 Data sorted using a stable sort algorithm:

 Data sorted using an unstable sort algorithm:

33
Source: https://www.freecodecamp.org/news/stability-in-sorting-algorithms-a-treatment-of-equality-fa3140a5a539/
 Conclusion – Simple Sorting Algorithms
 Insertion sort versus Selection sort
Efficient: for small n’s
More efficient in practice than most other simple
quadratic (i.e., O(n2)) algorithms
Stable: does not change the relative order of elements
with equal keys
 Both sorting algorithms: Selection sort and Insertion sort
In-place: only requires a constant amount of additional
memory space, i.e., O(1)
Both are from a class of sorting algorithm called
34 Comparison sort
Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University
 √ Learning Check
 Quick Review of Searching algorithms
 Sorting
Selection sort
Insertion sort
 Analyze their best, worst, and average case running time
and space efficiency of these sorting algorithms

35

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University

 Next Lecture

 More efficient sorting algorithm -> Quick sort

36

Copyright © Anne Lavergne, School of Computing Science, Simon Fraser University