Data Structures and

Algorithms
(CMP 4272)
Lecture 8 – Sorting
Outline
• Sorting
• Bubble Sort
• Selection Sort
• Insertion Sort
• Quick Sort
• Merge Sort
• Heap Sort
Sorting
• Put simply, sorting requires us to put things in order; either
• in-place or
• out-of-place.

• In-place sorting takes a list or other ADT as input where items must support
a comparison operation [where they can be compared numerically like
integers or alphabetically like strings]. In-place sorting does not have an
output.

• Out-of-place sorting creates a new list to contain sorted items for output
without modifying the input list.
Sorting in Python
• Python has a built-in sorting operation for lists called
Timsort.
• We’ll learn more about this later, but for the exercises in this
lesson use this operation only to test your own code.
Let’s Do It:
1 2 3 4 5
Sort this list yourself – by hand or on a computer, but do the
• Sweep • Select • Sort the • Split the • Pick a
sorting yourself:
through the leftmost list, sort value
the list min/max item into it then and
and sort and a new bring the pivot
[ 5, 13, 7, 1, 0, 3, 4, 19, 8, 7]
numbers
one at a
remove
them
list and
remove
sorted
sub-lists
others
around
time in from the from the back this as
place list one original together lower /
Most importantly, by oneyour
show list
working so you knowhigher
how you
did it.
 These are Standard Sorting Algorithms

Selection Insertion
Bubble Sort Merge Sort Quicksort
Sort Sort
• Sweep • Select the • Sort the • Split the • Pick a
through min/max leftmost list, sort it value and
the list and and item into a then bring pivot
sort remove new list the sorted others
numbers them from and sub-lists around this
one at a the list one remove back as lower /
time in by one from the together higher
place original list
Sorting Algorithms
We’re going to be looking at 6 main sorting algorithms before
seeing how the built-in Python algorithm works.

Sorting
Algorithms

Bubble Selection Insertion Merge

QuickSort Heap Sort
Sort Sort Sort Sort
Swapping in Python
• Directly swapping values at two different
indices of an array or list in Python is
impossible.
• If we try and swap the values at index 1 and 2
then when we copy the value at index 1 to
index 2 we’ve overwritten the 2nd value so
can’t copy this back to index 1.

10 23 97 10 23 23 10 23 23
Swapping in Python
• Swapping values requires us to use a temporary variable to hold the swap value
to then insert or set it.

10 23 97 10 23 23 10 97 23

97

• This can be hardcoded with an auxiliary variable getting python to do this for you:

arr[i], arr[j] = arr[j], arr[i]

Bubble Sort: Brute Force
• Bubble Sort is a very basic sorting algorithm that works by
swapping two adjacent elements if they are in the wrong order.

• This is done until the algorithm makes a full sweep through the
list and repeated until a full sweep occurs without swapping any
elements.
Bubble Sort Example: [5,1,4,2,8]
First Pass:
[ 5 1 4 2 8 ] –> [ 1 5 4 2 8 ]: 5 > 1 so swap.
[ 1 5 4 2 8 ] –> [ 1 4 5 2 8 ]: 5 > 4 so swap.
[ 1 4 5 2 8 ] –> [ 1 4 2 5 8 ]: 5 > 2 so swap.
[ 1 4 2 5 8 ] –> [ 1 4 2 5 8 ]: 5 < 8 so don’t swap.

Second Pass:
[ 1 4 2 5 8 ] –> [ 1 4 2 5 8 ]: 4 > 1 so don’t swap.
[ 1 4 2 5 8 ] –> [ 1 2 4 5 8 ]: 4 > 2 so swap.
[ 1 2 4 5 8 ] –> [ 1 2 4 5 8 ]: 4 < 5 so don’t swap.
[ 1 2 4 5 8 ] –> [ 1 2 4 5 8 ]: 5 < 8 so don’t swap.

Third Pass (check):

[ 1 2 4 5 8 ] –> [ 1 2 4 5 8 ]: 1 < 2 so don’t swap.
[ 1 2 4 5 8 ] –> [ 1 2 4 5 8 ]: 2 < 4 so don’t swap.
[ 1 2 4 5 8 ] –> [ 1 2 4 5 8 ]: 4 < 5 so don’t swap.
[ 1 2 4 5 8 ] –> [ 1 2 4 5 8 ]: 5 < 8 so don’t swap.
Selection Sort

• In this sorting algorithm the list is divided into two parts:

• the sorted part on the left end and
• the unsorted part on the right.
• Initially, the sorted part is empty, and the unsorted part is the entire list.
• The smallest element is selected from the unsorted list and swapped with the
leftmost element, then that element becomes a part of the sorted list.
• This process continues moving the unsorted list boundary by one element to the
right.
Selection Sort
Selection Sort
The following are the remaining
steps in the selection sort
algorithm. This is the in-place
version of the selection sort.
Think of how this generalises to an
algorithm.

Selection Sort
Selection Sort Algorithm Steps
1. Set min_index to location 0
2. Find the minimum element in the list
3. Swap with value at location min_index
4. Increment min_index to point to next element
5. Repeat until list is sorted
Bubble Sort vs. Selection Sort
When can bubble sort be more efficient (having less operations)
than selection sort?
Selection Sort Out-of-Place
1. Create a new empty list lst_new
2. Find the minimum element in the original list
[lst_origin]
3. Append the minimum element to list_new.
4. Remove it from lst_origin.
5. Repeat until lst_origin is empty.
Insertion Sort
• Insertion Sort is typically performed in-place by maintaining a sub-list of
sorted elements.
• Each element needs to be inserted into the sorted sub-list [usually on the
left of a list], has to find its appropriate place and is then inserted.
Insertion Sort
• 14 is already in the sorted sub-list.
• We then check where 33 goes, but
it’s already in the correct place.
• Next, we must check 27 and move
this to the middle of the sub-list, etc.

• Continue until each element is

placed at its appropriate place in the
sorted sub-list
10 14 27 33 35 19 42 44
10 14 27 33 35 19 42 44
10 14 19 27 33 35 42 44
10 14 19 27 33 35 42 44
10 14 19 27 33 35 42 44
Insertion Sort Algorithm Steps
1. Add the first element to the sorted sub-list
2. Pick the next unsorted element
3. Compare to all elements in the sorted sub-list
4. Shift all elements in the sorted sub-list that are greater than the
value to be sorted [swap]
5. Repeat steps 2-4 until list is sorted
Insertion Sort Out-of-Place
1. Create a list called list_sort containing the first element
2. Pick the next unsorted element
3. Compare to all elements in list_sort
4. Shift all elements in list_sort that are greater than the value to be
sorted [swap]
5. Repeat steps 2-4 until list is sorted
Insertion Sort Out-of-Place
What’s the difference?

• Selection sort finds the ‘right’ item for each location.

• Insertion sort finds the ‘right’ location for each item.

Selection Insertion
Merge Sort: Divide & Conquer
• For long lists Bubble Sort, Insertion Sort and Selection Sort are very
slow and inefficient due to the size of the internal loops.
• These iterative algorithms are deficient compared to recursive
sorting algorithms such as Merge Sort.
• Rather than sorting a single large list, this algorithm recursively
sorts shorter lists and combines the outputs. This is an out-of-place
sorting algorithm.

1) Split the list data apart into two shorter lists

2) Sort the shorter lists recursively
3) Combine the sorted lists into a single sorted list (Merge)
 Merge Sort

• Firstly, we split the list (or

array) into a set of smaller
lists as per step 1 of the
algorithm.
 Merge Sort
• Then we sort each smaller list as
per step 2 of the algorithm, and
finally merge these into a sorted
list.
Merge Sort Code
QuickSort: Divide & Conquer
• Merge Sort works by the process of divide and conquer allowing for the
algorithm to solve a series of smaller problems before combining to
solve a larger problem.
• Quicksort does the same, but instead of arbitrarily splitting lists in half, it
sorts elements into a tree using pivots.
• There are 4 common ways of choosing pivots that effect sorting speed:

1. Choose the first element

2. Choose the last element
3. Choose a random element
4. Choose the median element
QuickSort Example
The example here uses the last element of a list as a pivot to partition the data into a tree.
The sorting then just involves collating the subtree leaves from left, parent and right nodes.
[L-P-N] (In order)
QuickSort Example: The Tree

70

50 80

40 [] [] 90

30 []

10 []
QuickSort Code

• This code performs the tree

partition first and recursively
calls this in the actual
quicksort function. The
actual code can be copied
from the notes on this slide.
Heap Sort: Divide & Conquer

• Heap Sort, like QuickSort, creates a tree to partition the

data, but instead of arbitrarily choosing the pivot point,
the constructed tree satisfies the Heap property.
• A heap is a special type of binary tree that satisfies the
heap property, which ensures a specific order among
parent and child nodes. There are two types of heap
properties:

Max-Heap Property:
• In a max-heap, for every node i, the value of the
parent node is greater than or equal to the values of
its children.
• The largest element is always at the root.

Min-Heap Property:
• In a min-heap, for every node i, the value of the
parent node is less than or equal to the values of its
children.
• The smallest element is always at the root.
Heap Sort - Example
• Once the initial Max Heap is constructed the root node
will be the largest value so this is removed from the tree.

• Then, the right-most leaf node is placed at the root of

the tree as a placeholder and then replaced with the
next largest value which will be at level 1.
Heap Sort Code
• The Heap Sort algorithm in
Python is split into two separate
functions, the actual heapsort(.)
function that recursively calls the
heapify(.) function to re-organise
the Max Heap.
• The actual code can be copied
from the notes on this slide.
How Does Python Sort Data?
• Python’s built-in sorting algorithm list.sort() is very efficient.
• If the array length is less than 64 then the built-in sort uses the Insertion Sort algorithm.
• For arrays greater than 64 elements the Timsort algorithm is used.
• Timsort is named after Tim Peters, a programmer who created the algorithm in 2001.
• There are a few different parts to the algorithm, but on first pass it checks for runs in
data that are in increasing or decreasing order and don’t require (much) sorting for
example: [9, 17, 1, 2, 3, 34, 56] contains the run 1, 2, 3.
• Timsort then merges runs based on the number of elements using a stack and checks
for winning runs using galloping.
Testing
Apart from Bubble Sort you’ve been provided with the code for the rest of
the algorithms.
The question is: which is best? Run tests using the below situations and
compare results after a few independent runs:

1) check they all work with: arr = [64, 34, 25, 12, 22, 11, 90]
2) import time and use t = time.time() and t2 = time.time() and t2-t will tell
you how many seconds the sort took.
3) import copy and import numpy as np and use the following commands:
lst = np.random.permutation(1000) lst_bubble = copy.deepcopy(lst) to
test on a large array.
THANK YOU