219

‫ةد ا م يف تارضاحم‬:

‫تانايبلا لكايهو تايمزراوخال‬

Algorithms and Data
Structures

Lecture # 10

Sorting

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Sorting 220

Outline
:Sorting •
.Introduction 

:Simple sorting algorithms •

.Bubble Sort 

.Selection Sort 

.Insertion Sort 

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Sorting 221

Sorting
• Sorting is the process of rearranging a given set of objects in a
specific order (ascending or descending).

• There are many commonly-used sorting algorithms.

 Some of them are good for sorting small sets of data, while
others are good for sorting large sets of data.

 Some are simple, others are a bit more complicated.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Sorting 222

Sorting (cont…)
• In this article, you'll learn about:

 Simple sorting algorithms which are good for sorting small

sets of data (but slow) such as:

 Bubble Sort.

 Selection Sort.

 Insertion Sort.

 A bit more complicated sorting algorithms which are good

for sorting large sets of data such as:

 Merge Sort.

 Quick Sort.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Sorting 223

Sorting (cont…)
• The goal is to come up with better and more efficient sorts.

• Because sorting a large number of elements can be extremely time

consuming, a good sorting algorithm is very desirable.

• Measuring performance for different sort algorithms:

1) Space/storage requirements:
 Additional/temporary/intermediate arrays required?

2) Algorithm running time:

 Number of compares, moves, swaps of data items
required.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort 224

Bubble Sort )1(

• Bubble sort orders a list of values by repeatedly comparing
neighboring (adjacent) elements and swapping their positions if
necessary.

• Bubble sort algorithm:

 Scan the list, exchanging adjacent elements if they are not in

relative order; this bubbles the highest value to the end.

 Scan the list again, bubbling up the second highest value.

 Repeat until all elements have been placed in their proper
order.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 225

Bubbling" largest element"

• Traverse a collection of elements:

 Move from the front to the end.

 "Bubble" the largest value to the end using pair-wise comparisons
and swapping.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 226

Bubble Sort Example 1

1 2 3 4 5 6
:Original 77 42 35 12 101 5

1 2 3 4 5 6
77Sw ap4
42 77 35 12 101 5
2

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 227

Bubble Sort Example 1 (cont…)

1 2 3 4 5 6
Sw 77 ap3
42 35 77 12 101 5
5

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 228

Bubble Sort Example 1 (cont…)

1 2 3 4 5 6
Sw 77 ap1
42 35 12 77 101 5
2

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 229

Bubble Sort Example 1 (cont…)

1 2 3 4 5 6
42 35 12 77 101 5

No need to swap

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 230

Bubble Sort Example 1 (cont…)

1 2 3 4 5 6
ap 5
42 35 12 77 5 Sw 101
101

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 231

Bubble Sort Example 1 (cont…)

1 2 3 4 5 6
42 35 12 77 5 101

Largest value correctly placed

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 232

Bubble Sort Example 2

Original: 25 57 48 37 12 92 86 33

Pass 1: 25 57 48 37 12 92 86 33
33 86 92 12 37 57 48 25
33 86 92 12 57 37 48 25
33 86 92 57 12 37 48 25
33 92 86 57 12 37 48 25
92 33 86 57 12 37 48 25
.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 233

Bubble Sort Example 2 (cont…)

Pass 2: 25 48 37 12 57 86 33 92
92 33 86 57 12 48 37 25
92 33 86 57 48 12 37 25
92 86 33 57 48 12 37 25

Pass 3: 25 37 12 48 57 33 86 92
92 86 33 57 48 37 12 25
92 86 57 33 48 37 12 25
.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort example 234

Bubble Sort Example 2 (cont…)

Pass 4: 25 12 37 48 33 57 86 92
92 86 57 33 48 37 25 12
92 86 57 48 33 37 25 12

Pass 5: 12 25 37 33 48 57 86 92
92 86 57 48 37 33 25 12

Pass 6: 12 25 33 37 48 57 86 92

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Bubble Sort code 235

Bubble Sort code

void swap(int &x, int &y)
{
int temp = x;
x = y;
y = temp;
}

void
BubbleSort(int
a[])
{
for (int i =
0; i < size;
i++)
{
for (int j
= 1; j <
size - i;
j++)
{
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Bubble Sort runtime 236

Bubble Sort runtime

• Running time is the number of comparisons for input size n:
n1 ni n1

  1   (n  i)
i 0 j 1 i 0
n 1

  i
i 0

( n 21 ) n

 O(n2 )

 Number of actual swaps

performed depends on the data.

 Out-of-order data performs many

Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Bubble Sort 237

Bubble Sort summary

• Worst case: Inverse sorting
 Passes: (n - 1)
 Comparisons each pass: (n - k) , where k pass number.
 Total number of comparisons:
(n - 1) + (n - 2) + (n - 3) + … + 1 = n2/2 - n/2
= O(n2)

• Best case: already sorted

 O(n) : one pass over set, verifying sorting.

Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Bubble Sort 238

Bubble Sort summary (cont…)

• Total number of exchanges:

 Best case: None.
 Worst case: O(n2)
 Lots of exchanges: a problem with large items.

Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Selection Sort 239

Selection Sort )2(

• Selection sort orders a list of values by repeatedly putting a particular
value (smallest or largest) into its final position.

• Selection sort algorithm:

 Find the smallest value in the list.

 Switch it with the value in the first position.
 Find the next smallest value in the list.
 Switch it with the value in the second position.
 Repeat until all values are in their proper places.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Selection Sort example 240

Selection Sort Example 1

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Selection Sort example 241

Selection Sort Example 2

Figure 10.1: Example of Selection Sort (Sorted elements are shaded)

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Selection Sort example 242

Selection Sort Example 3

.Find the largest value in the list, switch it with the value in the last position 

Initial: 33 86 92 12 37
9248 57 25
:Pass 1 I 92 86 33 12 37 48
8657 25
:Pass 2 I 86 92 33 12 37 48 57 25
57
:Pass 3 I 5748
86 92 12 37 48 33 25
:Pass 4 I 48 5737
86 92 37 12 33 25
:Pass 5 I33
37 48 57 86 92 12 33 25
:Pass 6 25 I 33 37 48 57 86 92 12 25
:Pass 7 I 25 33 37 48 57 86 92 12
.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Selection Sort code 243

Selection Sort code

void SelectionSort(int a[])
{
for (int i = 0; i < size; i++)
{
// find index of smallest element
int min = i;
for (int j = i + 1; j < size; j+
+)
{
if (a[j] < a[min])
{
min = j;
}
}

// swap smallest element with a[i]

swap(a[i], a[min]);
}
}
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Selection Sort runtime 244

Selection Sort runtime

• Running time for input size n:

 in practice, a bit faster than bubble sort.

n1 n n1

  1  ( n  (i  1)  1)
i0 j i1 i0
n 1

  (n  i)
i0
n1
 i
i0

( n 21 ) n

 O(n2 )
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Selection Sort runtime 245

Selection Sort runtime (cont…)

• Note that the particular arrangement of values in the array does not affect
the amount of work done.

• Even if the array is in sorted order before the call to SelectionSort,

the
function still makes n(n-1)/2 comparisons.

Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Selection Sort example 246

Selection Sort summary

• Best case: Already sorted
 Passes: (n - 1)
 Comparisons each pass: (n-k) where k pass number.
 Number of comparisons:
(n - 1) + (n - 2) + (n - 3) + … + 1 = n2/2 - n/2
= O(n2)

• Worst case: Same.

• Number of exchanges:
 Always (n - 1) (better than Bubble Sort).
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Insertion Sort 247

Insertion Sort )3(

• Insertion sort orders a list of values by repeatedly inserting a particular
value into a sorted subset of the list.

• Selection sort algorithm:

 Consider the first item to be a sorted sublist of length 1.

 Insert the second item into the sorted sublist, shifting the first item
if needed.
 Insert the third item into the sorted sublist, shifting the other
items as needed.
 Repeat until all values have been inserted into their proper
positions.
.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Insertion Sort 248

Insertion Sort (cont…)

• Simple sorting algorithm.

 (n - 1) passes over the array.

 At the end of pass i, the elements that occupied A[0] … A[i]
originally are still in those spots and in sorted order.

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Insertion Sort example 249

Insertion Sort Example 1

Original: 2 15 8 1 17 10 12 5
0 1 2 3 4 5 6 7

2 15 8 1 17 10 12 5
0 1 2 3 4 5 6 7

Pass 1: 2 15 8 1 17 10 12 5
0 1 2 3 4 5 6 7

Pass 2: 2 8 15 1 17 10 12 5
0 1 2 3 4 5 6 7
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Insertion Sort example 250

Insertion Sort Example 1 (cont…)

Pass 3: 1 2 8 15 17 10 12 5
0 1 2 3 4 5 6 7

Pass 4: 1 2 8 15 17 10 12 5
0 1 2 3 4 5 6 7

Pass 5: 1 2 8 10 15 17 12 5
0 1 2 3 4 5 6 7

Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Insertion Sort example 251

Insertion Sort Example 1 (cont…)

Pass 6: 1 2 8 10 12 15 17 5
0 1 2 3 4 5 6 7

Pass 7: 1 2 5 8 10 12 15 17
0 1 2 3 4 5 6 7

Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Insertion Sort example 252

Insertion Sort Example 2

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Insertion Sort example 253

Insertion Sort Example 3

Figure 10.3: Example of Insertion Sort (Sorted elements are shaded)

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Insertion Sort code 254

Insertion Sort code

void InsertionSort(int a[])
{
for (int i = 1; i < size; i++)
{
int temp = a[i];

// slide elements down to make room for a[i]

;int j = i
while (j > 0 && a[j - 1] { )temp >
; a[j] = a[j - 1]
;--j
}

;a[j] = temp
}
}

.Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept Algorithms and Data Structures: Lecture (10)
Insertion Sort runtime 255

Insertion Sort runtime

• Worst case: reverse-ordered elements in array.
n1

i 1 2
 O(n2 )

• Best case: array is in sorted ascending order.

n 1

 1  n  1  O (n )
i1

• Average case: each element is about half way in order.

n1

i 1 2 2 4
 O(n2 )
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Insertion Sort 256

Insertion Sort summary

• Best case: Already sorted  O(n)
comparisons.

• Worst case: O(n2) comparisons.

• Number of exchanges:
 Best case: O(n).
 Worst case: O(n2).

• In practice, best for small sets (<30 items).

• Very efficient on nearly-sorted inputs.
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)
Comparing Simple Sorts 257

Comparing Simple Sorts

• We've seen "simple" sorting algorithms, such as:

 Bubble sort, Selection sort, and Insertion sort.

Comparisons Swaps

Worst n(n-1)/2 n(n-1)/2

Bubble
Best n-1 n-1

Selection n(n-1)/2 n-1

Worst n(n-1)/2 n(n-1)/2

Insertion
Best n-1 n-1

• They all use nested loops and perform approximately n2 comparisons.

• They are relatively inefficient.
Dr. Asim Abdallah Elshiekh, UofK, Fac. of Math. Sc., Computer Dept. Algorithms and Data Structures: Lecture (10)