Chapter 2: Sorting Algorithms

2.1. Introduction

In computer science, sorting algorithms play a critical in data processing and retrieval. Their
significance is rooted in their ability to efficiently organize and manage data, enhancing the overall
efficiency of various operations. Beyond their efficiency, sorting algorithms find utility in numerous
aspects of computing:

 Efficiency: Sorting algorithms are pivotal for organizing data efficiently, making searches faster
and facilitating operations like binary search.

 Data Retrieval: In databases and information systems, sorting accelerates data retrieval,
especially through sorted indexes, reducing the time required to find specific records.

 Search Engines: Sorting is vital for search engines, enabling them to deliver precise and timely
results. When web pages are systematically indexed and sorted according to relevance, users
benefit from more accurate search outcomes.

 Data Analysis: Sorting is essential in data analysis tasks, facilitating the identification of trends,
outliers, and patterns within data, crucial in domains like finance, healthcare, and marketing.

The areas and uses of sorting algorithms are not limited to these reasons alone. Sorting algorithms have
a wide range of importance in various domains and applications, including efficiency, data retrieval,
optimizing algorithms, user experience, search engines, data analysis, resource allocation, real-time
systems, security, decision making, etc.

In the context of this chapter, we explore diverse sorting algorithms. This exploration allows us to gain a
comprehensive understanding of how different sorting algorithms function and the various factors that
impact their efficiency and effectiveness.

12
 2.2. Bubble Sort

a) Principle:
Bubble Sort, also known as Exchange Sort, is a simple sorting algorithm that works by repeatedly
traversing the list to be sorted. It compares two items at a time and swaps them if they are in the wrong
order. This process continues until no more swaps are needed, indicating that the list is sorted. It's
considered the easiest method among all sorting algorithms.

h) Code

The simple implementation of bubble sort is given by the following c program:

Code 2.1 : Bubble sort
void bubbleSort(int A[], int n) {
for (int i = 0; i < n - 1; i++) {
for (int j = 0; j < n - i - 1; j++) {
if (A[j] > A[j + 1]) {
// Swap A[j] and A[j + 1]
int temp = A[j];
A[j] = A[j + 1];
A[j + 1] = temp;
}
}
}
}

i) Complexity
There are two nested loops. The outer loop runs for (n-1) passes through the array, and the inner loop
compares and potentially swaps elements. In the worst case, the inner loop makes (n-1) comparisons
during the first pass, (n-2) comparisons during the second pass, (n-3) comparisons during the third pass,
and so on. The total number of comparisons is :

(n-1) + (n-2) + (n-3) + ... + 2 + 1 = (n-1)*n/2

Therefore, the bubble sort algorithm encompasses a worst case time complexity of O(n2).
In the best Case, The bubble sort algorithm has a time complexity of O(n) for the already sorted array.

Problem: In the provided code above, the best-case time complexity stands at O(n2). Make
adjustments to achieve a best-case time complexity of O(n).

13
 j) Advantages and limits
The time complexity in the worst is the main limitation of the algorithm and this inefficiency making it
impractical for sorting large datasets. Furthermore, it does not necessitate any extra memory.
In the other hand, Bubble Sort has some advantages. :
 Easily understandable.
 Does not necessitate any extra memory.

2.3. Selection Sort

a) Principle:
The selection sort enhances the bubble sort by making only a single swap for each pass through the
rundown. In order to do this, a selection sort searches for the smallest value as it makes a pass and, after
finishing the pass, places it in the left place. Similar to Bubble Sort, after the first pass, the smallest item
is correctly positioned. This process continues, requiring n-1 passes to sort n items since the last item
must be positioned correctly after the (n-1)th pass.

k) Code

In c language, the selection sort can be provided as below:

Code 2.2 : Selection sort

void selectionSort(int A[], int n) {
int minIndex, temp;
for (int i = 0; i < n - 1; i++) {
minIndex = i;
for (int j = i + 1; j < n; j++) {
if (A[j] < A[minIndex]) {
minIndex = j;
}
}
// Swap A[i] and A[minIndex]
temp = A[i];
A[i] = A[minIndex];
A[minIndex] = temp;
}
}

14
 l) Complexity
The number of comparisons decreases sequentially in each pass. For example, during the first pass, it
performs n-1 comparisons, in the second pass, n-2 comparisons, and so on. Consequently, we can
determine the total number of comparisons by:

(n-1) + (n-2) + (n-3) + ... + 2 + 1 = (n-1)*n/2

So, the Selection Sort has a time complexity of O(n2) in the worst case. Even if the array is already
sorted, Selection Sort doesn't take advantage of this best case. It will still perform the same number of
comparisons and swaps as in the worst-case scenario, making its best case time complexity O(n²).

m) Advantages and limits

Advantages of Selection Sort:
 Simplicity: Selection Sort is one of the simplest sorting algorithms to understand and implement.
 Minimal Memory Usage: It doesn't require additional memory for sorting, making it efficient in
terms of space complexity.

Limits of Selection Sort:

 Selection Sort's time complexity of O(n²) makes it highly inefficient for large lists, as it involves
a significant number of comparisons and swaps.
 The algorithm's time complexity remains O(n²) in both best and worst cases, so it doesn't
perform well even when the list is partially sorted.

2.4. Insertion Sort

a) Principle:
Insertion Sort is simple sorting algorithms. It is not the best algorithm, but it's slightly more efficient
than selection sort and bubble sort.

The algorithm iterates through the unsorted segment, taking one element at a time and inserting it into its
correct position within the sorted segment. It does this by comparing the current element (key) with the
elements in the sorted segment. When it finds the appropriate position, it moves larger elements to the

15
right to make space for the element being inserted (key). This process repeats until all elements are in
their correct sorted positions (See Figure 5).

Figure 5: Insertion Sort principle

n) Code

The Insertion Sort can be give as follows:

Code 2.3 : Insertion sort

void insertionSort(int A[], int n) {
int key, j;
for (int i = 1; i < n; i++) {
key = A[i];
j = i - 1;
while (j >= 0 && A[j] > key) {
A[j + 1] = A[j];
j = j - 1;
}
A[j + 1] = key;
}
}

o) Complexity
In insertion sort, the number of comparisons decreases sequentially in each pass. During the first pass, it
performs only one comparison. In the second pass, it makes two comparisons, and so on. The total
number of comparisons can be calculated by summing up this decreasing sequence:

1 + 2 + 3 + ... + (n-3) + (n-2) + (n-1) = (n-1)*n/2

16
As a result, the time complexity of Insertion Sort is O(n²) in the worst-case scenario. However, in the
best case, when the array is already sorted, the time complexity is O(n), as it requires minimal
comparisons and no swaps.

p) Advantages and limits

Insertion sort offers simplicity in implementation and efficiency for small datasets. It is an in-place
sorting algorithm, which requires minimal extra memory (O(1)).
However, Insertion sort proves inefficient for sorting larger datasets and exhibits a worst-case time
complexity of O (n2). When compared to more advanced sorting algorithms, it may not perform as well.

2.5. Merge Sort

q) Principle:
Merge sort is a divide-and-conquer sorting algorithm that offers improved efficiency over simpler
sorting methods. It works by dividing the unsorted list into two equal halves, recursively sorting each
half, and then merging the sorted halves back together. During the merge step, it compares and
combines elements from the two halves, producing a single, sorted list. So, its principle involves the
following steps:

 Divide: The unsorted list is divided into two equal halves, or as close to equal as possible. This
division continues recursively until the list is broken down into its smallest components,
typically single elements. Each element can be considered a sorted list of one element.

 Conquer: After dividing the list, the sorting process begins. Merge sort recursively sorts each of
these smaller lists, combining them into sorted lists of larger and larger sizes. This is
accomplished by repeatedly dividing the list into two halves, sorting each half, and merging them
back together. The sorting of small lists is relatively straightforward, and the merging of sorted
lists into larger sorted lists is the key to merge sort's efficiency.

 Merge: During the merging step, elements from the two halves are compared and combined to
create a single, sorted list. This is where the algorithm demonstrates its efficiency. By comparing
and merging two sorted lists, merge sort avoids the need for extensive comparisons during the

17
 merging process. Elements are taken from the two lists and placed in the correct order in the
merged list. This process continues recursively until the entire list is sorted.

The figure 6 illustrates this principle through the use an example

Figure 6 : Illustrative example of using the merge sort principle

r) Code

The merge sort algorithm can be implemented as follows:

Code 2.3 : Merge sort

void merge(int A[], int left, int mid, int right) {
int n1 = mid - left + 1;
int n2 = right - mid;

int L[n1], R[n2];

for (int i = 0; i < n1; i++)

L[i] = A[left + i];
for (int j = 0; j < n2; j++)

18
 R[j] = A[mid + 1 + j];

int i = 0, j = 0, k = left;

while (i < n1 && j < n2) {

if (L[i] <= R[j]) {
A[k] = L[i];
i++;
} else {
A[k] = R[j];
j++;
}
k++;
}
while (i < n1) {
A[k] = L[i];
i++;
k++;
}
while (j < n2) {
A[k] = R[j];
j++;
k++;
}
}

void mergeSort(int A[], int left, int right) {

if (left < right) {
int mid = left + (right - left) / 2;

mergeSort(A, left, mid);

mergeSort(A, mid + 1, right);

merge(A, left, mid, right);

}
}

s) Complexity
Merge sort's time complexity is consistently efficient. It operates in O(n log n) time for all cases,
including the best, worst, and average scenarios. This makes it a reliable choice for sorting large datasets
efficiently.

t) Advantages and limits

Advantages:
 Merge sort can efficiently sort a list in O(n*log(n)) time.
 It can be used with linked lists without taking up any more space.
Limits:

19
  For small datasets, merge sort is slower than other sorting algorithms.
 For the temporary array, merge sort requires an additional space of O(n).
 Even if the array is sorted, the merge sort goes through the entire process.

2.6. Quick Sort

u) Principle:
Quick sort is a widely used sorting algorithm known for its efficiency. It employs a divide-and-conquer
strategy and is based on the selection of a 'pivot' element. The algorithm partitions the list into two
sublists, one with elements less than the pivot and another with elements greater than the pivot. This
process is repeated recursively for each sublist until the entire list is sorted.

Figure 7: Illustrative example of using the merge sort priniciple

b) Code

The quick sort algorithm can be implemented as follows:

20
 Code 2.3 : Quick sort
void swap(int* a, int* b) {
int temp = *a;
*a = *b;
*b = temp;
}

int partition(int A[], int low, int high) {

int pivot = A[high];
int i = (low - 1);

for (int j = low; j <= high - 1; j++) {

if (A[j] <= pivot) {
i++;
swap(&A[i], &A[j]);
}
}

swap(&A[i + 1], &A[high]);

return (i + 1);
}

void quickSort(int A[], int low, int high) {

if (low < high) {
int pi = partition(A, low, high);

quickSort(A, low, pi - 1);

quickSort(A, pi + 1, high);
}
}

c) Complexity
Worst Case: The worst-case scenario for Quick Sort occurs when the partition process consistently
selects either the largest or the smallest element as the pivot. For instance, if we follow the partition
strategy where the last element is always chosen as the pivot, the worst case arises when the array is
already sorted in ascending or descending order. In this case, the recurrence relation for the worst-case
time complexity is O(n2).

Best Case: The best-case scenario for any sorting algorithm occurs when there's only one element to
sort. In this situation, no comparisons between elements are necessary, as the array is already sorted.
Therefore, in the best case, Quick Sort exhibits a time complexity of O(n log n).

Average Case Analysis: The average-case time complexity of Quick Sort is O(n log n). This analysis
considers a typical scenario in which the pivot selection and partitioning lead to efficient sorting.

21
 d) Advantages and limits
Advantages of Quick Sort:
 Efficient Average Case: Quick sort offers excellent average-case performance, making it a
popular choice for sorting large datasets.
 In-Place Sorting: Quick sort sorts the data in-place without requiring additional memory.
 Faster than Merge Sort for Small Lists: In practice, quick sort can be faster than merge sort for
small lists due to its lower constant factors.
Limits of Quick Sort:
 In the worst case, when pivot selection is unbalanced, quick sort's time complexity can be O(n2),
making it inefficient for specific datasets.
 More Complex to Implement: Implementing quick sort can be more complex than simpler
sorting algorithms, and careful pivot selection is crucial to its performance.

2.7. Summary and comparison

In the comparison of sorting algorithms, the concepts of "in-place" and "stability" are two key
characteristics that can make one sorting algorithm more suitable for a specific use case than another.
In-place sorting means that the algorithm doesn't need to use additional memory or create a new copy of
the data it's sorting.
Stability in sorting means that when two elements in the original list have equal values, their relative
order is preserved in the sorted list.
Here's why these characteristics are relevant in the comparison of sorting algorithms:

a) In-Place
i. Memory Efficiency
Sorting large datasets can require a significant amount of memory if the algorithm needs to create a
separate copy of the data. In scenarios with limited memory resources, using an in-place sorting
algorithm can be crucial to avoid memory constraints.

ii. Performance
In-place algorithms typically use less memory, which can lead to better cache performance and reduced
data movement, resulting in faster sorting, especially for large datasets.

22
 b) Stability:

i. Preserving Order
In some applications, it's essential to preserve the original order of elements with equal values. Stable
sorting algorithms ensure that elements with the same value maintain their relative order in the sorted
list. This is important when sorting data by multiple criteria or when preserving a specific order, such as
first come, first served.

ii. Predictable Behavior

Stability provides predictability in sorting results. Users can rely on the fact that the algorithm won't
change the relative positions of equal elements. This predictability can be crucial in various applications,
like databases and user interfaces.

c) Comparison
In the following table, the five sort algorithms are compared using some important characteristics.

Table 3: summary comparison of the five sort algorithms

Time complexity In-place

Name stable Some advantages
best average worst (memory)
Simple Implementation
Bubble n n2 n2 Yes (1) yes
educational purposes
minimal additional memory
Selection n2 n2 n2 Yes (1) no
small datasets
good choice for real-time
Insertion n n2 n2 Yes (1) yes applications where data is
frequently updated.
Efficient for sorting large
Merge n log n n log n n log n No (n) yes
datasets.
Can be adapted for sorting data
Quick n log n n log n n2 No (log n) no
in parallel.

23
 2.8. Conclusion

In this chapter, we have explored five fundamental sorting algorithms: Bubble Sort, Selection Sort,
Insertion Sort, Merge Sort, and Quick Sort. Each of these algorithms offers unique characteristics and
trade-offs that make them suitable for different scenarios.

Bubble Sort, Selection Sort, and Insertion Sort, while not the most efficient sorting algorithms, are easy
to understand and implement. They can perform admirably on small datasets and are particularly
valuable when dealing with partially sorted data.

On the other hand, Merge Sort and Quick Sort provide efficient solutions for sorting larger datasets.
Merge Sort, known for its stability and consistent O(n log n) time complexity, excels in scenarios where
data must be sorted reliably. Quick Sort, with its adaptability for parallel processing and efficient
average-case performance, is often favored in practical applications.

The choice of which sorting algorithm to use should be influenced by the specific requirements of your
task. Consider the size of the dataset, its initial order, and the available memory when selecting a sorting
method. Remember that there is no one-size-fits-all solution, and it's essential to match the algorithm to
the problem at hand.

Sorting is a fundamental operation in computer science, and understanding these algorithms is a

valuable asset for any programmer. Whether you are optimizing for speed, memory usage, or ease of
implementation, the knowledge gained in this chapter will aid you in selecting the most suitable sorting
algorithm for your needs

24