Lecture 6

Merging two
sorted arrays

Merge Sort Advanced Analysis of Algorithms

Partitioning
arrays (Divide and Conquer II)
Quicksort

Randomization

The Selection
Problem
Abbas Khalid
Conclusion
COMSATS Institute of Information Technology
 Lecture 6

1 Merging two sorted arrays

Merging two
sorted arrays
2 Merge Sort
Merge Sort

Partitioning
arrays 3 Partitioning arrays
Quicksort

Randomization 4 Quicksort
The Selection
Problem
5 Randomization
Conclusion

6 The Selection Problem

7 Conclusion
 Merging two sorted arrays

Lecture 6

Merging two
sorted arrays
The merging of two sorted arrays can be done as follows:
Merge Sort 1 Two pointers are initialized to point to the first elements
Partitioning of the arrays to be merged.
arrays

Quicksort
2 The elements pointed to are compared and the smaller of
Randomization them is added to a new array.
The Selection 3 The index of the smaller element is incremented.
Problem

Conclusion 4 This process is repeated until end of any array is reached.

5 The remaining elements of the other array are copied to
the end of the new array.
 Example

Lecture 6

First List Second List

Merging two
14 20 36 10 12 30 40 44
sorted arrays 10
Merge Sort 14 20 36 10 12 30 40 44
Partitioning
arrays
10 12
Quicksort 14 20 36 10 12 30 40 44
Randomization 10 12 14
The Selection 14 20 36 10 12 30 40 44
Problem
10 12 14 20
Conclusion
14 20 36 10 12 30 40 44
10 12 14 20 30
14 20 36 10 12 30 40 44
10 12 14 20 30 36
10 12 14 20 30 36 40 44
 Algorithm

Lecture 6

Algorithm 1: Merges two sorted arrays into one sorted array (1)
Merging two
sorted arrays Input: B[0..p − 1], C [0..q − 1]: Two sorted arrays
Merge Sort Output: A[0..p + q − 1]: Merged array
Partitioning 1 i ← 0; j ← 0; k ← 0;
arrays
2 while i < p and j < q do
Quicksort
3 if B[i] ≤ C [j] then
Randomization

The Selection
4 A[k] ← B[i]; i ← i + 1;
Problem 5 else
Conclusion 6 A[k] ← C [j]; j ← j + 1;
7 end
8 k ← k + 1;
9 end
 Lecture 6

Algorithm 1: Merges two sorted arrays into one sorted array (2)
Merging two
sorted arrays 1 while i < p do
Merge Sort 2 A[k] ← B[i]; i ← i + 1;k ← k + 1;
Partitioning
arrays
3 end
Quicksort
4 while j < q do
Randomization 5 A[k] ← C [j]; j ← j + 1;k ← k + 1;
The Selection 6 end
Problem

Conclusion
Complexity
There are three independent loops to execute. Therefore, time
to execute merge is θ(n).
 Merge Sort

Lecture 6

was invented by John von Neumann in 1945.

Merging two sorts a given array by
sorted arrays

Merge Sort
1 dividing the array into two equal halves.
2 recursively sorting the first half of the input array.
Partitioning
arrays 3 recursively sorting the second half of the input array.
Quicksort 4 merging two sorted arrays into one.
Randomization Perfect example to understand the divide and conquer
The Selection
Problem
technique.
Conclusion Better performance than selection sort and bubble sort.

Demo
 Example

Lecture 6

Merging two
sorted arrays

Merge Sort

Partitioning
arrays

Quicksort

Randomization

The Selection
Problem

Conclusion
 Algorithm

Lecture 6
Algorithm 2: MergeSort(A[0..n − 1])
Input: A[0..n − 1]:unsorted array
Merging two Output: A[0..n − 1]: sorted array in ascending order
sorted arrays

Merge Sort
1 if n > 1 then
Partitioning
2 copy A[0..bn/2c − 1] to B[0..bn/2c − 1];
arrays 3 copy A[bn/2c..n − 1] to C [0..dn/2e − 1];
Quicksort
4 MergeSort(B[0..bn/2c − 1]);
Randomization
5 MergeSort(C [0..dn/2e − 1]);
The Selection
Problem 6 Merge(B, C , A);
Conclusion 7 end

Definitions
dxe: largest integer less than or equal to x e.g. d3.14e = 4.
bxc: smallest integer greater than or equal to x e.g.
b3.14c = 3.
 Analysis

Lecture 6

Merging two
sorted arrays
Consider the following recurrence T (n):
Merge Sort n
Partitioning T (n) = aT + θ(n) (b > 1) (1)
arrays b
Quicksort
In Merge sort,
Randomization

The Selection d = 1.
Problem
a = 2.
Conclusion
b = 2.
a = b d ⇒ T (n) = θ(nd log n) = θ(n log n).
 Partitioning an array around a pivot

Lecture 6

Partitioning of an array can be achieved as follows:

Merging two 1 Select an element (called pivot) around which partition
sorted arrays
will take place.
Merge Sort

Partitioning
2 Rearrange the array such that
arrays The values on the left of the pivot are smaller than pivot.
Quicksort The values on the right of the pivot are greater than pivot.
Randomization

The Selection
Problem Values < pivot pivot Values > pivot
Conclusion

Example
{3, 8, 2, 5, 1, 4, 7, 6} can be partitioned as {2, 1, 3, 6, 7, 4, 5, 8}
around the pivot 3.
 Partitioning using a naive method

Lecture 6

1 Select the first element of the array to be partitioned

Merging two (original array) as a pivot.
sorted arrays

Merge Sort
2 Take an auxiliary array of the same length as of original
Partitioning array. Point to the first and last indices of this array by a
arrays
pointers i and j respectively.
Quicksort
3 In original array, compare the second element with the
Randomization
pivot element.
The Selection
Problem If it is smaller than pivot, copy this element to the auxiliary
Conclusion array at position pointed to by i. Move i one position right.
If it is greater than pivot, copy this element to the auxiliary
array at position pointed to by j. Move j one position left.
4 Repeat step 3 for remaining elements in the original array.
5 Finally, copy the pivot element at the position i = j.
 Example

Lecture 6

Original array Auxiliary array

Merging two
sorted arrays
3 8 2 5 1 4 7 6 8
Merge Sort
3 8 2 5 1 4 7 6 2 8
Partitioning 3 8 2 5 1 4 7 6 2 5 8
arrays
3 8 2 5 1 4 7 6 2 1 5 8
Quicksort
3 8 2 5 1 4 7 6 2 1 4 5 8
Randomization
3 8 2 5 1 4 7 6 2 1 7 4 5 8
The Selection
Problem 3 8 2 5 1 4 7 6 2 1 6 7 4 5 8
Conclusion 3 8 2 5 1 4 7 6 2 1 3 6 7 4 5 8

# of comparisons: n − 1
# of swaps: 1
 A better approach

Lecture 6
1 Select the first element of the array to be partitioned as a
pivot.
Merging two
sorted arrays 2 Point to the second element of this array by two pointers;
Merge Sort i and j.
Partitioning
arrays
3 Compare the element pointed to by j with the pivot
Quicksort
element.
Randomization If it is greater than pivot, the element at position pointed
The Selection
to by j is already partitioned. Move j one position forward.
Problem If it is smaller than pivot, swap this element with the
Conclusion element pointed to by i. Move i and j one position
forward.
Do not increment j if it is pointing to the last element.
4 Repeat step 3 for the remaining elements in the array.
5 Finally, swap the pivot with the element pointed to by
i − 1.
 Example

Lecture 6
3 8 2 5 1 4 7 6
i, j
Merging two
sorted arrays
3 8 2 5 1 4 7 6
Merge Sort
i j
Partitioning 3 2 8 5 1 4 7 6
arrays
i j
Quicksort
3 2 8 5 1 4 7 6
Randomization
i j
The Selection
Problem 3 2 1 5 8 4 7 6
Conclusion i j
3 2 1 5 8 4 7 6
i j
3 2 1 5 8 4 7 6
i j
1 2 3 5 8 4 7 6
 Algorithm

Lecture 6
Algorithm 3: Partitioning an array around a pivot
Input: A[0..n − 1]: Array to be partitioned
Merging two
sorted arrays
Output: A[0..n − 1]: Partitioned array
Merge Sort
1 p ← A[0]; i ← 1 ;
Partitioning 2 for j ← 1 to n − 1 do
arrays
3 if A[j] < p then
Quicksort
4 Swap A[j] and A[i] ;
Randomization
5 i ←i +1 ;
The Selection
Problem 6 end
Conclusion 7 end
8 Swap A[0] and A[i − 1] ;

# of comparisons: n − 1
# of swaps: depends on the input (How?)
 Quicksort

Lecture 6

Merging two
sorted arrays Algorithm 4: Quicksort
Merge Sort Input: A[0..n − 1]: Array to be sorted
Partitioning
arrays
Output: A[0..n − 1]: sorted array
Quicksort
1 if n > 1 then
Randomization 2 pi ← Partition(A[0..n − 1]);
The Selection 3 Quicksort(A[0..pi − 1]);
Problem
4 Quicksort(A[pi + 1..n − 1])
Conclusion
5 end
pi is index containing the pivot.
 Example

Lecture 6
1 2 3 5 8 4 7 6
1 2 2 > 1; Already partitioned
Merging two
sorted arrays
i, j
Merge Sort
1 2 Self swap
Partitioning
2
arrays
5 8 4 7 6
Quicksort
i, j
Randomization
5 8 4 7 6
The Selection
Problem i j
Conclusion 5 4 8 7 6
i j
5 4 8 7 6
i j
4 5 8 7 6
4
 Example

Lecture 6

Merging two
sorted arrays
8 7 6
Merge Sort i, j
Partitioning 8 7 6
arrays
i, j
Quicksort
8 7 6
Randomization
6 7 8
The Selection
Problem 6 7
Conclusion i, j
7
1 2 3 4 5 6 7 8
 Analysis

Lecture 6
Best case If all the partitions happen in the middle of the
array. The total number of comparisons will be
Merging two θ(n log n).
sorted arrays

Merge Sort
Worst case If all the partitions are skewed to the extreme,
Partitioning one of the two subarrays will be empty. The size
arrays
of the other subarray will be one less than the
Quicksort
size of the original array. The total number of
Randomization
comparisons will be θ(n2 ).
The Selection
Problem Average case The average number of key comparisons made by
Conclusion quicksort on a randomly ordered array is
θ(1.38n log n) i.e. on average quicksort makes
only 38% more comparisons than in the best case.

Importance of the pivot

Running time of quicksort depends on the quality of the pivot.
 How to choose a good pivot?

Lecture 6

Merging two In every recursive call, choose the pivot randomly such
sorted arrays
that each element is equally likely to be chosen.
Merge Sort

Partitioning Not necessarily 50-50 partition is needed for n log n time.

arrays
A 25-75 split is sufficient to achieve this rate.
Quicksort

Randomization
Randomization can also make algorithms more elegant,
The Selection
simpler, easy to code and faster.
Problem

Conclusion
Quicksort Theorem

For every input array of length n, the average running time of

quicksort with random pivots is O(n log n).
 Randomization

Lecture 6

Merging two
sorted arrays

Merge Sort

Partitioning
arrays Randomly choose a pivot.
Quicksort
Exchange it with the first element.
Randomization

The Selection
Perform partition
Problem

Conclusion
 The Selection Problem

Lecture 6
Problem Given an array A with n distinct numbers and an
integer k, find the k th smallest element (i.e. k th
Merging two
sorted arrays order statistic).
Merge Sort Solution 1 Use a sorting algorithm to sort the array in
Partitioning
arrays
increasing order (θ(n log n)).
Quicksort
2 The element at the k th position is the k th
Randomization smallest element.
The Selection
Problem
Running time at least θ(n log n)
Conclusion Note Selecting the smallest element can be done in
θ(n) time without sorting.

Can we do better?
Can we solve the selection problem without sorting? This can
make the selection process faster by reducing the work.
 Using partitioning to solve the problem

Lecture 6

Values < pivot pivot Values > pivot

Merging two
sorted arrays

Merge Sort
1 If the pivot is placed at the i th position and i < k, the k th
Partitioning
arrays smallest element must be among the elements that are to
Quicksort the right of the i th position. Call partition again using the
Randomization right side of array as input.
The Selection
Problem 2 If the pivot is placed at the i th position and i > k, the k th
Conclusion smallest element must be among the elements that are to
the left of the i th position. Call partition again using the
left side of array as input.
3 Continue until partition places the pivot at the k th
position.
 Example

Lecture 6
Find the 4th smallest element in the array
{17, 21, 5, 23, 9, 37, 15, 3, 11, 25}.
Merging two
sorted arrays
The sorted array {3,5,9,11,15,17,21,23,25,37} ⇒ 11
Merge Sort

Partitioning
arrays

Quicksort 1↓ 2 3 4↓ 5 6 7 8 9 10↓
Randomization 17 21 5 23 9 37 15 3 11 25
The Selection
Problem
11 5 9 15 3 17 23 21 37 25
Conclusion 11 5 9 15 3
3 5 9 11 15

Exercise
Find the median in array {15,3,17,2,9,10,8,4,33,1,25}.
 Algorithm

Lecture 6
Algorithm 5: The Select Algorithm
Input: A[1..n], k: Order statistic
Merging two
sorted arrays
Output: Pi : position of k
Merge Sort
1 if n = 1 then
Partitioning
2 return A[1] ;
arrays
3 end
Quicksort
4 pi = partition(A[1..n]) ;
Randomization
5 if pi = k then
The Selection
Problem 6 return pi ;
Conclusion 7 end
8 if pi > k then
9 return select(A[1..pi − 1]) ;
10 else
11 return select(A[pi + 1..n]);
12 end
 Analysis

Lecture 6
The running time of the select algorithm depends on
The quality of the pivot.
Merging two
sorted arrays

Merge Sort In the worst-case running time is θ(n2 ) (similar to

Partitioning
arrays
quicksort).
Quicksort The best pivot
Randomization

The Selection
median! (A fortunate split)
Problem

Conclusion
Random pivots
For every input array of length n, the average running time of
select algorithm is O(n) (Theorem)

Assume a random partition by randomly choosing a pivot.

a = 1, b = 2 and d =1 ⇒ θ(n).
 Conclusion

Lecture 6

Merging two
sorted arrays

Merge Sort In today’s lecture we talked about

Partitioning
arrays
Mergesort
Quicksort

Randomization Quicksort
The Selection The Selection problem
Problem

Conclusion
In next lecture we will study more on Sorting algorithms.