Data Structure and Algorithm Analysis

Chapter 2:
Simple Sorting & Searching Algorithms

1 Prepared By Wasyhun Abdela

 Sorting and searching algorithms

 There are some very common problems that we use

computers to solve:

 Searching: Looking for specific data item/record

 from list of data items or set of records.
Sorting : placing records/items in order

2
Searching
 Searching is an operation or a technique that helps
finds the place of a given element or value in the list.

 Any search is said to be successful or unsuccessful

depending upon whether the element that is being
searched is found or not.

 There are two simple searching algorithms:

1. Linear Search,
2. Binary Search

3
Linear Searching (Sequential )
 It is a very basic and simple search algorithm.
 In Linear search, we search an element or value in a given
array by traversing the array from the starting, till the
desired element or value is found.

 It compares the element to be searched with all the

elements present in the array and when the element is
matched successfully, it returns the index of the element
in the array, else it return -1.

 Linear Search is applied on unsorted/unordered lists,

4
when there are fewer elements in a list.
Cont...Sequential Searching (Linear)
Algorithm

 Linear Search ( Array A, Value x)

 Step 1: Set i to 1
 Step 2: if i > n then go to step 7
 Step 3: if A[i] = x then go to step 6
 Step 4: Set i to i + 1
 Step 5: Go to Step 2
 Step 6: Print Element x Found at index i and go to step 8
 Step 7: Print element not found
 Step 8: Exit
5
Cont...Sequential Searching (Linear)
Pseudocode

 procedure linear_search (list, value)

 for each item in the list
 if match item == value
 return the item‟s location
 end if
 end for
 end procedure

6
 Linear Search (Sequential Search)

7
Sequential Search Example 1

int linear_search(int list[], int n, int key){

for (int i=0;i<n; i++){
if(list[i]==key)
return i;
}
return -1;
}
Time complexity O(n)
--Unsuccessful search --- n times
--Successful search (worst) --- n times
--Successful search (Best) --- 1 time
--Successful search (average) --- n/2 times
Binary Search

 Linear Search is applied on sorted list

 Sequential search is not efficient for large lists because, on
average, the sequential search searches half the list.
 The binary search gets its name because the
algorithm continually divides the list into two parts.
 Uses the divide-and-conquer technique to search
the list
Binary Search
How Fast is a Binary
Search?
 Worst case: 11 items in the list took 4 tries
 How about the worst case for a list with 32 items ?
1st try - list has 16 items
2nd try - list has 8 items
3rd try - list has 4 items
4th try - list has 2 items
5th try - list has 1 item
Efficienc
y
Binary search is one of the fastest Algorithms
 The computational time for this algorithm is
proportional to log2n
 Lg n means the log to the base 2 of some value of n.

• 8 = 23 lg 8 = 3 16 = 24 lg 16 = 4
 Therefore, the time complexity is O(logn)
How Binary Search Works?

 The binary search is a very fast search algorithm.

•  But, the list has to be sorted before we can search it
with binary search.

 The following is our sorted array and let us assume

that we need to search the location of value 31 using
binary search.
Algorithm of the binary search in C++
• beg = 0;
• end = size - 1;
• while ( beg <= end)
• { // calculate mid value
• mid = (beg + end) / 2;
• If (arr[mid] == num)
• { return mid + 1; }
• else if (arr[mid] > num)
• { End = mid - 1; }
• Else if (arr [mid] < num)
• { Beg = mid + 1; } }
• Return -1;
Linear Search
• workis for both Ordered/Unordered array
• O(n)
• Sequential access

Binary Search
• workis for both Ordered array
• log2n
• random access
Sort Algorithms

 It is a process of reordering a list of items in either

increasing or decreasing order.
 Sorting is the first step in more complex algorithms.
 Bubble sort is the slowest, running in n2 time. Quick
sort is the fastest, running in n lg n time.

 We will examine three sorting algorithms:

1. Bubble sort
2. Insertion sort
3. Selection sort
4. Quick sort
Bubble
Sort

 It is one of the most straightforward sorting

algorithms.
 It is the simplest sorting algorithm that works by
repeatedly swapping the adjacent elements if they are in
the wrong order.
 This algorithm is not suitable for large data sets as its
average and worst-case time complexity is quite high
Bubble Sort Idea

 Given an array of n items

 Compare pair of adjacent items
 Swap if the items are out of order
 Repeat until the end of array
• The largest item will be at the last position
 Reduce n by 1 and go to Step 1
Bubble
Sort

void bubbleSort(int a[], int n) {

for (int i = n-1; i >= 1; i--) {
Step 1:
for (int j = 1; j <= i; j++) { Compare adjacent pairs of
if (a[j-1] > a[j]) numbers
swap(a[j], a[j-1]);
}
Step 2 :
} Swap if the items are out of order
}
Step
Bubble Sort: Illustration
Bubble Sort Exercise

Illustrate the below unsorted array using Bubble Sort

Bubble Sort Illustration

 Consider the unsorted

array to the right
 We start with the
element in the first
location, and move
forward:
 if the current and next
items are in order,
continue with the next
item, otherwise swap
the two entries
…Bubble Sort Illustration

After one loop, the largest element is in the last

location Repeat the procedure
…Bubble Sort Illustration

Now the two largest elements are at the end

Repeat again
…Bubble Sort Illustration

With this loop, 5 and 7 are

swapped
…Bubble Sort Illustration

Finally, we swap the last

two entries to order them
At this point, we have a
sorted array
Bubble sort complexity
 Best Case Complexity - It occurs when there is no
sorting required, i.e. the array is already sorted.
 O(n).
 Average Case Complexity - It occurs when the array
elements are in jumbled order that is not properly
ascending and not properly descending.
 O(n2).
 Worst Case Complexity - It occurs when the array
elements are required to be sorted in reverse order. That
means suppose you have to sort the array elements in
ascending order, but its elements are in descending
order.
 O(n2).
Insertion Sort Algorithm

 It is a simple sorting algorithm that works similarly to

the way you sort playing cards in your hands.
 The array is virtually split into a sorted and an
unsorted part.
 Values from the unsorted part are picked and placed at
the correct position in the sorted part.
 Insertion sort is fast and best suitable either when the
problem size is small (because it has low overhead) or
when the data is nearly sorted (because it is adaptive).
Insertion Sort: Idea

 Similar to how most people arrange a hand of poker cards

 Start with one card in your hand
 Start with one card in your hand
 Pick the next card and insert it into its proper sorted
order
 Repeat previous step for all cards
Insertion Sort: Illustration
Insertion Sort: Implementation

for (i = 1; i < n; i++)

{
temp = a[i];
j = i-1;
while (j >= 0 && a[j] > temp)
{
a[j+1] = a[j];
j--;
}
a[j+1] = temp;
}
Arranging Your Hand

5 7
Arranging Your Hand

5 7

5 6 7

5 6 7 K

5 6 7 8 K
Insertion Sort
7 K Unsorted - shaded
Look at 2nd item - 5.
Compare 5 to 7.
7 1 5 is smaller, so move 5 to temp, leaving
5
an empty slot in position 2.
v Move 7 into the empty slot, leaving
position 1 open
7 . 5

Move 5 into the open position

7 .

2 >

5 7 3
<
Insertion Sort (con’t)
5 7 6 K Look at next item - 6.
Compare to 1st - 5.
6 is larger, so leave 5. Compare to next - 7.

1 6 is smaller, so move 6 to temp, leavingan empty slot.

5 7 Move 7 into the empty slot, leaving position 2 open.

Move 6 to the open 2nd

v position.
5 7 6

5 7
2 >

5 6 7 3
<
Insertion Sort (con’t)
Look at next item - King.
Compare to 1st - 5.
5 6 7 K King is larger, so leave 5 where it
is.
Compare to next - 6. King is larger,
leave 6 where it so
is. Compare to next - 7. King is larger,
so
leave 7 where it is.
Insertion Sort (con’t)
5 6 7 K 8

5 6 8 1
7 K
v
5 6 7 K 8

5 6 7 K
>
2
5 6 7 8 K 3
<
Insertion Sort Exercise

Illustrate the below unsorted array using Insertion Sort

5 2 4 5 1 3
Insertion Sort Illustration
Insertion sort complexity
 Best Case Complexity - It occurs when there is no
sorting required, i.e. the array is already sorted.
 O(n).
 Average Case Complexity - It occurs when the array
elements are in jumbled order that is not properly
ascending and not properly descending.
 O(n2).
 Worst Case Complexity - It occurs when the array
elements are required to be sorted in reverse order.
That means suppose you have to sort the array
elements in ascending order, but its elements are in
descending order
 O(n2).
Selection Sort Algorithm

 In selection sorting technique, the smallest element is

fetched by comparing itself with the rest of the
elements and sorted at the array's first position.
 The complete array is divided into two halves, the
sorted subarray on the left and the unsorted subarray
on the right.
 Once the first element is sorted, the search for the
second minimum element begins from the rest of the
array and is positioned at second place.
Selection Sort: Idea

Given an array of n items

1. Find the largest item x, in the range of [0…n−1]
2. Swap x with the (n−1)th item
3. Reduce n by 1 and go to Step 1
 Algorithm

• Step 1 − Set MIN to location 0

• Step 2 − Search the minimum element in the list
• Step 3 − Swap with value at location MIN
• Step 4 − Increment MIN to point to next element
• Step 5 − Repeat until list is sorted
Selection Sort: Illustration
Selection Sort: Implementation
void selectionSort(int a[], int n)
{
for (int i = 0; i >= n-2; i++)
{
int temp = i; Step 1:
for (int j = i+1; j < n-1; j++) Search for
Maximum element
if (a[i] > a[j])
temp = j; Step 2:
swap(a[i], a[temp ]); Swap maximum
} element with the last
item i
}
Selection Sort: Implementation
void selectionSort(int a[], int n)
{
for (int i = n-1; i >= 1; i--)
{
int temp = i; Step 1:
for (int j = 0; j < i; j++) Search for
Maximum element
if (a[j] >= a[temp ])
temp = j; Step 2:
swap(a[i], a[temp ]); Swap maximum
} element with the last
item i
}
Selection Sort Exercise:

• Illustrate the below unsorted array using

Selection Sort

12 31 25 8 32 17
Selection Sort Exercise:

for (i = 0; i < n-1; i++)

{
small = i;
for (j = i+1; j < n; j++)
if (arr[j] < arr[small])
small = j;
int temp = arr[small];
arr[small] = arr[i];
arr[i] = temp;
}
Selection sort complexity

Time Complexity
 Best Case Complexity - It occurs when there is no sorting
required, i.e. the array is already sorted.
 O(n2).
 Average Case Complexity - It occurs when the array
elements are in jumbled order that is not properly
ascending and not properly descending.
 O(n2).
 Worst Case Complexity - It occurs when the array
elements are required to be sorted in reverse order. That
means suppose you have to sort the array elements in
ascending order, but its elements are in descending order.
 O(n2).
Quick Sort
 It is the most widely used algorithm and the most
efficient sorting algorithm.
 It works on the divide and conquer approach, i.e., the
array is divided into subarrays, and when these subarrays
are sorted, they are combined to form a complete sorted
array.
Quick Sort: Idea
In this approach, a pivot element is selected, and the
array is partitioned into two halves based on the pivot
element.
The elements that are smaller than the pivot element are
shifted to the left side of it, and
the elements greater than the pivot element are moved to
the right side.

Apply the same algorithm to each half

< pivot > pivot
 Step 1 − Choose the highest index value has pivot
 Step 2 − Take two variables to point left and right of the
list excluding pivot
 Step 3 − left points to the low index
 Step 4 − right points to the high
 Step 5 − while value at left is less than pivot move right
 Step 6 − while value at right is greater than pivot move
left
 Step 7 − if both step 5 and step 6 does not match swap
left and right
 Step 8 − if left ≥ right, the point where they met is new
pivot
Insertion Sort: Illustration
 Quick Sort: Implementation
partition(l,h)
pivot a[s];
i=s,j=h;
while(i<j) {
do
{ i++;
}while(a[i]<=pivot);
do
{ j--;
}while(a[j]>=pivot);
if(i<j);
swap(a[i],a[j]); }
swap(a[s],a[j]);//to swap pivot number
return j;
}
Quick Sort Exercise:

• Illustrate the below unsorted array using

Quick Sort

5 15 1 8 22 6
Quick Sort Illustration:
Quicksort complexity
 Best Case Complexity - In Quicksort, the best-case
occurs when the pivot element is the middle element or
near to the middle element.
 O(n*logn).
 Average Case Complexity - It occurs when the array
elements are in jumbled order that is not properly
ascending and not properly descending.
 O(n*logn).
 Worst Case Complexity - In quick sort, worst case occurs
when the pivot element is either greatest or smallest
element. Suppose, if the pivot element is always the last
element of the array, the worst case would occur when
the given array is sorted already in ascending or
descending order.
 O(n2).