Simple Sorting and Searching

Algorithms
 Introduction
• In this chapter, we will illustrate
– the use of the data structures introduced in the preceding chapter
– how the choice of structure for the underlying data profoundly influences
the algorithms that perform a given task.
• Sorting and searching are a good example to show existence of
different algorithm on different data structure
• Each one having certain advantages and disadvantages that have to
be weighed against each other in the light of the particular
application.
• Moreover, sorting and searching are vital component of several
real world applications.
• In this chapter, we will discuss simple sorting algorithm and
simple searching algorithms

2
 Simple Sorting Algorithm
• Sorting is generally understood to be the process of rearranging a
given set of objects in a specific order.
• Sorting is an almost universally performed, fundamental activity.
• Objects are sorted in telephone books, in tables of contents, in
libraries, in dictionaries and in warehouses
• Even small children are taught to put their things "in order", and
they are confronted with some sort of sorting long before they
learn anything about arithmetic.

3
 Simple Sorting Algorithm
• One of the most important purpose of sorting is to facilitate later
search from the sorted set.
• Hence, sorting is a relevant and essential activity, particularly in
data processing.

• Moreover, sorting is an ideal subject to demonstrate a great

diversity of algorithms, all having the same purpose, many of
them being optimal in some sense, and most of them having
advantages over others.

• It is therefore an ideal subject to demonstrate the necessity of

performance analysis of algorithms.

4
 Simple Sorting Algorithm
1. Selection sort
• It is the most natural and easiest sorting algorithm
• Given an array of length n,
– Search elements 0 through n-1 and select the smallest
• Swap it with the element in location 0
– Search elements 1 through n-1 and select the smallest
• Swap it with the element in location 1
– Search elements 2 through n-1 and select the smallest
• Swap it with the element in location 2
– Search elements 3 through n-1 and select the smallest
• Swap it with the element in location 3
– Continue in this fashion until there’s nothing left to search
5
 Elementary Sorting Algorithm
1. Selection sort
• As stated above, at the jth step, selection sort searches the
position of the jth minimum element (the element that has to be
at that position) from the collection with index > j and swap
with the elements which were initially at the jth position and
with the newly searched element position.

6
 Elementary Sorting Algorithm
1. Selection sort algorithm

SelectionSort (A, n)
for j  0 to n-1
min  A[j]
index  j
//find the index of the jth min value
for i j+1 to n
if(A[i] < min)
index  i
min A[i]
//swap A[j]and A[index]
temp  min
A[index]  A[j]
A[j]  temp

7
 Elementary Sorting Algorithm
Selection sort example

7 2 8 5 4
• Selection Sort might swap an array
element with itself--this is harmless,
2 7 8 5 4
and not worth checking for

2 4 8 5 7

2 4 5 8 7

2 4 5 7 8

8
 Elementary Sorting Algorithm
SelectionSort (A, n)
for j  0 to n-1
min  A[j]
index  j
//find the index of the jth min value
for i j+1 to n-1
if(A[i] < min)
index  i
min A[i]
//swap A[j]and A[index]
temp  min
A[index]  A[j]
A[j]  temp

Note:
Both worst and best case has the same growth rate

9
 Simple Sorting Algorithm
2. Insertion sort
• Given an array of length n,
– Take the element at index j and call is key
– Let the element at index i is greater than key and the element
at index i-1 is less than the key
– Push all the element from index j-1 to i one step to the right
– Insert the key into index I

– Repeat this for all j from index 1 to n

10
 Elementary Sorting Algorithm
2. insertion sort
• Basic philosophy: at the jth step, take the jth element in
the order and find appropriate position for it in the (j-
1)th list which was already sorted. At the end of this
step, we will have all the j elements sorted and the
remaining elements unprocessed

11
 Elementary Sorting Algorithm
1. Insertion sort algorithm

InsertionSort (A, n)
for j  1 to n-1
key A[j]
//Insert key into the list A[0…j-1] so that A[0..j] becomes
sorted
i  j-1
while (i >= 0 and A[i] > key)// i is valid index and A[i] >
the key
A[i+1]  A[i]
i  i-1
A[i+1]  key

12
 Elementary Sorting Algorithm
One step of insertion sort
sorted next to be inserted

3 4 7 12 14 14 20 21 33 38 10 55 9 23 28 16
temp
less than 10 10

3 4 7 10 12 14 20
12 14 14 21 21 38
20 33 33 10
38 55 9 23 28 16

sorted

13
 Elementary Sorting Algorithm
Insertion sort algorithm analysis (worst case)
•In this case the inner loop will iterate j times for every j.
•This is the case where the data is sorted in descending order

InsertionSort (A, n)
for j  1 to n-1
key A[j]
i  j-1
while (i >= 0 and A[i] >
key)
A[i+1]  A[i]
i  i-1
A[i+1]  key

14
 Elementary Sorting Algorithm
Insertion sort algorithm analysis (best case)
•In this case the inner loop will iterate one times for every j.
•This is the case where the data is sorted in ascending order

InsertionSort (A, n)
for j  1 to n-1
key A[j]
i  j-1
while (i >= 0 and A[i] >
key)
A[i+1]  A[i]
i  i-1
A[i+1]  key

15
 Elementary Sorting Algorithm
3. Bubble sort
• Compare each element (except the last one) with its neighbor
to the right
– If they are out of order, swap them
– This puts the largest element at the very end
– The last element is now in the correct and final place
• Compare each element (except the last two) with its neighbor
to the right
– If they are out of order, swap them
– This puts the second largest element next to last
– The last two elements are now in their correct and final places
• Compare each element (except the last three) with its neighbor
to the right
– Continue as above until you have no unsorted elements on the left

16
 Elementary Sorting Algorithm
3. Bubble sort
• Basic philosophy: at the jth step, all the j-1 element
from the end are already sorted. Hence this step
compares the element with its neighbor and if they are
not in sorted fashion, it will swap them until it reaches
to the region which are already in appropriate place

17
 Elementary Sorting Algorithm
3. Bubble sort algorithm

BubbleSort (A, n)
for j  0 to n-2
for i 1 to n -1 -j
if( A[i-1] > A[i])
//Swap A[i] and A[i-1]
temp  A[i-1]
A[i-1]A[i]
A[i]  temp
i  i-1
A[i+1]  key

18
 Elementary Sorting Algorithm

Example of Bubble Sort

7 2 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8

2 7 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8

2 7 8 5 4 2 5 7 4 8 2 4 5 7 8 (done)

2 7 5 8 4 2 5 4 7 8

2 7 5 4 8

19
 Elementary Sorting Algorithm
Bubble sort algorithm analysis

BubbleSort (A, n)
for j  0 to n-2
for i 1 to n -1 -j
if( A[i-1] > A[i])
temp  A[i-1]
A[i-1]A[i]
A[i]  temp
i  i-1
A[i+1]  key

• Note: both worst and best case have the

same complexity
20
 Elementary Searching Algorithm
• The task of searching is one of most frequent operations in
computer programming.

• Searching is the process of looking for a particular value in a

Collection

• Searching also provides an ideal ground for application of the data

structures so far encountered.

• There exist several basic variations of the theme of searching, and

many different algorithms have been developed on this subject.

21
 Elementary Searching Algorithm
• The basic assumption in the following presentations is that the
collection of data, among which a given element is to be searched,
is fixed
• In this chapter we will focus on simple search algorithms: linear
search and binary search
• We will assume the search element is key and searching is made
on data set structured on an array A having total of N elements

22
 Elementary Searching Algorithm
Linear Search
•Basic philosophy: at the jth step, the search algorithm compare the
key value with jth element value and
– if they are the same it return the array index which is j
– If they are different increment the value of j by one and repeat the steps

Function linearSearch(key, Array A): return integer index

for I = 0 to N-1
do
if(A[I].key = key) return I
end do
return -1
End function

23
 Elementary Searching Algorithm
Linear Search
•What is the growth rate for the linear serach (best and worst case)
Function linearSearch(key, Array A): return integer index
for I = 0 to N-1
do
if(A[I].key = key) return I
end do
return -1
End function

24
 Simple Searching Algorithm
Binary Search
•Binary search operates on array whose elements are already in
sorted fashion (say in ascending order i.e.
a[0]≤a[1]≤a[2]≤. . . ≤ a[finalIndex])
•Binary search uses a recursive method to search an array to find a
specified value
•If the value is found, its index is returned
•If the value is not found, -1 is returned
•Note: Each execution of the recursive method reduces the search
space by about a half

25
 Simple Searching Algorithm
Binary Search
•An algorithm to solve this task looks at the middle of the array or
array segment first
•If the value looked for is smaller than the value in the middle of the
array
• Then the second half of the array or array segment can be ignored
• This strategy is then applied to the first half of the array or array segment
•If the value looked for is larger than the value in the middle of the
array or array segment
• Then the first half of the array or array segment can be ignored
• This strategy is then applied to the second half of the array or array
segment

26
 Simple Searching Algorithm
Binary Search
•If the value looked for is at the middle of the array or array segment,
then it has been found

•If the entire array (or array segment) has been searched in this way
without finding the value, then it is not in the array

27
 Simple Searching Algorithm
Binary Search
•Basic philosophy: at each iteration, the algorithm work on segment
of the array with lower index low to upper index upper and compare
the key with the element at the middle of the segment
– If they are equal return the index
– If the key is smaller that the middle element key, modify upper to be
middle index – 1
– If the key is larger that the middle element key, modify lower to be middle
index +1
– If lower > upper return ‘the key doesn’t found message’

28
 Simple Searching Algorithm
Binary Search (non-recursive implementation)
Function BinarySearch(key, Array A): return integer
lower = 1
upper = N
while (lower <= upper)
do
middle = (lower + upper)/2
if (A[middle].key = key) return middle
else if (A[middle].key > key) upper = middle -1
else lower = middle + 1
end do
return -1
End function

29
 Simple Searching Algorithm
Binary Search (recursive implementation)
Function BinarySearch(key, Array A, from, to): return integer
if(from = to)
if(A[from].key = key) return from
return -1
middle  (from + to)/2
if(A[middle].key = key) return middle
else if(A[middle].key > key)
return BinarySearch(key, A, from, middle-1)
else
return BinarySearch(key, A,middle + 1, to)
End function

30
 Simple Searching Algorithm
Analysis of Binary Search algorithm
Let n be power of 2 ( n = 2k)
The best case scenario is when the element is at the middle which is O(1)
The worst case scenario is when the key doesn’t exits
Iteration 1 works on 2k-1 elements //as it works on the first or second half
Iteration 2 works on 2k-2 elements //as it works on the first or second half
:
:
Iteration k works on 2k-(k) = 1 elements //as it works on the first or second half

Hence it require k comparison

As n = 2k
log2n = log22k = k
Hence it is O(logn)

31