CMP305_AlgorithmLecture2computer couse that will help you know and learn algorithm
CMP305_AlgorithmLecture2computer couse that will help you know and learn algorithm
LINEAR SEARCH
Let A[1..n] be a sequence of n elements. Consider the problem of determining whether a
given element x is in A. The problem can be rephrased as follows:
Find an index j, , such that x=A[j] if x is in A, and j=0 otherwise.
A straightforward approach is to scan the entries in A and compare each entry with x. If after j
comparisons, , the search is successful, i.e. x=A[j], j is returned, otherwise a value of
0 is returned indicating an unsuccessful search. This method is referred to as sequential search. It
is also called linear search because the maximum number of element comparison grows linearly
with the size of the sequence. The algorithm is shown as Algorithm LINEARSEARCH.
Algorithm: LINEARSEARCH.
Input: An array A[1..n] of n elements & an element x.
Output: j if x=A[j], , and 0 otherwise.
1. j=1
2. while (j<n)and (x≠A[j])
3. j=j+1
4. end while
5. if x==A[j] the return j else return 0
Illustrate Linear with the following inputs are supply to Linear Search
X=10, A= 5 2 10 0
BINARY SEARCH
For LINEARSEARCH, scanning all entries of A is inevitable if no more information about the
ordering of the elements in A is given. If we are also given that the elements in A are sorted, say
in nondecreasing order, then there is a much more efficient algorithm. The following example
illustrates this efficient search method
1 2 3 4 5 6 7 8 9 10 11 12 13 14
A[1..14]= 1 4 5 7 8 9 10 12 15 22 23 27 32 35
↑ ↑ ↑
L M H
First, we shall determine the middle element by calculating the middle index (M) using this
formula: M (L H ) / 2 (1)
Then, we compare X22 with the middle element A[7]=10. Since X22 > A[7], it means X22 cannot
be in A[1..7] (i.e. portion mark in green), and therefore, this portion of the array can be
discarded.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
A[1..14] = 1 4 5 7 8 9 10 12 15 22 23 27 32 35
↑ ↑ ↑
L M H
So we are left with the following sub array:
8 9 10 11 12 13 14
A[8..14] = 12 15 22 23 27 32 35
↑ ↑
L H
Secondly, we determine the middle element by calculating the middle index (M) using Equation
1,
Here, M = (8+14)/2 = 22/2 = 11.0 ≈ 11. So, the middle element is A[11]=23.
8 9 10 11 12 13 14
A[8..14] = 12 15 22 23 27 32 35
↑ ↑ ↑
L M H
Thus, the remaining portion of the array to be searched is now reduced to
8 9 10
A[8..10]= 12 15 22
↑ ↑
L H
Thirdly, we determine the middle element by calculating the middle index (M) using Equation 1,
Here, M = (8+10)/2 = 18/2 = 9 ≈ 9. So, the middle element is A[9]=15.
Then, we compare X22 with the middle element A[9]=15. Since X22 >15, X22 cannot be in A[8..9],
therefore A[8..9] is discard.
8 9 10
A[8..10]= 12 15 22
↑ ↑ ↑
L M H
10
A[10]= 22
↑
LH
Finally, M = (10+10)/2 = 20/2 = 10 ≈ 10. So, the middle element is A[10]=22.
Then, we compare X22 with the middle element A[10]=22. Since we find that X22 == A[10]22, and the
search is successfully completed.
In general, let A[low..high] be an array of elements sorted in nondecreasing order. Let A[mid]
be the middle element, and suppose that x>A[mid]. If x is in A, then it must be one of the
elements A[mid+1],A[mid+2],…, A[high]. It follows that we only need to search for x in
A[mid+1..high]. In other words, the entries in A[low..mid-1] are discarded in subsequent
comparisons since, by assumption, A is sorted in nondecreasing order, which implies that x
cannot be in this half of array. Similarly, if x<A[mid], then we only need to search for x in
A[low..mid-1].This results in an efficient strategy which, because of its repetitive halving, is
referred to as binary search. Algorithm BINARYSEARCH gives a more formal description of
this method.
i. First, many tasks run concurrently on a computer. The execution time of a particular
program depends on the system load.
ii. Second, the execution time depends on specific input. Consider, for example, linear
search and binary search. If an element to be searched happens to be the first in the array,
linear search will find the element quicker than binary search.
Order of Growth
The change in the behavior of algorithm as the input size increases is called order of growth of
the algorithm. It shows how fast an algorithm grows with the input size. The growth of execution
time as a function of the size of the problem data provides a basis for comparing ways of solving
the problems.
A simple approach that is frequently adequate for our purposes can be applied to the following
program segment:
statement 1
for i=1 to n do
[Statement2]
[Statement3]
next i
An analysis may begin by assuming that [Statement k] takes one unit of time for k=1, 2, 3.
Then this segment requires one unit of time for [Statement 1] followed by two units of
time for each pass through the for-loop. Therefore, the execution time is
This can be easily generalized by assuming that [Statement i] requires ti units of time, this
means
Equation 2 remains true even if [Statement 2] is itself a loop that always requires the same
amount of time to execute. For example, if [Statement 2] is
Then t2 = m.t4
( ) ∑( )
At this point, in order to sole simplify Equation 3, we need to document some of the properties of
summations, because loop execution times are a sum of the times required for each pass through
them. These properties are as follows:
i. ∑ ( )
For e.g. ∑ ( )
( ) ( )
ii. ∑
( )
For example, ∑
( ( )
iii. ∑
iv. ∑ [ () ] [ ∑ ( )] ( )
Where a and b are not function of i
A( )
{
int i, j, k, n,
for (i = 1, i <=n, i++){
for ( j=1, j<=i , j++){
for (k = 1, k<=100, k++){
print ("It")
}
}}
1 2 3 4 ... n
i=1 i=2 i=3 i=4 i=n
j = 1 time j = 2 times j = 3 times j = 4 times ... j = n times
k = 100 k = 100 * 2 k = 100 * 3 k = 100 * 4 k = 100 * n
times
The total time for the algorithm is:
T(n) = 100 + 100 * 2 + 100 * 3 + ... + 100 * n
= 100 (1+ 2 + 3 + ... + n)
= 100 (n (n + 1))
2
2
= 0 (n )
( ) ∑∑∑
∑∑
( )
( )
Selection Sort
* Select 1 (the smallest) and swap it with 2 (the first element in the list).
2 9 5 4 8 1 6
The number 1 is now in the correct position and thus, no longer need to be
considered.
1 9 5 4 8 2 6
* Select 2 (the smallest) and swap it with 9 (the first number) in the remaining
list of numbers.
1 9 5 4 8 2 6
The number 2 is now in the right position and thus, no need to be considered.
1 2 5 4 8 9 6
* Select 4 (the smallest) and swap it with 5 (first number) in the remaining list of
numbers.
1 2 5 4 8 9 6
The number 4 is now in the right position and thus, no need to be considered.
1 2 4 5 8 9 6
Considering the above sorted list, element 5 is in its right position and need not to
be considered.
1 2 4 5 8 9 6
The number 6 is now in the right position and thus, no need to be considered.
1 2 4 5 6 9 8
* Select 8 (the smallest) and swap it with 9 (first number) in the remaining list of
numbers.
1 2 4 5 6 9 8
The number 8 is now in the right position and thus, no need to be considered.
1 2 4 5 6 8 9
Since, there is only one element remaining in the list, the sorting is completed.
This method is described in Algorithm SELECTIONSORT.
1 2 3 4 5 6 7
A 2 9 5 4 8 1 6
1 2 3 4 5 6 7
A 1 2 5 4 8 9 6
i=3, k= i=3, j=4, is A [4] < A [3]? yes => k=j=4
j=5, is A [5] < A [4]? No
j=6, is A [6] < A [4]? No
j=7, is A [7] < A [4]? No
j=8,
is k≠ i? yes => swap A[i3] with A[k4]. This gives
1 2 3 4 5 6 7
A 1 2 4 5 8 9 6
i=4, k=i=4, j=5, is A [5] < A [4]? No
j=6, is A [6] < A [4]? No
j=7, is A [7] < A [4]? No
is k ≠ i? No => No swapping. This gives
1 2 3 4 5 6 7
A 1 2 4 5 6 8 9
One of the disadvantages of this algorithm is that, it doesn’t mind the arrangement of elements in the
array.
n 1
T (n) (t1 t 2 t 3 ) ----------------------------- (1)
i 1
Where
t1 1 ,
n
t2 2 2(n (i 1) 1) 2n 2i ,
j i 1
t3 4
T (n) n 2 4n 5
Step1: initially, the sorted sublist contains the first element in the list. Insert 9 into
the list sublist
2 9 5 4 8 1 6
Step2: the sorted sublist is [2, 9]. Insert 5 into the sublist
2 9 5 4 8 1 6
Step3: the sorted sublist is [2, 5, 9]. Insert 4 into the sublist.
2 5 9 4 8 1 6
Step4: the sorted sublist is [2, 4, 5, 9]. Insert 8 into the sublist.
2 4 5 9 8 1 6
Step5: the sorted sublist is [2, 4, 5, 8, 9]. Insert 1 into the sublist.
2 4 5 8 9 1 6
1 2 4 5 8 9 6
1 2 4 5 6 8 9
1. for i = 2 to n
2. x = A[i]
3. j = i–1
4. while (j>0) and (A[j]>x)
5. A[j+1]= A[j]
6. j = j–1
7. End while
8. A[j+1]= x
9. End For
i =2, x =A[i] =9, j=1, while j1>0 and A[j1]>x9? No => goto line8
A[j+1] =A[2]= x=9
1 2 3 4 5 6 7
2 9 5 4 8 1 6
i=3, x= A[i] =5, j= 2, while j2>0 and A[j2] > x5? yes
A [j+1] = A[j]
1 2 3 4 5 6 7
2 9 9 4 8 1 6
j=j-1=1
while j1>0 and A[j1] > x5? No =>Goto line8
A [j+1] = A[2] =x=5
1 2 3 4 5 6 7
1 2 3 4 5 6 7
2 5 5 9 8 1 6
j=j-1=1
while j1 > 0 and A[j1]>x4? No => goto line8
A [j+1] =A[2] = x= 4
1 2 3 4 5 6 7
2 4 5 9 8 1 6
TO BE COMPLETED
i =2, x =A[i] =20, j=1, while j1>0 and A[j1]>X20? No => goto line8
A[j+1] =A[2]= X=20
1 2 3 4 5
15 20 25 4 5
i=3, x= A[i] =25, j= 2, while j2>0 and A[j2] >X25? No => goto line8
A[j+1] =A[3]= X=25
1 2 3 4 5
15 20 25 4 5
Recursive Method
A recursive method is one that calls itself. This causes automatic repetition, a virtual loop. In
fact, most algorithm that use iteration (for loop, while loop, e.t.c) can be recast, replacing each
loop with recursive call. So recursion can be viewed as an alternative to iteration.
The "work toward base case" is where we make the problem simpler. The recursive call is where
we use the same method to solve a simpler version of the problem. The base case is the solution
to the "simplest" possible problem. (For example, the base case to adding a list of numbers
would be if the list had only one number... thus the answer is the number).
The following sections introduce the concepts and techniques of recursive programming and
illustrate with examples of how to “think recursively”.
Many mathematical functions are defined using recursion. Let’s begin with a simple example.
The factorial of a number n can be recursively define as follows:
{
( )
How do you find n! For a given n? To Find 1! is easy because you know that 0! is 1, and so 1! =
1×0! Assuming that you know (n-1)! , you can obtain n! immediately by using n × (n-1)
thus, the problem of computing n! is reduced to computing (n-1)!. When computing (n-
1)!, you can apply the same idea recursively until n is reduced to 0.
Let factorial(n) be the method for computing n! if you call the method with n=0, it
immediately returns the result. The method knows how to solve the simplest case, which is
referred to as the base case or the stopping condition. if you call the method with n > 0, it reduces
the problem into a sub problem for computing the factorial of n-1. The sub problem is
essentially same as the original problem, but it is simpler or smaller. Because the sub problem
Series 0 1 1 2 3 5 8 13
Indexes 0 1 2 3 4 5 6 7
The Fibonacci-series begins with 0 and 1, and each subsequence number is the sum of the
preceding two. The series can be recursively defined as
( ) {
( ) ( )
How do you find fib(index) for a given index? it is easy to find fib(2), because you know
fib(0) and fib(1). Assuming that you know fib(index-2) and fib(index-1), you
can obtain fib(index) immediately. Thus, the problem of computing fib(index) is
reduced to computing fib(index-2) and fib(index-1). When doing so, you apply the
idea recursively until index is reduced to 0 or 1.
The base case is index=0 or index=1. If you call the method with index=0 or index=1, it
immediately returns the result. If you call the method with index>=2, it divides the problem
into two sub problems for computing fib(index-1) and fib(index-2) using recursive
calls. The recursive method for computing the fib(index) can be written as follows:
Case study 3: Algorithm for finding the k-th even natural number
Note here that this can be solved very easily by simply outputting 2*(k - 1) for a given k .
Let us see how the same problem can be solved by an iterative algorithm.
By way of comparison, let us illustrate the recursion solution, which is the purpose of the lecture.
if k = 1, then return 0;
else return Even(k-1) + 2 .
Here the computation of Even(k) is reduced to that of Even for a smaller input value, that
is Even(k-1). Even(k) eventually becomes Even(1) which is 0 by the first line. For example, to
compute Even(3), Algorithm Even(k) is called with k = 2. In the computation
of Even(2), Algorithm Even(k) is called with k = 1. Since Even(1) = 0, 0 is returned for the
computation of Even(2), and Even(2) = Even(1) + 2= 2 is obtained. This value 2 for Even(2) is
now returned to the computation of Even(3), and Even(3) = Even(2) + 2 = 4 is obtained.
Exercise
Consider the following recursive method that receives an array data and lower index (low) as
well as the higher index (high) of the array.
What will be the output when dothing2Array (data,0,7) is executed with the array
data =4,3,6,2,7,8,9,5.
MERGE SORT
The merge sort algorithm can be described recursively as follows: the algorithm divides the array
into 2 halves and applies a merge sort on each half recursively. After the 2 halves are sorted,
merge them.
1. mergeSort (A, p, r) {
2. if P < R {
3. q = (p + r)/2
4. mergeSort (A, p, q)
5. mergeSort (A, p, q+1, r)
6. merge (A, p, q, r)
7. }
8. }
2 9 5 4 8 1 6 7
2 9 5 4 8 1 6 7
2 9 5 4 8 1 6 7
1 6 7
2 9 5 4 8
2 9 4 5 1 8 6 7
2 4 5 9 1 6 7 8
1 2 4 5 6 7 8 9
Figure ms: Merge sort employs a divide-and-conquer approach to sort the array.
Another way to demonstrate how merge-sort can be used to sort an array is shown in the figure
below. The figure shows how merge-sort is used to sort array A. where;
A=
9 6 5 0 8 2
ms(1 6)
A= 1 2 3 4 5 6
9 6 5 0 8 2
Suppose we have an array A[1..m] and the three indices p, q and r, with 1≤p≤q<r≤m, such
that both the sub arrays A[p,q] and A[q+1,r] are individually sorted in nondecreasing order.
We want to rearrange the elements in A so that the elements in the sub array A[p..r] are
sorted in non-decreasing order. This process is referred to as merging A[p..q] with
A[q+1..r]. An algorithm to merge these two sub arrays works as follows:
We maintain two pointers s and t that initially point to A[p] and A[q+1], respectively. We
prepare an empty array B[p..r] which will be used as temporary storage. Each time, we
compare the element A[s] and A[t] and append the smaller of the two to the auxiliary array B;
if they are equal we will choose to append A[s]. Next, we update the pointers: if A[s]<=A
[t], then we increment s, otherwise we increment t. This process ends when s=q+1 or t=r+1.
In the first case, we append the remaining elements A[t..r] to B, and in the second case, we
append A[s..q] to B. Finally, the array B [p..r] is copied back to A[p..r].This
procedure is given in Algorithm MERGE
Algorithm: MERGE
Input: An array A[1..m] of elements and three indices p, q, and r, with 1≤p≤q<r≤m, such
that both the sub arrays A[p..q] and A[q+1, r] are sorted individual in non
decreasing order.
Output: A[p..r] Contains the result of merging the sub arrays A[p..q] and
A[q+1..r].
= 1 and r =5.
s = p=1; t = q+1; k=p
1 2 3 4 5
A → 21 34 50 20 40
↑ ↑ ↑
p,s t r,m
Iteration 1
while s1 q3 and t 4 r 5 yes
Is A[s 1]21 A[t 4]20 ? No
B[k 1] A[t 4] 20
s 1
t t 1 4 1 5
k k 1 11 2
Iteration 2
while s1 q3 and t 5 r 5 yes
Is A[s 1]21 A[t 5]40 ? yes
B[k 2] A[s 1] 21
s s 1 11 2
t 5
k k 1 2 1 3
Iteration 3
while s 2 q3 and t 5 r 5 yes
Is A[s 2]34 A[t 5]40 ? yes
B[k 2] A[s 2] 34
s s 1 2 1 3
t 5
k k 1 3 1 4
Iteration 4
while s3 q3 and t 5 r 5 yes
Is A[s 3]50 A[t 5]40 ? NO
B[k 4] A[t 5] 40
s3
t t 1 5 1 6
k k 1 4 1 5
Is s== q+1? No
B[k5..r5] = A[s3..q3]
A[p..r]=B[p..r]