II Module EIT
II Module EIT
Problem solving Techniques: In order to instruct a computer correctly, the user must have
clear understanding of the problem to be solved.
Apart from this the user should be able to develop a method in the form of series of
sequential steps, to solve it.
Once the problem is well defined and a method of solving it is developed, then instructing the
computer to solve the problems becomes relatively easier task.
Thus, before attempt to write a computer program to solve a given problem it is necessary to
formulate or define the problem in a precise manner. Once the problem is defined, the steps
required to solve it, must be stated clearly in the required order.
Steps to problem solving:
A Computer cannot solve a problem on its own. One has to provide step by step solutions of
the problem to the computer. In fact, the task of problem solving is not that of the computer.
It is the programmer who has to write down the solution to the problem in terms of simple
operations which the computer can understand and execute.
In order to solve a problem by the computer, one has to pass through certain stages or steps.
They are:
i) Understanding the problem:
Here we try to understand the problem to be solved in totally. Before with the next stage or
step, we should be absolutely sure about the objectives of the given problem.
ii) Analyzing the problem:
After understanding thoroughly, the problem to be solved, we look different ways of solving
the problem and evaluate each of these methods. The ideal here is to search an appropriate
solution to the problem. The end result of this stage is a broad overview of the sequence of
operations that are to be carries out to solve the given problem.
iii) Developing the solution:
Here the overview of the sequence of operations that was the result of analysis stage is
expanded to form a detailed step by step solution to the problem consideration.
iv) Coding and implementation:
The last stage of the problem solving is the conversion of the sequence of operations into a
language that the computer can understand.
Computational Problems: it is a mathematical object, representing a collection of questions
that computer might be able to solve.
Computational problems are one of the main objects of study in computer sciences.
The field of algorithms studies methods of solving computational problems efficiently.
A computational problem can be viewed as an infinite collection of instances together with a
solution for every instances.
Types of computational problems:
i) A decision problem:
Is a computational problem where the answer for every instance is either yes or no. An
example of a decision problem is primality testing: “Given a positive integer n, determine if n
is prime” A decision problem is typically represented as the set of all instance for which the
answer is yes.
ii) A search problem:
In a search problem, the answers can be arbitrary strings for example, factoring is a search
problem where the instances are positive integers and the solution are collections of primes.
iii) Counting problems:
A counting problems ask for the number of solutions to given search problem.
iv) An optimization problems:
An optimization problems ask for finding a “best possible” solution among the set of all
possible solutions to a search problem.
v) Function problems:
In a function problem a single output is expected for every input but the output is more
complex than that of a decision problem, that is it is not just Yes or No.
Introduction to algorithm:
A set of sequential steps usually written in ordinary language to solve a given problem is
called algorithm.
Algorithm is defined as a formula or set of steps for a solving a particular problem in a finite
amount of time. The set of steps in an algorithm must be unambiguous and have a clear
stopping point.
Using algorithm to solve problems:
Algorithm are used in our daily life. For example: a recipe for making coffee is an algorithm.
All programs consist of algorithm. Designing algorithm which are simple and require the
fewest number of steps is one of the biggest challenges in programming.
Steps involved in algorithm development / solving the problem.
Example: Algorithm for Find the average of three numbers
i) Identify the inputs and outputs:
In the above problem statement, average of three numbers is the desired output. The three
numbers are the input.
All the algorithms should have some input. The logic of the algorithm should work on this
input to give the desired result. The input is to be identified first for any specified problem.
At least one output should be produced from the algorithm based on the input given.
ii) Identify any other data and constants required to solve the problem:
The average of number is defined as : sum of all inputs/total number of inputs
Here the other data that has to be computed is the sum. The total number of inputs is ‘3’
which is a constant in this case.
In this step identify other data and constants other than the input and output, required for
solving the problem. And also type of constants.
Properties of Algorithm:
Professor Donald Knuth has given list of five properties of an algorithm.
1) Finiteness: An algorithm must always terminate after a finite number of steps. The
algorithm can’t lead to an infinite condition.
2) Definiteness: Each step of an algorithm must be precisely defined, the actions to be carried
out must be rigorously and unambiguously specified for each case.
3) Effectiveness: All of the operations to be performed in the algorithm must be sufficiently
basic that they can in principle be done exactly and in a finite length of time using the given
input.
In other words the operations must be sufficiently basic to convert correct input into expected
output.
Example: find the square root of a number
BEGIN
Step 1: Read number
Step 2: calculate the square root
Step 3: display square root
END
In the above example step 2 is not a basic operation of an algorithm. (square root = number *
number)
4) Input: Quantities which are given to it initially before the algorithm begins. These inputs
are taken from specified sets of objects.
5) Output : Quantities which have a specified relation to input.
Patterns in algorithm:(Constructs of Algorithm)
An algorithm can have different patterns (constructs) in it:
i) Sequential: A sequential pattern is one where different steps occur in a sequence.
The steps written in the algorithm will execute the order in which they appear in the
algorithm. Eg. The above algorithm to find the average of three number is sequential.
ii) Conditional (selectional): A conditional pattern is one where the flow of control of
statement execution is based on condition in order to achieve the required result.
For Example:
BEGIN
Step 1: Read marks1, marks2, marks3
Step 2: Total = marks1+marks2+marks3
Step 3: Average = total / 3
Step 4: set status = “Student passes”
Step 5: If average = 40 then set status = “student failed”
Step 6: Display status.
END
In step5, the execution of the statement set status = “student failed”, depends upon the value
of the average. The value which is displayed to the user will change on the basis of the value
of average.
iii) Iterational : in an iterational pattern, a task (one or more steps) is repeated more than
once. In programming languages, an iterational pattern is implemented using loops. It is also
known as “repetitive” construct.
Iterational construct are used when a part of the algorithm is to be executed several times
depending upon some condition. The condition will decide how many time a part of the
algorithm will execute.
For eg:
BEGIN
Step 1: FACT = 1
Step 2: COUNT = 1
Step 3: FACT = FACT * COUNT
Step 4: COUNT = COUNT + 1
Step 5: IF COUNT <=10 GO TO Step 3.
Step 6: DISPLAY FACT
END
In the above algorithm calculating the factorial of a number is represents an iterational
construct at step 5 which decides how many times the steps 3, 4, 5 will execute.
Flowcharts:(Implementation of algorithm using flow charts)
A flowchart is an organized combination of shapes, lines and text which graphically illustrate
a process. A flowchart can be used to pictorially represent an algorithm. Flowchart is one of
the ways software engineers document an algorithm.
Flowchart symbols:
Flowcharts use standardized symbols to visually represent various aspects of an algorithm or
a process.
In this section, we will study the different flowchart symbols and their use.
Terminator: A terminator symbols is used to represent the beginning and end of an algorithm
or process. Usually it can be either “START” or “STOP”.
START Terminator
(Start/stop)
END
Connector lines: Connector lines are used to connect the symbols in a flow chart. The
direction of the arrow indicates the next step.
Connector lines
Process Sum=Num1+N
um2+Num3
Data: a data symbol represents data used in the algorithm. It is also used to represent the
input and output. The symbol contains text which describes the operation on data. When
writing flowcharts, if required, multiple inputs can be read or multiple data can be initialized
in the same symbol.
Decision: A decision symbol is used to branch into different steps based on condition.
Usually the decision contains the test condition. And based on whether the condition
succeeds or fails, connector lines connect to different points in the flowchart.
Decision (used true
For branching Count
Based on condition) < 10 ?
False
On page and Off page references: reference symbols are used when entire flowchart cannot
fit on the same page fully. The reference indicates from where the flow continues.
On-page reference symbol is used to continue flowchart on the same page.
Off page reference symbol is used to continue flowchart on the next page.
On page reference P2 off page reference
A
4) Analyzing algorithm: Any algorithm, when implemented, uses the computers memory
and CPU for execution. To analyze an algorithm, there is a need to determine how
much time an algorithm takes when implemented and how much space it occupies in
primary memory.
Analysis of algorithm needs great mathematical skills, it enables:
Quantitative studies on algorithms (refers to study with some facts and
number)
Knowledge if the software will meet the qualitative requirements.
The following steps are recommended for analysis:
Given an algorithm, determine the kind of operation it is performing (Eg.
Arithmetic, floating point, assignment operations, procedure execution)
Get the set of data which helps the algorithm to exhibit all kinds of behaviour.
(eg. In finding the roots of the quadratic equation ax 2 + bx + c =0, get the
combinations of a, b, c such that b2 < 4*a*c, b2 = 4*a*c, b2 = 4*a*c. )
The analysis is carried out in two stages: i) Priori analysis ii) Posteriori analysis
i) Priori Analysis: Analysis of the algorithm done before running the algorithm on any
computer is called priori analysis.
ii) Posteriori analysis: During this stage of analysis, the target machine is identified,
algorithm is converted to a program and run on the machine.
The principle of priori analysis was introduced by Donal Knuth. To perform the priori
analysis there is a need of some tools. Tools includes the instruction count and problem size.
While analysing the algorithms based on priori analysis, three different cases are identified.
i) Worst case: Refer to maximum number of instruction/operations that could be
executed by the algorithm in order to give the desired output
Eg. For sorting algorithm, the number of inputs is total number of un sorted
elements, the number of outputs is the total number of sorted elements. The
number of operations involved in the algorithm (total number of comparisons).
ii) Average case: Refers to number of instructions/ operations that could be executed
by the algorithm on an average in order to give the desired output.
iii) Best case: Refer to maximum number of instruction/operations that could be
executed by the algorithm in order to give the desired output.
Worst case:
The goodness of an algorithm is expressed usually in terms of its worst case running time.
Worst case running time of an algorithm determined using Big ‘O’ notation is the upper bond
for time of execution of that algorithm for different problem sizes under the worst case
problem conditions. O( ) its meaning is ‘grown as’ or ‘behave as’ – it is defined as the rate at
which the time of execution grows in terms of the problem size.
For Eg. In searching if the number to be searched is in the last position of the array, we call
this condition as the worst case.
Average case:
The average case execution time of an algorithm is arrived at by considering the average
condition of the problem.
For eg: in searching if the number to be searched is in the mid position in the search
elements, we call this condition as the average case.
Best case:
Refers to ideal working condition for an algorithm. Algorithm will have best possible
condition for execution.
For eg: in searching if the number to be searched is in the first position in the search
elements, we call this condition as the best case.
Algorithm techniques:
Algorithm techniques help in designing different algorithm for problem solving. Five well
known algorithm techniques are:
i) Brute force technique: This is one of the most popular problem solving technique.
In this approach, the algorithm is based on the problem statement and considers
complete range of input values for execution.
Eg: counting the total number of orders placed by customers.
Some of the algorithms based on this principle are:
a) Selection sort b) Bubble sort c) Linear search
ii) Greedy techniques: This technique is primarily used in optimization problems,
optimization problems are ones where in there is need to find the best among all
possible solutions.
In the greedy approach, each step chosen has to satisfy the constraints given in the
problem.
The name greedy technique is given to this approach because each step of the
technique has to satisfy the constraints.
The choice of each step in this method is done based on the following:
a) It must be possible to solve b) It must be irrevocable (not reversed back)
Eg: Traveling sales person. Finding the shortest distance a sales person need
to travel so that he can covers all the places once).
iii) Divide and conquer techniques:
This technique has three steps:
a) The first step is called divide, the given problem is divided into smaller sub
problem and these sub problems are of same size.
b) The second step is called conquer where in we solve these sub problems
separately.
c) The third step is to combine, we combine the solutions of the sub problems to
get the solution.
Algorithms based on these techniques are: quick sort and binary search
algorithm.
iv) Dynamic programming techniques:
Dynamic programming is a design principle which is used to solve problems with
overlapping sub problem. Algorithm based on this techniques are: Fibonacci
series computation and matrix multiplication.
Searching algorithms:
Two types of searching algorithms:
In linear search, the searching process starts from the first item. Searching is continued till
either the item is found or the end of the list reached indicating the it is not found.
Algorithm:
Assume the list of data to be 58, 62, 75, 88, 92 and 105
Problem: data to be searched is 75
Step1: Compare 75 with the first data item. That is compare 75 and 58. Since they are not
equal proceed to the next step.
Step2: Now compare 75 and 62. Since they are not equal proceed to the next step
Step 7: Now the end of the list is reached. There are no more elements in the list. So the
item is “not found” in the list.
Analysis:
i) Worst case analysis: The worst case condition is that the number to be searched is present
as the last element or not present at all. In any of the cases, the number of searches needed is
‘n’ , where ‘n’ is the number of items (problem size) . so the worst case complexity is given
by T(n) = O(n)
ii) Best case analysis: in the best case condition, the number to be searched is present as the
first element in the given set of numbers. The best case complexity of linear search algorithm
is T(n)=O(1).
iii) Average case analysis: the given number is typically or on average expected to be in the
centre of an array, hence n/2. This gives average case complexity T(n/2) =O(n)
T(n) is the time of execution of an algorithm for different problem sizes ‘n’
ii) Binary search Algorithm:
A binary search is a searching technique that can be applied only to a sorted list of items.
This searching technique is similar to dictionary search.
Algorithm:
Step1: Set First – 0 and Last = number of items -1
Step2: Find the middle of the list as
Mid = (First – Last)/2
Step3: Compare the middle item with the searching item. If they are equal than “Item is
found” and go to step 8.
Step4: If the searching item is less than the middle item then the searching item comes before
this middle element. So, set Last = mid – 1 and there is no change in the value of first. Go to
step 6.
Step5: if the above condition is false the searching element should be greater than the middle
element. So First = mid+1 and there is no change in the value of last. Go to the next step.
Step6: if First<=Last than go to step2. Else go to step7.
Step7: Since end of the list is reached, the searching item is “not found” in the list.
Step8: End of the algorithm
Analysis:
The worst case : the number of steps to search a given element in an array of size ‘n’ is equal
to the number of time the array of elements is split and scanned. T(n) = O(log2 n). (log2 n is
the number of split array)
Average case : Based on the assumption that the element can be present in first position or
second position or third position (or not at all present). T(n) = O(log2 n)
Best case: In best case condition the element to be searched is present in mid position of the
given array of ‘n’ element. So number of split in best case is 1. T(n) = O(1).
Comparison of linear and binary search:
Linear Search Binary search
Can be applied on sorted and unsorted list Can be applied only on sorted list of items
of items
Search time is more. Searching time is less.
Since binary search searches only half of the
list, it works faster compared to linear
search.
Sorting Algorithms:
Sorting refers to the arrangement of data items in a particular order. The data items can be
arranged either in the ascending order (increasing order) or in the descending order
(decreasing order).
Bubble sort:
In bubble sort, there are ‘N’ data items to be sorted and then main objective is to move the
heavy weight items towards the end of the list. This is repeated till the entire list is sorted.
i) Bubble Sort:
In bubble sort, there are ‘N’ data items to be sorted and the main objectives is to move the larger
items towards the end of the list. This is repeated till the entire list is sorted.
Analysis:
The worst case condition is that all the elements to be sorted in ascending order are initially
present in descending order. T(n) = O(n2).
Average case : For a given set of ‘n’ array elements. It is assumed that on an average, half of
the pairs of numbers are in reverse order. T(n) = O(n2)
Best case: In best case condition of bubble sort algorithm all the elements of the given array
are already sorted in desired order and no swapping is required.
i) Selection sort algorithm:
In selection Sort, each pass selects the smallest data item from the unsorted set and move it to its
position.
Step1: From the data items in positions 1 to n, select the smallest data item and interchange with 0 th
data item. Now first data item is sorted.
Step2: From the data items in positions 2 to n, select the smallest data item and interchange with 1 st
data item. Now second data item is sorted.
Step3: The steps are repeated n-1 times. At the end of n-1th time the entire data is sorted.
Now pick up first element and compare with items on the list
Analysis:
Worst case: The basic operation in the selection sort algorithm is comparison and swapping. Since
there is a nested for loop it is necessary to do inside out analysis. T(n) = O(n 2).
Average case: In average case condition of the algorithm, half of the elements are arranged in
proper sequence while other half are not proper in sequence. T(n) = O(n 2).
Best case : Best case: In best case condition of bubble sort algorithm all the elements of the
given array are already sorted in desired order. Due to two loops the number of times the
elements are compared with other for sorting is again in the order of n2 where n is the number
of comparisons. T(n) = O(n2).
Insertion Sort:
Analysis:
Merge sort is yet another sorting technique which works on the divide and conquer design principle.
The steps followed in merge sort are:
1. Merge sort works by the dividing the given array into two sub arrays of equal size.
2. The sub arrays are sorted independently using recursion.
3. The sorted sub arrays are then merged to get the solution for the original array.
In this algorithm, breaking of the given input array into two sub arrays of equal size is part of the
divide step. The recursive calls to sort the sub arrays are part of the conquer step. The merging of
the sub arrays to get the solution for the orginal array is part of the combine step.
Algorithm for Merge sort:
1. Begin
2. Set left = 1, right = n
3. If (left = right) then
3.1 set m = floor(left + right / 2)
3.2 merge sort a[left…m]
3.3 merge sort a[m+1…right]
3.4 merge a[left…m] and a[m+1…right] into an auxiliary array b
3.5 copy the elements of b to a[left…right]
4. End.
Example:
Initially merge sort starts by considering the inputs as ‘n’ sorted groups each of length one. These
are merged pair wise to obtain n/2 files of size two. If n is odd one file is of size one. These n/2 files
are then merged pair wise and so on until we are left with only one file. The figure below illustrates
the process of merge sort.
Analysis:
The analysis of worst case analysis is covered in this section.
Again the basic operation in merge sort is comparison and swapping. Merge
sort algorithm calls it self recursively. Merge sort divides the array into sub
arrays based on the position of the elements where as quick sort divides the
array into sub arrays based on the value of the elements.
Merge sort requires an auxiliary array to do the merging. The merging of two
sub arrays, which are already sorted, int an auxiliary array can be done in O(n)
where n is the total number of elements in both the sub arrays. This is
possible because both the sub arrays are sorted.
The whole merge sort when simplified gives worst case as O(n log 2 n).
1) The left sub array contains all elements which are less than or equal to the pivot.
3) The right sub array contains all elements which are greater than or equal to the pivot.
Now the two sub arrays, namely the left sub array and the right sub array are sorted
recursively.
To sort a[left…right] into ascending order:
1 If left < right
1.1 Partion a[left..right] such that
a[left…p-1] are all less than or equal to a[p] and
2. Terminate
Eg: (Important points: Left mark moves toward right if the condition is true. If it is false stop checking and go to
right mark checking. Right mark moves towards left If condition is false. If it is true stop checking and go to
left mark checking. When both stop swap the two number where it is stopped. This will continue till Right
mark > Left mark and stop the process. At that point insert the number i.e 54 and this is the pivot element )
54 26 93 17 77 31 44 55 20 Compare 26 and 54
Left mark -- ------ right mark
54 26 93 17 77 31 44 55 20 26<54 true move right
93<54 false stop & go
to right mark
- - left mark right mark
54 26 20 17 77 31 44 55 93 Swap 20 & 93
Analysis:
i) Worst case analysis: can arise when the partition algorithm divides the array in such a way
that either the left sub array or the right sub array is empty. T(n) = O(n2)
ii) Best case analysis: in the best case condition, the partition algorithm will split the given
array into two equal sub arrays. The pivot turns out to be the medium value in the array.
Then both left and right sub array are of equal size (n/2). So the best case analysis of quick
sort is T(n) = (n log2n).
iii) Average case analysis: the average number of comparison for quick sort is (n log n) This
gives average case complexity T(n) =O(n log n).
Data structures:
Different kinds of data structures are suited to different kinds of applications and some are
highly specialized to specific tasks.
Data structures can be broadly classified in two categories : i) Linear data structures ii) Non
linear data structures.
Data structure is a collection of data that related to each other. Data structure can be divided
in to two types. They are:
i) Primitive data structure: (built in data types) ii) Non Primitive data structures:(Derived
data types)
Linear data structures: Linear data structures elements form a sequence. In Linear data
structures,the data items are arranged in a linear sequence . Example: Arrays, linked list,
stacks, queues etc.
Graphs:
A Graph is a non-linear data structure consisting of nodes and edges. The nodes are
sometimes also referred to as vertices and the edges are lines or arcs that connect any two
nodes in the graph. More formally a Graph can be defined as,
A Graph consists of a finite set of vertices(or nodes) and set of Edges which connect a pair of
nodes.
In the above Graph, the set of vertices V = {0,1,2,3,4} and the set of edges E = {01, 12, 23,
34, 04, 14, 13}.
Graphs are used to solve many real-life problems. Graphs are used to represent networks. The
networks may include paths in a city or telephone network or circuit network. Graphs are also
used in social networks like linkedIn, Facebook. For example, in Facebook, each person is
represented with a vertex(or node). Each node is a structure and contains information like
person id, name, gender, locale etc.