DAA Final
DAA Final
Explain
the different areas
Algorithm is a step-by-step procedure, which defines a set of instructions to be executed
in a certain order to get the desired output. Algorithms are generally created independent
of underlying languages, i.e. an algorithm can be implemented in more than one
programming language.
From the data structure point of view, following are some important categories of
algorithms −
Search − Algorithm to search an item in a data structure.
Sort − Algorithm to sort items in a certain order.
Insert − Algorithm to insert item in a data structure.
Update − Algorithm to update an existing item in a data structure.
Delete − Algorithm to delete an existing item from a data structure.
Characteristics of an Algorithm
Not all procedures can be called an algorithm. An algorithm should have the following
characteristics −
Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or
phases), and their inputs/outputs should be clear and must lead to only one
meaning.
Input − An algorithm should have 0 or more well-defined inputs.
Output − An algorithm should have 1 or more well-defined outputs, and should
match the desired output.
Finiteness − Algorithms must terminate after a finite number of steps.
Feasibility − Should be feasible with the available resources.
Independent − An algorithm should have step-by-step directions, which should be
independent of any programming code.
2. Prim's Algorithm
In this article, we will discuss the prim's algorithm. Along with the algorithm, we will also
see the complexity, working, example, and implementation of prim's algorithm.
Before starting the main topic, we should discuss the basic and important terms such as
spanning tree and minimum spanning tree.
Minimum Spanning tree - Minimum spanning tree can be defined as the spanning tree
in which the sum of the weights of the edge is minimum. The weight of the spanning tree
is the sum of the weights given to the edges of the spanning tree.
Prim's Algorithm is a greedy algorithm that is used to find the minimum spanning tree
from a graph. Prim's algorithm finds the subset of edges that includes every vertex of the
graph such that the sum of the weights of the edges can be minimized.
Prim's algorithm starts with the single node and explores all the adjacent nodes with all
the connecting edges at every step. The edges with the minimal weights causing no
cycles in the graph got selected.
.
3. Merge Sort Algorithm
Merge Sort follows the rule of Divide and Conquer to sort a given set of
numbers/elements, recursively, hence consuming less time.
Before moving forward with Merge Sort, check these topics out first:
Selection Sort
Insertion Sort
Space Complexity of Algorithms
Time Complexity of Algorithms
In the last two tutorials, we learned about Selection Sort and Insertion Sort, both of
which have a worst-case running time of O(n2). As the size of input grows, insertion and
selection sort can take a long time to run.
Merge sort , on the other hand, runs in O(n*log n) time in all the cases.
In Merge Sort, the given unsorted array with n elements, is divided into n subarrays,
each having one element, because a single element is always sorted in itself. Then, it
repeatedly merges these subarrays, to produce new sorted subarrays, and in the end,
one complete sorted array is produced.
2. Conquer the subproblems by solving them. The idea is to break down the
problem into atomic subproblems, where they are actually solved.
3. Combine the solutions of the subproblems to find the solution of the actual
problem.
4. Elaborate various algorithm design strategies.
. Divide and Conquer Approach: It is a top-down approach. The algorithms which follow
the divide & conquer techniques involve three steps:
o Greedy Algorithm always makes the choice (greedy criteria) looks best at the
moment, to optimize a given objective.
o The greedy algorithm doesn't always guarantee the optimal solution however it
generally produces a solution that is very close in value to the optimal.
. Branch and Bound: In Branch & Bound algorithm a given subproblem, which cannot
be bounded, has to be divided into at least two new restricted subproblems. Branch and
Bound algorithm are methods for global optimization in non-convex problems. Branch and
Bound algorithms can be slow, however in the worst case they require effort that grows
exponentially with problem size, but in some cases we are lucky, and the method
coverage with much less effort.
. Backtracking Algorithm: Backtracking Algorithm tries each possibility until they find
the right one. It is a depth-first search of the set of possible solution. During the search, if
an alternative doesn't work, then backtrack to the choice point, the place which presented
different alternatives, and tries the next alternative.
5. Greedy Algorithm
The greedy method is one of the strategies like Divide and conquer used to solve the
problems. This method is used for solving optimization problems. An optimization problem
is a problem that demands either maximum or minimum results. Let's understand through
some terms.
The Greedy method is the simplest and straightforward approach. It is not an algorithm,
but it is a technique. The main function of this approach is that the decision is taken on
the basis of the currently available information. Whatever the current information is
present, the decision is made without worrying about the effect of the current decision in
future.
This technique is basically used to determine the feasible solution that may or may not
be optimal. The feasible solution is a subset that satisfies the given criteria. The optimal
solution is the solution which is the best and the most favorable solution in the subset. In
the case of feasible, if more than one solution satisfies the given criteria then those
solutions will be considered as the feasible, whereas the optimal solution is the best
solution among all the solutions.
6. Backtracking
is a technique based on algorithm to solve problem. It uses recursive calling to find the
solution by building a solution step by step increasing values with time. It removes the
solutions that doesn't give rise to the solution of the problem based on the constraints
given to solve the problem.
Backtracking algorithm is applied to some specific types of problems,
Decision problem used to find a feasible solution of the problem.
Optimization problem used to find the best solution that can be applied.
Enumeration problem used to find the set of all feasible solutions of the problem.
In backtracking problem, the algorithm tries to find a sequence path to the solution
which has some small checkpoints from where the problem can backtrack if no
feasible solution is found for the problem.
Example,
Here,
Green is the start point, blue is the intermediate point, red are points with no
feasible solution, dark green is end solution.
Here, when the algorithm propagates to an end to check if it is a solution or not, if
it is then returns the solution otherwise backtracks to the point one step behind it
to find track to the next point to find solution.
7. Algorithm performance analysis
Performance analysis of an algorithm depends upon two factors i.e. amount of memory
used and amount of compute time consumed on any CPU. Formally they are notified as
complexities in terms of:
Space Complexity.
Time Complexity.
Therefore the total space requirement of any algorithm 'A' can be provided as
Among both fixed and variable component the variable part is important to be
determined accurately, so that the actual space requirement can be identified for an
algorithm 'A'. To identify the space complexity of any algorithm following steps can be
followed:
1. Determine the variables which are instantiated by some default values.
2. Determine which instance characteristics should be used to measure the space
requirement and this is will be problem specific.
3. Generally the choices are limited to quantities related to the number and magnitudes of
the inputs to and outputs from the algorithms.
4. Sometimes more complex measures of the interrelationships among the data items can
used
8. Knapsack Problem
Given a set of items, each with a weight and a value, determine a subset of items to
include in a collection so that the total weight is less than or equal to a given limit and the
total value is as large as possible.
The knapsack problem is in combinatorial optimization problem. It appears as a
subproblem in many, more complex mathematical models of real-world problems. One
general approach to difficult problems is to identify the most restrictive constraint, ignore
the others, solve a knapsack problem, and somehow adjust the solution to satisfy the
ignored constraints.
Applications
In many cases of resource allocation along with some constraint, the problem can be
derived in a similar way of Knapsack problem. Following is a set of example.
The graph has two types of traversal algorithms. These are called the Breadth First
Search and Depth First Search.
The Breadth First Search (BFS) traversal is an algorithm, which is used to visit all of the
nodes of a given graph. In this traversal algorithm one node is selected and then all of
the adjacent nodes are visited one by one. After completing all of the adjacent vertices,
it moves further to check another vertices and checks its adjacent vertices again.
Algorithm
bfs(vertices, start)
Input: The list of vertices, and the start vertex.
Output: Traverse all of the nodes, if the graph is connected.
Begin
define an empty queue que
at first mark all nodes status as unvisited
add the start vertex into the que
while que is not empty, do
delete item from que and set to u
display the vertex u
for all vertices 1 adjacent with u, do
if vertices[i] is unvisited, then
mark vertices[i] as temporarily visited
add v into the queue
mark
done
mark u as completely visited
done
End
The Depth First Search (DFS) is a graph traversal algorithm. In this algorithm one starting
vertex is given, and when an adjacent vertex is found, it moves to that adjacent vertex
first and try to traverse in the same manner.
Algorithm
dfs(vertices, start)
Input: The list of all vertices, and the start node.
Output: Traverse all nodes in the graph.
Begin
initially make the state to unvisited for all nodes
push start into the stack
while stack is not empty, do
pop element from stack and set to u
display the node u
if u is not visited, then
mark u as visited
for all nodes i connected to u, do
if ith vertex is unvisited, then
push ith vertex into the stack
mark ith vertex as visited
done
done
End
8. Minimum Spanning Tree
In the above graph, we have shown a spanning tree though it’s not the minimum spanning
tree. The cost of this spanning tree is (5 + 7 + 3 + 3 + 5 + 8 + 3 + 4) = 38.
We will use Prim’s algorithm to find the minimum spanning tree.
Prim’s algorithm is a greedy approach to find the minimum spanning tree. In this
algorithm, to form a MST we can start from an arbitrary vertex.
For linear search, the worst case occurs when the element to search for is not present
in the array. When x is not present, the search () function compares it with all the
elements of arr [] one by one. Therefore, the temporal complexity of the worst case of
linear search would be Θ (n).
We need to predict the distribution of cases. For the linear search problem, assume that
all cases are uniformly distributed. So we add all the cases and divide the sum by (n +
1).
The number of operations in the best case is constant. The best-case time complexity
would therefore be Θ (1) Most of the time, we perform worst-case analysis to analyze
algorithms. In the worst analysis, we guarantee an upper bound on the execution time
of an algorithm which is good information.
The average case analysis is not easy to do in most practical cases and is rarely done.
In the average case analysis, we need to predict the mathematical distribution of all
possible inputs. The Best Case analysis is wrong. Guaranteeing a lower bound on an
algorithm does not provide any information because in the Worst Case scenario an
algorithm can take years to run.
13. State the greedy method.
A greedy Algorithm solves problems by making the choice that seems to be the best at
that particular moment. There are many optimization problems that can be determined
using a greedy algorithm. A greedy algorithm may provide a solution that is close to
optimal to some issues that might have no efficient solution. A greedy algorithm works if
a problem has the following two properties:
Greedy Choice Property: By creating a locally optimal solution we can reach a globally
optimal solution. In other words, by making “greedy” choices we can obtain an optimal
solution. Optimal substructure: Optimal solutions will always contain optimal
subsolutions. In other words, we say that the answers to subproblems of an optimal
solution are optimal.
Examples:
Machine scheduling
Huffman Code
Job Sequencing
A selection function − It is used to choose the best candidate that is to be added to the
solution.
A feasibility function − A feasibility function is useful in determining whether a candidate
can be used to contribute to the solution or not.
An objective function − This is used to assign a value to a solution or a partial solution.
A solution function − A solution function is used to indicate whether a complete solution
has been reached or not.
14. Differentiate between the subset paradigm and ordering paradiam
Subset Paradigm: To solve a problem (or possibly find the optimal/best solution),
greedy approach generate subset by selecting one or more available choices. Eg.
includes Knapsack problem, job sequencing with deadlines. In both of the problems
greedy create a subset of items or jobs which satisfies all the constraints.
Merge sort is yet another sorting algorithm that falls under the category of Divide and
Conquer technique. It is one of the best sorting techniques that successfully build a
recursive algorithm.
Divide: Rearrange the elements and split arrays into two sub-arrays and an element in
between search that each element in left sub array is less than or equal to the average
element and each element in the right sub- array is larger than the middle element.
3. If the Pivot Element (the item to be searched) is less than the item in the middle of the
interval, We discard the second half of the list and recursively repeat the process for the
first half of the list by calculating the new middle and last element.
4. If the Pivot Element (the item to be searched) is greater than the item in the middle of
the interval, we discard the first half of the list and work recursively on the second half by
calculating the new beginning and middle element.
In this context, using Strassen’s Matrix multiplication algorithm, the time consumption can
be improved a little bit.
Strassen’s Matrix multiplication can be performed only on square matrices where n is
a power of 2. Order of both of the matrices are n × n.
Divide X, Y and Z into four (n/2)×(n/2) matrices as represented below −
Z=[IKJL]Z=[IJKL] X=[ACBD]X=[ABCD] and Y=[EGFH]Y=[EFGH]
Using Strassen’s Algorithm compute the following −
M1:=(A+C)×(E+F)M1:=(A+C)×(E+F)
M2:=(B+D)×(G+H)M2:=(B+D)×(G+H)
M3:=(A−D)×(E+H)M3:=(A−D)×(E+H)
M4:=A×(F−H)M4:=A×(F−H)
M5:=(C+D)×(E)M5:=(C+D)×(E)
M6:=(A+B)×(H)M6:=(A+B)×(H)
M7:=D×(G−E)M7:=D×(G−E)
Then,
I:=M2+M3−M6−M7I:=M2+M3−M6−M7
J:=M4+M6J:=M4+M6
K:=M5+M7K:=M5+M7
L:=M1−M3−M4−M5L:=M1−M3−M4−M5
Analysis
Kruskal's Algorithm is used to find the minimum spanning tree for a connected weighted
graph. The main target of the algorithm is to find the subset of edges by using which we
can traverse every vertex of the graph. It follows the greedy approach that finds an
optimum solution at every stage instead of focusing on a global optimum.
The traveling salesman problem (TSP) is an algorithmic problem tasked with finding the
shortest route between a set of points and locations that must be visited. In the problem
statement, the points are the cities a salesperson might visit. The salesman‘s goal is to
keep both the travel costs and the distance traveled as low as possible.
Focused on optimization, TSP is often used in computer science to find the most
efficient route for data to travel between various nodes. Applications include identifying
network or hardware optimization methods. The creation of a game that was solvable
by finding a Hamilton cycle, which is a non-overlapping path between all nodes.
TSP has been studied for decades and several solutions have been theorized. The
simplest solution is to try all possibilities, but this is also the most time consuming and
expensive method. Many solutions use heuristics, which provides probability outcomes.
However, the results are approximate and not always optimal. Other solutions include
branch and bound, Monte Carlo and Las Vegas algorithms.
Rather than focus on finding the most effective route, TSP is often concerned with
finding the cheapest solution. In TSPs, the large amount of variables creates a
challenge when finding the shortest route, which makes approximate, fast and cheap
solutions all the more attractive.