Unit I: Introduction to Artificial Intelligence (AI)

What Is Artificial Intelligence?

Artificial Intelligence is a method of making a computer, a computer-

controlled robot, or a software think intelligently like the human mind. AI is
accomplished by studying the patterns of the human brain and by analyzing
the cognitive process. The outcome of these studies develops intelligent
software and systems.

A Brief History of Artificial Intelligence

Here’s a brief timeline of the past six decades of how AI evolved from its
inception.

1956 - John McCarthy coined the term ‘artificial intelligence’ and had the first
AI conference.

1969 - Shakey was the first general-purpose mobile robot built. It is now able
to do things with a purpose vs. just a list of instructions.

1997 - Supercomputer ‘Deep Blue’ was designed, and it defeated the world
champion chess player in a match. It was a massive milestone by IBM to
create this large computer.

2002 - The first commercially successful robotic vacuum cleaner was

created.

2005 - 2019 - Today, we have speech recognition, robotic process

automation (RPA), a dancing robot, smart homes, and other innovations
make their debut.

2020 - Baidu releases the LinearFold AI algorithm to medical and scientific

and medical teams developing a vaccine during the early stages of the
SARS-CoV-2 (COVID-19) pandemic. The algorithm can predict the RNA
sequence of the virus in only 27 seconds, which is 120 times faster than
other methods.

Types of Artificial Intelligence

Below are the various types of AI:

1. Purely Reactive

These machines do not have any memory or data to work with, specializing
in just one field of work. For example, in a chess game, the machine
observes the moves and makes the best possible decision to win.
2. Limited Memory

These machines collect previous data and continue adding it to their

memory. They have enough memory or experience to make proper
decisions, but memory is minimal. For example, this machine can suggest a
restaurant based on the location data that has been gathered.

3. Theory of Mind

This kind of AI can understand thoughts and emotions, as well as interact

socially. However, a machine based on this type is yet to be built.

4. Self-Aware

Self-aware machines are the future generation of these new technologies.

They will be intelligent, sentient, and conscious.

Introduction to AI Searching techniques

Search algorithms are important because they solve various problems in

artificial intelligence. There are various search algorithms in AI. This article

takes you through search algorithms, including their importance and

properties. It will also provide the classification of search algorithms and

identify the popular search algorithms in AI.

What is a search algorithm?

Search algorithms are algorithms that help in solving search problems. A

search problem consists of a search space, start state, and goal state.
Search algorithms help the AI agents to attain the goal state through the
assessment of scenarios and alternatives.

The algorithms provide search solutions through a sequence of actions that

transform the initial state to the goal state. Without these algorithms, AI
machines and applications cannot implement search functions and find
viable solutions.

Properties of search algorithms

Search algorithms have the following main properties:

 Completeness: A search algorithm can be said to be complete if it

provides a solution for a given input when there exists at least one
solution for this input.
 Optimality: Search algorithms are also characterized by optimal
solutions. These are the best solutions given by the search algorithms
at the lowest path cost.
 Time Complexity: These algorithms have a maximum time needed to
accomplish a task or provide a solution. The time taken is usually
based on the complexity of the task.
 Space Complexity: They have a maximum memory or storage space
needed when conducting a search operation. This memory is also
based on the complexity of the task.

How search algorithms work

Artificial intelligence is made possible through AI agents. These agents

perform tasks towards achieving a certain goal and establish the actions that
can result in the desired goal. The series of these actions provide the
solution to the problem.

The AI agents find the best solution for the problem by searching for all the
possible alternatives or solutions. The process of searching is done using
search algorithms.

Search algorithms work in two main phases: defining the problem and
searching in the search space.

Defining the problem

Before formulating a problem, various factors need to be defined to enable

the search algorithms to perform the required course of action. Defining
these factors provides the basis for searching and providing a solution. The
following are the factors that need to be defined:

 Initial state: This is the start state in which the search starts.
 State space: These are all the possible states that can be attained
from the initial state through a series of actions.
 Actions: These are the steps, activities, or operations undertaken by
AI agents in a particular state.
 Goal state: This is the endpoint or the desired state.
 Goal test: This is a test conducted to establish whether a particular
state is a goal state.
 Path cost: This is the cost associated with a given path taken by the
agents.

Types of search algorithms

Based on the search problems we can classify the search algorithms

into uninformed (Blind search) search and informed search
(Heuristic search) algorithms.
Depth-first Search (https://www.simplilearn.com/tutorials/data-structure-
tutorial/dfs-algorithm#:~:text=first%20search%20algorithm.-,What%20is
%20a%20Depth%2DFirst%20Search%20Algorithm%3F,far%20as%20possible
%20before%20backtracking. )
o Depth-first search is a recursive algorithm for traversing a tree or
graph data structure.
o It is called the depth-first search because it starts from the root node
and follows each path to its greatest depth node before moving to the
next path.
o DFS uses a stack data structure for its implementation.
o The process of the DFS algorithm is similar to the BFS algorithm.

Note: Backtracking is an algorithm technique for finding all possible solutions

using recursion.

Advantage:

o DFS requires very less memory as it only needs to store a stack of the
nodes on the path from root node to the current node.
o It takes less time to reach to the goal node than BFS algorithm (if it
traverses in the right path).

Disadvantage:

o There is the possibility that many states keep re-occurring, and there is
no guarantee of finding the solution.
o DFS algorithm goes for deep down searching and sometime it may go
to the infinite loop.

Example:

In the below search tree, we have shown the flow of depth-first search, and it
will follow the order as:

Root node--->Left node ----> right node.

It will start searching from root node S, and traverse A, then B, then D and E,
after traversing E, it will backtrack the tree as E has no other successor and
still goal node is not found. After backtracking it will traverse node C and
then G, and here it will terminate as it found goal node.
Completeness: DFS search algorithm is complete within finite state space
as it will expand every node within a limited search tree.

Time Complexity: Time complexity of DFS will be equivalent to the node

traversed by the algorithm. It is given by:

T(n)= 1+ n2+ n3 +.........+ nm=O(nm)

Where, m= maximum depth of any node and this can be much larger
than d (Shallowest solution depth)

Space Complexity: DFS algorithm needs to store only single path from the
root node, hence space complexity of DFS is equivalent to the size of the
fringe set, which is O(bm).

Optimal: DFS search algorithm is non-optimal, as it may generate a large

number of steps or high cost to reach to the goal node.

What is BFS Algorithm (Breadth-First Search)?

(https://www.vtupulse.com/artificial-intelligence/breadth-first-search-
example-advantages-and-disadvantages/ )

Breadth-first search (BFS) is an algorithm that is used to graph data or

searching tree or traversing structures. The full form of BFS is the Breadth-
first search.
The algorithm efficiently visits and marks all the key nodes in a graph in an
accurate breadthwise fashion. This algorithm selects a single node (initial or
source point) in a graph and then visits all the nodes adjacent to the selected
node. Remember, BFS accesses these nodes one by one.

Once the algorithm visits and marks the starting node, then it moves
towards the nearest unvisited nodes and analyses them. Once visited, all
nodes are marked. These iterations continue until all the nodes of the graph
have been successfully visited and marked.

What is Graph traversals?

A graph traversal is a commonly used methodology for locating the vertex
position in the graph. It is an advanced search algorithm that can analyze
the graph with speed and precision along with marking the sequence of the
visited vertices. This process enables you to quickly visit each node in a
graph without being locked in an infinite loop.

The architecture of BFS algorithm

1. In the various levels of the data, you can mark any node as the starting
or initial node to begin traversing. The BFS will visit the node and mark
it as visited and places it in the queue.
2. Now the BFS will visit the nearest and un-visited nodes and marks
them. These values are also added to the queue. The queue works on
the FIFO model.
3. In a similar manner, the remaining nearest and un-visited nodes on the
graph are analyzed marked and added to the queue. These items are
deleted from the queue as receive and printed as the result.
Why do we need BFS Algorithm?
There are numerous reasons to utilize the BFS Algorithm to use as searching
for your dataset. Some of the most vital aspects that make this algorithm
your first choice are:

 BFS is useful for analyzing the nodes in a graph and constructing the
shortest path of traversing through these.
 BFS can traverse through a graph in the smallest number of iterations.
 The architecture of the BFS algorithm is simple and robust.
 The result of the BFS algorithm holds a high level of accuracy in
comparison to other algorithms.
 BFS iterations are seamless, and there is no possibility of this algorithm
getting caught up in an infinite loop problem.

Generate and Test Search:

Generate and Test Search is a heuristic search technique based on Depth

First Search with Backtracking which guarantees to find a solution if done
systematically and there exists a solution. In this technique, all the
solutions are generated and tested for the best solution. It ensures that the
best solution is checked against all possible generated solutions.
It is also known as British Museum Search Algorithm as it’s like looking for
an exhibit at random or finding an object in the British Museum by
wandering randomly.

The evaluation is carried out by the heuristic function as all the solutions
are generated systematically in generate and test algorithm but if there are
some paths which are most unlikely to lead us to result then they are not
considered. The heuristic does this by ranking all the alternatives and is
often effective in doing so. Systematic Generate and Test may prove to be
ineffective while solving complex problems. But there is a technique to
improve in complex cases as well by combining generate and test search
with other techniques so as to reduce the search space. For example in
Artificial Intelligence Program DENDRAL we make use of two techniques,
the first one is Constraint Satisfaction Techniques followed by Generate and
Test Procedure to work on reduced search space i.e. yield an effective
result by working on a lesser number of lists generated in the very first
step.

Algorithm

1. Generate a possible solution. For example, generating a particular point

in the problem space or generating a path for a start state.
2. Test to see if this is a actual solution by comparing the chosen point or
the endpoint of the chosen path to the set of acceptable goal states
3. If a solution is found, quit. Otherwise go to Step 1

Properties of Good Generators:

The good generators need to have the following properties:

 Complete: Good Generators need to be complete i.e. they should
generate all the possible solutions and cover all the possible states. In
this way, we can guaranty our algorithm to converge to the correct
solution at some point in time.
 Non Redundant: Good Generators should not yield a duplicate solution
at any point of time as it reduces the efficiency of algorithm thereby
increasing the time of search and making the time complexity
exponential. In fact, it is often said that if solutions appear several times
in the depth-first search then it is better to modify the procedure to
traverse a graph rather than a tree.
 Informed: Good Generators have the knowledge about the search
space which they maintain in the form of an array of knowledge. This
can be used to search how far the agent is from the goal, calculate the
path cost and even find a way to reach the goal.

Hill Climbing | Artificial Intelligence

Hill climbing is a simple optimization algorithm used in Artificial Intelligence
(AI) to find the best possible solution for a given problem. It belongs to the
family of local search algorithms and is often used in optimization problems
where the goal is to find the best solution from a set of possible solutions.
 In Hill Climbing, the algorithm starts with an initial solution and then
iteratively makes small changes to it in order to improve the solution.
These changes are based on a heuristic function that evaluates the
quality of the solution. The algorithm continues to make these small
changes until it reaches a local maximum, meaning that no further
improvement can be made with the current set of moves.
 There are several variations of Hill Climbing, including steepest ascent
Hill Climbing, first-choice Hill Climbing, and simulated annealing. In
 steepest ascent Hill Climbing, the algorithm evaluates all the possible
moves from the current solution and selects the one that leads to the
best improvement. In first-choice Hill Climbing, the algorithm randomly
selects a move and accepts it if it leads to an improvement, regardless
of whether it is the best move. Simulated annealing is a probabilistic
variation of Hill Climbing that allows the algorithm to occasionally accept
worse moves in order to avoid getting stuck in local maxima.
Hill Climbing can be useful in a variety of optimization problems, such as
scheduling, route planning, and resource allocation. However, it has some
limitations, such as the tendency to get stuck in local maxima and the lack
of diversity in the search space. Therefore, it is often combined with other
optimization techniques, such as genetic algorithms or simulated annealing,
to overcome these limitations and improve the search results.

Advantages of Hill Climbing algorithm:

1. Hill Climbing is a simple and intuitive algorithm that is easy to

understand and implement.
2. It can be used in a wide variety of optimization problems, including those
with a large search space and complex constraints.
3. Hill Climbing is often very efficient in finding local optima, making it a
good choice for problems where a good solution is needed quickly.
4. The algorithm can be easily modified and extended to include additional
heuristics or constraints.

Disadvantages of Hill Climbing algorithm:

1. Hill Climbing can get stuck in local optima, meaning that it may not find
the global optimum of the problem.
2. The algorithm is sensitive to the choice of initial solution, and a poor
initial solution may result in a poor final solution.
3. Hill Climbing does not explore the search space very thoroughly, which
can limit its ability to find better solutions.
4. It may be less effective than other optimization algorithms, such as
genetic algorithms or simulated annealing, for certain types of problems.
Hill Climbing is a heuristic search used for mathematical optimization
problems in the field of Artificial Intelligence.
Given a large set of inputs and a good heuristic function, it tries to find a
sufficiently good solution to the problem. This solution may not be the
global optimal maximum.
 In the above definition, mathematical optimization problems imply
that hill-climbing solves the problems where we need to maximize or
minimize a given real function by choosing values from the given inputs.
Example-Travelling salesman problem where we need to minimize the
distance traveled by the salesman.
 ‘Heuristic search’ means that this search algorithm may not find the
optimal solution to the problem. However, it will give a good solution in
a reasonable time.
 A heuristic function is a function that will rank all the possible
alternatives at any branching step in the search algorithm based on the
available information. It helps the algorithm to select the best route out
of possible routes.
Features of Hill Climbing
1. Variant of generating and test algorithm:
It is a variant of generating and testing algorithms. The generate and test
algorithm is as follows:
 Generate possible solutions.
 Test to see if this is the expected solution.
 If the solution has been found quit else go to step 1.
Hence we call Hill climbing a variant of generating and test algorithm as it
takes the feedback from the test procedure. Then this feedback is utilized
by the generator in deciding the next move in the search space.
2. Uses the Greedy approach:
At any point in state space, the search moves in that direction only which
optimizes the cost of function with the hope of finding the optimal solution
at the end.
1. Introduction
The best-first search is a class of search algorithms aiming to find
the shortest path from a given starting node to a goal node in a
graph using heuristics. The AO* algorithm is an example of the best-
first search class. In this tutorial, we’ll discuss how it works.
This class should not be mistaken for breadth-first search, which is just an
algorithm and not a class of algorithms. Moreover, the best-first search
algorithms are informed, which means they have prior knowledge about the
distances from heuristics. However, the breadth-first search algorithm is
uninformed.
Introduction:
AO* algorithm is a best first search algorithm. AO* algorithm uses the
concept of AND-OR graphs to decompose any complex problem given into
smaller set of problems which are further solved. AND-OR graphs are
specialized graphs that are used in problems that can be broken down into
sub problems where AND side of the graph represent a set of task that need
to be done to achieve the main goal , whereas the or side of the graph
represent the different ways of performing task to achieve the same main
goal.

In the abovc figure we can see an example of a simple AND-OR graph

wherein, the acquisition of speakers can be broken into sub problems/tasks
that could be performed to finish the main goal. The sub task is to either
steal speakers which will directly helps us achieve the main goal "or" earn
 some money "and" buy speakers which helps us achieve the main goal. The
AND part of the graphs are represented by the AND-ARCS, referring that all
the sub problems with the AND-ARCS need to be solved for the predecessor
node or problem to be completed. The edges without AND-ARCS are OR sub
problems that can be done instead of the sub problems with And-arcs. It is to
be noted that several edges can come from a single node as well as the
presence of multiple AND arcs and multiple OR sub problems are possible.

The AO* algorithm is a knowledge-based search technique, meaning the

start state and the goal state is already defined, and the best path is found
using heuristics. The time complexity of the algorithm is significantly
reduced due to the informed search technique. Compared to the A*
algorithm, AO* algorithm is very efficient in searching the AND-OR trees very
efficiently.

Working of AO algorithm:
The AO* algorithm works on the formula given belowx:
f(n) = g(n) + h(n)
where,
g(n): The actual cost of traversal from initial state to the current state.

h(n): The estimated cost of traversal from the current state to the goal state.

f(n): The actual cost of traversal from the initial state to the goal state.

o Best First Search Algorithm(Greedy search)

o A* Search Algorithm

1.) Best-first Search Algorithm (Greedy Search):

Greedy best-first search algorithm always selects the path which appears
best at that moment. It is the combination of depth-first search and breadth-
first search algorithms. It uses the heuristic function and search. Best-first
search allows us to take the advantages of both algorithms. With the help of
best-first search, at each step, we can choose the most promising node. In
the best first search algorithm, we expand the node which is closest to the
goal node and the closest cost is estimated by heuristic function, i.e.

1. f(n)= g(n).

Were, h(n)= estimated cost from node n to the goal.

The greedy best first algorithm is implemented by the priority queue.

Best first search algorithm:

 o Step 1: Place the starting node into the OPEN list.
o Step 2: If the OPEN list is empty, Stop and return failure.
o Step 3: Remove the node n, from the OPEN list which has the lowest
value of h(n), and places it in the CLOSED list.
o Step 4: Expand the node n, and generate the successors of node n.
o Step 5: Check each successor of node n, and find whether any node is
a goal node or not. If any successor node is goal node, then return
success and terminate the search, else proceed to Step 6.
o Step 6: For each successor node, algorithm checks for evaluation
function f(n), and then check if the node has been in either OPEN or
CLOSED list. If the node has not been in both list, then add it to the
OPEN list.
o Step 7: Return to Step 2.

Advantages:

o Best first search can switch between BFS and DFS by gaining the
advantages of both the algorithms.
o This algorithm is more efficient than BFS and DFS algorithms.

Disadvantages:

o It can behave as an unguided depth-first search in the worst case

scenario.
o It can get stuck in a loop as DFS.
o This algorithm is not optimal.

Example:

Consider the below search problem, and we will traverse it using greedy
best-first search. At each iteration, each node is expanded using evaluation
function f(n)=h(n) , which is given in the below table.
In this search example, we are using two lists which
are OPEN and CLOSED Lists. Following are the iteration for traversing the
above example.

Expand the nodes of S and put in the CLOSED list

Initialization: Open [A, B], Closed [S]

Iteration 1: Open [A], Closed [S, B]

Iteration 2: Open [E, F, A], Closed [S, B]

: Open [E, A], Closed [S, B, F]

Iteration 3: Open [I, G, E, A], Closed [S, B, F]

: Open [I, E, A], Closed [S, B, F, G]

Hence the final solution path will be: S----> B----->F----> G

Time Complexity: The worst case time complexity of Greedy best first
search is O(bm).

Space Complexity: The worst case space complexity of Greedy best first
search is O(bm). Where, m is the maximum depth of the search space.

Complete: Greedy best-first search is also incomplete, even if the given

state space is finite.

Optimal: Greedy best first search algorithm is not optimal.

2.) A* Search Algorithm:

A* search is the most commonly known form of best-first search. It uses

heuristic function h(n), and cost to reach the node n from the start state g(n).
It has combined features of UCS and greedy best-first search, by which it
solve the problem efficiently. A* search algorithm finds the shortest path
through the search space using the heuristic function. This search algorithm
expands less search tree and provides optimal result faster. A* algorithm is
similar to UCS except that it uses g(n)+h(n) instead of g(n).

In A* search algorithm, we use search heuristic as well as the cost to reach

the node. Hence we can combine both costs as following, and this sum is
called as a fitness number.

At each point in the search space, only those node is expanded which have
the lowest value of f(n), and the algorithm terminates when the goal node is
found.
Algorithm of A* search:

Step1: Place the starting node in the OPEN list.

Step 2: Check if the OPEN list is empty or not, if the list is empty then return
failure and stops.

Step 3: Select the node from the OPEN list which has the smallest value of
evaluation function (g+h), if node n is goal node then return success and
stop, otherwise

Step 4: Expand node n and generate all of its successors, and put n into the
closed list. For each successor n', check whether n' is already in the OPEN or
CLOSED list, if not then compute evaluation function for n' and place into
Open list.

Step 5: Else if node n' is already in OPEN and CLOSED, then it should be
attached to the back pointer which reflects the lowest g(n') value.

Step 6: Return to Step 2.

Advantages:

o A* search algorithm is the best algorithm than other search algorithms.

o A* search algorithm is optimal and complete.
o This algorithm can solve very complex problems.

Disadvantages:

o It does not always produce the shortest path as it mostly based on

heuristics and approximation.
o A* search algorithm has some complexity issues.
o The main drawback of A* is memory requirement as it keeps all
generated nodes in the memory, so it is not practical for various large-
scale problems.

Example:

In this example, we will traverse the given graph using the A* algorithm. The
heuristic value of all states is given in the below table so we will calculate the
f(n) of each state using the formula f(n)= g(n) + h(n), where g(n) is the cost
to reach any node from start state.
Here we will use OPEN and CLOSED list.
Solution:

Initialization: {(S, 5)}

Iteration1: {(S--> A, 4), (S-->G, 10)}

Iteration2: {(S--> A-->C, 4), (S--> A-->B, 7), (S-->G, 10)}

Iteration3: {(S--> A-->C--->G, 6), (S--> A-->C--->D, 11), (S--> A-->B, 7),
(S-->G, 10)}

Iteration 4 will give the final result, as S--->A--->C--->G it provides the

optimal path with cost 6.

Points to remember:

o A* algorithm returns the path which occurred first, and it does not
search for all remaining paths.
o The efficiency of A* algorithm depends on the quality of heuristic.
o A* algorithm expands all nodes which satisfy the condition f(n)<=""
li="">

Complete: A* algorithm is complete as long as:

o Branching factor is finite.

o Cost at every action is fixed.

Optimal: A* search algorithm is optimal if it follows below two conditions:

o Admissible: the first condition requires for optimality is that h(n)

should be an admissible heuristic for A* tree search. An admissible
heuristic is optimistic in nature.
o Consistency: Second required condition is consistency for only A*
graph-search.

If the heuristic function is admissible, then A* tree search will always find the
least cost path.

Time Complexity: The time complexity of A* search algorithm depends on

heuristic function, and the number of nodes expanded is exponential to the
depth of solution d. So the time complexity is O(b^d), where b is the
branching factor.

Space Complexity: The space complexity of A* search algorithm is O(b^d)