UNIT2 PROBLEMSOLVINGAGENTS

Search Algorithms, Heuristic Search–Heuristic Functions Local Search and OptimizationProblems –

Local Search in Continuous Space – Search with Non–Deterministic Actions –Search in Partially
Observable Environments– Online Search Agents and UnknownEnvironments.
ProblemSolving ApproachtoTypicalAIproblems
Herearesomeof themostcommon problem-solving approaches used in AI:

 Search:Thisisageneral-purposeapproachthatcanbeusedtosolveawidevarietyofproblems.Thebasic idea is to
start with a set of possible solutions and then explore the space of solutions until you find one that meets
your criteria.
 Heuristics: These are rules of thumb that can be used to improve the efficiency of search algorithms.
Heuristics can be very effective in practice, but they are not guaranteed to find the optimal solution.
  Rule-based reasoning: This approach uses a set of rules to determine the best course of action. Rule-
basedreasoningisoftenusedinexpertsystems,whicharecomputerprogramsthatcanmakedecisions in a
particular domain.
 Machine learning: This approach uses data to train a model that can make predictions or decisions.
Machinelearningisapowerfultoolthatcanbeusedtosolveawidevarietyofproblems,butitcan be complex
and time-consuming to develop a good model.

Herearesomeexamplesofhow theseapproachesareusedtosolve typicalAIproblems:

 Search:Searchcanbeusedtosolveavarietyofproblems,suchasfindingtheshortestpathbetween two points,

finding the best way to pack a suitcase, or finding the optimal move in a game.
 Heuristics: Heuristics can be used to improve the efficiency of search algorithms. For example, in the
gameofchess,aheuristiccouldbeusedtoestimatethevalueofaparticularposition.Thisinformation could
then be used to prune the search tree, which would make the search algorithm more efficient.
 Rule-basedreasoning:Rule-basedreasoningisoftenusedinexpertsystems.Forexample,amedical expert
system could use rules to diagnose diseases or recommend treatments.
 Machinelearning:Machinelearningcanbeusedtosolveavarietyofproblems,suchasspamfiltering, image
recognition, and natural language processing.

SearchAlgorithmsinArtificial Intelligence

SearchalgorithmsareoneofthemostimportantareasofArtificial Intelligence.Thistopicwillexplain all about

the search algorithms in AI.

Problem-solving agents:

InArtificialIntelligence,Searchtechniquesareuniversalproblem-solvingmethods.Rationalagentsor
Problem-solving agents in AI mostly used these search strategies or algorithms to solve a specific
problem and provide the best result. Problem-solving agents are the goal-based agents and use atomic
representation. In this topic, we will learn various problem-solving search algorithms.

SearchAlgorithmTerminologies:
CurrentTime0:00/Duration18:10

Search:Searchingisastepbystep proceduretosolveasearch-probleminagivensearchspace.A search

problem can have three main factors:

1. SearchSpace:Searchspacerepresentsasetofpossiblesolutions,whichasystemmay have.
2. StartState:Itisastatefromwhereagentbegins thesearch.
3. Goaltest: Itisafunctionwhichobservethecurrentstateandreturnswhetherthegoalstate is
achieved or not.
 Searchtree:AtreerepresentationofsearchproblemiscalledSearchtree.Therootofthesearch tree is
the root node which is corresponding to the initial state.
 Actions:Itgivesthedescriptionofalltheavailableactionstotheagent.
  Transitionmodel: Adescription ofwhateachaction do,canberepresented asatransition model.
 PathCost: Itisafunctionwhichassignsanumericcosttoeach path.
 Solution:It isan actionsequencewhichleads fromthestartnode tothe goal node.
 OptimalSolution:If asolutionhas thelowest costamong all solutions.

PropertiesofSearch Algorithms:

Followingarethefouressentialpropertiesofsearchalgorithmstocomparetheefficiencyofthese
algorithms:

Completeness:Asearch algorithmissaidtobecompleteifitguaranteestoreturnasolutionifatleast any

solution exists for any random input.

Optimality:Ifasolutionfoundforan algorithmisguaranteedtobethebestsolution(lowestpathcost) among all

other solutions, then such a solution for is said to be an optimal solution.

TimeComplexity: Timecomplexity isa measureof timeforanalgorithm tocompleteits task.

SpaceComplexity:Itisthemaximumstoragespacerequiredatanypointduringthesearch,asthe
complexity of the problem.

Typesofsearchalgorithms

Basedonthesearchproblemswecanclassifythesearchalgorithmsintouninformed(Blindsearch) search and

informed search (Heuristic search) algorithms.
 Uninformed/BlindSearch:
Theuninformedsearchdoesnotcontainanydomainknowledgesuchascloseness,thelocationofthegoal. It
operates in a brute-force way as it only includes information about how to traverse the tree and how to
identifyleafandgoalnodes.Uninformedsearchappliesawayinwhichsearchtreeissearchedwithoutany
information about the search space like initial state operators and test for the goal, so it is also called blind
search. It examines each node of the tree until it achieves the goal node.

Itcanbedividedintofivemain types:

 Breadth-firstsearch
 Uniformcostsearch
 Depth-firstsearch
 Iterativedeepeningdepth-firstsearch
 BidirectionalSearch

Informed Search

Informed search algorithms use domain knowledge. In an informed search, problem information is
availablewhichcanguidethesearch.Informedsearchstrategiescanfindasolutionmoreefficientlythan an
uninformed search strategy. Informed search is also called a Heuristic search.

A heuristic is a way which might not always be guaranteed for best solutions but guaranteed to find a
goodsolutioninreasonabletime.Informedsearchcansolvemuchcomplexproblemwhichcouldnotbe solved in
another way.

Anexampleofinformedsearch algorithmsisatravelingsalesmanproblem.

1. GreedySearch
2. A* Search

UninformedSearch Algorithms

Uninformed search is a class of general-purpose search algorithms which operates in brute force-
way.Uninformedsearchalgorithmsdonothaveadditionalinformationaboutstateorsearchspace other
than how to traverse the tree, so it is also called blind search.

Followingarethe varioustypesofuninformedsearch algorithms:

1. Breadth-firstSearch
2. Depth-firstSearch
3. Depth-limitedSearch
4. Iterativedeepeningdepth-firstsearch
5. Uniformcostsearch
6. Bidirectional Search

1. Breadth-firstSearch:
Breadth-firstsearchisthemostcommonsearchstrategyfortraversingatreeorgraph.Thisalgorithm searches
breadthwise in a tree or graph, so it is called breadth-first search.
  BFSalgorithmstartssearchingfromtherootnodeofthetreeandexpandsallsuccessornodeatthe current
level before moving to nodes of next level.
 Thebreadth-firstsearchalgorithmisanexample ofageneral-graphsearchalgorithm.
 Breadth-firstsearchimplementedusingFIFOqueuedatastructure.

Advantages:

 BFSwillprovide asolutionif anysolution exists.

 Iftherearemorethanonesolutionsforagivenproblem,thenBFSwillprovidetheminimal solution
which requires the least number of steps.

Disadvantages:

 Itrequireslotsofmemorysinceeachlevelofthetreemustbesavedintomemorytoexpandthe next level.

 BFSneedslotsoftime ifthesolutionisfar awayfromtheroot node.

Example:

In the below tree structure, we have shown the traversing of the tree using BFS algorithm from the root
nodeStogoalnodeK.BFSsearchalgorithmtraverseinlayers,soitwillfollowthepathwhichisshown by the dotted
arrow, and the traversed path will be:

1. S--->A--->B---->C--->D---->G--->H--->E---->F---->I----->K

Time Complexity: Time Complexity of BFS algorithm can be obtained by the number of nodes
traversedinBFSuntiltheshallowestNode. Wherethed=depthofshallowestsolutionandbisanode at every
state.
 T (b) = 1+b2+b3+.....+bd= O (bd)

SpaceComplexity:SpacecomplexityofBFSalgorithmisgivenbytheMemorysizeoffrontierwhich is O(bd).

Completeness:BFSiscomplete,whichmeansiftheshallowestgoalnodeisat somefinitedepth,then BFS will

find a solution.

Optimality:BFS isoptimal if pathcost is anon-decreasing functionofthedepthof thenode.

2. Depth-firstSearch

 Depth-firstsearchisa recursivealgorithm fortraversingatreeorgraphdatastructure.

 Itiscalledthedepth-firstsearchbecauseitstartsfromtherootnodeandfollowseachpathtoits greatest
depth node before moving to the next path.
 DFSusesastackdata structureforits implementation.
 Theprocess ofthe DFS algorithmis similar tothe BFS algorithm.

Note:Backtrackingisanalgorithmtechniqueforfindingallpossiblesolutionsusingrecursion.

Advantage:

 DFSrequiresverylessmemoryasitonlyneedsto storeastackofthenodesonthepathfrom root node

to the current node.
 Ittakesless timetoreachtothegoal nodethan BFSalgorithm(if ittraversesin theright path).

Disadvantage:

 Thereisthepossibilitythatmanystateskeep re-occurring,andthereisnoguaranteeoffinding the

solution.
 DFSalgorithmgoesfordeep downsearchingandsometimeitmay gototheinfinite loop.

Example:

Inthebelowsearchtree, wehaveshowntheflowofdepth-firstsearch, and itwillfollowtheorderas: Root

node--->Left node-----------> right node.

It will start searching from root node S, and traverse A, then B, then D and E, after traversing E, it will
backtrackthetreeasEhasnoothersuccessorand stillgoalnodeisnotfound.Afterbacktrackingitwill traverse node C
and then G, and here it will terminate as it found goal node.
 Completeness:DFSsearchalgorithmiscompletewithinfinitestatespaceasitwillexpandeverynode within a
limited search tree.

TimeComplexity:TimecomplexityofDFSwillbeequivalenttothenodetraversedbythealgorithm. It is
given by:

T(n)= 1+ n2+n3+..........+ nm=O(nm)

Where,m=maximumdepthofanynodeandthiscanbemuchlargerthand(Shallowestsolution depth)

SpaceComplexity:DFSalgorithmneedstostoreonlysinglepathfromtherootnode,hencespace complexity of
DFS is equivalent to the size of the fringe set, which is O(bm).

Optimal:DFSsearchalgorithmisnon-optimal,asitmaygeneratealargenumberofstepsorhighcost to reach to
the goal node.

3. Depth-LimitedSearchAlgorithm:

A depth-limited search algorithm is similar to depth-first search with a predetermined limit. Depth-
limitedsearchcansolvethedrawbackoftheinfinitepathintheDepth-firstsearch.Inthisalgorithm, the node
at the depth limit will treat as it has no successor nodes further.

Depth-limitedsearchcan beterminatedwithtwo Conditionsoffailure:

 Standardfailurevalue:Itindicatesthatproblemdoesnothaveanysolution.
 Cutofffailurevalue:Itdefines nosolutionfortheproblem withinagivendepth limit.

Advantages:

Depth-limitedsearchisMemoryefficient.
Disadvantages:

 Depth-limitedsearchalsohasadisadvantageofincompleteness.
 Itmay not be optimal if theproblem has morethan onesolution.

Example:

Completeness:DLSsearchalgorithmiscompleteif thesolutionisabovethe depth-limit.

Time Complexity: Time complexity of DLS algorithm is O(bℓ).

SpaceComplexity:SpacecomplexityofDLSalgorithmisO(b×ℓ).

Optimal:Depth-limitedsearch canbeviewed as aspecialcaseofDFS,anditisalsonotoptimaleven if ℓ>d.

4. Uniform-costSearchAlgorithm:

Uniform-costsearchisasearchingalgorithmusedfortraversingaweightedtreeorgraph.Thisalgorithm comes
into play when a different cost is available for each edge. The primary goal of the uniform-cost search
is to find a path to the goal node which has the lowest cumulative cost. Uniform-cost search expands
nodes according to their path costs form the root node. It can be used to solve any graph/tree where the
optimal cost is in demand. A uniform-cost search algorithm is implemented by the priority queue. It
gives maximum priority to the lowest cumulative cost. Uniform cost search is equivalent to BFS
algorithm if the path cost of all edges is the same.

Advantages:

 Uniformcostsearch isoptimalbecauseateverystatethepath withtheleast costis chosen.

Disadvantages:

 Itdoesnotcareaboutthenumberofstepsinvolveinsearchingandonlyconcernedaboutpath cost.
Due to which this algorithm may be stuck in an infinite loop.
 Example:

Completeness:

Uniform-costsearchiscomplete,suchasif there isasolution,UCSwillfind it.

Time Complexity:

LetC*isCostoftheoptimalsolution,andεiseachsteptogetclosertothegoalnode.Thenthe number of
steps is = C*/ε+1. Here we have taken +1, as we start from state 0 and end to C*/ε.

Hence,theworst-casetimecomplexityof Uniform-costsearchisO(b1+ [C*/ε])/.

Space Complexity:

Thesame logicis forspacecomplexity so,theworst-casespace complexity ofUniform-cost searchis

O(b1+[C*/ε]).

Optimal:

Uniform-costsearchisalwaysoptimal asitonlyselectsa pathwiththelowestpath cost.

5. Iterativedeepeningdepth-first Search:

The iterative deepening algorithm is a combination of DFS and BFS algorithms. This search
algorithmfinds out the best depth limit and does it by gradually increasing the limit until a goal is
found.

This algorithm performs depth-first search up to a certain "depth limit", and it keeps increasing the depth
limit after each iteration until the goal node is found.

This Search algorithm combines the benefits of Breadth-first search's fast search and depth-first search's
memory efficiency.
The iterative search algorithm is useful uninformed search when search space is large, and depth of goal
node is unknown.
Advantages:

 ItcombinesthebenefitsofBFSandDFSsearchalgorithmintermsoffastsearchandmemory
efficiency.

Disadvantages:

 Themaindrawback of IDDFSis thatitrepeatsall theworkof theprevious phase.

Example:

Followingtreestructureisshowingtheiterativedeepeningdepth-firstsearch.IDDFSalgorithmperforms
various iterations until it does not find the goal node. The iteration performed by the algorithm is given
as:

1'st Iteration----->A
2'nd Iteration---->A, B, C
3'rdIteration------>A,B, D, E, C, F, G
4'th Iteration------>A,B,D,H,I, E,C,F,K,G

Completeness:

Thisalgorithmis completeis ifthebranching factoris finite.

Time Complexity:

Let'ssupposebisthebranchingfactoranddepthisdthentheworst-casetimecomplexityis O(bd). Space

Complexity:

ThespacecomplexityofIDDFSwillbeO(bd). Optimal:

IDDFSalgorithm isoptimal ifpath costis anon-decreasingfunction ofthedepthof the node.

6. BidirectionalSearch Algorithm:

Bidirectional search algorithm runs two simultaneous searches, oneform initial state called as forward-
search and other from goal node called as backward-search, to find the goal node. Bidirectional search
replacesonesinglesearchgraphwithtwosmallsubgraphsinwhichonestartsthesearchfromaninitial vertex and
other starts from goal vertex. The search stops when these two graphs intersect each other.

BidirectionalsearchcanusesearchtechniquessuchasBFS,DFS,DLS,etc.

Advantages:

 Bidirectionalsearchisfast.
 Bidirectionalsearchrequiresless memory

Disadvantages:

 Implementation of the bidirectional search tree is difficult.

 In bidirectional search, one should know the goal state in advance.

Example:

In the below searchtree, bidirectional search algorithm is applied .This algorithm divides one graph/tree
into two sub-graphs. It starts traversing from node 1 in the forward direction and starts from goal node
16 in the backward direction.

The algorithm term in at esat node 9 where twosearches meet.

Completeness: Bidirectional Search is complete if we use BFS in both

searches. TimeComplexity: Time complexity of bidirectional search using BFS

is O(bd). Space Complexity: Space complexity of bidirectional search is O(bd).

Optimal: Bidirectional search is Optimal.

Informed Search Algorithms

So far we have talked about the uninformed search algorithms which looked through search space for
all possible solutions of the problem without having any additional knowledge about search space. But
informed search algorithm contains an array of knowledge such as how far we are from the goal, path
cost, how to reach to goal node, etc. This knowledge help agents to explore less to the search space and
find more efficiently the goal node.
Theinformedsearchalgorithmismoreusefulforlargesearchspace.Informedsearchalgorithmusesthe idea of
heuristic, so it is also called Heuristic search.

Heuristics function: Heuristic is a function which is used in Informed Search, and it finds the most
promising path. It takes the current state of the agent as its input and produces the estimation of how
closeagentisfromthegoal.Theheuristicmethod,however,mightnotalwaysgivethebestsolution,but
itguaranteedtofindagoodsolutioninreasonabletime.Heuristicfunctionestimateshowcloseastateis to the
goal. It is represented by h(n), and it calculates the cost of an optimal path between the pair of states.
The value of the heuristic function is always positive.

Admissibilityoftheheuristicfunctionisgivenas:

1. h(n)<=h*(n)

Hereh(n)isheuristiccost,andh*(n)istheestimatedcost.Henceheuristiccostshouldbelessthan
or equal to the estimated cost.

PureHeuristicSearch:

Pureheuristicsearchisthesimplestformofheuristicsearchalgorithms.Itexpandsnodesbasedontheir
heuristicvalueh(n). Itmaintainstwolists,OPENandCLOSEDlist.IntheCLOSEDlist,itplacesthose nodes
which have already expanded and in the OPEN list, it places nodes which have yet not been expanded.

Oneachiteration,eachnodenwiththelowestheuristicvalueisexpandedandgeneratesallitssuccessors and n is
placed to the closed list. The algorithm continues unit a goal state is found.

Intheinformedsearchwewilldiscuss twomainalgorithms whicharegiven below:

 BestFirstSearchAlgorithm(Greedysearch)
 A*SearchAlgorithm

1.) Best-firstSearchAlgorithm(Greedy Search):

Greedy best-first search algorithm always selects the path which appears best at that moment. It is the
combination of depth-first search and breadth-first search algorithms. It uses the heuristic function and
search. Best-first search allows us to take the advantages of both algorithms. With the help of best-first
search,ateachstep,wecanchoosethemostpromisingnode.Inthebestfirstsearchalgorithm,weexpand the
node which is closest to the goal node and the closest cost is estimated by heuristic function, i.e.
 1. f(n)=g(n).

Were,h(n)= estimated costfromnode ntothe goal.

Thegreedy bestfirst algorithm isimplemented bythe priority queue.

Bestfirstsearch algorithm:

 Step1:Place the starting node into the OPEN list.

 Step2:If the OPEN list is empty ,Stop and return failure.
 Step3:Remove then oden ,from the OPEN list which has the lowest value of h(n), and places
it in the CLOSED list.
 Step4:Expand the node n, and generate the successors of noden.
 Step5:Check each successor of noden, and find whether any node is ago al node or not. If any
successor node is goal node, then return success and terminate the search, else proceed to Step
6.
 Step6:For each successor node, algorithm checks for evaluation function f(n), and then check
If the node has been in either OPEN or CLOSED list.If then ode has not been in both list, then
add it to the OPEN list.
 Step7:Return to Step 2.

Advantages:

 Best first search can switch between BFS and DFS by gaining the advantages of both
the algorithms.
 This algorithm is more efficient than BFS and DFS algorithms.

Disadvantages:

 It can behave as an unguided ddepth-first search in the worst case scenario.

 It can get stuck in a loop as DFS.
 This algorithm is not optimal.

Example:

Consider the below search problem, and we will traverse it using greedy best-first search. At each
iteration, each node is expanded using evaluation function f(n)=h(n),which is given in the below table.
In this search example, weare using two lists which are OPEN and CLOSED Lists .Following are the iteration
for traversing the above example.

Expand the nodes of Sand put in the CLOSED list

Initialization: Open [A, B], Closed [S]

Iteration1: Open [A], Closed[S, B]

Iteration 2: Open[E,F,A],Closed[S, B]
:Open[E,A],Closed[S,B,F]

Iteration 3: Open[I, G,E,A],Closed[S, B, F]

:Open [I,E, A], Closed [S,B,F,G]

Hence the final solution path will be:S---->B----->F----->G

TimeComplexity:The worst case time complexity of Greedybestfirstsearch is O(bm).

Space Complexity: The worst case space complexity of Greedy best first search is O(b m). Where, m is
the maximum depth of the search space.

Complete: Greedy best-first search is also incomplete, even if the given states pace is finite.

Optimal:Greedy best first search algorithm isnot optimal.

2.) A*Search Algorithm:

A*searchisthemostcommonlyknownformofbest-firstsearch.Itusesheuristicfunctionh(n),andcost
toreachthenodenfromthestartstateg(n).IthascombinedfeaturesofUCSandgreedybest-firstsearch,
bywhichitsolvetheproblemefficiently.A*searchalgorithmfindstheshortestpaththroughthesearch space
using the heuristic function. This search algorithm expands less search tree and provides optimal result
faster. A* algorithm is similar to UCS except that it uses g(n)+h(n) instead of g(n).

In A* search algorithm, we use search heuristic as well as the cost to reach the node. Hence we can
combine both costs as following, and this sum is called as a fitness number.

At each point in the search space, only that node is expanded which have the lowest value of f(n), and
the algorithm terminates when the goal node is found.

Algorithmof A*search:

Step1:Placethe startingnodein theOPEN list.

Step2:Check iftheOPENlist isempty ornot,ifthelist isempty thenreturn failureand stops.

Step3:SelectthenodefromtheOPENlistwhichhasthesmallestvalueofevaluationfunction(g+h),if node n is
goal node then return success and stop, otherwise

Step4:Expandnodenandgenerateallofitssuccessors,andputnintotheclosedlist.Foreachsuccessor
n',checkwhethern'isalreadyintheOPENorCLOSEDlist,ifnotthencomputeevaluationfunctionfor n' and
place into Open list.

Step 5: Else if node n' is already in OPEN and CLOSED, then it should be attached to the back pointer
which reflects the lowest g(n') value.

Step6:ReturntoStep2.
 Advantages:

 A*searchalgorithmisthebestalgorithmthanother search algorithms.

 A*searchalgorithmisoptimaland complete.
 Thisalgorithmcansolve verycomplexproblems.

Disadvantages:

 Itdoesnotalwaysproducetheshortestpathasitmostlybasedonheuristicsand approximation.
 A*searchalgorithm has somecomplexity issues.
 ThemaindrawbackofA*ismemoryrequirementasitkeepsallgeneratednodesinthememory, so it is
not practical for various large-scale problems.

Example:
Inthisexample,wewilltraversethegivengraphusingtheA*algorithm.Theheuristicvalueofallstates
isgiveninthebelowtablesowewillcalculatethef(n)ofeachstateusingtheformulaf(n)=g(n)+h(n), where g(n)
is the cost to reach any node from start state.
Herewewill useOPENandCLOSEDlist.

Solution:

Initialization:{(S, 5)}
Iteration1:{(S-->A,4),(S-->G,10)}

Iteration2:{(S-->A-->C,4),(S-->A-->B,7), (S-->G,10)}

Iteration3:{(S-->A-->C--->G,6),(S-->A-->C--->D,11),(S-->A-->B,7),(S-->G,10)}

Iteration4 :will give the final result, as S--->A--->C--->G it provides the optimal path with cost 6.

Point store member:

 A* algorithm returns the path whichoccurredfirst,anditdoesnotsearchforallremaining

paths.
 The efficiency of A* algorithm depend son the quality of heuristic.
 A* algorithm expand sall nodes which satisfy the condition f(n)<=""li="">

Complete: A* algorithm is complete as long as:

 Branching factoris finite.

 Costate very action is fixed.

Optimal: A* search algorithmisoptimal ifitfollowsbelowtwo conditions:

 Admissible:thefirstconditionrequiresforoptimalityisthath(n)shouldbeanadmissible
heuristic for A* tree search. An admissible heuristic is optimistic in nature.
 Consistency:Secondrequired conditionisconsistencyforonlyA* graph-search.

Iftheheuristicfunctionisadmissible, thenA* treesearchwill alwaysfindthe leastcost path.

TimeComplexity:ThetimecomplexityofA*searchalgorithmdependsonheuristicfunction,and the number of

nodes expanded is exponential to the depth of solution d. So the time complexity is O(b^d), where b is the
branching factor.

SpaceComplexity:ThespacecomplexityofA*searchalgorithmisO(b^d)
1. PROBLEMSOLVINGMETHODS

First, we must reduce it to one for which a precise statement can be given. This is done by defining
theproblem's state space, which includes the start and goal states and a set of operators for moving
aroundinthat space.

The problem can then be solved by searching for a path through the space from an initial state to
agoalstate.Theprocess ofsolvingtheproblem canusefullybemodelledas aproduction system.

The production rules are also known as condition – action, antecedent – consequent, pattern –
action,situation– response, feedback–result pairs.

Forexample,
If (youhaveanexam tomorrow)THEN(studythewholenight)

Theproductionsystemcanbe classifiedasmonotonic,non-monotonic, partiallycommutativeandcommutative

Inproductionsystemwehavetochoosetheappropriatecontrolstructuresothatthesearchcanbeasefficient as
possible.
A production system is a system that adapts a system with production
rules.Aproduction systemconsists of:
• Asetofproduction Rules
Asetofruleseachconsistingofaleftsideandarighthandside.Lefthandsideorpatterndetermin
es the applicability of the rule and a right side describes the operation to be
performediftheruleis applied.
• Knowledge/databases
One or more knowledge/databases that contain whatever information is appropriate
forthe particular task. Some parts of the database may be permanent, while other parts of it
maypertainonlytothesolutionofthecurrentproblem.Theinformationinthesedatabasesmaybestru
cturedinanyappropriate way.
• Controlstrategy
A control strategy that specifies the order in which the rules will be compared to
thedatabase and awayof resolvingthe conflicts that arisewhen severalrulesmatchat once.
• Aruleapplied
Production System also encompasses a family of general production system
interpreters,including:

• Basicproduction systemlanguages,suchasOPS5andACT*
• Morecomplex,oftenhybridsystemscalled
expertsystemshells,whichprovidecomplete(relatively speaking) environments for the
construction of knowledge-basedexpertsystems.
• General problem-solving architectures like SOAR, a system based on a specific set
ofcognitivelymotivated hypotheses about thenatureof problem solving.

FeaturesofProductionSystem
Someofthemainfeaturesofproductionsystemare:
1. Expressivenessandintuitiveness: In real world, many times situation comes like ―if this
happen-you will do that‖, ―if thisisso-then thisshould happen‖and many more. The
productionrules essentiallytell us what to do in agiven situation.
2. Simplicity:
Thestructureofeachsentenceinaproductionsystemisuniqueanduniformastheyuse―IF-
THEN‖structure.Thisstructureprovidessimplicityinknowledgerepresentation.Thisfeatureofprodu
ctionsystem improves the readabilityofproduction rules.
3. Modularity:
Thismeansproductionrulecodetheknowledgeavailableindiscretepieces.
Information can be treated as a collection of independent facts which may be added or
deletedfromthesystem with essentiallyno deleterious side effects.
4. Modifiability:
This means the facility of modifying rules. It allows the development of production rules in
askeletalform first andthenit is accurate to suit aspecificapplication.
5. Knowledgeintensive:
Theknowledgebaseofproductionsystemstorespureknowledge.Thispartdoesnotcontainanytype of
control or programming information. Each production rule isnormally written as anEnglish
sentence;theproblem ofsemantics is solvedbytheverystructureoftherepresentation.

20
Disadvantagesofproduction system
1. Opacity:
Thisproblem isgeneratedbythecombinationofproduction
rules.Theopacityisgeneratedbecauseoflessprioritizationof rules. Morepriorityto a rulehas theless opacity.
2. Inefficiency:
During execution of a program several rules may active. A well devised control strategy reduces
thisproblem. As the rules of the production system are large in number and they are hardly written
inhierarchical manner, it requires some forms of complex search through all the production rules for
eachcycleof control program.
3. Absenceof learning:
Rule based production systems do not store the result of the problem for future use. Hence, it does
notexhibit any type of learning capabilities. So for each time for a particular problem, some new
solutionsmaycome.
4. Conflictresolution:
The rules in a production system should not have any type of conflict operations. When a new rule
isaddedto a database, it shouldensure that it does nothaveanyconflicts with the existingrules.

GraphSearch
– Representationofstatesandtransitions/actionsbetweenstates→graph
– Exploredexplicitlyor implicitly
• ConstraintSolving
– Representproblem byvariablesandconstraints
– Usespecificsolvingalgorithms tospeedupsearch
• LocalSearchandMetaheuristics
– Evaluation function to check ifstateis―good‖or not
– Optimizationoftheevaluation function

GRAPHSEARCH
 Alargevarietyofproblems can berepresentedbyagraph
 A(directed)graphconsists ofasetofnodesandaset ofdirectedarcsbetweennodes.
 Theideaisto find apathalongthesearcs froma startnodeto a goal node.
 Adirectedgraphconsistsof
 aset Nofnodes and

Apath from nodes to nodeg is asequenceofnodes ⟨n0,n1,...,nk⟩

 asetAofordered pairsofnodescalledarcs.

Atreeis
 a DAG where there is one node with no incoming arcs and every other node has exactly
oneincomingarc.
 Nodewith noincomingarcsiscalled therootofthetree
 Nodes with no outgoing arcs are called
leaves.Toencodeproblems asgraphs,
 Onesetofnodesis referred to asthestart nodes
 Anothersetiscalledthe goalnodes.
Asolution isapathfromastart nodeto a goal node.

21
Anoptimal solutionis oneoftheleast-cost solutions;that is, it isa pathpfromastart
nodetoagoalnodesuchthatthereisnopathp'fromastartnode toagoalnodewherecost(p')<cost(p).
Theforwardbranchingfactorofanodeis thenumberof arcs leavingthe node.
Thebackwardbranchingfactorof a node isthenumber of arcsenteringthe node.These
factorsprovidemeasures of thecomplexityofgraphs.

• Solutionscan beconsideredas definingspecificnodes

• Solvingtheproblemisreducedto searchingthe graph forthosenodes
– startingfrom aninitial node
– eachtransition inthegraphcorrespondstoapossibleaction
– endingwhenreachingafinal node(solution)

DefiningSearchProblems
 AstatementofaSearchproblemhas4 components
1. Aset ofstates
2. Aset of―operators‖which allow onetoget from onestateto another
3. Astart state S
4. A set of possible goal states, or ways to test for goal
states4a.Cost path
 Searchsolution consistsofasequenceofoperatorswhichtransform SintoagoalstateG

BASICGRAPHSEARCHALGORITHM

22
The8-puzzle

• canbegeneralizedto15-puzzle,24-puzzle,etc.
• Any(n2-1)-puzzle for n ≥ 3
• state = permutation of (0, 1, 2, 3, 4, 5, 6, 7,
8)Eg.state aboveis:(1,2,3,8,0,4,7,6,5)
• 9!=362,880possiblestates
• Solution:(1,2,3,4,5,6,7,8,0)
• Actions:possiblemoves

Search:
•Expandoutpossiblenodes
•Maintainafringeof asyetunexpandednodes
•Trytoexpandasfewtreenodesas possible

WaterJug Problem
A Water Jug Problem: You are given two jugs, a 4-gallon one and a 3-gallon one, a
pumpwhich has unlimited water which you can use to fill the jug, and the ground on which
water maybepoured.Neitherjughasanymeasuringmarkingsonit.Howcanyougetexactly2
23
gallons ofwater inthe4-gallonjug?

State Representation and Initial State–we will represent a state of the problem as a tuple
(x,y)where x represents the amount of water in the 4-gallon jug and y represents the
amountofwaterinthe3-gallonjug.Note0≤x≤4,and0≤y≤3.Ourinitialstate:(0,0)

•State:(x,y) x=0,1,2,3,or 4 y=0,1,2,3

•Startstate:(0,0).
•Goalstate:(2,n)foranyn.
• Attemptingtoendupinagoalstate

TOYPROBLEMS
VacuumWorldExample

o States:Theagentis inone of twolocations.Each of which mightor might

notcontaindirt.Thusthereare2 x22 =8 possible world states.
o Initialstate:Anystatecan bedesignated asinitialstate.
o Successorfunction:Thisgeneratesthelegalstatesthatresultsfromtryingthethreeactions(left,rig
ht, and suck). Thecompletestatespaceisshown in figure2.3
o GoalTest:Thistests whetherall thesquaresareclean.
o Pathtest:Eachstep costs one,sothat thepathcostis thenumberofsteps inthepath.

24
VacuumWorld StateSpace

Thestatespaceforthe vacuum world.

Arcsdenoteactions:L= Left,R=Right,S=Suck

CONSTRAINTMODELING
Search can be made easier in cases where the solution of corresponding to an optimal path, is
onlyrequired to satisfy local consistency conditions. We call such problems Constraint Satisfaction
(CS)Problems.

Manyproblemscanbestatedasconstraints satisfactionproblems.
Two‐stepprocess:
1. Constraintsarediscoveredandpropagatedasfaraspossible.
2. Ifthereisstill notasolution,thensearchbegins,addingnew constraints.

Twokindsofrules:
1. Rulesthatdefinevalidconstraintpropagation.
2. Rulesthatsuggest guesseswhennecessary.
.
 Declarativelanguage : modelingis easy
• localspecificationofthe problem
• globalconsistencyachieved (orapproximated)byconstraint solvingtechniques
• compositionality:constraintsarecombinedimplicitlythroughsharedlogicalvariables

BasicObjects
Variable : aplaceholderforvalues X,Y,Z, L,U,List3 21
FunctionSymbol:mappingofvariablestovalues:Sin(x),Cos(x),x+y, x-y
RelationSymbol:
RelationbetweenvariablesArithmeti
c
relation:+,-,x,/Symbolicrelation:all_
different

Constraints
Declarativerelationsbetweenvariables
• Constraintsusedto bothmodel andsolve theproblem
• specificalgorithmsforefficientcomputation Constraints
• Constraintscouldbenumericorsymbolic:
25
X≤5 , X+ Y= Z
all_different(X1,X2,…,Xn)at_most(N,[X1,X2,X3],V)
• multi-directionalrelations

Crypto-arithmeticsas CSP
SE ND
+M O R E

MO NE Y
eachletterrepresentsa(different)digitandthea
ddition should becorrect!
...Solution?

C4 C3 C2
C1SEND
+MORE

MON EY
variables:{S,E,N,D,M,O,R,Y,R1, R2, R3,R4}
domains:{0,...,9}fortheletters,
{0,1}forthecarriesconstraints:all_different(S,E,N,D,M,O,
R,Y}S≠0M≠0

Wehaveto replaceeachletterbyadistinct digit so that the resultingsum is correct.

In the above problem, M mustbe 1. You can visualize that, thisis an addition problem. The
sumoftwofourdigitnumberscannotbemorethan10,000.AlsoMcannotbezeroaccordingtotherules,sinceitis
thefirst letter.

Nowinthecolumns10,s+1≥10.Smustbe8becausethereisa1carriedoverfromthecolumnEONor
9. Omustbe0(ifs=8andthereisa1carriedors=9andthereisno1carried)or1(ifs=9andthereisa 1 carried).But 1
is alreadytaken, so O must be0.

There cannot be carry from column EON because any digit +0 < 10, unless there is a carry from
thecolumn NRE, and E=9; But this cannot be the case because then N would be 0 and 0 is already
taken.SoE <9 and thereis nocarryfrom thiscolumn. ThereforeS=9 because9+1=10.

In the column EON, E cannot be equal to N. So there must be carry from the column NRE;
E+1=N.WenowlookatthecolumnNRE,weknowthatE+1=N.Sinceweknowthatcarryfromthiscolumn,

26
N+R=1E (if there is no carry from the column DEY) or N+R+1=1E (if there is a carry from the
columnDEY).

Now just think what are the digits we have left? They are 7, 6, 5, 4, 3 and 2. We know there must be
acarryfrom thecolumn DEY. So D +E%10. N=E+1, SoE cannot be7 becausethen N would be 8

Nowwehave gottenthisimportantdigit,itgetsmuchsimplerfromhere.N+8+1=15,N=6

The digits left are 7, 4, 3 and 2. We know there is carry from the column D5Y, so the only pair
thatworksis D=7 and Y=2.

Whichisfinal solution oftheproblem.

2. SEARCHSTRATEGIES
Problem solving in artificial intelligence may be characterized asa systematic search through a rangeof
possible actions in order to reach some predefined goal or solution. In AI problem solving by
searchalgorithmsisquitecommontechnique.InthecomingageofAIitwillhavebigimpactonthetechnologieso
f the robotics and path finding. It is also widely used in travel planning. A search algorithm takes
aproblemasinput and returnsthesolution in theformof an action sequence.

Once the solution is found, the actions it recommends can be carried out. This phase is called as
theexecution phase. After formulating a goal and problem to solve the agent cells a search procedure
tosolve it .

Asearch strategyis defined bypickingtheorder of nodeexpansion

• Strategies areevaluatedalongthefollowingdimensions:
– completeness:doesit alwaysfindasolution ifoneexists?
27
– timecomplexity:numberofnodes generated
– spacecomplexity:maximum numberofnodesinmemory
– optimality:doesitalwaysfindaleast-costsolution?
– systematicity:doesit visiteachstateatmostonce?

DIFFERENTTYPES OFSEARCHING
Thesearchingalgorithmscanbevarioustypes.Whenanytypeofsearchingisperformed,theremaysomeinform
ationaboutthesearchingormayn’tbe.Alsoitispossiblethatthesearchingproceduremaydepend upon any
constraints or rules. However, generally searching can be classified into two types i.e.uninformed
searching and informed searching. Also some other classifications of these searches
areSearchAlgorithms
Uninformed
SearchDFS
BFS
IterativeDeepeningDFSUn
iformcostsearchBidirectio
nal

SearchBranchandboundsea
rch
InformedSearch
Generate and test
SearchBestFirst Search
A*A
O*
GreedySearch

• Strategiesforsearch
Somewidelyusedcontrol strategiesforsearcharestated below.
 Forwardsearch:Here,thecontrolstrategiesforexploringsearchproceedsforwardfrominitialstateto
wards asolution; themethods arecalleddata-directed.
 Backward search : Here, the control strategies for exploring searchproceeds backward from
agoalorfinalstatetowardseitherasolvablesubproblemortheinitialstate;themethodsarecalledgoaldir
ected.
 Bothforwardandbackwardsearch:Here,thecontrolstrategiesforexploringsearchisa
mixtureofbothforwardandbackwardstrategies.
 Systematic search : Where search space is small, a systematic (but blind) method can be
usedtoexplorethe whole search space.
Onesuchsearchmethodisdepth-first searchandtheotherisbreath-first search.

3. UNINFORMEDSEARCH(BlindSearch)
 BreadthFirstSearch(BFS)–Referprevioussection
 DepthFirstSearch(DFS)–Referprevioussection
 Depth-limitedsearch(DLS)
 Iterativedeepeningdepthfirstsearch
 Uniform-costsearch(UCS)

28
BreadthFirstSearch
Breadth first search is a general technique of traversing a graph. Breadth first search may use
morememory but will always find the shortest path first. In this type of search the state space is
representedinformof atree.

Thesolutionisobtainedbytraversingthroughthetree.Thenodesofthetreerepresentthestartvalueor starting
state, various intermediate states and the final state. In this search a queue datastructure isused and it is
level by level traversal. Breadth first search expands nodes in order of their distance fromtheroot.Itis
apath findingalgorithm thatis capableofalwaysfindingthesolution ifoneexists.

The solution which is found is always the optional solution. This task is completed in a very
memoryintensivemanner. Eachnodeinthesearchtreeis expandedin abreadthwiseat each level.

Concept:
Step 1:Traversetherootnode
Step 2:Traverseallneighbours ofrootnode.
Step 3:Traverseall neighboursofneighboursoftherootnode.
Step4:Thisprocesswill continueuntilweare gettingthe goalnode.

Figure : Search tree of Breadth first

searchExample:
 Construct atreewiththeinitial stateasitsroot.
 Generateall theoffspringofthe rootbyapplyingeach oftheapplicablerules totheinitialstate

29
  Nowforeachleafnode,generateall itssuccessors byapplyingalltherulesthatareappropriate

 Continuethisprocess until someruleproducesa goalstate.

Evaluation
– Complete?-Yes(bisfinite)
– TimeComplexity?O(b^d)
– SpaceComplexity?O(b^d)
– Optimal? Yes,if stepcost=1

Advantages:
Inthis procedureat anywayit willfindthegoal.
Itdoesnotfollow asingle unfruitfulpathfor a
longtime.Itfindstheminimalsolutionincaseofmultiple
paths.
Disadvantages:
BFSconsumeslargememoryspace.Itsti
mecomplexityis more.
Ithaslongpathways,whenallpathstoadestinationareonapproximatelythesamesearchdepth.

DEPTHFIRSTSEARCH(DFS)
DFS is also an important type of uniform search. DFS visits all the vertices in the graph. This type
ofalgorithm always chooses to go deeper into the graph. After DFS visited all the reachable vertices
froma particular sources vertices it chooses one of the remaining undiscovered vertices and continues
thesearch. DFS reminds the space limitation of breath first search by always generating next a child of
thedeepest unexpanded nodded. The data structure stack or last in first out (LIFO) is used for DFS.
Oneinteresting property of DFS is that, the discover and finish time of each vertex from a
parenthesisstructure. If we use one open parenthesis when a vertex is finished then the result is
properly nested setofparenthesis.

Concept:
Step1:Traversetherootnode.
Step2:Traverseanyneighbourof theroot node.
Step 3:Traverseanyneighbourof neighbourof theroot node.
Step4:Thisprocesswill continueuntilwearegettingthe goalnode.

Algorithm:
1. If the initial state is a goal state, quit and return
success.2.Otherwise,loopuntil success or
failureissignalled.
a) Generate a state, say E, and let it be the successor of the initial state. If there is
nosuccessor,signal failure.

30
 b) CallDepth-FirstSearchwithEastheinitialstate.
c) Ifsuccessisreturned,signalsuccess.Otherwisecontinueinthisloop.

Figure:Searchtreefor depthfirstsearch
Implementation:
LetustakeanexampleforimplementingtheaboveDFSalgorithm.

(0,0)

(4,0)

(4,3)
FigureExamplesofDFS

Evaluation
– Complete?-No
– TimeComplexity?-O(b^m)
– SpaceComplexity?-O(bm)

Advantages:
 DFSconsumes verylessmemoryspace.
 Itwill reach atthe goalnodeinalesstimeperiodthanBFSifit traversesinarightpath.
 Itmayfindasolutionwithoutexaminingmuchofsearchbecausewemaygetthedesiredsolutioninthe
veryfirstgo.

Disadvantages:
 Itispossiblethatmaystateskeepreoccurring.
 Thereisnoguaranteeof findingthe goalnode.
 Sometimesthestates mayalso enterinto infiniteloops.

31
DifferencebetweenBFS
andDFSBFS
1. Itusesthedatastructure queue.
2. BFSiscompletebecauseitfindsthesolutionifoneexists.
3. BFStakesmorespacei.e.equivalenttoo(b0)wherebisthemaximumbreathexistina searchtreeand d is
the maximum depth exit in asearch tree.
4. Incaseofseveralgoals,itfindsthebest one.
DFS
1. Itusesthedatastructurestack.
2. Itisnotcompletebecauseitmaytakeinfinitelooptoreachatthegoalnode.
3. ThespacecomplexityisO(d).
4. Incaseofseveralgoals,itwillterminatethesolutioninanyorder.

DEPTH-LIMITEDSEARCH(DLS)
Description:
We can avoid examining unbounded branches by limiting the search to depth l. The nodes
atlevellaretreadedasiftheyhavenosuccessors.Wewillcalldepthlimitl,thissolvestheinfinitepath
problem.
PerformanceMeasure:
Completeness:
Thelimitedpathintroducesanotherproblemwhichisthecasewhenwechoosel<d,inwhichisourDLSwi
ll neverreacha goal,inthiscasewecansaythat DLSisnotcomplete.
Optimality:
OnecanviewDFSasaspecialcaseofthedepthDLS,thatDFSisDLSwithl=infinity.DLSisnot
optimalevenif l >d.
TimeComplexity:O(bl)SpaceC
omplexity:O(bl)

ITERATIVEDEEPENINGDEPTHFIRSTSEARCH(IDDFS)
One way to combine the space efficiency of depth-first search with the optimality of breadth-
firstmethods is to use iterative deepening. The idea is to recompute the elements of the frontier rather
thanstoringthem. Eachrecomputation can beadepth-first search,which thususes lessspace.

Consider making a breadth-first search into an iterative deepening search. This is carried out by
havingadepth-firstsearcher,whichsearchesonlytoalimiteddepth.Itcanfirstdoadepth-firstsearchtodepth 1
by building paths of length 1 in a depth-first manner. Then it can build paths to depth 2, thendepth 3,
and so on. It can throw away all of the previous computation each time and start again.Eventually it
will find a solution if one exists, and, as it is enumerating paths in order, the path with thefewestarcs
will always be foundfirst.
Whenimplementinganiterativedeepeningsearch,youhavetodistinguishbetween
 failurebecausethedepthboundwasreachedand
 failurethat does not involvereachingthedepth bound.

32
 Inthefirstcase,thesearchmustberetriedwithalargerdepthbound.Inthesecondcase,itisawasteoftimetotryaga
inwithalargerdepthbound,becausenopathexistsnomatterwhatthedepth.Wesaythatfailure due to reaching
the depth bound is failing unnaturally, and failure without reaching the depthboundis failing
naturally.

The iterative-deepening search fails whenever the breadth-first search would fail. When asked
formultiple answers, it only returns each successful path once, even though it may be rediscovered
insubsequentiterations.Haltingisachievedbykeepingtrackofwhenincreasingtheboundcouldhelpfindanans
wer:

 The depth bound is increased if the depth bound search was truncated by reaching the
depthbound. In this case, the search failed unnaturally. The search failed naturally if the search
didnot prune any paths due to the depth bound. In this case, the program can stop and report
no(more)paths.
 Thesearchonlyreportsasolutionpathifthatpathwouldnothavebeenreportedinthepreviousiteration.
Thus,it onlyreports pathswhoselengthis thedepthbound.

The obvious problem with iterative deepening is the wasted computation that occurs at each step.
This,however,maynotbeasbadasonemightthink,particularlyifthebranchingfactorishigh.Consider
therunning time of the algorithm. Assume a constant branching factor of b>1. Consider the search
wheretheboundisk.Atdepthk,therearebknodes;eachofthesehasbeengeneratedonce.Thenodesatdepthk-1
have been generated twice, those at depth k-2 have been generated three times, and so on, and
thenodesat depth 1 havebeengeneratedktimes.Thus,the total numberofnodesgeneratedis

bk+2bk-1+3bk-
+···+kb
= bk(1+2b-1+3b-2+···+kb1-k)
≤ bk(∑i=1∞ib(1-i))
= bk(b/(b-1))2.

Asthereare(b/(b-1))bknodesinthetree,iterativedeepeninghasaconstantoverheadof(b/(b-1))timesthe cost of
generating the nodes at depth n. When b=2 there is an overhead factor of 2, and whenb=3there is an
overhead factor of 1.5 over generating the frontier. This algorithm is O( bk) and there cannotbe an
asymptotically better uninformed search strategy. Note that, if the branching factor is close to
1,thisanalysisdoes notwork (because then thedenominator would beclose to zero).

33
IterativedeepeningcanalsobeappliedtoanA*search.IterativedeepeningA*(IDA*)performsrepeateddepth-
bounded depth-first searches. Instead of the bound being on the number of arcs in the path, it is abound
on the value of f(n). The threshold starts at the value of f(s), where s is the starting node withminimal
h-value. IDA*then carries out a depth-first depth-bounded search butneverexpands a nodewith a higher
f-value than the current bound. If the depth-bounded search failsunnaturally, the nextbound is the
minimum of the f-values that exceeded the previous bound. IDA*thus checks the samenodesas A*but
recomputes them withadepth-firstsearch instead ofstoringthem.

Our starting node (A) is at a depth of 0. Our goal node (R) is at a depth of 4. The above example is
afinite tree, but think of the above tree as an infinitely long tree and only up to depth = 4 is shown in
thediagram.

InIDDFS,weperformDFSuptoacertaindepthandkeepincrementingthisalloweddepth.PerformingDFS
upto a certain allowed depth is called Depth Limited Search (DLS). As Depth Limited
Search(DLS)isimportant forIDDFS.

34
Let us understand DLS, by performing DLS on the above example. In Depth Limited Search, we
firstset a constraint on how deep (or how far from root) will we go. Let’s say our limit (DEPTH) is 2.
Now,in the above diagram, place your hand to cover the nodes at depth 3 and 4. Now, by looking at the
restofthenodes,canyou telltheorderin whichanormalDFS wouldvisitthem?It wouldbeasfollows–

1ABEFC G DH

Canyoudoit forDEPTH={0,1,2,3,4} ?Justcover thenodesyoudon’t needwithyourhand andtryto perform

DFS inyou mind. Youshould get answers like this–

DEPTH DLStraversal
0 A
1 ABC D
2 ABEFC G DH
3 ABEIFJK C GLDHM N
4 ABEIFJK OP C GLR D HM N S
PropertiesofIterativedeepeningSearch
Space complexity= O(bd)
Time Complexity B+(b+b2)+ … (b+…bd) =
O(bd)Complete? Yes
Optimal Onlyif path cost is a non-decreasingfunction ofdepth
IDScombinesthesmallmemoryfootprintofDFS, andhasthecompleteness guaranteeofBFS

UNIFORM-COSTSEARCH(UCS)
Description:
Uniform-cost is guided by path cost rather than path length like in BFS, the algorithms starts
byexpanding the root, then expanding the node with the lowest cost from the root, the
searchcontinuesin this manner for all nodes.
where there is no goal state and processing continues until the shortest path to all nodes
hasbeendetermined.
PerformanceMeasure:
Completeness:
It isobviousthatUCSiscomplete if thecostof eachstepexceedssome smallpositiveinteger,this
to prevent infiniteloops.
Optimality:
UCSis always optimal in the sensethatthe nodethat it alwaysexpands isthenode with
theleastpath cost.
TimeComplexity:
UCS is guided by path cost rather than path length so it is hard to determine its
complexityin terms of b and d, so if we consider C to be the cost of the optimal solution,
and everyactioncosts at least e, then the algorithm worst case isO(bC/e).
SpaceComplexity:
ThespacecomplexityisO(bC/e) as the time complexityof UCS.

35
BIDIRECTIONALSEARCH
Description:
If node predecessors are easy to compute – eg pred(n) = S(n)- then search can
proceedsimultaneously forward from theinitial state and backward from the goal state,
those 2searches stop when they meet each other at some point in the middle of the
graph. Themotivation forthisideais that 2b d/2<<bd
Thefollowingpictures illustratesabidirectionalsearch

PerformanceMeasure:
Completeness:
Bidirectional search is complete when weuse BFSin bothsearches, thesearch
thatstartsfromtheinitial state andtheotherfrom thegoal state.
Optimality:
o Like the completeness, bidirectional search is optimal when BFS is used and paths
areofauniform cost – all steps of the samecost.
o Other search strategies can be used like DFS, but this will sacrifice the optimality
andcompleteness, any other combination than BFS may lead to a sacrifice in
optimality orcompletenessor maybeboth of them.
TimeandSpaceComplexity:
o May be the most attractive thing in bidirectional search is its performance, because
bothsearches will run the same amount of time meeting in the middle of the graph, thus
eachsearch expands O(bd/2) node, in total both searches expand O(bd/2 + bd/2) node which
istoofar better than theO(bd+1) ofBFS.
o If a problem with b = 10, has a solution at depth d = 6, and each direction runs
withBFS, then at the worst case they meet at depth d = 3, yielding 22200 nodes
comparedwith11111100 forastandard BFS.
o Wecan saythat thetime and spacecomplexityofbidirectional searchisO(bd/2).

36
4. INFORMEDSEARCH(HEURISTICSEARCH)
Heuristicisatechniquewhichmakesoursearchalgorithmmoreefficient.Someheuristicshelptoguideas
earchprocess withoutsacrificinganyclaimto completenessandsomesacrificing it.

Heuristicisaproblemspecificknowledgethatdecreasesexpectedsearchefforts.Itisatechnique
whichsometimesworksbutnotalways.Heuristicsearchalgorithmusesinformationabouttheproblemtoh
elp directing thepath through the search space. These searches uses some functions that estimatethe
cost from the currentstate to the goal presuming that such function is efficient. A heuristic
functionisafunctionthatmapsfromproblemstatedescriptionstomeasureofdesirabilityusuallyrepresentedas
number.Thepurposeofheuristicfunctionistoguidethesearchprocessinthemostprofitabledirectionsbysugge
stingwhich pathto follow first when morethanisavailable.

Generallyheuristicincorporatesdomainknowledgetoimproveefficiencyoverblindsearch.InAIheuristichasage
neralmeaningandalsoamorespecializedtechnicalmeaning.Generallyatermheuristicisusedforanyadvicethat is
effectivebutis notguaranteed towork in everycase.

For example in case of travelling sales man (TSP) problem we are using a heuristic to calculate
statedescriptions→measuresofdesirability
thenearestneighbour.Heuristicisamethodthatprovidesabetterguessaboutthecorrectchoicetomakeat any
junction that would beachieved by random guessing. This technique is useful in solving
thoughproblemswhich couldnot besolved inanyotherway.Solutions takeaninfinite time tocompute.

Letusseesomeclassificationsofheuristicsearch.
1. GenerateandTestSearch
2. Best-firstSearch
3. A*Search
4. ConstraintSearch
5. GreedySearch
6. Means-endsanalysis

Generateand TestSearch
Generate-and-test search algorithm is a very simple algorithm that guarantees to find a solution if
donesystematicallyand thereexists asolution.

37
 1. Generateapossiblesolution.
2. Testto seeif this is actuallyasolution.
3. Quit if a solution has been
found.Otherwise,returntoste
p1.
Potential solutions that need to be generated vary depending on the kinds of problems. For
someproblems the possible solutions may be particular points in the problem space and for some
problems,pathsfrom thestart state.

Generate-and-test,likedepth-firstsearch,requiresthatcompletesolutionsbegeneratedfortesting.Initsmost
systematic form, it is only an exhaustive search of the problem space. Solutions can also begenerated
randomly but solution is not guaranteed. This approach is what is known as British
Museumalgorithm:findingan object in the BritishMuseum bywanderingrandomly.

While generating complete solutions and generating random solutions are the two
extremesthereexistsanother approach that lies in between. The approach is that the search process
proceeds systematicallybut some paths that unlikely to lead the solution are not considered. This
evaluation is performed by aheuristicfunction.

Depth-firstsearchtreewithbacktrackingcanbeusedtoimplementsystematicgenerate-and-testprocedure. As
per this procedure, if some intermediate states are likely to appear often in the tree, itwouldbebetter to
modifythatprocedureto traverse agraphratherthan atree.

Advantages: Acceptable for simple

problems.Disadvantage:
Inefficientforproblemswithlargespace.

• Exhaustivegenerate-and-test.
• Heuristic generate-and-test: notconsiderpathsthatseemunlikelyto leadtoasolution.
• Plangenerate-test:
−Create alist ofcandidates.
−Applygenerate-and-test to that list.

Example:
―Arrangefour6-sidedcubesinarow,witheachsideofeachcubepaintedoneoffourcolours,suchthat onallfoursides
ofthe row oneblockfaceofeachcolouris showing.‖

38
Heuristic:iftherearemoreredfacesthanothercoloursthen,whenplacingablockwithseveralredfaces,usefew
ofthemaspossible as outside faces.

GREEDYBEST –FIRST SEARCH

 Greedybest-firstsearchtriestoexpandthenodethatisclosesttothegoal,onthegroundsthatthis is
likelyto lead to a solution quickly
• Thus,theevaluation functionisf(n)=h(n)
• E.g.inminimizingroaddistancesaheuristiclowerboundfordistancesofcitiesistheirstraight-linedistance
• Greedysearchignores thecost ofthepaththathasalreadybeentraversedtoreachn
• Therefore,thesolutiongivenis notnecessarilyoptimal
• Ifrepeatingstatesarenotdetected,greedy best-
firstsearchmayoscillateforeverbetweentwopromisingstates
 Becausegreedybest-
firstsearchcanstartdownaninfinitepathandneverreturntotryotherpossibilities,it is incomplete
 Becauseofitsgreedinessthesearchmakeschoicesthatcanleadtoadeadend;thenonebacksupin
thesearch treeto thedeepest unexpanded node
 Greedybest-firstsearchresemblesdepth-firstsearchinthewayitpreferstofollowasinglepathall
thewayto the goal, butwill back up whenit hits adead end
 Theworst-casetime andspacecomplexityis O(bm)
 Thequalityof theheuristicfunction determines the practicalusabilityofgreedysearch

GREEDY:BestFirstSearch
Combiningthetwoistofollowasinglepathatatime,butswitchpathswheneversomecompetingpathlooksmorepro
misingthan thecurrent one.

 Bestfirstsearchisaninstanceofgraphsearchalgorithminwhichanodeisselectedforexpansionbasedon
evaluation function f (n).
 The node which is the lowest evaluation is selected for the explanation because the
evaluationmeasuresdistance to thegoal.
 Best first search can be implemented within general search frame work via a priority queue,
adatastructurethat will maintain the fringein ascendingorder offvalues.
 Thissearchalgorithmservesascombinationofdepthfirstandbreadthfirstsearchalgorithm. Best
first search algorithm is often referred greedy algorithm this is because
theyquicklyattackthemostdesirablepathassoonasitsheuristicweightbecomesthemostdesirable.
 For both depth-first and breadth-first search, which node in the search tree will be
considerednext onlydepends on thestructureof thetree.
 •Therationaleinbest-
firstsearchistoexpandthosepathsnextthatseemthemost“promising”.Making
thisvagueideaofwhatmay bepromisingprecisemeansdefiningheuristics.
 Heuristicsby meansof a heuristic function h that isused to estimate the ―distance‖of the
currentnoden to agoal node:
h(n)=estimatedcostfrom noden toagoalnode

39
Thedefinitionofhishighlyapplication-dependent.Intheroute-planningdomain,forinstance,wecoulduse the
straight-line distance to the goal location. For the eight-puzzle, we might use the number
ofmisplacedtiles.

Concept:
Step 1:Traversetherootnode
Step2:Traverseanyneighbouroftherootnode,thatismaintainingaleastdistancefromtherootnodeandinsert them
in ascendingorder into thequeue.
Step3:Traverseanyneighbourofneighbouroftherootnode,thatismaintainingaleastdistancefromtheroot
nodeand insert them in ascendingorder into thequeue
Step4:Thisprocesswill continueuntilwearegettingthe goalnode

 OPEN:nodesthathavebeengenerated,buthavenotexamined.Thisisorganizedasapriorityqueue.
 CLOSED:nodesthathavealreadybeenexamined.Wheneveranewnodeisgenerated,checkwhetherit
has beengenerated before.

Algorithm
1. OPEN={initial state}.
2. Loopuntil agoal isfound ortherearenonodesleftin OPEN:
−Pick thebest node in OPEN
−Generateitssuccessors
−Foreachsuccessor:
new

evaluateit,addittoOPEN,recorditsparentgeneratedbef
ore changeparent,updatesuccessors

Greedysearch:
h(n)=costof thecheapest pathfrom noden toagoal state.
Neitheroptimalnorcomplete
• Uniform-costsearch:
g(n)=costof thecheapest path from theinitial stateto noden.
Optimalandcomplete,butveryinefficient

Advantage:
 ItismoreefficientthanthatofBFS andDFS.

40
  Time complexityofBestfirst search is much less than Breadthfirst search.
 The Best first search allows us to switch between paths by gaining the benefits of both
breadthfirst and depth first search. Because, depth first is good because a solution can be found
withoutcomputingallnodesandBreadthfirstsearchisgoodbecauseitdoesnotgettrappedindeadends.
Disadvantages:
Sometimes,itcoversmoredistancethanourconsideration.

A*SEARCH
 A*isacornerstonenameofmanyAIsystemsandhasbeenusedsinceitwasdevelopedin1968byPeterHa
rt; NilsNilsson and Bertram Raphael.
 ItisthecombinationofDijkstra’salgorithmandBestfirstsearch.Itcanbe usedtosolvemanykindsof
problems.
 A*searchfindstheshortestpaththroughasearchspacetogoalstateusingheuristicfunction.Thistechni
quefindsminimal cost solutions andisdirectedtoagoal statecalledA*search.
 InA*, the*is written foroptimalitypurpose.
 The A*algorithmcombinesfeaturesofuniform-
costsearchandpureheuristicsearchtoefficientlycompute optimal solutions.

The A* algorithm also finds the lowest cost path between the start and goal state, where
changingfromonestateto anotherrequires somecost.

A*requiresheuristicfunctiontoevaluatethecostofpaththatpassesthroughtheparticularstate.Thisalgorith
m is complete if the branching factor is finite and every action has fixed cost. It can be
definedbyfollowingformula.
f(n)=g(n)+h(n)
Wher
e g(n): The actual cost path from the start state to the current
state.h (n): The estimated cost path from the current state to goal
state.f(n): Thecost pathfromthestartstate tothegoalstate.
If h(n)isan underestimateofthe path costsfrom noden toagoal node,then f(p)isan underestimate ofapath
cost ofgoingfromastart node to agoal nodeviap.

Functionf*(n):Atany noden,itistheactualcostof anoptimalpathfromnodesto nodenplusthecostofan

optimal path fromnoden to a goal node.
f*(n)=g*(n)+h*(n)
Whereg*(n)=costoftheoptimalpathinthesearchtreefromston;h*(n)=costoftheopti
mal pathinthesearchtreefromn toagoalnode;

FortheimplementationofA*algorithmwewillusetwoarraysnamelyOPENandCLOSE.OPEN:Anarra
ywhichcontainsthenodesthathasbeengeneratedbuthasnotbeenyetexamined.CLOSE:Anarraywhichc
ontainsthe nodes that havebeenexamined.

Algorithm:
1.CreateasearchgraphG,consistingsolelyof thestart nodes. Putson alist called OPEN.

41
2 .Create a list called CLOSED that is initially
empty.3.LOOP: ifOPENisempty,exit with failure.
4. Selectthefirstnode onOPEN,remove itfromOPEN andputit onCLOSED.Callthis noden.
5.Ifnisagoalnode,exitsuccessfullywiththesolutionobtainedbytracingapathalongthepointersfromn to s in
G.
6.Expandnoden,generatingtheset,M,ofitssuccessorsandinstallthemassuccessorsofninG.7.Establishapoin
tertonfromthosemembersofMthatwerenotalreadyinG(I.e,notalreadyoneitherOPENorCLOSED).Addthes
emembersofMtoOPEN.ForeachmemberofMthatwas
already on OPEN or CLOSED,decide whether or not to redirect its pointer to n. For
eachmember of M already on CLOSED, decide for each of its descendents in G whether or
not toredirectits pointer.
8. Reorder the list OPEN, either according to some scheme or some heuristic
merit.9.GotoLOOP

Example:

:8 :9

D
A

:4 :3 :0
:0
E
G’ G H

:10

PathcostforS-D-G
f(S)=g(S)+h(S)
=0+10 10
f(D)=(0+3)+
912
f(G)=(0+3+3)+0 6
Totalpathcost=f(S)+f(D)+f(G) 28

PathcostforS-A-G’
f(S)=0 +10 10
f(A)=(0+2)+8 10
f(G’)=(0+2+2)+0 4
Totalpathcost=f(S)+f(A)+f(G’) 24
*PathS-A-Gischosen=Lowestcost
42
Implementation:
TheimplementationofA*algorithmis8-puzzlegame.
Advantages:
- Itiscompleteandoptimal.
- Itisthebestonefrom othertechniques.
- Itisused tosolveverycomplexproblems.
- Itisoptimallyefficient,i.e.thereisnootheroptimalalgorithmguaranteedtoexpandfewernodesthan
A*.
Disadvantages:
- Thisalgorithmis completeif thebranchingfactorisfiniteandeveryactionhas fixedcost.
- The speed execution of A* search is highly dependant on the accuracy of the
heuristicalgorithmthatis used tocompute h (n).
- It hascomplexityproblems.

Admissibility
The property that A* always finds an optimal path, if one exists, and that the first path found to a
goalis optimal is called the admissibility of A*. Admissibility means that, even when the search space
isinfinite, if solutions exist, a solution will be found and the first path found will be an optimal solution
-alowest-cost pathfrom astart node to agoal node.

Exampleheuristicfunctions:
• Leth1(n) be thestraight-
linedistancetothegoallocation.Thisisanadmissibleheuristic,becausenosolutionpath will ever beshorter
than thestraight-lineconnection.

43
 5. HEURISTICFUNCTIONS
 A heuristic function or simply a heuristic is a function that ranks alternatives in various
searchalgorithmsateachbranchingstepbasingonanavailableinformationinordertomakeadecisionw
hichbranch is to befollowed duringasearch

Atypicalinstanceofthe8-puzzle.
Thesolutionis26stepslong.

The8-puzzle
 The8-puzzleisanexampleofHeuristicsearchproblem.
 Theobjectofthepuzzleistoslidethetileshorizontallyorverticallyintotheemptyspaceuntiltheconfigur
ation matches the goalconfiguration
 Theaveragecost forarandomlygenerated 8-puzzleinstanceis about22 steps.
 The branching factoris about3.(Whentheempty tileis in the middle,thereare four
possiblemoves;whenit isin the cornertherearetwo;and when it isalonganedgetherearethree).
 This means that an exhaustive search to depth 22 would look at about 322 approximately
=X 1010states.
 Bykeepingtrackofrepeatedstates,wecouldcutthisdownbyafactorofabout170,000,becausethereare
only9!/2 = 181,440 distinct states that arereachable.
 Thisis a manageable number, but thecorrespondingnumber forthe15-puzzle is roughly
 1013.
 IfwewanttofindtheshortestsolutionsbyusingA*,weneedaheuristicfunctionthatneveroverestimates
the number ofsteps to thegoal.
 Thetwocommonlyusedheuristicfunctionsforthe15-puzzleare:(1)h1=thenumberofmisplacedtiles.
 Inthe above figureall of theeighttiles areoutof position,
sothestart statewould haveh1=8. h1 isanadmissible heuristic.
(2)h2=thesumofthedistancesofthetilesfromtheirgoalpositions.ThisiscalledthecityblockdistanceorManhattan
distance.
h2isadmissible ,becauseall anymovecan doismove onetile onestepclosertothegoal.

Tiles1to8instartstategivea Manhattandistance ofh2=3

+1 +2+2 +2 +3 +3 +2 =18.

Neitheroftheseoverestimates thetruesolutioncost, whichis26.

44
 Inventingadmissibleheuristicfunctions

Relaxedproblems
o Aproblemwithfewerrestrictions ontheactionsiscalledarelaxedproblem
o The cost of an optimal solution to a relaxed problem is an admissible heuristic for the
originalproblem
o If the rulesof the 8-puzzlearerelaxedsothata tilecanmoveanywhere,then h1(n)givestheshortestsolution
o Iftherulesarerelaxedsothatatilecanmovetoanyadjacentsquare,thenh2(n)givestheshortestsolution

6. LOCALSEARCH &METAHEURISTICS
TheLocalsearchalgorithmoperateusingasinglecurrentstateandgenerallymoveonlytoneighboursofthat
state.It is not systematic, theyhavetwo keyadvantages
1. Theyuse verylittle memory-usuallyaconstantamount
2. Theycanoftenfindreasonablesolutionsinlargeorinfinitestatespacesforwhichsystematicalgorithms
areunsuitable.

Heuristicmethods
–―guided‖, but incomplete…
• Tobeusedwhensearchspaceistoobigand cannotbesearchedexhaustively
• So-called≪Metaheuristics≫:
• generaltechniquestoguidethesearch
• Experimentedinvariousproblems :
– TravelingSalesmanProblem (since60 ’s)
– scheduling,vehiclerouting,cutting
• Simplebut experimentallyveryefficient ...

TravellingSalesPerson(TSP)byLocalSearch
Atour can berepresented byapermutation of thelist of citynodes
• 2-opt: Swapthevisitof2nodes
• Example:
(a,b,e,d,c,f,g>→<a,b,c,d,e,f, g>

-
NaiveLocalSearchalgorithm
– Startbyarandom tour
– Considerall tours formedbyexecutingswapsof2nodes

45
– Taketheonewith best (lower)cost
– Continueuntiloptimumortime-limitreached

LocalSearch-IterativeImprovement

solvingas optimization:
Optimizationproblemwithobjectivefunction
– e.g. fitnessfunctiontomaximize,orcosttominimize
• Basicalgorithm:
– startfromarandom assignment
– Explorethe≪neighborhood≫
– moveto a≪better≫candidate
– continueuntil optimal solution isfound
• iterativeimprovement
• anytimealgorithm
– outputsgoodifnotoptimalsolution

WhyHill-Climbing/GradientDescent
TheLocalsearchalgorithmoperateusingasinglecurrentstateandgenerallymoveonlytoneighboursofthat
state.It is not systematic, theyhavetwo keyadvantages
1. Theyuseverylittlememory-usuallya constantamount
2. Theycanoftenfindreasonablesolutionsinlargeorinfinitestatespacesforwhichsystematic
algorithms areunsuitable.

HILL CLIMBING
Hill climbing search algorithm is simply a loop that continuously moves in the direction of
increasingvalue. It stopswhen it reachesa ―peak‖where no neighbour hashigher value. Thisalgorithm
is considered to be one of the simplest procedures for implementing heuristic search. The hill
climbingcomes from that idea if you are trying to find the top of the hill and you go up direction from
whereever you are. This heuristic combines the advantages of both depth first and breadth first
searches intoasinglemethod.
Hillclimbingsearch

- Itissimplyaloopthatcontinuallymovesinthedirectionofincreasingvalue–
thatisuphill.Itterminateswhen itreaches a―peak‖whereno neighborhas ahigher value.

- Thealgorithmdoesnotmaintainasearchtree,sothecurrentnodedatastructureneedonlyrecordthestate
anditsobjectivefunctionvalue.Hill-climbingdoesnotlookaheadbeyondthe

46
 immediateneighborsofthecurrentstate.ThisresemblestryingtofindthetopofMountEverestin athick
fogwhilesufferingfrom amnesia.
Hillclimbingsearchalgorithm

- Localsearchalgorithmstypicallyuseacomplete-stateformulation,whereeachstatehas8queenson the
board, one per column. The successor function returns all possible states generated bymoving a
single queen to another square in the same column (so each state has 8X 7 = 56successors)

- Theheuristiccostfunctionhisthenumberofpairsofqueensthatareattackingeachother;eitherdirectlyo
rindirectly.

- Theglobalminimumofthisfunctioniszero,whichoccursonlyatperfect solutions.

- The figure shows a state with h = 17. The figure also shows the values of all its successors
withthebest successors havingh =12.

- Hill-climbing typically chooses randomly among the set of best successor, if there is more
thanone.

- Hill-climbing is sometimes called greedy local search because it grabs a good neighbor
statewithout thinking ahead about where to go next. It often makes very rapid progress towards
asolution, becauseit isusuallyquiteeasyto improveabad state.

Figure: An 8-queens state with heuristic cost estimate h = 17, showing the value of h
foreach possible successor obtained by moving a queen within its column. The best moves
aremarked.

47
 ALocalminimuminthe8–queensstate space;thestat has h =1butevery successor has ahighercost

Forexample,fromstateinfigure(a),ittakesjustfivestepstoreachthestateinfigure(b),whichhash
=1andisverynearlya solution.

Unfortunately,hillclimbingoftengetsstuckforthefollowingreasons.LocalMa
xima:
Alocalmaximaisastatethatisbetterthaneachofitsneighbouringstates,butnotbetterthansomeotherstatesfurther
away.
Generally this state is lower than the global maximum. At this point, one cannot decide easily to
movein which direction! This difficulties can be extracted by the process of back tracking i.e.
backtrack toanyof oneearliernodeposition and trytogo onadifferentevent direction.
To implementthis strategy,maintaining in a list of pathalmosttaken and go back to one of
them.Ifthepath was taken that leads to adeadend, thengo back to oneofthem.

Local

MaximaRidges:
It is a special type of local maxima. It is a simply an area of search space. Ridges result in a
sequenceof local maxima that is very difficult to implement ridge itself has a slope which is
difficult totraverse. In this type of situation apply two or more rules before doing the test. This will
correspondtomoveinseveral directions at once.

48
 Plateau:
It is a flat area of search space in which the neighbouring have same value. So it is very difficult
tocalculate the best direction. So to get out of this situation, make a big jump in any direction,
whichwill help to movein anewdirection this isthebest waytohandle theproblem likeplateau.

• Hill climbing isa local method: Decides what to do next by looking only at the ―immediate‖
consequencesof its choices.
• Globalinformationmight beencodedinheuristicfunctions.
• Canbeveryinefficientinalarge,roughproblem space.
• Globalheuristicmayhavetopayforcomputationalcomplexity.Oftenusefulwhencombinedwithothermethod
s, gettingitstarted rightin therightgeneral neighborhood.
Example:

Localheuristic:
+1foreachblock thatisrestingonthe thingit issupposed to berestingon.
−1foreachblock thatis restingonawrongthing.

49
Globalheuristic:
Foreachblock thathasthe correctsupportstructure:+1toeveryblock inthe supportstructure.Foreach
blockthat hasawrongsupportstructure:−1 toeveryblockin thesupport structure.
Canbeveryinefficient in alarge,rough problemspace.
• Globalheuristicmayhaveto payforcomputational complexity.
• Often useful when combined with other methods, getting it started right in the right
generalneighbourhood.
5. ONLINESEARCHAGENTSANDUNKNOWNENVIRONMENT

- Offline search algorithms compute a complete solution before setting foot in the real world
andthenexecute the solutionwithout recourseto theirpercepts.

- Online search agent operates by interleaving computation and action : first it takes an
action,thenit observes the environment and computesthe next action.

- Onlinesearchisanevenbetterideaforstochasticdomains.

- In general, an offline search would have to come up with an exponentially large

contingencyplan that considers all possible happenings, while an online search need only
consider whatactuallydoes happen

- For example, a chess playing agent is well-advised to make its first move long before it
hasfiguredout thecompletecourseof thegame.

- Online search is a necessary idea for an exploration problem, where the states and actions
areunknownto theagent.

- An agent in this state of ignorance must use its actions as experiments to determine what to
donext,and hencemust interleavecomputationand action.

- The canonical example of online search is a robot that is placed in a new building and
mustexploreit to build a mapthat it can usefor gettingfrom A to B.

- Methods for escaping from labyrinths – required knowledge for aspiring heroes of antiquity –
arealsoexamples of online search algorithms.

- Spatial explorationisnottheonlyformofexploration,however.

- Consider a new born baby : it has many possible actions, but knows the outcomes of none
ofthem,and it has experienced onlyafewofthe possiblestates that itcanreach.

- Thebaby’sgradual discoveryofhowthe worldworksis,inpart, an onlinesearch process.

50
 Onlinesearchproblems

- Anonlinesearchproblemcanbesolvedonlybyanagentexecutingactions,ratherthanbyapurelycompu
tational process.

- Wewillassumethattheagentknows just thefollowing:

a. ACTIONS(S),whichreturnsalist ofactionsallowedinstateS.
b. Thestepcostfunctionc(s,a,ś)–notethatthiscannotbeuseduntiltheagentknowsthatśis
theoutcome, and
c. GOAL_TEST(S)

- The agentcannotaccessthe successorsofa stateexceptbyactuallytryingallthe actionsinthatstate.

- Forexample,inthemazeproblem,theagentdoesnotknowthatgoingupfrom(1,1)leadsto(1,2);nor,
havingdonethat,does it know thatgoingDownwilltakeitbackto (1,1).

- Thisdegreeofignorancecanbereducedinsomeapplications–
forexample,arobotexplorermightknowhowitsmovement
actionsworkandbeignorantonlyofthelocationsof obstacles.

AsampleMazeproblem

3 G
2
1 S
1 2 3

- wewillassumethattheagentcanalwaysrecognizeastatethatithasvisitedbefore,andwewill
assumethat the actions aredeterministic.

- Finally,theagentmighthaveaccesstoanadmissibleheuristicfunctionh(s)thatestimatesthedistancefr
om thecurrent state to agoalstate.

- For example, in maze problem, the agent might know the location of the goal and be able to
usetheManhattan distance heuristic.

- Typically,theagent’sobjectiveistoreachagoalstatewhileminimizingcost.Thecostisthetotalpath
cost of thepaththat the agent actuallytravels.

- Itiscommontocomparethiscostwiththepathcostofthepaththeagentwouldfollowifitknowthesearch
space inadvance–that is, the actual shortestpath

- Inthelanguageofonlinealgorithms,thisiscalledthecompetitiveratio;wewouldlikeittobeas smallas
possible.

- Althoughthissoundslikeareasonablerequest,itiseasytoseethat thebestachievablecompetitiveratio
is infinite in some cases.

- Forexample,ifsomeactionsareirreversible,theonlinesearchmightaccidentallyreachadead-
endstatefrom which no goal state isreachable.

- Noalgorithmcanavoid deadendsinallstatespaces.

51
 Figure :Twostatespacesthatmightleadanonlinesearchagentintoadeadend

- ToanonlinesearchalgorithmthathasvisitedstateSandA,thetwostatespaceslookidentical,soitmustm
akethesamedecisioninboth.Thereforeitwillfailinoneofthem–
thisisanexampleofanadversaryargument –
wecanimagineanadversarythatconstructsthestatespacewhiletheagentexplores it andcan put the
goalsand deadends whereverit likes

- Dead ends are a real difficulty for robot exploration – staircases, ramps, cliffs and all kinds
ofnaturalterrain present opportunities forirreversibleactions.

- Tomakeprogress,wewillsimplyassumethatthestatespaceissafelyexplorable–
thatis,somegoalstateis reachable from everyreachable state.

Statespaceswith reversible actions,such asmazes and8-puzzles, can be viewedas undirectedgraphsand

are clearlysafelyexplorable.

- Thebasicideaistostorea―currentbestestimate‖H(s)ofthecosttoreachthegoalfromeach statethat has

beenvisited.
- H(s) starts out being just the heuristic estimate h(s) and is updated as the agent gains
experienceinthestatespace.
52
 - In(a),theagent seemsto bestuckinaflat localmaximumattheshadedstate.

- Ratherthanstayingwhereitis,theagentshouldfollowwhatseemstobethebestpathtothegoalbased on
thecurrentcost estimatesforits neighbors.

Theestimatedcosttoreachthegoal througha neighbor śplustheestimatedcosttogettoagoalfromthere– that

is, C(S,a, ś)+H(ś)
6. EXAMPLE:REALWORLDPROBLEMS
llycareabout.

oftheir
formulation,

o RouteFindingProblem
o TouringProblems
o TravellingSalesmanProblem
o RobotNavigation

ROUTE-FINDINGPROBLEM
Route-findingproblemisdefinedintermsofspecifiedlocationsandtransitionsalonglinksbetweenthem.
Route-findingalgorithmsareusedinavarietyofapplications,

suchasroutingincomputernetworks,militaryoperationsplanning,andairlinetravelplanningsystems.

AIRLINETRAVELPROBLEM
The airlinetravelproblemisspecifiesasfollows:

o States: Eachisrepresentedbyalocation(e.g.,anairport)andthecurrenttime.
o Initialstate:Thisisspecifiedbytheproblem.
o Successor function : This returns the states resulting from taking any scheduled
flight(furtherspecified by seat class and location),leaving later than the current time plus the within-
airport transittime,fromthecurrentairporttoanother.
o GoalTest:Areweatthedestinationbysomeprespecifiedtime?
o Pathcost:Thisdependsuponthemonetarycost,waitingtime,flighttime,customsandimmigration
procedures,seat quality,time of dat,type of air plane,frequent-flyer mileage awards, andsoon.

TOURINGPROBLEMS
Touring problems are closely related to route-finding problems,but
withanimportantdifference.
Considerforexample,theproblem,"Visiteverycityatleastonce"asshowninRomaniamap.
Aswithroute-finding theactionscorrespondtotripsbetweenadjacent
cities.Thestatespace,however,isquitedifferent.

Initialstatewouldbe"InBucharest;visited{Bucharest}".
Intermediatestatewouldbe"InVaslui;visited{Bucharest,Vrziceni,Vaslui}".
Goaltestwouldcheckwhetherthe agentis inBucharestandall20citieshavebeenvisited.

53
THETRAVELLINGSALESPERSONPROBLEM(TSP)

 TSPisatouringproblemin whicheachcitymustbevisited exactlyonce.

 Theaimistofindtheshortesttour.TheproblemisknowntobeNP-hard.
 Enormous efforts have been expended to improve thecapabilitiesof
TSPalgorithms.
 These algorithms are also used in tasks such as
planningmovementsofautomaticcircuit-boarddrillsandofstockingmachineson shopfloors.

VLSIlayout
A VLSI layout problem requires positioning millions of components and connections on a chip
tominimize area ,minimizecircuitdelays,minimizestraycapacitances,and maximize
manufacturingyield.Thelayoutproblemis splitintotwo parts: cell layoutand channelrouting.

ROBOTnavigation
ROBOT navigation is a generalization of the route-finding problem. Rather than a discrete set
ofroutes,arobotcan movein a continuousspacewith an infinitesetof possibleactions and states.For
acircular Robotmovingon aflat surface,thespaceisessentiallytwo-dimensional.

Whentherobothasarmsandlegsorwheelsthatalsomustbecontrolled,thesearchspacebecomesmulti-
dimensional.Advancedtechniques arerequiredtomakethe searchspacefinite.

AUTOMATICASSEMBLYSEQUENCING
The example includes assembly of intricate objects such as electric motors. The aim in
assemblyproblems is to find the order in which to assemble the parts of some objects. If the wrong
order ischoosen,therewill benowaytoadd somepart laterwithout undoingsomework alreadydone.
Anotherimportantassemblyproblemisproteindesign,inwhichthegoalistofindasequenceofAminoacidsthat
will befold intoa three-dimensionalprotein withtherightpropertiesto curesomedisease.

INTERNETSEARCHING
InrecentyearstherehasbeenincreaseddemandforsoftwarerobotsthatperformInternetsearching,lookingforansw
ersto questions, forrelated information, or forshoppingdeals.
Thesearchingtechniquesconsiderinternetasagraphofnodes(pages)connectedbylinks.

7. SEARCHING WITH PARTIAL

OBSERVATIONSBeyondClassical Search
 Searchingwith NondeterministicActions
 SearchingwithPartialObservation
 OnlineSearchAgents andUnknownEnvironments
Whentheenvironmentisfullyobservableanddeterministic,andtheagentknowswhattheeffectsofeachactionare,p
erceptsprovideno newinformation aftertheagent determines theinitialstate.

Inpartiallyobservableenvironments,everypercepthelpsnarrowdownthesetofpossiblestatestheagentmight be
in, makingit easierfortheagent toachieveitsgoals.

54
  Innondeterministicenvironments,perceptstelltheagentwhichofthepossibleoutcomeshasactuallyoc
curred.
 Inbothcases,futureperceptscannotbedeterminedinadvanceandtheagent'sfutureactionswill
depend on thosefuturepercepts.
 Asolutiontothistypeofproblemisacontingencyplan(alsoknowasastrategy)thatspecifieswhat
to do dependingon what percepts arereceived.

If theenvironment isnotfullyobservableordeterministic,thenthefollowingtypesofproblemsoccur:
1. Sensorlessproblems
Iftheagenthasnosensors,thentheagentcannotknowit’scurrentstate,andhencewouldhavetomakemanyrepeateda
ction pathstoensurethatthegoalstateis reachedregardlessofit’s initialstate.

2. Contingencyproblems
This is when the environment is partially observable or when actions are uncertain. Then after
eachaction the agent needs to verify what effects that action has caused. Rather than planning for
everypossible contingency after an action, it is usually better to start acting and see which
contingencies doarise.
Thisiscalledinterleavingofsearchandexecution.

A problem is called adversarial if the uncertainty is caused by the actions of another

agent.SearchingwithNondeterministic Actions
Example:ErraticVacuumWorld,samestatespaceas before.Goal statesare7 and8.

Suckaction is:
Whenappliedtoadirtysquare,theactioncleansthesquareandsometimescleansupdirtinanadjacentsquare,tooWhe
n appliedtoacleansquare,theactionsometimes depositsdirton thecarpet.

Toformulatethisproblem,generalizenotionoftransitionmodelfrombefore.UseResultsfunctionthat
returns asetofpossible outcome states.
E.g.,Results (1,Suck)= { 5,7 }

Alsogeneralizenotion ofasolutiontoacontingencyplan.
E.g.,From state1, [Suck,ifState=5 then[Right,Suck]else []]

Augmentsearch trees in thefollowingway

Branchingcausedbyagent'schoice

ofactionarecalledORnodes.E.g.,invacuumworld,agentchoosesLeftorRight orSuck.

55
Branchingcausedbyenvironment'schoiceofactionarecalledANDnodes.E.g.,inerraticvacuumworld,Suck
action in state 1 leads to a state in { 5, 7 }, so agent would need to find a plan for state 5 and state7.

TwokindsofnodesalternategivingAND-OR tree.
Solutionisasubtreethathasagoalnodeateveryleafspecifiesoneactionateach-
ORnodeincludeseveryoutcomebranch ateachANDnode.

Function:AND-OR-Graph-Search
Receives: problem; Returns: conditional plan or
failureOR-Search(problem.InitialState,problem,
[])Function:OR-Search
Receives:state,problem,path
Returns:conditional planorfailure
1. Ifproblem.GoalTest(state)thenreturn emptyplan
2. Ifstateisonpaththenreturnfailure
3. Foreachactioninproblem.Actions(state)do
3.1 plan=AND-Search(Results(state,action),
problem,[state|path ])
3.2 ifplan failurethenreturn[action|plan]
4. Returnfailure

Function:AND-Search
Receives:states,problem,path
Returns:conditional planorfailure
1. Foreachsin states do
1.1 planii=OR-Search(s,problem,path)
1.2 Ifplanii=failurethenreturnfailure
2. Return [ if s1 then plan1 else if s2 then plan else …
ifsn-1thenplan n-1 elseplan n ]2
Noteloopsarehandledbylookingforastateinthecurrentpathandreturningfailure.Thisguaranteesterminationin
finite statespaces.

GivenalgorithmisDFS.CanalsosearchtreewithBFSorbest-
first,andgeneralizeheuristicfunctionforcontingencyplans foran analogtoA*.

56
An alternative agent design is for the agent act before it has a guaranteed plan and deal
contingenciesonlyastheyariseinexecution.Thistypeofinterleavingofsearchandexecutionisusefulforexplo
rationproblemsandgameplaying.

Asnoted,whenanenvironmentispartiallyobservable,anagentcanbeinoneofseveralpossiblestates.Anaction
leads tooneofseveral possibleoutcomes.

To solve these problems, an agent maintains a belief state that represent the agent's current belief
aboutthepossiblephysical stateit mightbein,giventhesequenceofactions andperceptsup tothatpoint.

When an agent's percepts provide no information at all, this is called a sensorless (or
conformant)problemExample: deterministic, assumeknow geography, butnotlocation or dirt.

Initiallycould be inanystate{1, 2, 3, 4,5, 6, 7,8}

DoRight,mustin{ 2,4,6,8 }.DoSuc
kresults in{ 4,8 }.
DoingLeft, thenSuck
coerces worldintostate7regardlessofinitialstate

Solve sensorless problems by searching space of belief states, which is fully observable
toagent. Suppose physical problem P with Actions P , Result P , GoalTest P and
StepCostDefinecorrespondingsensorless problem:

Beliefstates:powersetofstatesofP,althoughmanyareunreachablefrominitialstate.For N
statesofP, thereare2 Npossiblebeliefstates.

Initialstate:typically,allthestatesofP
Actions:ifillegalactionsaresafe,usethe unionof actionsofallstates inb;otherwiseusetheintersection.Useto
predict the next beliefstates.
Transition model:b' =Result(b,a)={s':s' =Result (s,a)ands b }.
Similarlyfornondeterministicenvironments usingResultsP

Goaltest:all ofthephysical states inbmust satisfyGoalTest P

Pathcost:Assumethatactioncosts sameinallstates,sotransferfromunderlyingproblem

Complete belief
statesearchspacefordeterministicenvironment.Noteonly12reachablesta
tes out of 2possible beliefstates.
57
Previous problems use offline search algorithms. Complete solution is computed, then solution
isexecuted.

In online search, agent interleaves computation with action: it first takes an action, then observes
theenvironmentand computesthe next action.
Good idea for dynamic environments where there is a penalty for computing too long. Helpful idea
fornondeterministicenvironments.Don'tspendtime planningforcontingenciesthat arerare.
Necessaryideaforunknownenvironments,whereagentdoesn'tknowwhatstatesexistsortheresultsofits
actions. Calledan exploration problem. Agent uses actions as experiments to learn
aboutitsenvironment
Assumedeterministic, fullyobservableenvironment. Agent onlyknows:
Actions(s) – list of actions allowed in state s Step-cost function c(s, a, s') – cannot be used until
agentknows that s' is the outcome of doing a GoalTest(s) In particular, it doesn't know Result (s,a)
except byactuallybeingin s and doinga, thenobservings'.

Example:mazeproblem.
AgentstartsinS.GoalistomovetoG.AgentinitiallydoesnotknowthatgoingUpfrom(1,1)leadsto(1,2)nor
thatgoing Downfrom there goes back to (1,1).
Agentmayhaveaccess to admissible heuristic.
E.g.,if agentknowswheregoalis,canuseManhattandistanceheuristic.

Typically,objectiveistoreachgoalstatewhileminimizingcost,wherecostistotalpathcost of

58
path an agent actually travels. Common to compare this cost with path cost of path agent
would followif it knew the search space in advance, i.e., the actual shortest path. Called the
competitive ratio, andwouldlikeit to be as smallas possible.

Niceidea,butin some casescompetitiveratioisinfinite,ifonline searchagent reachesa dead-

endstatefrom which nogoalstateis reachable.
Claim:noalgorithmcanavoiddeadendsinallstatespaces.Twostatesshownareindistinguishable,
Somustresultin sameaction sequence.
Will bewrongforone.

Tomakeprogress,weassumethatstatespaceissafelyexplorable.I.e.,somegoalstateisreachablef
romeveryreachablestate.Statespaceswithreversibleactions(mazes,8puzzles)areclearlysafely
explorable.
Even with this assumption, no bounded competitive ratio can be guaranteed if there are
paths
ofunboundedcost.Commontodescribeperformanceintermsofsizeofentirestatespaceinsteadof
depthoftheshallowestnodeOnlinesearchalgorithmsareverydifferentfromofflinesearchalgorit
hms.Sinceagent occupies specific physical node, can only expand immediate successors.
Needto expand nodes inlocalorder.DFS has thisproperty,but needsreversible
actionstosupportbacktracking.

Hill-climbing already is a local search algorithm, but can get stuck in local maxima. Add
memory tokeep track of a "current best estimate", H(s), of the cost to reach goal from each
state that has beenvisited.
H(s)startsoutwithh(s),theheuristicestimateand isupdatedasagent gainsexperience

59