-------------------------------------------------------------------------------------------------------

CS1351 ARTIFICIAL INTELLIGENCE

VI SEMESTER CSE

AUTHORS N.Vivekanantha !!"th# S.Sanka" $e%a"t ent !& CSE 'CG C!((e)e !& Te*hn!(!)#+Chennai-,-

UNIT-II ./0 SEARCHING TECHNI1UES

----------------------------------------------------------------------------------------/.1 INFORME$ SEARCH AN$ E23LORATION
/.1.1 In&!" e4.He5"i6ti*0 Sea"*h St"ate)ie6 /.1./ He5"i6ti* F5n*ti!n6 /.1.3 L!*a( Sea"*h A()!"ith 6 an4 O%ti i7ati!n 3"!8(e 6 /.1.9 L!*a( Sea"*h in C!ntin5!56 S%a*e6

/.1.5 On(ine Sea"*h A)ent6 an4 Unkn!:n Envi"!n ent6 -----------------------------------------------------------------------------------------------------------------------

/./ CONSTRAINT SATISFACTION 3RO;LEMS.CS30

/./.1 C!n6t"aint Sati6&a*ti!n 3"!8(e 6 /././ ;a*kt"a*kin) Sea"*h &!" CS36 /./.3 The St"5*t5"e !& 3"!8(e 6 ---------------------------------------------------------------------------------------------------------------------------------

/.3 A$VERSARIAL SEARCH

/.3.1 Ga e6 /.3./ O%ti a( $e*i6i!n6 in Ga e6 /.3.3 A(%ha-;eta 3"5nin) /.3.9 I %e"&e*t +Rea(-ti e $e*i6i!n6 /.3.5 Ga e6 that in*(54e E(e ent !& Chan*e -----------------------------------------------------------------------------------------------------------------------------

/.1 INFORME$ SEARCH AN$ E23LORATION

/.1.1 In&!" e4.He5"i6ti*0 Sea"*h St"ate)ie6

In&!" e4 6ea"*h 6t"ate)# is one that uses problem-specific knowledge beyond the definition of the problem itself. It can find solutions more efficiently than uninformed strategy. ;e6t-&i"6t 6ea"*h ;e6t-&i"6t 6ea"*h is an instance of general TREE-SEARC or !RA" -SEARC algorithm in which a node is selected for e#pansion based on an eva(5ati!n &5n*ti!n f$n%. The node with lowest e&aluation is selected for e#pansion'because the e&aluation measures the distance to the goal. This can be implemented using a priority-(ueue'a data structure that will maintain the fringe in ascending order of f-&alues. /.1./. He5"i6ti* &5n*ti!n6 A he5"i6ti* &5n*ti!n or simply a he5"i6ti* is a function that ranks alternati&es in &arious search algorithms at each branching step basing on an a&ailable information in order to make a decision which branch is to be followed during a search. The key component of )est-first search algorithm is a he5"i6ti* &5n*ti!n'denoted by h$n%* h$n% + e#timated cost of the *hea%e6t %ath from node n to a )!a( n!4e. ,or e#ample'in Romania'one might estimate the cost of the cheapest path from Arad to )ucharest &ia a 6t"ai)ht-(ine 4i6tan*e from Arad to )ucharest$,igure -..%. euristic function are the most common form in which additional knowledge is imparted to the search algorithm. G"ee4# ;e6t-&i"6t 6ea"*h G"ee4# 8e6t-&i"6t 6ea"*h tries to e#pand the node that is closest to the goal'on the grounds that this is likely to a solution (uickly. It e&aluates the nodes by using the heuristic function f$n% + h$n%. Taking the e#ample of R!5te-&in4in) %"!8(e 6 in Romania ' the goal is to reach )ucharest starting from the city Arad. /e need to know the straight-line distances to )ucharest from &arious cities as shown in ,igure -... ,or e#ample' the initial state is In$Arad% 'and the straight line distance heuristic hS01$In$Arad%% is found to be 233. 4sing the 6t"ai)ht-(ine 4i6tan*e heuristic hSL$ +the goal state can be reached faster.

Fi)5"e /.1 5alues of hS01 - straight line distances to )ucharest

Fi)5"e /./ stages in greedy best-first search for )ucharest using straight-line distance heuristic hS01. 6odes are labeled with their h-&alues.

,igure -.- shows the progress of greedy best-first search using hS01 to find a path from Arad to )ucharest. The first node to be e#panded from Arad will be Sibiu'because it is closer to )ucharest than either 7erind or Timisoara. The ne#t node to be e#panded will be ,agaras'because it is closest. ,agaras in turn generates )ucharest'which is the goal.

"roperties of greedy search

o C! %(ete<< 6o8can get stuck in loops' e.g.' Iasi 9 6eamt 9 Iasi 9 6eamt 9 Complete in finite space with repeated-state checking o Ti e<< :$bm%' but a good heuristic can gi&e dramatic impro&ement o S%a*e<< :$bm%;keeps all nodes in memory o O%ti a(<< 6o !reedy best-first search is not optimal'and it is incomplete. The worst-case time and space comple#ity is :$bm%'where m is the ma#imum depth of the search space.

A= Sea"*h
A= Sea"*h is the most widely used form of best-first search. The e&aluation function f$n% is obtained by combining $.% ).n0 > the cost to reach the node'and $-% h.n0 > the cost to get from the node to the )!a( *

f$n% + g$n% < h$n%.

A= Search is both optimal and complete. A= is optimal if h$n% is an admissible heuristic. The ob&ious e#ample of admissible heuristic is the straight-line distance hS01. It cannot be an o&erestimate. A= Search is optimal if h$n% is an admissible heuristic 8 that is'pro&ided that h$n% ne&er o&erestimates the cost to reach the goal. An ob&ious e#ample of an admissible heuristic is the straight-line distance hS01 that we used in getting to )ucharest. The progress of an A= tree search for )ucharest is shown in ,igure -.-. The &alues of >g > are computed from the step costs shown in the Romania map$ figure -..%. Also the &alues of hS01 are gi&en in ,igure -...

Re*5"6ive ;e6t-&i"6t Sea"*h.R;FS0

Recursi&e best-first search is a simple recursi&e algorithm that attempts to mimic the operation of standard best-first search'but using only linear space. The algorithm is shown in figure -.?. Its structure is similar to that of recursi&e depth-first search'but rather than continuing indefinitely down the current path'it keeps track of the f-&alue of the best alternati&e path a&ailable from any ancestor of the current node. If the current node e#ceeds this limit'the recursion unwinds back to the alternati&e path. As the recursion unwinds'R),S replaces the f-&alue of each node along the path with the best f-&alue of its children. ,igure -.@ shows how R),S reaches )ucharest.

Fi)5"e /.3 Stages in A= Search for )ucharest. 6odes are labeled with f + g < h . The h-&alues are the straight-line distances to )ucharest taken from figure -..

function RECURSIVE-BEST-FIRST-SEARCH(problem) return a solution or failure return RFBS(problem,MAKE-N !E(INITIA"-STATE#problem$),%) function RFBS( problem, node, f_limit) return a solution or failure an& a ne' fcost li(it if ) A"-TEST#problem$(STATE#node$) then return node successors E*+AN!(node, problem) if successors is e(,t- t.en return failure, % for each s in successors do f #s$ (a/(g(s) 0 h(s), f #node$) repeat best t.e lo'est f-1alue no&e in successors if f #best$ > f_limit then return failure, f #best$ alternative t.e se2on& lo'est f-1alue a(on3 successors result, f #best$ RBFS(problem, best, (in(f_limit, alternative)) if result failure then return result

Fi)5"e /.9 The algorithm for recursi&e best-first search

Fi)5"e /.5 Stages in an R),S search for the shortest route to )ucharest. The f-limit &alue for each recursi&e call is shown on top of each current node. $a% The path &ia Rimnicu 5ilcea is followed until the current best leaf $"itesti% has a &alue that is worse than the best alternati&e path $,agaras%. $b% The recursion unwinds and the best leaf &alue of the forgotten subtree $?.A% is backed up to Rimnicu 5ilceaBthen ,agaras is e#panded're&ealing a best leaf &alue of ?@C. $c% The recursion unwinds and the best leaf &alue of the forgotten subtree $?@C% is backed upto ,agarasB then Rimni 5icea is e#panded. This time because the best alternati&e path$through Timisoara% costs atleast ??A'the e#pansion continues to )ucharest R),S Eva(5ati!n * R),S is a bit more efficient than I1A= 4 Still e#cessi&e node generation $mind changes%

0ike A=' optimal if h(n) is admissible Space comple#ity is O(bd). 4 I1A= retains only one single number $the current f-cost limit% Time comple#ity difficult to characteriDe 4 1epends on accuracy if h$n% and how often best path changes. I1A= en R),S suffer from too little memory.

/.1./ He5"i6ti* F5n*ti!n6

A he5"i6ti* &5n*ti!n or simply a heuristic is a function that ranks alternati&es in &arious search algorithms at each branching step basing on an a&ailable information in order to make a decision which branch is to be followed during a search

Fi)5"e /.? A typical instance of the E-puDDle. The solution is -3 steps long. The @-%577(e The E-puDDle is an e#ample of euristic search problem. The obFect of the puDDle is to slide the tiles horiDontally or &ertically into the empty space until the configuration matches the goal configuration$,igure -.3% The a&erage cost for a randomly generated E-puDDle instance is about -- steps. The branching factor is about 2.$/hen the empty tile is in the middle'there are four possible mo&esBwhen it is in the corner there are twoBand when it is along an edge there are three%. This means that an e#hausti&e search to depth -- would look at about 2-- appro#imately + 2.. G .C.C states. )y keeping track of repeated states'we could cut this down by a factor of about .AC'CCC'because there are only H9I- + .E.'??C distinct states that are reachable. This is a manageable number 'but the corresponding number for the .@-puDDle is roughly .C.2. If we want to find the shortest solutions by using A='we need a heuristic function that ne&er o&erestimates the number of steps to the goal. The two commonly used heuristic functions for the .@-puDDle are * $.% h. + the number of misplaced tiles. ,or figure -.3 'all of the eight tiles are out of position'so the start state would ha&e h. + E. h. is an admissible heuristic.

$-% h- + the sum of the distances of the tiles from their goal positions. This is called the *it# 8(!*k 4i6tan*e or Manhattan 4i6tan*e. h- is admissible 'because all any mo&e can do is mo&e one tile one step closer to the goal. Tiles . to E in start state gi&e a Janhattan distance of h- + 2 < . < - < - < - < 2 < 2 < - + .E. 6either of these o&erestimates the true solution cost 'which is -3. The E&&e*tive ;"an*hin) &a*t!" :ne way to characteriDe the A5a(it# !& a he5"i6ti* is the e&&e*tive 8"an*hin) &a*t!" 8=. If the total number of nodes generated by A= for a particular problem is N'and the 6!(5ti!n 4e%th is 4+then b= is the branching factor that a uniform tree of depth d would ha&e to ha&e in order to contain 6<. nodes. Thus' 6 < . + . < b= < $b=%-<K<$b=%d ,or e#ample'if A= finds a solution at depth @ using @- nodes'then effecti&e branching factor is ..H-. A well designed heuristic would ha&e a &alue of b= close to .'allowing failru large problems to be sol&ed. To test the heuristic functions h. and h-'.-CC random problems were generated with solution lengths from - to -? and sol&ed them with iterati&e deepening search and with A= search using both h. and h-. ,igure -.A gi&es the a&eraghe number of nodes e#panded by each strategy and the effecti&e branching factor. The results suggest that h- is better than h.'and is far better than using iterati&e deepening search. ,or a solution length of .?'A= with h- is 2C'CCC times more efficient than uninformed iterati&e deepening search.

Fi)5"e /.- Comparison of search costs and effecti&e branching factors for the ITERATI5E1EE"E6I6!-SEARC and A= Algorithms with h.'and h-. 1ata are a&erage o&er .CC instances of the E-puDDle'for &arious solution lengths. Inventin) a4 i66i8(e he5"i6ti* &5n*ti!n6 Re(aBe4 %"!8(e 6 o A problem with fewer restrictions on the actions is called a relaxed problem o The cost of an optimal solution to a rela#ed problem is an admissible heuristic for the original problem o If the rules of the E-puDDle are rela#ed so that a tile can mo&e anywhere' then h1(n) gi&es the shortest solution

o If the rules are rela#ed so that a tile can mo&e to any adjacent square, then h2(n) gi&es the shortest solution

/.1.3 LOCAL SEARCH ALGORITHMS AN$ O3TIMICATION 3RO;LEMS

o In many optimiDation problems' the path to the goal is irrele&antB the goal state itself is the solution o ,or e#ample'in the E-(ueens problem'what matters is the final configuration of (ueens'not the order in which they are added. o In such cases' we can 56e (!*a( 6ea"*h a()!"ith 6. They operate using a 6in)(e *5""ent 6tate$rather than multiple paths% and generally mo&e only to neighbors of that state. o The important applications of these class of problems are $a% integrated-circuit design' $b%,actory-floor layout'$c% Fob-shop scheduling'$d%automatic programming' $e%telecommunications network optimiDation'$f%5ehicle routing'and $g% portfolio management. 'e# a4vanta)e6 !& L!*a( Sea"*h A()!"ith 6 $.% They use &ery little memory 8 usually a constant amountB and $-% they can often find reasonable solutions in large or infinite$continuous% state spaces for which systematic algorithms are unsuitable. O3TIMICATION 3RO;LEMS Inaddition to finding goals'local search algorithms are useful for sol&ing pure !%ti i7ati!n %"!8(e 6'in which the aim is to find the 8e6t 6tate according to an !8De*tive &5n*ti!n. State S%a*e Lan46*a%e To understand local search'it is better e#plained using 6tate 6%a*e (an46*a%e as shown in figure -.E. A landscape has both L(!*ati!nM $defined by the state% and Le(evati!nM$defined by the &alue of the heuristic cost function or obFecti&e function%. If ele&ation corresponds to *!6t'then the aim is to find the (!:e6t va((e# 8 a )(!8a( ini 5 B if ele&ation corresponds to an !8De*tive &5n*ti!n'then the aim is to find the hi)he6t %eak 8 a )(!8a( aBi 5 . 0ocal search algorithms e#plore this landscape. A complete local search algorithm always finds a )!a( if one e#istsB an !%ti a( algorithm always finds a )(!8a( ini 5 E aBi 5 .

Fi)5"e /.@ A one dimensional 6tate 6%a*e (an46*a%e in which ele&ation corresponds to the !8De*tive &5n*ti!n. The aim is to find the global ma#imum. ill climbing search modifies the current state to try to impro&e it 'as shown by the arrow. The &arious topographic features are defined in the te#t

Hi((-*(i 8in) 6ea"*h

The hi((-*(i 8in) search algorithm as shown in figure -.H' is simply a loop that continually mo&es in the direction of increasing &alue 8 that is'5%hi((. It terminates when it reaches a L%eakM where no neighbor has a higher &alue.

&5n*ti!n I00-C0IJ)I6!$ problem% "et5"n a state that is a local ma#imum in%5tF problem' a problem (!*a( va"ia8(e6F current+ a n!4e. neighbor+ a n!4e. current JANE-6:1E$I6ITIA0-STATEOproblemP% (!!% 4! neighbor a highest &alued successor of current i& 5A04E OneighborP 5A04EOcurrentP then "et5"n STATEOcurrentP current neighbor Fi)5"e /., The hill-climbing search algorithm $steepest ascent &ersion%'which is the most basic local search techni(ue. At each step the current node is replaced by the best neighborBthe neighbor with the highest 5A04E. If the heuristic cost estimate h is used'we could find the neighbor with the lowest h. ill-climbing is sometimes called greedy local search because it grabs a good neighbor state without thinking ahead about where to go ne#t. !reedy algorithms often perform (uite well.

3"!8(e 6 :ith hi((-*(i 8in)

ill-climbing often gets stuck for the following reasons * o L!*a( aBi a * a local ma#imum is a peak that is higher than each of its neighboring states'but lower than the global ma#imum. ill-climbing algorithms that reach the &icinity

of a local ma#imum will be drawn upwards towards the peak'but will then be stuck with nowhere else to go o Ri4)e6 * A ridge is shown in ,igure -..C. Ridges results in a se(uence of local ma#ima that is &ery difficult for greedy algorithms to na&igate. o 3(atea5B * A plateau is an area of the state space landscape where the e&aluation function is flat. It can be a flat local ma#imum'from which no uphill e#it e#ists'or a shoulder'from which it is possible to make progress.

Fi)5"e /.1G Illustration of why ridges cause difficulties for hill-climbing. The grid of states$dark circles% is superimposed on a ridge rising from left to right'creating a se(uence of local ma#ima that are not directly connected to each other. ,rom each local ma#imum'all th a&ailable options point downhill.

Hi((-*(i 8in) va"iati!n6

St!*ha6ti* hi((-*(i 8in) o Random selection among the uphill mo&es. o The selection probability can &ary with the steepness of the uphill mo&e. Fi"6t-*h!i*e hi((-*(i 8in) o cfr. stochastic hill climbing by generating successors randomly until a better one is found. Ran4! -"e6ta"t hi((-*(i 8in) o Tries to a&oid getting stuck in local ma#ima.

Si 5(ate4 annea(in) 6ea"*h

A hill-climbing algorithm that ne&er makes LdownhillM mo&es towards states with lower &alue$or higher cost% is guaranteed to be incomplete'because it can stuck on a local ma#imum.In contrast'a purely random walk 8that is'mo&ing to a successor choosen uniformly at random from the set of successors 8 is complete'but e#tremely inefficient. Simulated annealing is an algorithm that combines hill-climbing with a random walk in someway that yields both efficiency and completeness. ,igure -... shows simulated annealing algorithm. It is (uite similar to hill climbing. Instead of picking the best mo&e'howe&er'it picks the random mo&e. If the mo&e impro&es the situation'it is always accepted. :therwise'the algorithm accepts the mo&e with some probability less than .. The probability decreases e#ponentially with the LbadnessM of the mo&e 8 the amount E by which the e&aluation is worsened.

Simulated annealing was first used e#tensi&ely to sol&e 50SI layout problems in the early .HECs. It has been applied widely to factory scheduling and other large-scale optimiDation tasks.

Fi)5"e /.11 The simulated annealing search algorithm'a &ersion of stochastic hill climbing where some downhill mo&es are allowed.

Geneti* a()!"ith 6
A !enetic algorithm$or !A% is a &ariant of stochastic beam search in which successor states are generated by combining two parent states'rather than by modifying a single state. 0ike beam search'!as begin with a set of k randomly generated states'called the population. Each state'or indi&idual'is represented as a string o&er a finite alphabet 8 most commonly'a string of Cs and .s. ,or e#ample'an E E-(uuens state must specify the positions of E (ueens'each in acolumn of E s(uares'and so re(uires E # log- E + -? bits.

Fi)5"e /.1/ The genetic algorithm. The initial population in $a% is ranked by the fitness function in $b%'resulting in pairs for mating in $c%. They produce offspring in $d%'which are subFected to

mutation in $e%. ,igure -..- shows a population of four E-digit strings representing E-(ueen states. The production of the ne#t generation of states is shown in ,igure -..-$b% to $e%. In $b% each state is rated by the e&aluation function or the &itne66 &5n*ti!n. In $c%'a random choice of two pairs is selected for reproduction'in accordance with the probabilities in $b%. ,igure -..2 describes the algorithm that implements all these steps. &5n*ti!n !E6ETICQA0!:RIT J$ population, ,IT6ESS-,6% "et5"n an indi&idual in%5tF population' a set of indi&iduals ,IT6ESS-,6' a function which determines the (uality of the indi&idual "e%eat new_population empty set (!!% &!" i &"! . t! SI7E$population% 4! x RA61:JQSE0ECTI:6$population' ,IT6ESSQ,6% y RA61:JQSE0ECTI:6$population' ,IT6ESSQ,6% child RE"R:14CE$x,y% i& $small random probability% then child J4TATE$child % add child to new_population population new_population 5nti( some indi&idual is fit enough or enough time has elapsed "et5"n the best indi&idual Fi)5"e /.13 A genetic algorithm. The algorithm is same as the one diagrammed in ,igure -..-'with one &ariation*each mating of two parents produces only one offspring'not two.

/.1.9 LOCAL SEARCH IN CONTINUOUS S3ACES

/e ha&e considered algorithms that work only in discrete en&ironments' but real-world en&ironment are continuous 0ocal search amounts to ma#imiDing a continuous obFecti&e function in a multi-dimensional &ector space. This is hard to do in general. Can immediately retreat o 1iscretiDe the space near each state o Apply a discrete local search strategy $e.g.' stochastic hill climbing' simulated annealing% :ften resists a closed-form solution o ,ake up an empirical gradient o Amounts to greedy hill climbing in discretiDed state space Can employ 6ewton-Raphson Jethod to find ma#ima Continuous problems ha&e similar problems* plateaus' ridges' local ma#ima' etc.

/.1.5 On(ine Sea"*h A)ent6 an4 Unkn!:n Envi"!n ent6

On(ine 6ea"*h %"!8(e 6

 :ffline Search $all algorithms so far% Compute complete solution' ignoring en&ironment Carry out action se(uence :nline Search Interlea&e computation and action Compute;Act;:bser&e;Compute;R :nline search good ,or dynamic' semi-dynamic' stochastic domains /hene&er offline search would yield e#ponentially many contingencies :nline search necessary for e#ploration problem States and actions unknown to agent Agent uses actions as e#periments to determine what to do E#amples Robot e#ploring unknown building Classical hero escaping a labyrinth Assume agent knows Actions a&ailable in state s Step-cost function c$s'a'sS% State s is a goal state /hen it has &isited a state s pre&iously Admissible heuristic function h$s % 6ote that agent doesnTt know outcome state $s S % for a gi&en action $a% until it tries the action $and all actions from a state s % Competiti&e ratio compares actual cost with cost agent would follow if it knew the search space 6o agent can a&oid dead ends in all state spaces Robotics e#amples* Staircase' ramp' cliff' terrain Assume state space is safely e#plorable;some goal state is always reachable On(ine Sea"*h A)ent6 Interlea&ing planning and acting hamstrings offline search A= e#pands arbitrary nodes without waiting for outcome of action :nline algorithm can e#pand only the node it physically occupies )est to e#plore nodes in physically local order Suggests using depth-first search 6e#t node always a child of the current /hen all actions ha&e been tried' canTt Fust drop state Agent must physically backtrack :nline 1epth-,irst Search Jay ha&e arbitrarily bad competiti&e ratio $wandering past goal% :kay for e#plorationB bad for minimiDing path cost :nline Iterati&e-1eepening Search Competiti&e ratio stays small for state space a uniform tree On(ine L!*a( Sea"*h ill Climbing Search Also has physical locality in node e#pansions

Is' in fact' already an online search algorithm 0ocal ma#ima problematic* canTt randomly transport agent to new state in effort to escape local ma#imum Random /alk as alternati&e Select action at random from current state /ill e&entually find a goal node in a finite space Can be &ery slow' esp. if LbackwardM steps as common as LforwardM ill Climbing with Jemory instead of randomness Store Lcurrent best estimateM of cost to goal at each &isited state Starting estimate is Fust h$s % Augment estimate based on e#perience in the state space Tends to Lflatten outM local minima' allowing progress Employ optimism under uncertainty 4ntried actions assumed to ha&e least-possible cost Encourage e#ploration of untried paths Lea"nin) in On(ine Sea"*h o Rampant ignorance a ripe opportunity for learning Agent learns a LmapM of the en&ironment o :utcome of each action in each state o 0ocal search agents impro&e e&aluation function accuracy o 4pdate estimate of &alue at each &isited state o /ould like to infer higher-le&el domain model o E#ample* L4pM in maDe search increases y -coordinate Re(uires o ,ormal way to represent and manipulate such general rules $so far' ha&e hidden rules within the successor function% o Algorithms that can construct general rules based on obser&ations of the effect of actions

/./ CONSTRAINT SATISFACTION 3RO;LEMS.CS30

A C!n6t"aint Sati6&a*ti!n 3"!8(e $or CS"% is defined by a set of va"ia8(e6 +G.'G-'K.Gn'and a set of constraints C.'C-'K'Cm. Each &ariable Gi has a nonempty 4! ain 1'of possible va(5e6. Each constraint Ci in&ol&es some subset of &ariables and specifies the allowable combinations of &alues for that subset. A State of the problem is defined by an a66i)n ent of &alues to some or all of the &ariables'UGi + &i'GF + &F'KV. An assignment that does not &iolate any constraints is called a *!n6i6tent or (e)a( a66i)n ent. A complete assignment is one in which e&ery &ariable is mentioned'and a 6!(5ti!n to a CS" is a complete assignment that satisfies all the constraints. Some CS"s also re(uire a solution that ma#imiDes an !8De*tive &5n*ti!n. EBa %(e &!" C!n6t"aint Sati6&a*ti!n 3"!8(e F ,igure -..@ shows the map of Australia showing each of its states and territories. /e are gi&en the task of coloring each region either red'green'or blue in such a way that the neighboring regions ha&e the same color. To formulate this as CS" 'we define the &ariable to be the regions */A'6T'W'6S/'5'SA' and T. The domain of each &ariable is the set Ured'green'blueV.The constraints re(uire neighboring regions to ha&e distinct colorsBfor e#ample'the allowable combinations for /A and 6T are the pairs U$red'green%'$red'blue%'$green'red%'$green'blue%'$blue'red%'$blue'green%V.

The constraint can also be represented more succinctly as the ine(uality /A not + 6T'pro&ided the constraint satisfaction algorithm has some way to e&aluate such e#pressions.% There are many possible solutions such as U /A + red' 6T + green'W + red' 6S/ + green' 5 + red 'SA + blue'T + redV. It is helpful to &isualiDe a CS" as a constraint graph'as shown in ,igure -..@$b%. The nodes of the graph corresponds to &ariables of the problem and the arcs correspond to constraints.

Fi)5"e /.15 $a% "rinciple states and territories of Australia. Coloring this map can be &iewed as aconstraint satisfaction problem. The goal is to assign colors to each region so that no neighboring regions ha&e the same color.

Fi)5"e /.15 $b% The map coloring problem represented as a constraint graph. CS" can be &iewed as a standard search problem as follows * Initia( 6tate * the empty assignment UV'in which all &ariables are unassigned. S5**e66!" &5n*ti!n * a &alue can be assigned to any unassigned &ariable'pro&ided that it does not conflict with pre&iously assigned &ariables. G!a( te6t * the current assignment is complete. 3ath *!6t * a constant cost$E.g.'.% for e&ery step. E&ery solution must be a complete assignment and therefore appears at depth n if there are n &ariables. 1epth first search algorithms are popular for CS"s

Va"ietie6 !& CS36 .i0 $i6*"ete va"ia8(e6 Finite 4! ain6 The simplest kind of CS" in&ol&es &ariables that are 4i6*"ete and ha&e &inite 4! ain6. Jap coloring problems are of this kind. The E-(ueens problem can also be &iewed as finite-domain CS"'where the &ariables W.'W-'K..WE are the positions each (ueen in columns .'K.E and each &ariable has the domain U.'-'2'?'@'3'A'EV. If the ma#imum domain siDe of any &ariable in a CS" is d'then the number of possible complete assignments is :$dn% 8 that is'e#ponential in the number of &ariables. ,inite domain CS"s include ;!!(ean CS36'whose &ariables can be either true or alse. In&inite 4! ain6 1iscrete &ariables can also ha&e in&inite 4! ain6 8 for e#ample'the set of integers or the set of strings. /ith infinite domains'it is no longer possible to describe constraints by enumerating all allowed combination of &alues. Instead a constraint language of algebric ine(ualities such as StartFob. < @ X+ StartFob2. .ii0 CS36 :ith *!ntin5!56 4! ain6 CS"s with continuous domains are &ery common in real world. ,or e#ample 'in operation research field'the scheduling of e#periments on the ubble Telescope re(uires &ery precise timing of obser&ationsB the start and finish of each obser&ation and maneu&er are continuous-&alued &ariables that must obey a &ariety of astronomical'precedence and power constraints. The best known category of continuous-domain CS"s is that of (inea" %"!)"a in) problems'where the constraints must be linear ine(ualities forming a con!ex region. 0inear programming problems can be sol&ed in time polynomial in the number of &ariables. Va"ietie6 !& *!n6t"aint6 F .i0 5na"# *!n6t"aint6 in&ol&e a single &ariable. E#ample * SA Y green $ii% )inary constraints in&ol&e paris of &ariables. E#ample * SA Y /A $iii% igher order constraints in&ol&e 2 or more &ariables. E#ample * cryptarithmetic puDDles.

Fi)5"e /.1? $a% Cryptarithmetic problem. Each letter stands for a distinct digitBthe aim is to find a substitution of digits for letters such that the resulting sum is arithmetically correct'with the added restriction that no leading Deros are allowed. $b% The constraint hypergraph for the cryptarithmetic problem'showint the "lldi constraint as well as the column addition constraints. Each constraint is a s(uare bo# connected to the &ariables it

contains.

/././ ;a*kt"a*kin) Sea"*h &!" CS36

The term 8a*kt"a*kin) 6ea"*h is used for depth-first search that chooses &alues for one &ariable at a time and backtracks when a &ariable has no legal &alues left to assign. The algorithm is shown in figure -..A.

Fi)5"e /.1- A simple backtracking algorithm for constraint satisfaction problem. The algorithm is modeled on the recursi&e depth-first search

Fi)5"e /.1-.80 "art of search tree generated by simple backtracking for the map coloring problem. 3"!%a)atin) in&!" ati!n th"!5)h *!n6t"aint6
So far our search algorithm considers the constraints on a &ariable only at the time that the &ariable is chosen by SE0ECT-46ASSI!6E1-5ARIA)0E. )ut by looking at some of the constraints earlier in the search' or e&en before the search has started' we can drastically reduce the search space. F!":a"4 *he*kin)

:ne way to make better use of constraints during search is called &!":a"4 *he*kin). /hene&er a &ariable X is assigned' the forward checking process looks at each unassigned &ariable Y that is connected to X by a constraint and deletes from Y Ts domain any &alue that is inconsistent with the &alue chosen for X. ,igure @.3 shows the progress of a map-coloring search with forward checking.

C!n6t"aint %"!%a)ati!n
Although forward checking detects many inconsistencies' it does not detect all of them. C!n6t"aint %"!%a)ati!n is the general term for propagating the implications of a constraint on one &ariable onto other &ariables.

A"* C!n6i6ten*#

k-C!n6i6ten*#

L!*a( Sea"*h &!" CS36

/./.3 The St"5*t5"e !& 3"!8(e 6 3"!8(e St"5*t5"e

In4e%en4ent S58%"!8(e 6

T"ee-St"5*t5"e4 CS36

/.9 A$VERSARIAL SEARCH

Competeti&e en&ironments'in which the agentTs goals are in conflict'gi&e rise to a4ve"6a"ia( 6ea"*h problems 8 often known as )a e6. /.9.1 Ga

e6

Jathematical Ga e The!"#'a branch of economics'&iews any 5(tia)ent envi"!n ent as a )a e pro&ided that the impact of each agent on the other is LsignificantM'regardless of whether the agents are cooperati&e or competiti&e. In'AI'MgamesM are deterministic'turn-taking'twoplayer'Dero-sum games of perfect information. This means deterministic'fully obser&able en&ironments in which there are two agents whose actions must alternate and in which the 5ti(it# va(5e6 at the end of the game are always e(ual and opposite. ,or e#ample'if one player wins the game of chess$<.%'the other player necessarily loses$-.%. It is this opposition between the agentsT utility functions that makes the situation a4ve"6a"ia(. F!" a( $e&initi!n !& Ga e /e will consider games with two players'whom we will call MA2 and MIN. JAG mo&es first'and then they take turns mo&ing until the game is o&er. At the end of the game' points are awarded to the winning player and penalties are gi&en to the loser. A )a e can be formally defined as a 6ea"*h %"!8(e with the following components * o The initia( 6tate'which includes the board position and identifies the player to mo&e. o A 65**e66!" &5n*ti!n'which returns a list of $mo!e,state% pairs'each indicating a legal mo&e and the resulting state. o A te" ina( te6t'which describes when the game is o&er. States where the game has ended are called te" ina( 6tate6. o A 5ti(it# &5n*ti!n $also called an obFecti&e function or payoff function%'which gi&e a numeric &alue for the terminal states. In chess'the outcome is a win'loss'or draw'with &alues <.'-.'or C. he payoffs in backgammon range from <.H- to -.H-.

!ame Tree
The initia( 6tate and (e)a( !ve6 for each side define the )a e t"ee for the game. ,igure -..E shows the part of the game tree for tic-tac-toe $noughts and crosses%. ,rom the initial state'JAG has nine possible mo&es. "lay alternates between JAGTs placing an G and JI6Ts placing a C until we reach leaf nodes corresponding to the terminal states such that one player

has three in a row or all the s(uares are filled. e number on each leaf node indicates the utility &alue of the terminal state from the point of &iew of JAGBhigh &alues are assumed to be good for JAG and bad for JI6. It is the JAGTs Fob to use the search tree$particularly the utility of terminal states% to determine the best mo&e.

Fi)5"e /.1@ A partial search tree . The top node is the initial state'and JAG mo&e first'placing an G in an empty s(uare.

/.9./ O%ti a( $e*i6i!n6 in Ga e6 In normal search problem'the !%ti a( 6!(5ti!n would be a se(uence of mo&e leading to a )!a( 6tate 8 a terminal state that is a win. In a game'on the other hand'JI6 has something to say about it'JAG therefore must find a contingent 6t"ate)#'which specifies JAGTs mo&e in the initia( 6tate'then JAGTs mo&es in the states resulting from e&ery possible response by JI6'then JAGTs mo&es in the states resulting from e&ery possible response by JI6 those mo&es'and so on. An !%ti a( 6t"ate)# leads to outcomes at least as good as any other strategy when one is playing an infallible opponent.

Fi)5"e /.1, A two-ply game tree. The nodes are LJAG nodesM'in which it is AJGTs turn to mo&e'and the nodes are LJI6 nodesM. The terminal nodes show the utility &alues for JAGB the other nodes are labeled with their minima# &alues. JAGTs best mo&e at the root is a.'because it leads to the successor with the highest minima# &alue'and JI6Ts best reply is b.'because it leads to the successor with the lowest minima# &alue.

Fi)5"e /./G An algorithm for calculating minima# decisions. It returns the action corresponding to the best possible mo&e'that is'the mo&e that leads to the outcome with the best utility'under the assumption that the opponent plays to minimiDe utility. The functions JAG-5A04E and JI6-5A04E go through the whole game tree'all the way to the lea&es'to determine the backed-up &alue of a state. The ini aB A()!"ith

The minima# algorithm$,igure -.-C% computes the minima# decision from the current state. It uses a simple recursi&e computation of the minima# &alues of each successor state'directly implementing the defining e(uations. The recursion proceeds all the way down to the lea&es of the tree 'and then the minima# &alues are 8a*ke4 5% through the tree as the recursion unwinds. ,or e#ample in ,igure -..H'the algorithm first recourses down to the three bottom left nodes'and uses the utitliy function on them to disco&er that their &alues are 2'.-'and E respecti&ely. Then it takes the minimum of these &alues'2'and returns it as the backed-up &alue of node ). A similar process gi&es the backed up &alues of - for C and - for 1. ,inally'we take the ma#imum of 2'-'and - to get the backed-up &alue of 2 at the root node. The minima# algorithm performs a complete depth-first e#ploration of the game tree. If the ma#imum depth of the tree is m'and there are b legal mo&es at each point'then the time comple#ity of the minima# algorithm is :$bm%. The space comple#ity is :$bm% for an algorithm that generates successors at once. /.9.3 A(%ha-;eta 3"5nin) The problem with minima# search is that the number of game states it has to e#amine is eB%!nentia( in the number of mo&es. 4nfortunately'we canTt eliminate the e#ponent'but we can effecti&ely cut it in half. )y performing %"5nin)+we can eliminate large part of the tree from consideration. /e can apply the techni(ue known as a(%ha 8eta %"5nin) 'when applied to a minima# tree 'it returns the same mo&e as ini aB would'but %"5ne6 a:a# branches that cannot possibly influence the final decision. Alpha )eta pruning gets its name from the following two parameters that describe bounds on the backed-up &alues that appear anywhere along the path* Z * the &alue of the best$i.e.'highest-&alue% choice we ha&e found so far at any choice point along the path of JAG. o [* the &alue of best $i.e.' lowest-&alue% choice we ha&e found so far at any choice point along the path of JI6. Alpha )eta search updates the &alues of Z and [ as it goes along and prunes the remaining branches at anode$i.e.'terminates the recursi&e call% as soon as the &alue of the current node is known to be worse than the current Z and [ &alue for JAG and JI6'respecti&ely. The complete algorithm is gi&en in ,igure -.-.. The effecti&eness of alpha-beta pruning is highly dependent on the order in which the successors are e#amined. It might be worthwhile to try to e#amine first the successors that are likely to be the best. In such case'it turns out that alpha-beta needs to e#amine only :$bdI-% nodes to pick the best mo&e'instead of :$bd% for minima#. This means that the effecti&e branching factor becomes s(rt$b% instead of b 8 for chess'3 instead of 2@. "ut anotherway alpha-beta cab look ahead roughly twice as far as minima# in the same amount of time. o

Fi)5"e /./1 The alpha beta search algorithm. These routines are the same as the minima# routines in figure -.-C'e#cept for the two lines in each of JI65A04E and JAG-5A04E that maintain Z and [

/.9.9 I %e"&e*t +Rea(-ti e $e*i6i!n6

The minima# algorithm generates the entire game search space'whereas the alpha-beta algorithm allows us to prune large parts of it. owe&er'alpha-beta still has to search all the way to terminal states for atleast a portion of search space. ShannonTs .H@C paper'"rogramming a computer for playing chess'proposed that programs should *5t !&& the search earlier and apply a heuristic eva(5ati!n &5n*ti!n to states in the search'effecti&ely turning nonterminal nodes into terminal lea&es. The basic idea is to alter minima# or alpha-beta in two ways *

$.% The utility function is replaced by a heuristic e&aluation function E5A0'which gi&es an estimate of the positionTs utility'and $-% the terminal test is replaced by a *5t!&& te6t that decides when to apply E5A0.

/.9.5 Ga e6 that in*(54e E(e ent !& Chan*e

Eva(5ati!n &5n*ti!n6 An e&aluation function returns an estimate of the e#pected utility of the game from a gi&en position'Fust as the heuristic function return an estimate of the distance to the goal.

Ga e6 !& i %e"&e*t in&!" ati!n o Jinima# and alpha-beta pruning re(uire too much leaf-node e&aluations. Jay be impractical within a reasonable amount of time. o S A66:6 $.H@C%* o Cut off search earlier $replace TERJI6A0-TEST by C4T:,,-TEST% o Apply heuristic e&aluation function E5A0 $replacing utility function of alpha-beta% C5ttin) !&& 6ea"*h ChangeF 4 i& TERJI6A0-TEST$state% then "et5"n 4TI0IT\$state% into 4 i& C4T:,,-TEST$state,depth% then "et5"n E5A0$state% Introduces a fi#ed-depth limit depth 4 Is selected so that the amount of time will not e#ceed what the rules of the game allow. /hen cuttoff occurs' the e&aluation is performed. He5"i6ti* EVAL Idea* produce an estimate of the e#pected utility of the game from a gi&en position. "erformance depends on (uality of E5A0. Re(uirements* 4 E5A0 should order terminal-nodes in the same way as 4TI0IT\. 4 Computation may not take too long. 4 ,or non-terminal states the E5A0 should be strongly correlated with the actual chance of winning. :nly useful for (uiescent $no wild swings in &alue in near future% states Hei)hte4 Linea" F5n*ti!n The introductory chess books gi&e an appro#imate material &alue for each piece * each pawn is worth .'a knight or bishop is worth 2'a rook 2'and the (ueen H. These feature &alues are then added up toobtain the e&aluation of the position. Jathematically'these kind of e&aluation fuction is called weighted linear function'and it can be e#pressed as * #!al(s) + w. f.$s% < w- f-$s% < K < wn fn$s% ] e.g.' w. + H with f.$s% + $number of white (ueens% 8 $number of black (ueens%' etc.

Ga e6 that in*(54e *han*e

In real life'there are many unpredictable e#ternal e&ents that put us into unforeseen situations. Jany games mirror this unpredictability by including a random element'such as throwing a dice. ;a*k)a !n is a typical game that combines luck and skill. 1ice are rolled at the beginning of playerTs turn to determine the legal mo&es. The backgammon position of ,igure -.-2'for e#ample'white has rolled a 3-@'and has four possible mo&es.

Fi)5"e /./3 A typical backgammon position. The goal of the game is to mo&e all oneTs pieces off the board. /hite mo&es clockwise toward -@'and black mo&es counterclockwise toward C. A piece can mo&e to any position unless there are

multiple opponent pieces thereB if there is one opponent 'it is captured and must start o&er. In the position shown'white has rolled 3-@ and must choose among four legal mo&es [email protected]'@-..%'$@-..'.H--?%'[email protected]'.C-.3%'and $@-..'..-.3% /hite mo&es clockwise toward -@ )lack mo&es counterclockwise toward C A piece can mo&e to any position unless there are multiple opponent pieces thereB if there is one opponent' it is captured and must start o&er. /hite has rolled 3-@ and must choose among four legal mo&es* [email protected]' @-..%' $@-..' .H--?% [email protected]' .C-.3%' and $@-..' ..-.3% ------------------------------------------------------------------------------------------------------------------------

Fi)5"e /-/9 Schematic game tree for a backgammon position. EB%e*te4 ini aB va(5e EG"ECTE1-JI6IJAG-5A04E$n%+ 4TI0IT\$n% If n is a terminal ma#s successors(n) If n is a ma# node mins successors(n) If n is a ma# node

JI6IJAG-5A04E$s% JI6IJAG-5A04E$s%

s successors(n) $(s) % EG"ECTE1JI6IJAG$s% If n is a chance node These e(uations can be backed-up recursi&ely all the way to the root of the game tree.

------------------------------------------------------------------------------------------------------