Artificial intelligence

Search
 Introduction
• Predicate calculus expressions provide a
means of describing objects, relations
within a problem domain.
• Further inference rules allows us to infer
knowledge from the descriptions.
• Such inferences define a space that is
searched to find a problem solution.
• Hence it is important to understand the
theory of state space search.
 Important questions
• How can I represent information?
• What are my possibilities?
• Will my program terminate one day ?
• Is the solution an optimal one ?
• What is the complexity in terms of time and
memory?
• In short, we need a mathematical modelisation.
• Graphs are used to give answers, (mathematical
tool)
 Graphs
• A graph is a set of nodes and arcs (vertices)
that connect them.
• A graph can be labeled.
• A graph can be weighted.
• A graph can be directed.
 Directed graphs and path
• A graph is directed if arcs have an
associated directionality.
• Example:
• A path is a sequence of arcs between two
nodes.
 Example Graph - Travelling
Salesman Problem

3 7

4 5 5
6 6
4
2
5 9

• Example: The Traveling salesman problem.

(a map).
 Man, Wolf, Goat and Cabbage
• A man owns a wolf, a goat and
a cabbage
• He has to cross a river in a boat,
but he can only take one item at
a time
• If left alone, the wolf will eat
the goat or the goat will eat the
cabbage
• How can he cross the river
without anything being eaten?
 Searching
• Many problems can be solved by searching
for a good solution in a search space
• Examples of such problems are
– finite state systems where possible successor
states multiply
– large rule based systems where many different
rules can fire
 Representing a Search Space
• A convenient way of representing search
spaces is as a graph
• A graph consists of nodes and links
– nodes represent states in a problem solving
process
– links represent transitions or relationships
between nodes
 Trees
• A special case of a graph is a directed rooted tree
• A rooted tree has:
– a unique node (the root) from which it is possible to reach
all other nodes in the graph
– at most one link between any two nodes
– directional links
– no cycles
• Trees are common in search problems
• Terms: ancestor, successor, child, parent, root, leaf
Example
 Representing a problem with a
graph
• Typically when representing a problem with a
graph, we have:
– a start state (where we are now, for example the current
position of a game of chess)
– one or more goal states where the search for a solution
terminates (for example a check-mate position in chess)
• goal states have either a specific property in themselves, or a
measurable property of the path from the goal
• for example, we might be interested in the length of the path
 State Space Representation of
Problems
A state space is represented by a four tuple [N, A, S, GD]
where
– N is the set of states.
– A is the set of steps between states
– S is the start state(S) of the problem.
– GD is the goal state(S) of the problem.
The states in GD are described:
1. A measurable property of the states. (winning board in tic-tac-
toe)
2. A property of the path.(shortest path in traveling sales man
problem)
A solution path is a path through this graph from S to GD
 Provides a means of determining the complexity of the problem
 Tic-tac-toe
• The set of states are all different configurations of Xs and
Os that the game can have. [N]
– 39 ways to arrange {blank, X, O} in nine spaces.
• Arcs(steps) are generated by legal moves of the game,
alternating between placing an X and an O in an unused
location [A]
• The start state is an empty board. [S]
• The goal state is a board state having three Xs in a row,
column, or diagonal. [GD]
Tic-tac-toe
 Tic-tac-toe
• There are no cycles in the state space because the
directed arcs of the graph do not allow a move to
be undone.
• The complexity of the problem : 9!(362,880)
different path can be generated.
– Need heuristics to reduce the search complexity.
e.g. My possible winning lines – Opponent’s possible
wining lines (Fig 4.16, p149, tp14)
 Chess has 10120 possible game paths
 The 8-puzzle
• The set of states are all different configurations
of 9 tiles (9!).
• The legal moves are : move the blank tile up(↑),
right(→), down(↓), and the left(←).
• make sure that it does not move the blank off the
board.
• All four moves are not applicable at all times.
• The start state
• The goal state
• unlike tic-tac-toe, cycles are possible in the 8-
puzzle
Eight puzzle
Forward and Backward Chaining
• Defining the ”start state” can be done in two main
ways
• Forward Chaining
– also known as “data driven” approach
– start from what you know….
– work towards a solution
– used to prove new theorems
• Backward Chaining
– also known as a “goal driven” approach
– start from the optimum solution….
– work out what inputs yield this solution
– useful to check new things
Goal-directed search
Data-directed search
 Goal Driven
 A goal is given
• In Mathematics
• Diagnostic System
 A large number of rules are given that match the
facts
 Problem data are not given
• In Diagnostic
• Only a subset of medical tests are ordered
 Data Driven
• The initial problem statement contains most
of the data
• A large number of goals exist
– A few ways to use the facts
• Difficult to form the goal
 How to choose the right
approach?
• Choice of the approach
– Is determined by the properties of the problem
• Complexity of the rules
• Shape of the state space
• Nature and availability of problem data
• One has to analyse the graph to be able to make an
appropriate choice
• Often forward chaining, because often one would not
know where the solution is
 Backtracking
• In practice, we do not know in advance if
there is at least a solution or how we can
reach them in the graph.
• Thus, we try to build the correct path. If we
fail, we go back to a previous state. (you
may see why recursion is useful)
Backtracking
 Searching a Graph
• We want systematic methods for searching
graphs which yield optimum solutions with
as little computational effort as possible
• Two basic approaches:
– depth first search
– breadth-first search
• Both rely on some common machinery
 Search Machinery
• A list of closed nodes
– nodes which have been evaluated and their children placed on
the open list
• A list of open nodes
– nodes which have not been evaluated yet
– the first item on this list is evaluated next
• Note that nodes are usually stored as tuples, with their
parent node attached. This enables the path to a solution
(when we find it) to be reconstructed.
• Before placing a node on the open list, we need to check
that it is not already on the closed list
 Depth First Search
• When we evaluate a state and derive its children,
we put these children at the front of the open list
– the list operates as a stack
• The search goes deeper into the space whenever
possible and backtracks only at dead ends
• Can be very fast, especially if there are many
solutions, but all quite deep in the space
• Good for memory management (back tracking)
• May fail to terminate even if a solution exists
• Not guaranteed to return the shortest path
Implementing Depth First Search
• Implementing depth first search is very easy in
any modern programming language:
– Evaluate the current state using a function
– Recursively call the evaluation function for each of
the children states
– The recursion mechanism takes care of all the
necessary book-keeping and allows the solution
path to be generated as the recursion unwinds
 Example-DFS
1. open = [A]; closed =[]
2. open = [B,C,D]; closed =[A]
3. open = [E,F,C,D]; closed=[B,A]
……..
……..
 Breadth First Search
• When we evaluate a state and derive its children, we put these
children at the back of the open list
– the list operates as a queue
• The search systematically goes across each level
• Always finds a solution if one exists
• Always finds the shortest path
• Can be very slow if the branching factor is high
• Can use a huge amount of memory (the open list can get
enormous)
• Requires more a bit more work to implement
 Example-BFS
1. open = [A]; closed =[]
2. open = [B,C,D]; closed =[A]
3. open = [C,D, E,F]; closed=[B,A]
4. open = [ D,E,F,G,H];closed=[C,B,A]
5. open = [E,F,G,H,I,J];closed=[D,C,B,A]
……..
 “Blind search”
• DFS and BFS are blind in the sense that they
have no knowledge about the problem at all other
than the problem space
• Such techniques are also called brute-force
search, uninformed search, or weak methods
• Obviously, we can’t expect too much from these,
but they provide
– Worst-case scenarios
– A basis for more interesting algorithms later on
 DFS and BFS
• Worst case scenarios are equally bad (exponential)
• How to evaluate search algorithms
– Completeness: a search algorithm is complete if it is
guaranteed to find a solution when one exists
– Quality of solution: usually the path cost
– Time cost of finding the solution: usually in terms of
number of nodes generated or examined
– Memory cost of finding the solution: how many
nodes do we have to keep around during the search
 Evaluating BFS
• Complete?
Yes
• Optimal quality solution?
Yes
• Time required in the worst case
O(bd)
• Memory required in the worst case (in OPEN)
O(bd)
• where b is the branching factor, d is the depth of the
solution
 Evaluating DFS
• Complete?
Yes (only if the tree is finite)
• Optimal quality solution?
No
• Time required in the worst case
O(bm)
• Memory required in the worst case (in OPEN)
O(bm)
• where b is the branching factor, m is the
maximum depth of the tree
 Example search problems
• Puzzles:
– 8-puzzle, 15-puzzle, n-queens, the Tower of Hanoi,
• 2-player games:
– chess
• Proving theorems in logic and geometry
• Path finding
• “Industrial” problems:
– VLSI layout design, assembling a complex object
• “AI problems”: speech recognition, planning, …
The importance of the problem space
• The choice of a problem space makes a big
difference
– In fact, finding a good abstraction is half of the
problem
• Intelligence is needed to figure out what
problem space to use
• still poorly understood: the human problem
solver is conducting a search in the space of
problem spaces
 Refinements
• There are several possible refinements to
straightforward depth- and breadth- first
searching:
– Hybrid breadth/depth using iterative deeping
– Mixed forward/backward chaining
– Heuristics
 Iterative Deepening
• A way of carrying out a breadth-first search
without excessive memory needs and getting some
of the advantages of both methods
• Do a depth first search but limit the depth of the
search
• If no solution is found, repeat the depth first
search with an increased depth limit
 Mixed Chaining
• If what we are interested in is finding the optimum
path between two (known) nodes, it may make
sense to combine forward and backward chaining
– i.e. expand both simultaneously and try to meet in the
middle by matching the open lists
• This is because the number of nodes expands
exponentially
Effectiveness of Mixed Chaining
• This reduction in computational effort relies on
having efficient sorting or hashing algorithms so
as to be able to compare the open lists quickly
 Heuristics
• AI uses heuristics to address two main
difficulties:
– A problem may be ambiguous, with no exact
solution
– A problem may have an exact solution, but the
computational cost of finding it may be too large.
• This second problem is the subject of heuristic
search
State space graph of a set of implications
in propositional calculus
And/or graph of the expression
q∧r→p
And/or graph of the expression
q∨r→p
 Hypergraph
• A hypergraph consists of:
– N: a set of nodes
– H: a set of hyperarcs
• Hyperarcs are defined by ordered pairs in which
the first element of the pairs is a single node from
N and the second element is a subset of N.
• An ordinary graph is a special case of hypergraph
in which all the sets of descendant nodes have a
cardinality of 1.
And/or graph of a set of propositional
calculus expressions