AI Techniques

Properties of AI Problems

AI Problems have a very broad spectrum.

AI problems are hard to solve.
AI problems require intelligence.
Intelligence requires knowledge.
Fundamental Techniques of AI

Knowledge Representation
Intelligence/intelligent behaviour requires knowledge.

Properties of Knowledge
Voluminous
Hard to characterize
Constantly changing

How can one capture formally (i.e., computerize) everything

needed for intelligent behaviour? Some questions...
How do you store all of that data in a useful way?
How can you store decision making steps?
Fundamental Techniques of AI
Characteristics of good data representation
techniques:

Captures general situation rather than being

overly specific
Understandable by the people who provide it
Easily modified to handle errors, changes in data,
and changes in perception
general use
Three important AI Techniques
Search
Provides a way of solving problems for which
no direct approach is available
as well as a framework into which any direct
techniques that are available can be embedded.

Use of Knowledge
Provides a way of solving complex problems by
exploiting the structures of the objects that are
involved.

Abstraction
Provides a way of separating important features
and variations from the many unimportant ones
that would otherwise overwhelm any process.
Advantages of using AI Techniques

They are much less fragile. They will not be thrown off
completely by a small changes in their input.
People easily understand what the program knowledge
is.
These techniques can work for large problems when
direct methods fail.
 Search
Search is the fundamental technique of AI.
Possible answers, decisions or courses of action are structured into
an abstract space, which we then search.
Search is either "blind" or “uninformed":
blind
we move through the space without worrying about
what is coming next, but recognising the answer if
we see it
informed
we guess what is ahead, and use that information to
decide where to look next.
We may want to search for the first answer that satisfies our
goal, or we may want to keep searching until we find the
best answer.
Knowledge Representation & Reasoning
The second most important concept in AI is Knowedge
Representation and Reasoning.
If we are going to act rationally in our environment, then
we must have some way of describing that environment
and drawing inferences from that representation.
–how do we describe what we know about the world ?
–how do we describe it so that we can get hold of the
right piece of knowledge when we need it ?
–how do we generate new pieces of knowledge ?
–how do we deal with uncertain knowledge ?
 Types of Knowledge
Declarative Knowledge
Expressed as statements of facts about the world.
Ex.- Personnel data in a database is a declarative
knowledge.
Procedural Knowledge
It is a compiled knowledge related to the performance of
some task.
Ex.- The steps used to solve an algebraic equation are
expressed as procedural.
Heuristic
A special type of knowledge used by human to solve
complex problems. Heuristics are the knowledge used to
make good judgments or strategies, tricks or “rules of
thumb” used to simplify the solution of problem.
Heuristics are usually acquired with much experience.
Ex.- Fault finding in a TV set.
Problems, Problem Spaces, and Search
Building a Problem-Solving Program

1. Define the problem precisely.

2. Analyze the problem.
3. Represent the task knowledge.
4. Choose and apply representation and reasoning
techniques.
Defining the problem as State
Space Search
The state space representation forms the basis of most of
the AI methods.
Its structure corresponds to the structure of problem
solving in two important ways:
It allows for a formal definition of a problem as the need to convert
some given situation into some desired situation using a set of
permissible operations.
It permits us to define the process of solving a particular problem as
a combination of known techniques (each represented as a rule
defining a single step in the space) and search, the general
technique of exploring the space to try to find some path from
current state to a goal state.
Search is a very important process in the solution of hard problems
for which no more direct techniques are available.
Example: Playing Chess
To build a program that could “play chess”, we
could have to specify
the starting position of the chess board,
the rules that define the legal moves
and the board positions that represent a win for one
side or the other.
In addition, we must make explicit the previously
implicit goal of not only playing the legal game of
chess but also winning the game, if possible,
One Legal Chess Move
Playing chess
The starting position can be described as an 8 by 8 array
where each position contains a symbol for appropriate
piece.
We can define as our goal the check mate position.
The legal moves provide the way of getting from initial
state to a goal state.
They can be described easily as a set of rules consisting of
two parts:
A left side that serves as a pattern to be matched against the
current board position.
And a right side that describes the change to be made to reflect the
move
However, this approach leads to large number of rules
10120 board positions !!
Using so many rules poses problems such as:
– No person could ever supply a complete set of such rules.
– No program could easily handle all those rules. Just storing so
many rules poses serious difficulties.
Defining chess problem as State
Space search
We need to write the rules describing the
legal moves in as general a way as possible.
The less work we will have to do to provide
the knowledge and more efficient the
program.

For example:
White pawn at Square( file e, rank 2) AND
Square( File e, rank 3) is empty AND Square(file
e, rank 4) is empty, then move the pawn from
Square( file e, rank 2) to Square( file e, rank 4).
Way to Describe Chess Moves
A Water Jug Problem
Water Jug Problem
The state space for this problem can be described as the
set of ordered pairs of integers (x,y) such that x = 0, 1,2,
3 or 4 and y = 0,1,2 or 3; x represents the number of
gallons of water in the 4-gallon jug and y represents the
quantity of water in 3-gallon jug
The start state is (0,0)
The goal state is (2,n)
Production Rules for the
Water Jug Problem
A Water Jug Problem (Cont’d)
To solve the water jug problem

Require a control structure that loops through a simple

cycle in which some rule whose left side matches the
current state is chosen, the appropriate change to the state
is made as described in the corresponding right side, and
the resulting state is checked to see if it corresponds to
goal state.
One Solution to the water Jug Problem
Example: Water Pouring
To Specify a Problem

1. Define the state space.

2. Specify the initial states.
3. Specify the goal states.
4. Specify the operations.
Production Systems
1. A set of rules.
2. Knowledge/databases.
3. A control strategy.
4. A rule applier.
Production Systems
A production system consists of:
A set of rules, each consisting of a left side that
determines the applicability of the rule and a right side
that describes the operation to be performed if that rule
is applied.
One or more knowledge/databases that contain
whatever information is appropriate for the particular
task. Some parts of the database may be permanent,
while other parts of it may pertain only to the solution
of the current problem.
A control strategy that specifies the order in which the
rules will be compared to the database and a way of
resolving the conflicts that arise when several rules
match at once.
A rule applier
How to decide which rule to apply next during the process
of searching for a solution to a problem?

The two requirements of good control strategy are that

It should cause motion.

It should be systematic
Searching For Solutions
Searching For Solutions
Uninformed Search Strategies
• Uninformed strategies use only the
information available in the problem
definition
– Also known as blind searching

• Breadth-first search
• Uniform-cost search
• Depth-first search
• Depth-limited search
• Iterative deepening search
Comparing Uninformed Search
Strategies
• Completeness
– Will a solution always be found if one exists?
• Time
– How long does it take to find the solution?
– Often represented as the number of nodes searched
• Space
– How much memory is needed to perform the search?
– Often represented as the maximum number of nodes
stored at once
• Optimal
– Will the optimal (least cost) solution be found?
Algorithm:
Create a variable called NODE-LIST and set it to
initial state
Until a goal state is found or NODE-LIST is empty
do
Remove the first element from NODE-LIST and call it E. If
NODE-LIST was empty, quit
For each way that each rule can match the state described
in E do:
Apply the rule to generate a new state
If the new state is a goal state, quit and return this state
Otherwise, add the new state to the end of NODE-LIST
Breadth-First Search
Breadth-First Search
Breadth-First Search
Breadth-First Search
Properties of Breadth-First
Search
• Complete
– Yes if b (max branching factor) is finite
• Time
– 1 + b + b2 + … + bd + b(bd-1) = O(bd+1)
– exponential in d
• Space
– O(bd+1)
– Keeps every node in memory
– This is the big problem; an agent that generates nodes at
10 MB/sec will produce 860 MB in 24 hours
• Optimal
– Yes (if cost is 1 per step); not optimal in general
Lessons From Breadth First
Search
• The memory requirements are a bigger problem for
breadth-first search than is execution time

• Exponential-complexity search problems cannot be

solved by uniformed methods for any but the
smallest instances
 (0,0)

(4,0) (0,3)

(0,0) (1,3) (4,3) (0,0) (3,0)

(4,3)
If the initial state is a goal state, quit and return success
Otherwise, do the following until success or failure is
signaled:
Generate a successor, E, of initial state. If there are no more
successors, signal failure.
Call Depth-First Search, with E as the initial state
If success is returned, signal success. Otherwise continue in
this loop.
Depth-First Search
• Recall from Data Structures the basic algorithm for
a depth-first search on a graph or tree

• Expand the deepest unexpanded node

• Unexplored successors are placed on a stack until

fully explored
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
Depth-First Search
• Complete
– No: fails in infinite-depth spaces, spaces with loops
• Modify to avoid repeated spaces along path
– Yes: in finite spaces
• Time
– O(bm)
– Not great if m is much larger than d
– But if the solutions are dense, this may be faster than
breadth-first search
• Space
– O(bm)…linear space
• Optimal
– No
Uniform-Cost Search
• Same idea as the algorithm for breadth-first
search…but…
– Expand the least-cost unexpanded node
– FIFO queue is ordered by cost
– Equivalent to regular breadth-first search if all step
costs are equal
Uniform-Cost Search
• Complete
– Yes Time
– Complexity cannot be determined easily by b or
d
– Let C* be the cost of the optimal solution
– O(bceil(C*/ ε)) (assume that every action costs at least
ε.)
• Space
– O(bceil(C*/ ε))
• Optimal
– Yes, Nodes are expanded in increasing order
• In this search, we pursue a signal branch of the tree until it yields a
solution or until a decision to terminate the path is made.
• It makes sense to terminate a path if it reaches dead-end, produces a
previous state. In such a state backtracking occurs.
• Chronological Backtracking: Order in which steps are undone depends
only on the temporal sequence in which steps were initially made.
Specifically most recent step is always the first to be undone. This is also
simple backtracking.
Depth-Limited Search
• A variation of depth-first search that uses a depth
limit
– Alleviates the problem of unbounded trees
– Search to a predetermined depth l (“ell”)
– Nodes at depth l have no successors

• Same as depth-first search if l = ∞

• Can terminate for failure and cutoff
Depth-Limited Search
Depth-Limited Search
• Complete
– no if l < d
• Time
– O(bl)
• Space
– O(bl)
• Optimal
– No if l > d
Iterative Deepening Search
• Iterative deepening depth-first search
– Uses depth-first search
– Finds the best depth limit
• Gradually increases the depth limit; 0, 1, 2, …
until a goal is found
Iterative Deepening Search
Iterative Deepening Search
Iterative Deepening Search
Iterative Deepening Search
Iterative Deepening Search
Iterative Deepening Search
• Complete
– Yes
• Time
– O(bd)
• Space
– O(bd)
• Optimal
– Yes if step cost = 1
– Can be modified to explore uniform cost tree
Lessons From Iterative
Deepening Search
• Faster than BFS even though IDS generates
repeated states
– BFS generates nodes up to level d+1
– IDS only generates nodes up to level d

• In general, iterative deepening search is the

preferred uninformed search method when there is
a large search space and the depth of the solution is
not known
Avoiding Repeated States
• Complication of wasting time by expanding
states that have already been encountered
and expanded before
– Failure to detect repeated states can turn a
linear problem into an exponential one

• Sometimes, repeated states are unavoidable

– Problems where the actions are reversable
• Route finding
• Sliding blocks puzzles
Avoiding Repeated States

State Space Search Tree

Avoiding Repeated States
DFS requires less memory since only the nodes on the
current path are stored.

By chance, DFS may find a solution without examining

much of the search space at all.
BFS will not get trapped exploring a blind alley.

If there is a solution, BFS is guaranteed to find it.

If there are multiple solutions, then a minimal solution will

be found.
 A simple motion causing and systematic control
structure could solve this problem. Simply explore all
possible paths in the tree and return the shortest path.

If there are N cities, then number of different paths

among them is 1.2….(N-1) or (N-1)!. The time to examine
single path is proportional to N. So the total time
required to perform this search is proportional to N!

For 10 cities, 10! = 3,628,800.This phenomenon is called

Combinatorial explosion.
Begin generating complete paths, keeping track of
the shortest path found so far.
Give up exploring any path as soon as its partial
length becomes greater than the shortest path
found so far.
Using this algorithm, we are guaranteed to find
the shortest path.
It still requires exponential time. The time it
saves depends on the order in which paths are
explored.
A Heuristic is a technique that improves the efficiency of a
search process, possibly by sacrificing claims of completeness.
Heuristics are like tour guides. They are good to the extent that
they point in generally interesting directions;

They are bad to the extent that they may miss points of interest
to particular individuals.

On the average they improve the quality of the paths that are
explored.

Using Heuristics, we can hope to get good ( though possibly

non optimal ) solutions to hard problems.

There are good general purpose heuristics that are useful in a

wide variety of problem domains.

Special purpose heuristics exploit domain specific knowledge

 It works by selecting locally superior alternative at each
step.
Applying to TSP:
Arbitrarily select a starting city
To select the next city, look at all cities not yet visited and
select the one closest to the current city. Go to next step.
Repeat step 2 until all cities have been visited.

– This procedure executes in time proportional to N2

● This is a function that maps from problem state
descriptions to measures of desirability, usually
represented as numbers.

• which aspects of the problem state are considered.

• how those aspects are evaluated, and
• the weights given to individual aspects are chosen in
such a way that the value of the heuristic function at a
given node in the search process gives as good an
estimate as possible of whether that node is on the
desired path to a solution.

● Well designed heuristic functions can play an

important part in efficiently guiding a search process
toward a solution.
Chess : The material advantage of our side over opponent.

TSP: the sum of distances traveled so far

Tic-Tac-Toe: 1 for each row in which we could win and in

which we already have one piece plus 2 for each such row in
which we have two pieces