CS6659 ARTIFICIAL INTELLIGENCE

UNIT I

INTRODUCTION TO Al AND PRODUCTION SYSTEMS

Introduction to AI - Problem formulation, Problem Definition - Production systems, Control

strategies, Search strategies. Problem characteristics, Production system characteristics -
Specialized production system - Problem solving methods - Problem graphs, Matching, Indexing
and Heuristic functions - Hill Climbing - Depth first and Breath first, Constraints satisfaction -
Related algorithms, Measure of performance and analysis of search algorithms.

******

What is “Artificial Intelligence” ?

 Artificial Intelligence is a branch of Science which deals with helping machines to find
solutions for complex problems in a more human-like fashion.
 This generally involves borrowing characteristics from human intelligence, and applying
them as algorithms in a computer friendly way”.
 "Artificial Intelligence commonly abbreviated as AI, also known as machine intelligence.
 The process of developing algorithms that makes machines are to make seemingly
intelligent decisions or act as if possessing intelligence of a human scale”.

Conclusion about definition of AI

 AI is the science of making a machine think and act like an intelligent person.
 Term intelligent person is important to ensure that the actions and thinking that are being
imitated and incorporated in machine should be rational.

What is “Intelligence” ?
Intelligence
 Is the ability to acquire, understand and apply knowledge
 Intelligence embodies all of the knowledge and feats, both conscious and unconscious,
which we have acquired through study and experience
 Ability to think . . . ?
 Simulating the brain . . . ?
 Creativity . . . ?
 Ability to learn . . . ?
 Being good at maths . . . ?
 Playing good Chess . . . ?

Knowledge
 is the awareness and understanding of facts
AI
 Simple things turn out to be the hardest to automate:
o Recognising a face.
o Navigating a busy street.
o Understanding what someone says
 For more complex problems, computers have trouble in understanding situations, and
adapting to new situations
 AI aims to improve the machine behaviour to tackle such situations

→ ”Rational”: Performance-oriented as opposed to imitating humans.

Who does AI?

 Many disciplines contribute to goal of creating/modelling intelligent entities:
o Computer Science, Maths
o Psychology (human reasoning)
o Philosophy (nature of belief, rationality, etc)
o Linguistics (structure and meaning of language)
o Human Biology (how brain works)
 AI draws an idea from each discipline.

So, What Does Modern AI Do?

Typical AI Problems
 Intelligent entities (or “agents”) need to be able to do both “mundane” and “expert” tasks:
 Mundane tasks - consider going shopping:
o Planning a route, and sequence of shops to visit!
o Recognising (through vision) buses, people.
o Communicating (through natural language).
o Navigating on the street, and manipulating objects for purchase.
 Expert tasks are things like:
o Medical diagnosis
o Equipment repair
o Inventory control

History of AI
 1943 : McCulloch & Pitts Boolean circuit model of the brain
o A simplified model of how neurons work in the brain
 1949 : Hebbian learning
o Simple rule to enable neural networks to learn new behaviour
 1950 : Turing test
o “Computing machinery and intelligence”
 1956 : The term ‘Artificial Intelligence’ adopted
o AI researchers agreed on this term
 1958 LISP invented (List processing)
o John McCarthy developed the LISP language
o Currently Prolog is familiar
 1960’s Computational complexity
 1970’s Early works in Expert Systems
 1980’ AI becomes an industry
o First commercial ES, R1- to configure orders for DEL
 1990’s AI becomes Science
 1997 IBM’s Deep Blue beats Garry Kasparov at chess
o ‘ A new kind of intelligence’

Branches of AI
 Natural Language processing
o Machines to communicate in any language
 Knowledge representation
o To represent the facts in mathematical logic
 Pattern recognition
o Comparing a scene with given pattern
 Expert Systems
o Expert level competence in a particular problem domain
 Combinatorial Search
o Examining large number of possibilities
  Machine learning
o To adapt to new circumstances, detect and extrapolate patterns
 Computer Vision
o Three dimensional view of any object
 Genetic Algorithms
o Heuristic algorithms based on natural selection and genetic
 Robotics
 Neural networks
o Interconnected group of neurons

Applications of AI
 Autonomous control, target identification in Military
 Game playing Ex. chess
 Speech recognition Ex. UN airlines
 Understanding natural language- the domain of text
 Heuristic classification-
 Robotic pets

Future of AI?
 Copying human intelligence, and building human-like robots
 Currently, acting as surgeons, pilots, etc
 Eventually society will be left with machines working at every store and humans on
every beach
 Dr.Brooks prediction: by 2050 our lives will be populated with all kinds of intelligent
machines.

Acting Humanly: The Turing Test

The Turing Test, proposed by Alan Turing (Turing, 1950), was designed to provide a
satisfactory operational definition of intelligence. Turing defined intelligent behavior as the
ability to achieve human-level performance in all cognitive tasks, sufficient to fool an
interrogator. Roughly speaking, the test he proposed is that the computer should be interrogated
by a human via a teletype, and passes the test if the interrogator cannot tell if there is a computer
or a human at the other end.
Achievements in AI
 Deep Blue defeated the reigning world chess champion Garry Kasparov in 1997.
 During the 1991 Gulf War, US forces deployed an AI logistics planning and scheduling
program that involved up to 50,000 vehicles, cargo, and people.
 NASA's on-board autonomous planning program controlled the scheduling of operations
for a spacecraft.
 Proverb solves crossword puzzles better than most humans.

Problem Formulation
Definition of the state space.
Important: only the relevant aspects → level of abstraction!
Example: moving a robot?
Definition of the actions that can change the world state.
Definition of the action costs (cost incurred when executing this action). Assumption here: ≥ 0.
Definition of the goal: world states with certain properties.

Note: The problem formulation can have a serious influence on the difficulty of finding a
solution!

General Problem Solving

Write one program that can solve all problems.
X ∈ {algorithms} : for all Y ∈ {problems} : X solves Y
Write one program that can solve a large class of problems.

Problem Solving System

• Define the problem
• Analyze the problem
• Isolate & represent the task knowledge necessary to solve the problem
Choose the best problem- solving techniques & apply it to the problem

Romania example
• On holiday in Romania; currently in Arad.
• Flight leaves tomorrow from Bucharest
• Formulate goal:
– be in Bucharest
• Formulate problem:
– states: various cities
– actions: drive between cities
• Find solution:
– sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest
Problem formulation
A problem is defined by:
– An initial state, e.g. Arad
– Successor function S(X)= set of action-state pairs
• e.g. S(Arad)={<Arad  Zerind, Zerind>,…}
• intial state + successor function = state space
– Goal test, can be
• Explicit, e.g. x=‘at bucharest’
• Implicit, e.g. checkmate(x)
– Path cost (additive)
• e.g. sum of distances, number of actions executed, …
• c(x,a,y) is the step cost, assumed to be >= 0
A solution is a sequence of actions from initial to goal state.
Optimal solution has the lowest path cost.

A problem solving environment is one which decides what actions and states to consider in
completing a goal.
Algorithm - Three phases:
– Formulate the problem
– Search for a solution
– Execute the solution
State space : set of all possible states that the environment can be involved.
Search phase: searching the state space to perform a sequence of actions that can change the
environment to reach the goal state.
A solution is a sequence of actions leading from the initial state to a goal state.
Problem types
• Deterministic, fully observable  single-state problem
– Environment knows exactly which state it will be in; solution is a sequence
• Non-observable, known state space  sensor less problem
(conformant problem)
– Environment may not sense; solution is a sequence
• Nondeterministic and/or partially observable  contingency problem
– percepts provide new information about current state
– often interleave search, execution
• Unknown state space  exploration problem

Problem Types, Revisited

General Problem Solving, Pros and Cons

Powerful: In some applications, generality is absolutely necessary. (E.g. SAP)

Quick: Rapid prototyping: 10s lines of problem description vs. 1000s lines of C++ code. (E.g.
language generation)
Flexible: Adapt/maintain the description. (E.g. network security)
Intelligent: Determines automatically how to solve a complex problem effectively! (The ultimate
goal, no?!)
Efficiency loss: Without any domain-specific knowledge about Chess, you don’t beat Kasparov.
Example Problem Formulation

Goal: Cover the board completely with dominoes, each of which covers two neighbouring
squares. → Goal, state space, actions, search, . . .

Alternative Problem Formulation

Problem Formulation for the Vacuum Cleaner World

The Vacuum Cleaner Problem

• states? dirt and robot location

• Successor functions? Left, Right, Suck
• goal test? no dirt at all locations
• path cost? 1 per action

The Vacuum Cleaner World as a Partially Observable State Problem

Concepts Recap

Initial State: The state in which the agent is at the beginning.

State Space: Set of all possible states.
Actions: Description of possible actions. Available actions might be a function of the state.
Transition Model: Description of the outcome of an action (aka successor function).
Goal Test: Tests whether the state description matches a goal state.
Path: A sequence of actions leading from one state to another.
Path Costs: Cost function g over paths. Here (and usually) the sum of the costs of the actions
along the path.
Solution: Path from an initial to a goal state.
Search Costs: Time and storage requirements to find a solution.

General Game Playing

Write one program that can play all (2-player, 0-sum, . . . ) games.
Describe “game” in a declarative language (based on logics).
Variables (board position/cards), operators (moves), win/loss.
Chess.

Planning

Write one program that can solve all classical search problems.

What is a “classical search problem”?

Variables: x with finite number of possible values D(x);

state gives value to each variable (e.g., x = 1).
Actions: precondition (some variable values); effect (changes to some variable values);
e.g., action “increment x: pre x = 1, eff x := 2”.
Initial state: What the world is like now (e.g., x = 1).
Goal: What we want the world to change into (some variable values, e.g., x = 2).
Plan: Sequence of actions achieving the goal (e.g., “increment x”)
Natural Language Sentence Generation

Input: Tree-adjoining grammar, intended meaning.

Output: Sentence expressing that meaning.

Production Systems
Production Systems: Systems that generate (produce) rules (states) to reach a solution.
A production system commonly consists of following four basic components:
1. A set of rules of the form Ci → Ai where Ci refers to starting state and Ai represents
consequent state. Also Ci the condition part and Ai is the action part.
2. One or more knowledge databases that contain whatever information is relevant for the given
problem.
3. A control strategy that ascertains the order in which the rules must be applied to the available
database.
4. A rule applier which is the computational system that implements the control strategy and
applies the rules to reach to goal (if it is possible).

Elements of Production Systems

• Representation :
– State Space.
– Goal States.
– Initial States.
– Transformation Rules.
• Control/ Search Algorithms :
– Uninformed.
– “Heuristic”.
State Space
• State space search is a process used in which successive or states of an instance are
considered, with the goal of finding a goal state with a desired property.
• State space search often differs from traditional search (sequential, indexed sequential,
binary search etc) methods because the state space is implicit: the typical state space
graph is much too large to generate and store in memory. Instead, nodes are generated as
they are explored, and typically discarded thereafter.
• E.g. in a tic tack toe game, every move by a player forms a state space and the three
similar (O or X) consecutive (row, column or diagonal) symbols forms goal state.
• In a chess game also state space forms a set of moves by players.

State Space Search

The water jug problem: You are given two jugs, a 4-litre one and a 3-litre one. Neither has any
measuring marker on it. There is a pump that can be used to fill the jugs with water. How can
you get exactly 2-litre of water into 4-litre jug?
Let x and y be the amounts of water in 4-Lt and 3-Lt Jugs respectively. Then (x,y) refers to water
available at any time in 4-Lt and 3-Lt jugs. Also (x,y) → (x-d,y+dd) means drop some unknown
amount d of water from 4-Lt jug and add dd onto 3-Lt jug.
All possible production rules can be written as follows:
1. (x, y) → (4, y) fill it to 4; y remains unchanged if x < 4
if x<4
2. (x, y) → (x, 3) fill it to 3; x remains unchanged if y < 3
if y<3
3. (x, y) → (x − d, y) if there is some water in 4 Lt jug, drop some more water from it
if x > 0
4. (x, y) → (x, y − d) if there is some water in 3 Lt jug, drop some more water from it
if y > 0
5. (x, y) → (0, y) if there is some water in 4-Lt, empty it, y remains unchanged
if x > 0
6. (x, y) → (x, 0) if there is some water in 3-Lt, empty it, x remains unchanged
if y > 0
7. (x, y) → (4, y − (4 − x)) if there is some water in 3-Lt, the sum of water of 4-Lt and 3-Lt
if x + y ≥ 4, y > 0 jug is >=4, then fill water in 4-Lt jug to its capacity from 3-Lt jug
8. (x, y) → (x − (3 − y), 3) same as 7 with suitable change in x,y
if x + y ≥ 3, x > 0
9. (x, y) → (x + y, 0) if sum of water in both jugs <=4, then drop
if x + y ≤ 4, y > 0 whole water from 3-Lt into 4-Lt
10. (x, y) → (0, x + y) if sum of water in both jugs <=3, then drop
if x + y ≤ 3, x > 0 whole water from 4-Lt into 3-Lt
11. (0, 2) → (2, 0) Transfer 2-Lt from 3-Lt jug into empty 4-Lt jug
12. (2, y) → (0, y) Empty 2 Lt water onto ground from 4-Lt jug without disturbing
3Lt jug
Solution of Water Jug Problem
Obviously to solve water jug problem, we can perform following sequence of actions,
(0,0) (0,3) (3,0) (3,3) (4,2) (0,2) (2,0) by applying rules 2, 9, 2, 7, 5 and 9 with initial empty jugs.
Note: There are NO hard and fast rules to follow this sequence. In any state space search
problem, there can be numerous ways to solve, your approach can be different to solve a problem
and sequence of actions too.

Control Strategies
• How to decide which rule to apply next during searching a solution
Examples
– Breadth first search - Hill Climbing
– Depth first search - Best First search
– Back Tracking

Issues in the design of Search programs

• Direction in which to conduct the search (forward Vs Backward reasoning)
• How to select applicable rules (matching)
• How to represent each node of the search process

Examples of Problems
“Toy” Problems : “Real” Problems :
• Water jug. • Schedules.
• Cannibals. • Traveling Salesman.
• 8 – Queens. • Robot navigation.
• 8 Puzzle. • Language Analysis (Parsers, Grammars).
• VLSI design.
Problem-solving agent
Four general steps in problem solving:
– Goal formulation
• What are the successful world states
 – Problem formulation
• What actions and states to consider given the goal
– Search
• Determine the possible sequence of actions that lead to the states of known
values and then choosing the best sequence.
– Execute
• Given the solution perform the actions.

Example: Romania

• On holiday in Romania; currently in Arad

• Flight leaves tomorrow from Bucharest
• Formulate goal
• Be in Bucharest
• Formulate problem
• States: various cities
• Actions: drive between cities
• Find solution
• Sequence of cities; e.g. Arad, Sibiu, Fagaras, Bucharest, …

Search Through State-Space

Search strategies
• A strategy is defined by picking the order of node expansion.
• Problem-solving performance is measured in four ways:
– Completeness; Does it always find a solution if one exists?
– Optimality; Does it always find the least-cost solution?
– Time Complexity; Number of nodes generated/expanded?
– Space Complexity; Number of nodes stored in memory during search?
• Time and space complexity are measured in terms of problem difficulty defined by:
– b - maximum branching factor of the search tree
– d - depth of the least-cost solution
– m - maximum depth of the state space (may be )

Search strategies (Uninformed Vs Informed)

Uninformed search strategies
• (a.k.a. blind search) = use only information available in problem definition.
• When strategies can determine whether one non-goal state is better than another 
informed search.
Examples
• Breadth-first search (BFS)
• Depth-first search (DFS)
Informed search strategies
• Uses problem specific knowledge beyond the definition of the problem.
• Categories defined by expansion algorithm:
– Greedy Best-first search
– A* search
– Memory bounded heuristics search

BF-search :Breadth-first search

• Expand shallowest unexpanded node
• Implementation: fringe is a FIFO queue
BF-search; evaluation
• Completeness:
– YES (if b is finite)
• Time complexity:
– Total number of nodes generated:

• Space complexity:
– Idem if each node is retained in memory
– Memory requirements are a bigger problem than its execution time.
• Optimality:
– Does it always find the least-cost solution?
– In general YES unless actions have different cost.

DF-search: Depth-first search

• Expand deepest unexpanded node
• Implementation: fringe is a LIFO queue (=stack)
DF-search; evaluation
• Completeness;
– NO unless search space is finite.
• Time complexity;
• Space complexity;
– Backtracking search uses even less memory
• One successor instead of all b.
• Optimallity; No
– Same issues as completeness
Heuristics
• Where the exhaustive search is impractical, heuristic methods are used to speed up the
process of finding a satisfactory solution via mental shortcuts to ease the cognitive load of
making a decision. Examples of this method include using a rule of thumb, an educated
guess, an intuitive judgment, stereotyping, or common sense.
• In more precise terms, heuristics are strategies using readily accessible, though loosely
applicable, information to control problem solving in human beings and machines. Error
and trial is simplest form of heuristics. We can fit some variables in an algebraic equation
to solve it.

Local Search (LS) Algorithms

Generate and Test
1. Generate a possible solution. 2. Test to see if this is actually a solution. 3. Quit if a solution
has been found. Otherwise, return to step 1.
Features: 1.Acceptable for simple problems. 2. Inefficient for problems with large space.
LS Just operate on a single current state rather than multiple paths. It is generally moved only to
neighbors of that state. The paths followed by the search are not retained hence the method is not
systematic.
Benefits: 1. uses little memory –a constant amount for current state and some information 2. can
find reasonable solutions in large or infinite (continuous) state spaces, where systematic
algorithms are unsuitable.

Hill Climbing (HC)

Hill Climbing is an iterative algorithm that starts with an arbitrary solution to a problem, then
attempts to find a better solution by incrementally changing a single element of the solution. If
the change produces a better solution, an incremental change is made to the new solution,
repeating until no further improvements can be found. In simple hill climbing, the first closer
node is chosen, whereas in steepest ascent hill climbing all successors are compared and the
closest to the solution is chosen. Both forms fail if there is no closer node, which may happen if
there are local maxima in the search space which are not solutions. Steepest ascent hill climbing
is similar to best first search, which tries all possible extensions of the current path instead of
only one.

Stochastic hill climbing does not examine all neighbors before deciding how to move. Rather, it
selects a neighbor at random, and decides (based on the amount of improvement in that
neighbor) whether to move to that neighbor or to examine another.

Local Search: Hill Climbing Pseudo Algorithm

1.Pick initial state s
2.Pick t in neighbors(s) with the largest f(t)
3.IF f(t) <= f(s) THEN stop, return s
4. s = t. GOTO 2.
Features:
Not the most sophisticated algorithm in the world.
 Very greedy.
 Easily stuck.

Simple Hill Climbing

1. Evaluate the initial state.
2. Loop until a solution is found or there are no new operators left to be applied:
 Select and apply a new operator
 Evaluate the new state: if goal then quit
 Otherwise better than current state ← new current state
Drawbacks: it not try all possible new states!
Gradient Descent:
Considers all the moves from the current state.
Selects the best one as the next state.
Example of TSP

Note:
• Generate & Test - Exhaustive Random Search – British Museum Algorithm
• Variant of G&T based on Feedback
• In G& T, Test Procedure  Yes (or) No
• Here, “Test Function is Augmented with heuristic”
 • HC often used when a good heuristic Function is available & when no useful knowledge
available
• Example : To reach a place in Unfamiliar city without a map

• Consider all possible successors as “one step” from the current state on the landscape.
• At each iteration, go to
• The best successor (steepest ascent)
• Any uphill move (first choice)
• Any uphill move but steeper is more probable (stochastic)
• Simple hill climbing  Better node next (No of Move longer)
• Steepest Ascent Hill Climbing Best node next (time required to select a move longer)

Basic and Steepest –fail to find the solution in the following situation
• Local maxima = no uphill step - Allow “random restart”
• Plateau = all steps equal (flat or shoulder)
• Must move to equal state to make progress, but no indication of the correct
direction
• Ridge = narrow path of maxima, but might have to go down to go up (e.g. diagonal ridge
in 4-direction space)
• Backtrack to some earlier node
• Make a big jump in some direction
• Apply two or more rules before doing the test

Like climbing Everest in thick fog with amnesia

Informed (Heuristic) Search Strategies

• Informed Search – a strategy that uses problem-specific knowledge beyond the definition
of the problem itself
 • Best-First Search – an algorithm in which a node is selected for expansion based on an
evaluation function f(n)
o node with the lowest evaluation function is selected
o Choose the node that appears to be the best
• g(n) = cost from the initial state to the current state n
• h(n) = estimated cost of the cheapest path from node n to a goal node
• f(n) = evaluation function to select a node for expansion (usually the lowest cost node)
Example: in route planning the estimate of the cost of the cheapest path might be the straight
line distance between two cities

Greedy Best-First Search

Let h(n) be the heuristic function in a graph. In simple case, let it be the straight line distance
SLD from a node to destination.
1. Start from source node S, determine all nodes outward from S and queue them.
2. Examine a node from queue (as generated in 1) .
 If this node is desired destination node, stop and return success.
 Evaluate h(n) of this node. The node with optimal h(n) gives the next successor, term this
node as S.
3. Repeat steps 1 and 2.

Note:
• Expand the node that is closest to the goal assuming it will lead to a solution quickly –
a.k.a. “Greedy Search”
o f(n) = h(n)
• Example: consider the straight-line distance heuristic hSLD
o Expand the node that appears to be closest to the goal
Example: hSLD(In(Arid)) = 366

A* Search Algorithm
Let h(n) be the heuristic function in a graph. In simple case, let it be the straight line distance
SLD from a node to destination. Let g(n) be the function depending on the distance from source
to current node. Thus f(n) = g(n) + h(n)
1.Start from source node S, determine all nodes outward from S and queue them.
2.Examine a node from queue (as generated in 1) .
 If this node is desired destination node, stop and return success.
 Evaluate f(n) at this node. The node with optimal f(n) gives thenext successor, term this
node as S.
3. Repeat steps 1 and 2.
Time = O(log f(n)) where h(n) is the actual distance travelled from n to goal.
A* (A star) is the most widely known form of Best-First search
– It evaluates nodes by combining g(n) and h(n)
– f(n) = g(n) + h(n)
where
 • g(n) = cost so far to reach n
• h(n) = estimated cost to goal from n
• f(n) = estimated total cost of path through n
When h(n) = actual cost to goal
• Only nodes in the correct path are expanded
• Optimal solution is found
When h(n) < actual cost to goal
• Additional nodes are expanded
• Optimal solution is found
When h(n) > actual cost to goal
• Optimal solution can be overlooked
A* is optimal if it uses an admissible heuristic
– h(n) <= h*(n) the true cost from node n
– if h(n) never overestimates the cost to reach the goal
Example
– hSLD never overestimates the actual road distance
Searching And-Or graphs
 The DFS and BFS strategies for OR trees and graphs can be adapted for And-Or trees.
 The main difference lies in the way termination conditions are determined, since all goals
following an And node must be realized, whereas a single goal node following an Or
node will do.
 A more general optimal strategy is AO* (O for ordered) algorithm.
 As in the case of the A* algorithm, we use the open list to hold nodes that have been
generated but not expanded and the closed list to hold nodes that have been expanded.
 The algorithm is a variation of the original given by Nilsson.
 It requires that nodes traversed in the tree be labeled as solved or unsolved in the solution
process to account for And node solutions which require solutions to all successors
nodes.
 A solution is found when the start node is labeled as solved.

The AO* algorithm

Step 1: Place the start node s on open
Step 2: Using the search tree constructed thus far, compute the most promising solution tree T0
Step 3: Select a node n that is both on open and a part of T0. Remove n from open and place it on
closed
Step 4: If n is a terminal goal node, label n as solved. If the solution of n results in any of n’s
ancestors being solved, label all the ancestors as solved. If the start node s is solved, exit
with success where T0 is the solution tree. Remove from open all nodes with a solved
ancestor
Step 5: If n is not a solvable node, label n as unsolvable. If the start node is labeled as
unsolvable, exit with failure. If any of n’s ancestors become unsolvable because n is,
label them unsolvable as well. Remove from open all nodes with unsolvable ancestors.
Step 6: Otherwise, expand node n generating all of its successors. For each such successor node
that contains more than one subproblem, generate their successors to give individual
subproblems. Attach to each newly generated node a back pointer to its predecessor.
Compute the cost estimate h* for each newly generated node and place all such nodes
thst do not yet have descendents on open. Next recomputed the values oh h* at n and
each ancestors of n
Step 7: Return to step 2

It can be shown that AO* will always find a minimum-cost solution tree if one exists, provided
only that h*(n) ≤ h(n), and all arc costs are positive. Like A*, the efficiency depends on how
closely h* approximates h

Constraint Satisfaction Problem (CSP)

Design must be created within fixed limits on time, cost and materials.
Idea: Solve a problem by stating constraints on acceptable solutions

Constraint programming methodology

Ex. Place numbers 1 through 8 on nodes, where each number appears exactly once and no
connected nodes have consecutive numbers
A Solution:

A CSP is defined by:

– a set of variables {x1, …, xn}
– a set of values for each variable dom(x1), …, dom(xn)
– a set of constraints {C1, …, Cm}
A solution to a CSP is a complete assignment to all the variables that satisfies the constraints
– state is defined by variables Xi with values from domain Di
– goal test is a set of constraints specifying allowable combinations of values for subsets of
variables
For a given CSP
– Determine whether it has a solution or not
– Find one/ all solution
– Find an optimal solution, given some cost function

Example: Map-Coloring
– Variables WA, NT, Q, NSW, V, SA, T
– Domains Di = {red,green,blue}
 – Constraints: adjacent regions must have different colors
e.g., WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red),
(green,blue),(blue,red),(blue,green)}

Solutions are complete and consistent assignments

e.g., WA = red, NT = green, Q = red, NSW = green,V = red,SA = blue,T = green

Constraint graph
• Binary CSP: each constraint relates two variables
• Constraint graph: nodes are variables, arcs are constraints

Varieties of constraints
• Unary constraints involve a single variable,
– e.g., SA ≠ green
• Binary constraints involve pairs of variables,
– e.g., SA ≠ WA
• Higher-order constraints involve 3 or more variables,
– e.g., cryptarithmetic column constraints
Real-world CSPs
• Assignment problems
– e.g., who teaches what class
• Timetabling problems
– e.g., which class is offered when and where?
• Transportation scheduling
• Factory scheduling
Notice that many real-world problems involve real-valued variables.

Standard search formulation (incremental)

States are defined by the values assigned so far
• Initial state: the empty assignment { }
• Successor function: assign a value to an unassigned variable that does not conflict with
current assignment
 fail if no legal assignments
• Goal test: the current assignment is complete

Constraint propagation
• A backtracking search is a depth-first traversal of a search tree
• A generic scheme to maintain a level of local consistency in a backtracking search is to
perform constraint propagation at each node in the search tree
– if any domain of a variable becomes empty, inconsistent so backtrack

Backtracking Example

Heuristics for backtracking algorithms

• Variable ordering (very important)
– what variable to branch on next
– Which variable should be assigned next?
– Most Constrained Variable (most likely to fail first)
 • Value ordering (can be important)
– given a choice of variable, what order to try values
– In what order should its values be tried?
– Least Constraining Value (most likely to succeed)

Most constrained variable

• Most constrained variable:
– choose the variable with the fewest legal values

• a.k.a. minimum remaining values (MRV) heuristic

• A good idea is to use it as a tie-breaker among most constrained variables

– Most constraining variable i.e. choose the variable with the most constraints on
remaining variables (Degree heuristics)

Value Ordering Least Constraining Value

Choose first the values - most likely to succeed - choose the least constraining value:
o the one that rules out the fewest values in the remaining variables
Forward checking
The main ideas of forward checking are:
– keep track of remaining legal values for unassigned variables
– terminate search when any variable has no legal values
– propagates information from assigned to unassigned variables, but doesn't provide
early detection for all failures
– constraint propagation algorithms repeatedly enforce constraints locally.

Application areas
• scheduling • networks
• logistics • configuration
• planning • assembly line sequencing
• supply chain • cellular frequency assignment
management • airport counter and gate allocation
• rostering • airline crew scheduling
• timetabling • optimize placement of
• vehicle routing transmitters for wireless
• bioinformatics • …

Problem Characteristics
Heuristic search is a very general method applicable to a large class of problems. It encompasses
a variety of specific techniques, each of which is particularly effective for a small class of
problems. In order to choose the most appropriate method (or combination methods) for a
particular problem, it is necessary to analyze the problem along several key dimensions:
 Is the problem decomposable into a set of (nearly) independent smaller or easier
subproblems?
 Can solution steps be ignored or at least undone if they prove unwise?
 Is the problem’s universe predictable?
 Is a good solution to the problem obvious without comparison to all other possible
solutions?
 Is a desired solution a state of the world or a path to a state?
 Is a large amount of knowledge absolutely requires to solve the problem, or is knowledge
important only to constrain the search?
 Can a computer that is simply given the problem return the solution, or will the solution
of the problem require interaction between the computer and a person?

Is the problem Decomposable?

Suppose we want to solve the problem of computing the expression
∫(𝑥 2 + 3𝑥 + 𝑠𝑖𝑛2 𝑥. 𝑐𝑜𝑠 2 𝑥)𝑑𝑥
We can solve this problem by breaking it down into three smaller problems, each of which we
can then solve by using a small collection of specific rule. The following figure shows the
problem tree that will be generated by the process of problem decomposition as it can be
exploited by a simple recursive integration problem that works as follows: At each step it checks
to see whether the problem it is working on is immediately solvable. If so, then the answer is
returned directly.

A decomposable Problem
If the problem is not easily solvable, the integrator checks to see whether it can decompose the
problem into smaller problems. If it can, it creates those problems and calls itself recursively on
them. Using this technique of problem decomposition, we can often solve very large problems
easily.

Production System Characteristics

Production systems are good way to describe the operations that can be performed in a search for
a solution to a problem. There are four categories of production systems.
Monotonic Nonmonotonic
Partially
Theorem Proving Robot Navigation
Commutative
Not Partially
Chemical Synthesis Bridge
Commutative

- A monotonic production system is a production system in which the application of a rule

never prevents the later application of another rule that could also have been applied at
the time the first rule was selected.
- A nonmonotonic production system is one in which this is not true.
 - A partially commutative production system is a production system with the property that
if the application of a particular sequence of rules transforms state x into state y, then any
permutation of those rules that is allowable (i.e. each rule precondition’s are satisfies
when it is applied) also transforms state x into state y.
- A commutative production system is a production system that is both monotonic and
partially commutative.
The significance of these categories of production systems lies in the relationship between
the categories and appropriate implementation strategies.

Matching and Indexing

Matching is the process of comparing two or more structures to discover their likenesses or
differences. The structures may represent a wide range of objects including physical entities,
words or phrases in some language, complete classes of things, general concepts, relations
between complex entities, and the like.
Matching is used in a variety of programs for different reasons. It may serve to control the
sequence of operations, to identify or classify objects, to determine the best of a number of
different alternatives, or to retrieve items from a database. It is an essential operation such
diverse programs as speech recognition, natural language understanding, vision, learning,
automated reasoning, planning, automatic programming, and expert systems, as well as many
others.
Measure for Matching
- The problem of comparing structures without the use of pattern matching variables. This
requires consideration of measures used to determine the likeness or similarity between
two or more structures
- The similarity between two structures is a measure of the degree of association or
likeness between the object’s attributes and other characteristics parts.
- If the describing variables are quantitative, a distance metric is used to measure the
Proximity.

How to extract from the entire collection of rules that can be applied at a given point? =>
Matching between current state and the precondition of the rules.
Indexing – A large number of rules => too slow to find a rule
• Use the current state as an index into rules and select the matching ones immediately
o Only works when preconditions of rules match exact board configuration
o It’s not always obvious whether a rule’s preconditions are satisfied by a particular
state.
• There’s a trade-off between the ease of writing rules (high-level descriptions) and the
simplicity of the matching process.

Indexed Organization
- While organization by content can help to reduce block transfers, an indexed organization
scheme can greatly reduce the time to determine the storage location of an item
- Indexing is accomplished by organizing the information in some way for easy access
- One way to index is by segregating knowledge into two or more groups and storing the
locations of the knowledge for each group in a smaller index file
- To build an indexed file, knowledge stored as units is first arranged sequentially by some
key value
- The key can be any chosen fields that uniquely identify the record
- A second file containing indices for the record locations is created while the sequential
knowledge file is being loaded
- Each physical block in this main file results in one entry in the index file
- The index file entries are pairs of record key values and block addresses
- The key value is the key of the first record stored in the corresponding block
- To retrieve an item of knowledge from the main file, the index file is searched to find the
desired record key and obtain the corresponding block address
- The block is then accessed using this address. Items within the block are then searched
sequentially for the desired record
- An indexed file contains a list of the entry pairs (k,b) where the values k are the keys of
the first record in each block whose starting address is b.