MODELING and Analysis

Mathematical Programming and Search

Approaches
Lecture 12
Modeling and Analysis Topics
 Optimization via mathematical programming
 Search Approaches
 Heuristic programming

Mathematical Programming
 Application of mathematical and computer programming techniques to the
construction of deterministic models, principally for business and economics.
 Optimization
 Linear Programming (LP) is the best known technique in a family of
optimization tools called mathematical Programming.
 Linear programming (LP) used extensively in DSS
 Tools for solving managerial problems
 Decision-maker must allocate scarce resources (distribution of the machine
time for example) amongst competing activities (various products)
 Optimization of specific goals
 Linear programming
 Consists of decision variables, objective function and coefficients,
uncontrollable variables (constraints), capacities, input and output
coefficients.
 Decision variables: whose values are unknown and are searched for.
 An objective function: a linear mathematical function that relate the
decision variable to the goal, measures goal attainment and is to be
optimized.
 Objective function coefficients: unit profit or cost coefficient
indicating the contribution to the objective of one unit of a decision
variable.
 Constrains: expressed in the form of linear inequalities or equalities
that limit resources and/or requirement, these relate the variables
through linear relationships.
 Capacities: which describe the upper and sometimes lower limits of
constrains and variables.
 Input and output coefficients: which indicate resource utilization for a
decision variables.
Search Approaches
 When problem-solving, the choice phase involves a search for an appropriate
course of action that can solve the problem.
 There are several search approaches, depending on the criteria (or criterion)
of choice and the type of modeling approach used.

- 101 -
Analytical Techniques
 Analytical techniques (algorithms) for structured problems (See next page)
1. Generate a step-by-step search process
2. Obtains an optimal solution
3. Solutions are generated and tested for possible improvements
4. Improvement is made whenever possible.
5. The new solution is subject to an improvement test based on the principle of
choice (objective value found).
6. The process continues until no further improvement is possible.
Blind Search
 In conducting a search, a description of a desired solution may be given. This
is called a goal. A set of possible steps leading from initial conditions to the
goal is called the search steps.
 Problem solving is done by searching through the space of possible solutions.
The first of these search methods is blind search. The second is heuristic
search.

 Blind search techniques are arbitrary search approaches that are not
guided.
 There are two types of blind search: a complete enumeration search,
for which all the alternatives are considered and therefore an optimal
solution is discovered.
 Incomplete, partial search, which continues until a good-enough
solution is found. The latter is form of suboptimization.
 There are practical limits on the amount of time and computer
storage available for blind searches.
 In principle, blind search methods can eventually find an optimal
solution in most search situations, and in some situations the scope of
the search can be limited;

- 101 -
  However, the method is not practical for solving very large problems
because too many solutions must be examined before an optimal
solution is found.
Problem Solving as State Space Searching
Problem solving can be described in terms of searching across its state space.
1. A state is a set of values to the elements of a given problem.
2. To start searching you need to create at least one initial state. (e.g.,
initial chessboard, current positions of objects in world, current
location, middle of game, unsolved puzzle)
3. You have operations which may be applied to an old state to generate
a new one. (e.g. chess move, robot action, simple change in location).
4. A sequence of operations forms a path through the state space.
5. There is some notion of a goal state, which you are trying to reach.
(e.g., solved puzzle, game won, shopping done, winning chess
position, target location)
 Sometimes the goal is to reach and find a goal state, other times it is
the path which is important (i.e. we want to know the sequence of
operations required to reach a goal state).
 When we have represented the problem in this way, we can apply an
algorithm to search across the range of all possible states.
1. Blind Search
 Search techniques are used in AI to find a sequence of steps that will
get us from some initial state to some goal state(s).
 You can either search backwards, from the goal state, or forwards,
from the initial state. And whichever direction you choose you can use
various search algorithms to do the search.

 Search Algorithms
 Breadth First search (FIFO)
 Depth First search (LIFO)
2. Tree Searches
 In artificial intelligence, we use trees to represent a lot of things, from
sentence structures, equations and even game states

- 101 -
  For some problems, any goal node is acceptable (N or J); for other
problems, you want a minimum-depth goal node, that is, a goal node
nearest the root (only J)
3. Problem Space Representation of Problem
 In the problem space representation of a problem
1. Nodes represent a problem to be solved.
2. Links/Arcs represent alternate decompositions of the problem.
Tree Searching Algorithms

 Nodes are explored in the order A B D E H L M N I O P C F G J K Q

N will be found before J
The Algorithm of Depth-first
list CLOSED, OPEN = [StartNode]
Mainloop:
If OPEN = NULL, Stop // No goal!
Set N = OPEN[0]
if N is a goal node, stop. // Solution!
Remove N from OPEN
f N is in CLOSED goto Mainloop
Add N to CLOSED

- 101 -
 Prepend any children of N to OPEN
Goto Mainloop
 The Algorithm of Depth-first : Explanation
 The OPEN list contains nodes that have not been looked at, and
CLOSED contains all the nodes that have been explored. So, initially
CLOSED is empty and OPEN is initialized to OPEN.
 Now, in the first iteration of the loop, we set N to the first element of
OPEN (in this case StartNode). StartNode is not a goal node, so we
continue.
 We remove N from the OPEN list, and check whether it is in CLOSED.
It won't be, so we add it. We then check for any children of N, and put
those on the OPEN list to be explored.

The Algorithm of Breadth-first

list CLOSED, OPEN = [StartNode]
Mainloop:
If OPEN = NULL, Stop // No goal!
Set N = OPEN[0]
if N is a goal node, stop. // Solution!
Remove N from OPEN

- 101 -
 If N is in CLOSED goto Mainloop
Add N to CLOSED
Append any children of N to OPEN
Goto Mainloop
Remark
Notice where the depth-first and breadth-first algorithms differ though, child
items are prepended on the list in depth-first, but appended in breadth-first.
 Depth- vs. breadth-first searching
 When a breadth-first search succeeds, it finds a minimum-depth
(nearest the root) goal node
 When a depth-first search succeeds, the found goal node is not
necessarily minimum depth
o For the recursive solution we have a path to the node.
 For a large tree, breadth-first search memory requirements may be
excessive. DFS uses less memory – it only has to record the current
path that it is on, not an entire “layer” of states.
 For a large tree, a depth-first search may take an excessively long time
to find even a very nearby goal node
 Breadth first search and depth first search. These are both systematic,
exhaustive search techniques that will eventually try all the nodes in
the search space (if it’s finite!).
 The appropriate direction of search and the appropriate algorithm
depend on the nature of the problem you are trying to solve, and in
particular the properties of the search space.
 Search Space
 The set of all possible states reachable from the initial state defines
the search space.
 We can represent the search space as a tree.
 We refer to nodes connected to and “under” a node in the tree as
“successor nodes”.
 Recursion vs. Iteration
 Recursion in computer science is a method where the solution to a

problem depends on solutions to smaller instances of the same

problem (as opposed to iteration). The approach can be applied to

many types of problems, and recursion is one of the central ideas

of computer science. The power of recursion evidently lies in the

possibility of defining an infinite set of objects by a finite

statement. In the same manner, an infinite number of

- 101 -
 computations can be described by a finite recursive program, even

if this program contains no explicit repetitions.

 Iteration in computing is the technique marking out of a block of

statements within a computer program for a defined number of
repetitions. That block of statements is said to be iterated; a
computer scientist might also refer to that block of statements as
“iteration".

Heuristic Search
 For many applications, it is possible to find rules to guide the search
process and reduce the amount of necessary computations. This is
done by heuristic search.
 Heuristics (from the Greek word for discovery) are decision rules
governing how a problem should be solved.

- 101 -
  Heuristic searches or programming are step-by-step procedures like
(algorithms) that are repeated until a satisfactory solution is found
(unlike algorithms).
 Much faster and cheaper than a blind search and the solutions can be
very close to the best ones.
 Heuristic programming is the approach of using heuristics to arrive at
feasible and “good enough” solutions to some complex problems.
Good enough is usually in the range of 90-99.9 percent of the
objective value of an optimal solution.

Review Questions
1. Define the term mathematical programming.
2. What are the components of linear programming?
3. What are the main steps of analytical techniques?
4. What is the difference between complete enumeration search and
partial search?
5. Why is complete enumeration search not practical for solving very
large problems?
6.

In what order will BFS and DFS consider these states?

7. Explain the Problem Solving as state space searching.
8. Complete:
a. Blind Search is used in AI to find a sequence of steps that
will get us from some ------------------- to some ------------------.
b. In blind search you can either search ----------------------, from
the goal state, or ---------------------------, from the initial state.
And whichever direction you choose you can use various ----
----------------------- to do the search.
9. Write the algorithm of Depth-First Searching.
10.Name two differences between DFS and BFS.
11.What is meant by the term successor nodes?

- 101 -
12. Complete: Heuristic searches or programming are --------------------
procedures like (algorithms) that are repeated until a -------------- --
-------------- is found (unlike algorithms).
13.T/F Question
1) Linear Programming (LP) is the best known technique in a family of
optimization tools called mathematical Programming.
2) Mathematical Programming is an application of mathematical and
computer programming techniques to the construction of stochastic
models, principally for business and economics.
3) Linear programming (LP) used extensively in DSS.
4) In linear programming decision-maker must allocate scarce resources
amongst competing programs.
5) Linear programming consists of decision variables, subjective function
and coefficients, uncontrollable variables (constraints), capacities, input
and output coefficients.
6) When problem-solving, the implementation phase involves a search for
an appropriate course of action that can solve the problem.
7) There are several search approaches, depending on the criteria (or
criterion) of choice and the type of modeling approach used.
8) Analytical techniques (algorithms) used for unstructured problems.
9) In analytical techniques the new solution is subject to an improvement
test based on the principle of choice.
10) In analytical techniques the process continues until further
improvement is possible.
11) In conducting a search, a description of a desired solution may be
given. This is called a goal.
12) A set of possible steps leading from initial conditions to the goal is
called the search loop.
13) Blind search techniques are arbitrary search approaches that are not
guided.
14) There are three types of blind search.
15) Complete enumeration search is a form of suboptimization.
16) There are practical limits on the amount of time and computer
storage available for algorithmic searches.
17) Blind search is not practical for solving very large problems because
too many solutions must be examined before an optimal solution is
found.
18) In blind search, a goal is a set of values to the elements of a given
problem.
19) In problem solving as state space searching, sometimes the goal is to
reach and find a goal state, other times it is the path which is
important.
20) Blind search is search techniques are used in BI to find a sequence of
steps that will get us from some initial state to some goal state(s).

- 101 -
21) In blind search, you can either search backwards, from the goal state,
or forwards, from the initial state. And whichever direction you choose
you can use various search agents to do the search.
22) In blind search algorithm Breadth First search means (FIFO) and Depth
First search means (LIFO).
23) In artificial intelligence, we use trees to represent a lot of things, from
accounting problems to sentence structures, equations and even game
states.
24) In problem space representation of problem, Nodes represent an
alternate decomposition of the problem.
25) In problem space representation of problem, links or arks represent
the problem to be solved.
26) A depth-first search (DFS) explores a path all the way to a leaf before
backtracking and exploring another path.
27) A breadth-first search (BFS) explores nodes nearest the root before
exploring nodes further away
28) Where the depth-first and breadth-first algorithms differ though,
child items are appended on the list in depth-first, but prepended in
breadth-first.
29) When a depth-first search succeeds, it finds a minimum-depth
(nearest the root) goal node.
30) When a depth-first search succeeds, the found goal node is not
necessarily minimum depth.
31) For a large tree, BFS memory requirements may be excessive.
32) DFS uses less memory because it only has to record an entire “layer”
of states.
33) Iteration in computer science is a method where the solution to a
problem depends on solutions to smaller instances of the same
problem.
34) Iteration in computing is the technique marking out of a block of
statements within a computer program for a defined number of
reputations.
35) For a large tree, a DFS may take an excessively long time to find even
a very nearby goal node
36) BFS and DFS. These are both systematic, partial search techniques
that will eventually try all the nodes in the search space (if it’s finite!).
37) We refer to nodes connected to and “under” a node in the tree as
“success nodes”
38) For many applications, it is possible to find rules to guide the search
process and reduce the amount of necessary computations. This is done
by heuristic search.
39) Heuristic searches or programming are step-by-step procedures like
(algorithms) that are repeated until a satisfactory solution is found (like
algorithms).
40) Heuristic search is much faster and cheaper than a blind search.
41) Heuristic programming is the approach of using heuristics to arrive at
feasible and “good enough” solutions to some simple problems.

- 110 -