Goal Programming
Goal Programming
Student name:
Student number:
J.M. de Rue
1163493
Supervisor:
S. Bhulai
Preface
This BMI paper is the last phase of the study Business Mathematics and
Informatics at the VU University in Amsterdam. This paper has the goal to use
the knowledge gained from the study to address a problem, formulate this in a
clear way, and to provide a solution.
Operations Research is a widely applied mathematical technique. It is used for
many different business problems. However, Operations Research has its
shortcomings. In this paper, I will give an example of these shortcomings and
describe several approaches to solve these shortcomings. I will apply one of
these approaches to an example in which the major shortcoming is solved by
recourse models.
I would hereby like to thank Sandjai Bhulai. Without his time, help, and
catching enthusiasm, this paper would not have been possible. Furthermore, I
would like to thank Maarten van der Vlerk for sending me his lecture notes on
Stochastic Programming. These lecture notes enabled me to learn a great deal on
the subject of Stochastic Programming.
Jaap de Rue
Haarlem, August 2007
Page 1
Page 2
Management summary
Linear programming is one of the many specializations of Operations Research
and it is one of the most applied mathematical techniques. The many different
algorithms of linear programming can be used for the optimization of a wide
range of problems. However, there are some drawbacks to using these
algorithms. The major drawback is that all coefficients (i.e., the quantity of
resources, time, and distance) need to be deterministic and known in advance.
However, in many practical situations one can not be certain of the true value of
these coefficients. These shortcomings can be solved by stochastic linear
programming, which uses random variables to approach the values of these
coefficients.
This paper describes several approaches that can be applied when the model
contains uncertainty on some of the parameters. In particular, recourse structures
are seen as the most important class within Stochastic Programming. This paper
therefore concentrates on the application of recourse structures. An example is
given of a model which contains uncertainty on some of the parameters. Adding
a recourse structure to this model shows that this leads to a model in which the
risk is explicitly taken care of.
Operations Research is already one of the mostly applied mathematical
techniques. Combining Stochastics and Mathematical Programming increases
the applicability enormously and may thus account for an even greater number
of application areas.
Page 3
Page 4
Table of contents
PREFACE............................................................................................................ 1
MANAGEMENT SUMMARY .......................................................................... 3
TABLE OF CONTENTS ................................................................................... 5
1
INTRODUCTION...................................................................................... 7
2.2.1
2.2.2
2.3
3
MODELLING ....................................................................................... 23
EXTENDING THE MODEL ..................................................................... 27
CONCLUSIONS ...................................................................................... 31
BIBLIOGRAPHY .................................................................................... 33
Page 5
Page 6
1 Introduction
Page 7
Page 8
2 Operations Research
The field of operations research arose during World War II, during which the
team of Patrick Blackett made numerous crucial analyses to aid the war effort.
An example of one of these analyses is the number of boats in a convoy. The
idea of ships travelling in convoys was introduced by Britain, this way warships
could be used to accompany and protect merchant ships. Since these convoys
travel at the speed of the slowest member, the question arose whether convoys
should be large or small. Since a small convoy will be faster, it will be harder to
detect by a U-boat. Large convoys may be slower, but they can deploy more
warships against such an attack.
Another example of the work of Blacketts team is given by a report they
analysed. This report was carried out by RAF Bomber Command, for which
they inspected all bombers returning from bombing raids in Germany. This
report contained a note of all the damages on the plane inflicted by Germanys
air defence. The RAF concluded from this report that the bombers should be
armoured more heavily on the damaged areas. However, they made the
remarkable conclusion that the armour should be placed on the areas that were
not damaged. Since only the planes that came back from Germany were
included in the survey, his team reasoned that the untouched areas were
apparently the most vital areas.
After the war, people realized that the planning methods that were developed for
the military could also be used in the profit- and non-profit sector. The study
after operations research flourished after the war and not long after that the
famous simplex-method was invented by George Bernard Dantzig that still is of
great practical use nowadays.
Dantzigs original example of finding the best assignment of 70 people to 70
jobs still explains its success. The computing power required to scan all the
permutations to select the best assignment is vast and impossible. He observed
that it takes only a moment to find the optimum solution using the simplex
method, which is effectively observing that a solution exists in the extreme
points of the polygon described by the equations formed from the given
constraints.
Two other founders are John van Neumann, who developed the theory of duality
in the same year, and Leonid Kantorovich, a Russian mathematician who used
similar techniques in economics before Dantzig and won the Nobel Prize in
1975 in economics. The linear programming problem was first shown to be
solvable in polynomial time by Leonid Khachiyan in 1979, but a larger major
Page 9
theoretical and practical breakthrough in the field came in 1984 when Narendra
Karmakar introduced a new interior point method for solving linear
programming problems.
The possibilities of applying linear programming only became bigger and bigger
after the introduction of the personal computer and nowadays it is one of the
most applied mathematical techniques.
cT x
Ax = b
x 0,
Page 10
maximize the income. Here one could think of, among others, inventory
management, resource allocation for human and machine resources, and
planning. Next to that, many of the central concepts of optimization theory are
based on ideas from linear programming. It can thus be said that linear
programming is an important field of optimization.
2.2 Techniques
This paragraph will give some of the most frequently used techniques to solve
linear programming problems.
2.2.1
Simplex method
cT x
Ax = b
x 0,
Maximize
Subject to
(1)
with n variables and m restrictions and for which holds that m<n. In the simplex
algorithm (an iterative procedure) bases {B} are formed with corresponding
basis solutions {x}. A basis B is a part of the matrix A consisting of m linear
independent columns of the matrix A. The remaining columns of A form the
matrix R. The basis solution x for j = 1,2,...., n is then defined by:
x j = B 1b j ,
for j B ,
xj = 0,
for j R .
Page 11
c = c BT B 1 A
c,
z ( x ) = c BT B 1b c j x j .
jR
The linear programming problem (1) is called the primary problem and the dual
problem is defined as follows:
bT y
AT y c.
Minimize
Subject to
In the primary problem it is taken care of that for every iteration it holds that x is
allowed for the primary problem, i.e., x j 0 j and a new iteration is started
(
(
) : (B
)
B 1b
l = arg min 1
jB
B a s
as
> 0 .
0.
This test ensures that for all new basis variables it holds that x new
j
In the dual problem it is taken care of that in every iteration x is feasible for the
dual problem, i.e., c j 0 j and a new iteration is started until x is also
feasible for the primary problem, i.e., x j 0 j .
If j B : x j < 0 , a basis column al B with a negative value
(x = (B b)
1
< 0 is chosen to leave the basis. A ratio-test has to point out which
column a s enters the basis B, with:
l
Page 12
cj
1
s = arg min 1
: (B a j )l < 0 .
jR
(B a j )l
This test ensures that for all new reduced costs it holds that c new
0 , where c new
j
j
is defined by:
new
j
= c j
(B
(B
1
1
)
c
a )
aj
for j = 1,2,..., n .
s l
2.2.2
Column generation
Page 13
Maximize
Subject to
cT x
Ax = b
x 0,
with n variables and m restrictions for which holds that m<n. Now, let
N 1 {1,...., n}; k = 1 ; and the matrix D = ( Ai : i N k ) be a part of the matrix A
for which holds that rank(D)=m, i.e., an optimal solution is possible. The vector
c N k and x N k are formed in a similar way. Then, the limited linear programming
problem is defined by:
Maximize
Subject to
c TN k x N k
Dx N k = b
x N k 0.
This limited linear programming problem is solved with the simplex method. If
an optimal solution is found for this limited linear programming problem, this
solution is also optimal for the original problem if all reduced costs are nonnegative, which are defined as follows:
c j = y T A j c j ,
for j = 1,...., n .
If the solution is not optimal for the original linear programming problem, the
matrix D is changed, i.e., N k is replaced with N k +1 ; k = k + 1 and the whole
process is repeated.
2.3 Drawbacks
It is obvious that linear programming is very popular, since so many practical
problems can be modelled as a linear program, or at least an approximation can
be made. When a problem is modelled as a linear program, powerful software is
available to solve these problems. However, the linear programming approach
does have some drawbacks.
One of these drawbacks is that numerical values have to be provided for each of
the coefficients. But, in most situations one can not be completely certain about
the true values of these coefficients. There are some approaches to take this
uncertainty into account when building the model. For instance, one could use
Page 14
ranges or solve the linear programming model for every possible realization of
the coefficients.
However, if the optimal solution depends heavily on the value of, some of, these
coefficients, this uncertainty will have to be taken into account in a more
fundamental way. Stochastic programming is characterized by interpreting the
uncertain coefficients as realizations of random variables. Hereby the framework
for the quantitative analysis of uncertainty provided by probability theory is
used.
Page 15
Page 16
3 Stochastic Programming
Stochastic programming is a framework for modelling optimization problems
that involve uncertainty (stoprog.org). Many practical problems can be modelled
with linear programming, in case these problems include known parameters.
But, real-world problems almost invariably include some parameters of which
the true values are unknown. Stochastic programming interprets these unknown
parameters by taking advantage of the fact that probability distributions
governing the data are known or can be estimated.
When a linear programming problem has been completely specified with the
exception of some parameters, which are assumed to be random variables with a
known joint distribution, two types of models are considered in stochastic
programming. These two types of models are the wait-and-see model and the
here-and-now model.
In the first case, the model builder is assumed to be able to wait for the
realization of the random parameters. In the second case, the model builder will
have to decide upon the value of these parameters without knowledge of the
realizations. So, in the here-and-now model some parameters are undetermined,
which makes the feasibility and/or optimality of the solution useless. Because of
this, additional specification is needed to deal with questions of risk and risk
aversion.
The following paragraph will describe some approaches to deal with these
questions of risk and risk aversion.
3.1 Approaches
Given is the following linear programming problem with random parameters in
the constraints:
Minimize
Subject to
cT x
Ax ~ b
Tx ~ h
x 0.
(2),
Page 17
Then assume that the real value of (T , h ) is not known, i.e., it is not known
which instance of the model occurs. Furthermore, assume that the uncertainty is
expressed by a probability distribution, e.g., so-called scenarios:
)}
Pr (T , h ) = T s , h s = ps ,
s = 1,2,...., S .
Fat solution
The idea is to replace Tx h by T s x h s , s = 1,2,...., S . This constraint is to
be satisfied in all scenarios. The advantage this approach has is that the
problem is deterministic again. But on the other hand, it is very conservative
and expensive. In addition to this, it results in a lot of constraints, which is
the reason this approach is called the fat solution.
Expected value
_
Scenario analysis
cT x
Page 18
Ax ~ b
Subject to
T s x ~ hs .
x0
Chance constraint
With chance constraints, Tx h is replaced by Pr (Tx h ) for some
prescribed reliability level 1 ,1 , where is to be determined by the
2
problem owner. The advantage this approach has is that the risk is explicitly
taken care of, i.e., risk := Pr{Tx / h}. So the maximum accepted risk is 1 .
The disadvantage this approach has is that a discrete probability distribution
leads to a mixed-integer linear programming problem.
( )
The final approach is called recourse actions. Since recourse actions are seen as
the most important class of stochastic programming, this concept will be
discussed in more detail in the following paragraph.
cT x
Ax ~ b
T ( )x ~ h( )
x 0.
(3).
Page 19
and h( ) is an m 1 vector. Both the matrix and the vector are dependent on a
random vector r . It is possible that the whole of T and h is random, but
more often than not only a restricted number of matrix and vector elements is
random. We assume that the joint distribution of the random vector is known.
Now consider the linear programming problem given by (3) and suppose this is
a so-called here-and-now problem. This problem needs a decision on x before
the real value of is known, i.e., only the joint distribution of on is
known. Moreover, in recourse models the random constraints of (3) are
reformulated as soft constraints, i.e., violation of the constraints is accepted but
not at any price. To describe how violated constraints are dealt with a secondstage linear programming model is introduced. This second stage model
contains second stage variables y p . The name recourse can now be
explained by the second stage, as its decisions are made after observations of the
value of .
A recourse structure is formally specified by a triple (Y , q,W ) , which is defined
as follows:
Y = {y p : y 0}, which describes the feasible set of recourse
actions y,
q is a 1 p vector of unit recourse costs,
W is an m p matrix, the recourse (technology) matrix.
When the recourse structure is applied to the linear programming problem (3),
the following decision problem can be defined:
Minimize
Subject to
c T x + min q T y
Ax
T ( )x
x X
first-stage
decisions
Wy
]
~b
~ h( ) ,
y Y
second-stage
decisions
first stage
constraints
second stage
constraints
This representation makes clear how the introduction of the recourse structure is
based on a relaxation of the constraints T ( ) ~ h( ) .
Page 20
The third possibility is to split the vector of decision variables in the original
linear programming problem in two parts. One set has to be determined
before obtaining , and the remaining set of variables may depend on the
value of . The constraints are also split into two parts. The first part of the
constraints does not include , and the second part of the constraints does.
Page 21
Page 22
4 The model
In this chapter an example will be worked out in which the theory of the
previous chapters will be applied. A linear programming model will be built to
schedule operations with a given duration over multiple days and operating
rooms in such a way that as much of the available time will be used for these
operations. First a simple model will be built, after which the model will be
expanded to deal with multiple operating rooms.
Operations almost always take more or less time than anticipated beforehand by
the model builder. Therefore, recourse actions will be applied to deal with a
random surplus or shortage of the duration of these operations.
For this first model, numerous assumptions have to be made regarding the
values of the parameters such as the duration of an operation, the number of
hours the operating staff works on one day, and more of such assumptions will
be made along the path of building the model.
4.1 Modelling
The problem that has to be modelled is how to plan operations such that the staff
and equipment are being scheduled most efficiently. In other words, if the staff
and equipment are available for eight hours per day, operations have to be
planned such that the staff and equipment are scheduled to be occupied during
these eight hours.
First the decision variable is defined:
xij = starting time of operation j on day i.
The assumption is thus made that the staff is expected to work eight hours per
day. Then assume that there are n operations, that are being planned over m
days, whereby operation j has a duration of dj hours. The decision variable is
equal to zero if operation j is not scheduled on day i and it is equal to or greater
than > 0 if operation j is scheduled on day i. Hereby, one could think of a
starting time for the staff of eight oclock in the morning, which would result in
= 8.
Page 23
First, the assumption is made that only one operating room is available to the
hospital. This assumption keeps the model somewhat simpler, but will
eventually be dropped later on. This assumption means that only one operation
at the time can be carried out. To make sure that an operation j that is scheduled
for day i is not scheduled again on a different day, a dummy-variable is needed.
This dummy-variable z ij is a 0-1 variable, which means it will only have the
values 1, if operation j is scheduled on day i, and the value 0 in all other cases.
Now, the following constraints can be formulated:
xij Nz ij ,
i, j ,
xij z ij ,
i, j ,
z
i =1
ij
= 1,
j ,
whereby N has a sufficiently large value, such that it always holds that xij N
i, j and k > j ,
xik + d k xij + My jk ,
i, j and k > j ,
Page 24
z
j =1
ij
d j 8 , i .
This constraint will make sure that the total duration of all operations scheduled
on day i will be eight hours at maximum.
What remains to be done is the objective function. To schedule the operations as
efficient as possible, the difference 8 z ij d j needs to be minimized. This is the
difference between the total duration of all operations planned on day i and the
total available time (eight hours) of the operating staff and equipment. So, to
optimize the schedule, the maximum difference has to be found. In other words,
the biggest difference needs to be as small as possible. The biggest difference is
given by the next expression:
n
n
n
max 8 z1 j d j ,8 z 2 j d j ,...,8 z mj d j .
j =1
j =1
j =1
Page 25
n
n
n
q = max 8 z1 j d j ,8 z 2 j d j ,...,8 z mj d j .
j =1
j =1
j =1
Subject to
q 8 z ij d j , for i = 1,..., m
j =1
xij Nz ij ,
i, j ,
xij z ij ,
i, j ,
z
i =1
ij
= 1,
j ,
x ij + d j xik + M (1 y jk ) ,
i, j and k > j ,
xik + d k xij + My jk ,
i, j and k > j ,
z
j =1
ij
d j 8,
xij 0 ,
y jk {0,1},
z ij {0,1} ,
i ,
i, j ,
j, k ,
i, j .
i, j , l ,
xijl z ijl ,
i, j , l ,
Page 26
z
i =1 l =1
ijl
= 1,
j ,
Minimize
Subject to
q 8 z ijl d j ,
i, l ,
xijl Oz ijl ,
i, j , l ,
xijl z ijl ,
i, j , l ,
j =1
z
i =1 l =1
ijl
= 1,
j ,
i, j , l and k > j ,
i, j , l and k > j ,
z
j =1
ijl
d j 8,
xijl 0 ,
y jkl {0,1},
zijl {0,1},
i, l ,
i, j , l ,
j , k , l ,
i, j , l .
Page 27
If an operation runs over time it means that personnel will have to work
overtime. Obviously, this leads to additional costs. If an operation takes less
time than anticipated, personnel is scheduled less efficiently than would have
been possible afterwards. This will also lead to additional costs.
Taking into account the aforementioned additional variables, the objective
function needs to be adapted. The adapted objective function is as follows:
c+ + + c .
Minimize
Next to the objective function, the following constraint also needs to be adapted:
n
z
j =1
ijl
d j + j = 8 j + +j ,
i, l .
j d j ,
j ,
+j , j 0 ,
j .
These two constraints give an upper and a lower bound to the increase and
decrease in duration of operation j.
Suppose that for the operations hold that the random demand of extra time is
assumed to be normally distributed. Then for every operation, the mean and
standard deviation of the random deviation j per operation has to be given. In
Page 28
addition to this, the different costs for an increase or decrease in duration need to
be specified.
The linear programming problem with the recourse structure then is as follows:
Minimize
c+ + + c
n
Subject to
z
j =1
ijl
d j + j = 8 j + +j ,
i, l ,
xijl Oz ijl ,
i, j , l ,
xijl z ijl ,
i, j , l ,
z
i =1 l =1
ijl
= 1,
j ,
i, j , l and k > j ,
i, j , l and k > j ,
xijl 0 ,
i, j , l ,
y jk {0,1},
j , k ,
z ijl {0,1} ,
i, j , l ,
j d j ,
j ,
+j , j 0 ,
j .
Page 29
Page 30
5 Conclusions
Linear programming is one of the many specializations of Operations Research
and is one of the most applied mathematical methods. The many different
algorithms of linear programming can be used for the optimization of a wide
range of problems.
The example that is worked out in this paper showed that there are drawbacks to
Linear Programming. If there are parameters of which the values are uncertain,
the measure of optimality is questionable. There are several approaches to take
this uncertainty into account in building the model. However, it shows that these
approaches lead to models in which the risk is not taken into account, or to
models that are very restrictive.
We have shown that adding a recourse structure to a well-defined linear
programming problem is relatively easy to do. Furthermore, adding a recourse
structure leads to a model in which the risk is explicitly taken care of. In
addition to this, it leads to a large scale Linear Programming problem. However,
it does have one disadvantage. The disadvantage is that the model may become
too large to solve.
Page 31
Page 32
6 Bibliography
H. Tijms
Operationele analyse, 2004
H. Tijms, A.A.N. Ridder
Mathematische Programmering, 2002
W.K. Klein Haneveld, M.H. van der Vlerk
Stochastic Programming, 2006
Wikipedia
http://www.wikipedia.org
Stoprog
http://stoprog.org
Google
http://www.google.com
Mathworld
http://www.mathworld.com
Page 33