Lecture Notes in Economics and Mathematical

Systems 347

David Rios Insua

Sensitivity Analysis
in Multi-objective
Decision Making
Lecture Notes
in Economics and
Mathematical Systems
Managing Editors: M. Beckmann and W. Krelle

347

David Rlos
RIOS Insua

Sensitivity Analysis in Multi-

objective Decision Making

Springer-Verlag Berlin Heidelberg GmbH

Editorial Board

H.Albach M. Beckmann (Managing Editor)

p. Dhrymes G. Fandel G. Feichtinger J. Green W. Hildenbrand W. Krelle (Managing
Editor) H. P. Kunzi K. Ritter R. Sato U. Schittko P. Schonfeld R. Selten

Managing Editors
Prof. Dr. M. Beckmann
Brown University
Providence, RI 02912, USA

Prof. Dr. W. Krelle

Institut fUr Gesellschafts· und Wirtschaftswissenschaften
der Universitat Bonn
Adenauerallee 24-42, 0-5300 Bonn, FRG

Author
Dr. David Rios Insua
School of Computer Studies
University of Leeds
Leeds LS2 9JT, UK
Opt. de Inteligencia Artificial, Facultad de Informatica
Universidad Politecnica de Madrid
Boadilla del Monte, 28660-Madrid, Spain

ISBN 978-3-540-52692-6 ISBN 978-3-642-51656-6 (eBook)

DOI 10.1007/978-3-642-51656-6

This work is subject 10 copyright. All rights are reserved, whether the whole or part of the materral
is concerned, spedlcally the rights of translation, reprinting, re·use of illustrations, recitation,
broadcasting, reproduction on microfilms or in other ways, and storage in data banks. Duplication
ofthis publicalion or Darts thereof is only permitted under the provisions of the German Copyright
Law of September 9. 1965, in its current version, and a copyright fee must always be paid.
Violations fall unCer Ihe prosecution act of the German Copyright Law.
© Springer-Verlag Berlin Heidelberg 1990
Originally published by Springer-Verlag Berlin Heidelberg New York in 1990

2142/3140·54321C - Printed on acid·free paper

Dedicated to my parents
 Introduction

The axiomatic foundations of the Bayesian approach to decision making assurne

precision in the decision maker's judgements. In practicc, dccision makers often
provide only partial and/or doubtful information. We unify and expand results to
deal with those cases introducing a general framework for sensitivity analysis in
multi-objective decision making.
We study first decision making problems under partial information. We provide
axioms leading to modelling preferences by families of value functions, in problems
under certainty, and moJelling beliefs by families of probability distributions and
preferences by familics of utility functions, in problems under uncertainty. Both
problems are treated in parallel with the same parametric model. Alternatives
are ordered in a Pareto sense, the solution of the problem being the set of non-
dominated alternatives. Potentially optimal solutions also seem acceptable, from
an intuitive point of view and due to their relation with the nondominated ones.
Algorithms are provided to compute these solutions in general problems and in
cases typical in practice: linear and bilinear problems. Other solution concepts
are criticised on the grounds of being ad hoc. In summary, we have a more ro-
bust theory of decision making based on a weaker set ofaxioms, but embodying
coherence, since it essentially implies carrying out a family of coherent dccision
anitlyses.
These results are used to detect when a current optimal solution has competi-
tors. In this case, sensitivity analyses are necessary as it means to help the DM
explore her thoughts to find which of her judgements are the most influential in de-
termining choice. This includes a criticism of the Flat Maxima Principle. Several
concepts of adjacent potential optimality help us to reduce the set of competitars
of the current optimal alternative, detecting which alternatives may share opti-
mality with it. A class of easily implcmcntable sensitivity tools based on gauge
functions is introduced, allowing us to identify critical judgements, competitors of
the current optimal alternative, and vitrious scnsitivity issues itccording to certain
scnsitivity indices. We then explore some questions on error modelling in judge-
mental mcitsurement, providing some other lools, based on Baycsian hypüthcsis
testing, to deepen the analysis.
Most of the above ideas are implemented in SEl\'SATO, a prototype library für
a sensitivity analysis package far dccision aids. Some computational experience is
 VI

described, proving the feasibility and success of our framework.

I wish to thank my supervisor, Professor Simon French, for his permanent en-
couragement, discussions and criticisms, his patience with my sometimes imprecise
approaches and for guiding my interest to my research area and my thinking to
the Bayesian path.
My first steps in Decision Theory were given under the guidance of Professor
Sixto Rios, who has provided invaluable criticisms to my manuscript. I have
received many suggestions for my research from Professor Doug White, Dr. Roger
Cooke and Dr. Les Proll.
My gratitude to staff and fellow students in Manchester University, for taking
part in my experiments, and Leeds University, for helping me with the computers
and supplying an enjoyable environment. Mike Wiper belongs to the intersection
of both sets; in addition, he has discussed many concepts and read most of the
manuscript. Valeria Rios Insua helped me with the figures.
Computer facilities were provided by the University of Manchester Regional
Computer Center and the University of Leeds Computing Service. This research
was supported by a grant of the Bank of Spain.
Finally, I wish to thank my family for their support and understanding, and
Cornelia for showing me that even the impossible is beatable.
Contents

1 Partial Information and Sensitivity Analysis in Decision Making.

Introduction 1
1.1 Bayesian decision analysis 2
1.1.1 Problems under certainty . 3
1.1.2 Problems und er uncertainty 4
1.1.3 Comment . . . . . . . . . 5
1.2 Some views on other decision aids 6
1.2.1 Fuzzy-set based decision analysis: Yager's method 6
1.2.2 The Analytic Hierarchy Process: a comment .. 9
1.2.3 Outranking methods: Comments on ELECTRE I 13
1.2.4 Interactive decision aids 16
1.2.5 A general comment 20
1.3 Our problem. . . . . . . 21
1.4 Some previous thoughts 22
1.4.1 The problem. . . 22
1.4.2 Foundations of decision making under partial information. 23
1.4.3 Ordering the alternatives. Decision making under partial
information . . . . 23
1.4.4 Additional criteria 23
1.4.5 Hierarchical approaches 24
1.4.6 Detection of sensitivity . 24
1.4.7 Sensitivity problems are not important 24
1.4.8 Sensitivity measures .. 25
1.4.9 Detection of competitors . . . 25
1.4.10 What to do about sensitivity? 25
1.4.11 Error modelling . . . . . . . 25
1.4.12 Sensitivity analysis in commercial decision aids 25
1.5 Comments. . .. 27
 VIII

2 Decision Making under Partial Information: Theory and Algo-

rithms 28
2.1 Some basic results . 29
2.2 Judgemental quasi orders. 29
2.2.1 General preference quasi orders under certainty 30
2.2.2 Additive models. . . . . . . .. . ... 32
2.2.3 Preference quasi orders under uncertainty 45
2.3 A parametric model . . . . 51
2.3.1 The certainty case 52
2.3.2 The uncertainty case 52
2.4 Orders and solution concepts . 53
2.5 General algorithms . . . . . . 59
2.5.1 Nondominated alternatives. 59
2.5.2 Potentially optimal alternatives 61
2.6 Some particular cases . 63
2.6.1 Linear models . 63
2.6.2 Bilinear models 67
2.7 Additional criteria .. 69
2.7.1 Some additional criteria 69
2.7.2 Hierarchical approaches 71
2.8 Comments . . . . . . . . . . . . 73

3 Sensitivity Analysis in Multi-objective Decision Making 74

3.1 Some basic results . . . . . . . . . . . 75
3.1.1 The maximum ranking function 75
3.1.2 The optimality subsets 80
3.2 Do we need sensitivity analysis? (Yes) 83
3.2.1 Strang optimality . . . . . . . . 83
3.2.2 A Flat Maxima Principle? ., 86
3.2.3 Demands of a sensitivity analysis tool . 88
3.3 Adjacent potentially optimal solutions 89
3.3.1 Adjacent potential optimality .. 89
3.3.2 Adjacent potential optimality according to w . 92
3.3.3 Straight adjacent potential optimality . 92
3.3.4 Relations between the concepts 93
3.4 Some sensitivity tools . . . " 93
3.4.1 A dass of sensiti,ity tools 94
 IX

3.4.2 Gauge function based sensitivity toels 96

3.5 Error modelling in decision analysis . . . 103
3.5.1 Errors in judgemental modelling . 104
3.5.2 An error model . . . . . . . . . . 106
3.5.3 Some thoughts on the elements of the problem. 112
3.6 Comments . . . . . . . . 126

4 Computational experience 121

4.1 An introduction to SENSATO · 127
4.2 Linear models: Flood-plain management 132
4.2.1 The model . . . . . 133
4.2.2 Input to SENSLIN 134
4.2.3 Results . . . . . . . · 134
4.3 Bilinear models: Portfollo selection · 143
4.3.1 The model . . . . . · 144
4.3.2 Input to SENSBIL · 145
4.3.3 Results....... · 146
4.4 General models: Technology-purchasing decision . · 147
4.4.1 The model . . . . . . · 149
4.4.2 Input to SENSGEN . · 150
4.4.3 Results...... 151
4.5 Dallas' problem revisited 151
4.5.1 The model . . . . 152
4.5.2 Results...... · 153
4.6 Questioning a model: Road selection · 153
4.6.1 Analysis 1 . . . . . . . . · 154
4.6.2 The model (Analysis 1) . 155
4.6.3 Results . . . . . . . . . . 156
4.6.4 Analysis 2 . . . . . . . . · 156
4.6.5 The model (Analysis 2) . 157
4.6.6 Results(Analysis 2) 158
4.7 Some counterexamples 158
4.7.1 The model. 159
4.7.2 Results. 160
4.8 Comments..... 161
 x

5 Conclusions 164
5.1 Summary . 164
5.2 Topics for further research . 165
5.2.1 Decision making under partial information 165
5.2.2 Sensitivity analysis .. . 167
5.2.3 Computation . . . . . . 168
5.2.4 Group decision making . 170
5.2.5 Imprecision in expert systems 170
5.2.6 Restructuring . . . . . . . . . 170
5.2.7 Integration with other methodologies 171

Bibliography 172

Appendix 186
 List oe abbreviations

- p.o. Potentiallyoptimal
- a.p.o. Adjacent potentially optimal
- s.a.p.o Straight adjacent potentially optimal
- a.p.o.w Adjacent potentially optimal according to w
a.w.o. additive weak order

a.q.o. additive quasi order

- w.a.w.o. weighted additive weak order

- w.a.q.o. weighted additive quasi order
Chapter 1

Partial Information and

Sensitivity Analysis in Decision
Making. Introduction

One of the defining capacities of humans is that of being able to make decisions.
We shall consider decision making problems in which adecision maker (DM) has to
choose among a finite number of alternatives according to some criteria. Since this
is not always a routine task, some kind of rational help may be of value to DM's.
Scientists have approached this problem developing more or less sound theories,
models and concepts leading to decision aids. Essentially, these are calculation
procedures or rules such that filled in with the proper data they give us a ranking
of the alternatives, or the best ones, or a sub set of good ones.
Thus, the excellence of an application of adecision aid will depend not only
on the goodness of the calculation procedure, its theoretical foundations and its
numerical precision, but also on the quality of the data input.
Simplifying, data input are of two types:

• 'objective' data: they refer to the performance characterisation of the alter-

natives. More and more decision aids tend to be multi-objective optimisers,
i. e., performance is measured by more than one attribute,

• 'subjective' data: they refer to the DM's judgemental inputs (weights, pair-
wise comparisons, beliefs, strength of preferences, ... ) required to derive the
decision aid output according to her thought.

As an attempt to formalise the problem mathematically, assurne that X represents

the set of alternatives, f :X --+ R m the performance function, Y the space of
judgemental inputs and F : f(X) x Y --+ Rq the calculation rule. The problem
 2

would be to optimise F(f(x),y) subject to the condition x E X. We shall see

various examples of this formalisation.
The output of adecision aid depends critically on data input, thus one should
criticise these data in some way. It is essential to check the sensitivity of the con-
clusions to the data, because data input is constantly revised as the DM comes to
understand the implications and possible inconsistencies of her judgements. Sen·
sitivity analysis can focus on those judgemental inputs which are most important
in determining choice and therefore, need to be revised most carefully.
This monograph deals with sensitivity analysis tools for decision aids, within
a Bayesian context. In this chapter, we start with a summary of the foundations
of these aids. Though their strength is precisely on their foundations, in practice
one has to confront several problems. This leads us to consider other decision
aids, with less than positive conclusions, so we face the need of developing results
to overcome the problems with the Bayesian aids. We have to study problems
of decision making under partial information and sensitivity analysis in decision
making, which are intimately related. A review of previous approaches in these
fields follow.

1.1 Bayesian decision analysis

Assurne that a DM has to choose among a set A = {al,' .. , a m }. The set of states
of nature is e = {OI, ... ,Od. The n-attributed consequence aij = (aIj, ... ,aij)
arises when ai is selected and Oj is the state; aij E C = {CI,'" , c,}, the space of
consequences. The problem is summarised in the following table.

States
° 1 .·····Ok

al all ...... alk

a2 a21 ...... a2k

Alternatives ...
. ..
am aml .... ·· amk

We give a brief review of some well-known results allowing us to handle the

problem. Only concepts and properties to be used later are defined.
Given a binary relation :<, we say x -.( y iff x:<y, -,(y:<x), x ~ y iff x:<y,y:<x.
 3

1.1.1 Problems und er certainty

When k = 1, the problem is under certainty. We assume a binary relation ~ on A,
representing the DM's preferences. We interpret it as: 'x~y' means 'x is at most
as preferred as y'. We shall use a value function v, modelling the DM's preferences,
to rank the alternatives.

Proposition 1.1 (Roberts,1979) Let A be afinite set of alternatives and ~ the

DM's preference relation. (A,~) is a weak order iff there is areal function v on
A such that
a~b ~ v(a) :::: v(b)

fora,bEA.

Additive models are popular in the multi-attribute case. Scott (1964) gives con-
ditions allowing this representation.

Definition 1.1 Let A = f1~lAi, I > 1,dj,bj E A,j = 1, ... ,1. (dl, ... ,d1)E1
(b l , .•• , b,) means that for each i (dL ... , di) is a permutation of (bL ... , bi).

For exarnple, in R 3

«1,3,2), (2, 6, 3), (3,2,4), (4,2,5) )E4 «1, 6,4), (2, 2, 5), (4,2,3), (3,3, 2».

Proposition 1.2 (Scott,1964) Let A = f1~1 Ai, with Ai finite. (A,~) satisfies:

• SC1. Fora,bEA,a~borb~a .

• SC2. Let I> 1, dj , bj E A,j = 1, ... ,1. (d l , ... , d,)El(~, ... , b1) and dj~bj,
for j = 1, ... ,1-1 =? bl~dl.

iff there are real functions Vi on Ai, i = 1, ... , n such that

n n
a::5b ~ LVi(ai) ::; LVi(bi)
i=l i=l

for a, bE A.

Fishburn (1970) gives other conditions. French (1986) provides a number of results
when A is an infinite set.
 4

1.1.2 Problems under uncertainty

If k > 1, there is uncertainty involved in the problem. The DM expresses her
beliefs over A, the algebra of subsets of G. They are modelled in terms of a binary
relation ~t on A interpreted as follows: 'B~tC' for B, CE A means 'B is at most
aB likely to occur as C'. We model these beliefs by a probability distribution p,
giving a representation
B~tC {=} p(B) ::; p(C).

Proposition 1.3 (Scott,1964) Let G, A, ~t be as above. (A, ~t) may be repre-

sented by a probability p iff

• S1. 0 -<t e, 0~tB, VB E A.

• S2. VB,C E A, B~tC or C~tB.
• SS. For all integer 1 2: 1 and AI, BI, ... , A1+I, B1+I E A, such that
Ai ~t Bi,i = 1, ... ,1, and

XA 1 + ... + XA! + XA!+! = XB, + ... + XB! + XB!+!

B1+dtAI+l (where XB is the characteristic function of B E A).

S3 simply requires that any element of G belong to the same number of Ai and Bi,
for i = 1, ... ,1 + 1. P need not be unique. A unique representation is achieved if
we add an axiom which intro duces as a primitive the notion of a uniform random
variable over [0,1] through a reference experiment, see De Groot (1970), French
(1982). Other ways are reviewed in Fishburn (1986), which includes results for
infinite spaces.
Having seen how to model the DM's beliefs, we associate a lottery with each
alternative. 'vVe assume that the DM may establish her preferences among these
lotteries. More generally, let Po(C) be the set of prob ability measures over C and
:::5 a binary relation on Pa(C) interpreted as folIows: let p, q E Po(C), 'P:::5q' means
'p is at most as preferred as q'. We model this relation via expected utility:

Proposition 1.4 (Fishburn,1970) Let Pa(C), ~ be as before. There is areal

function u on C satisfying

P:::5q {=} E(u,p)::;E(u,q),Vp,qEPo(C)

iff
 5

• Ul. (Po(C), j) is a weak order.

• U2. pjq,O < a < l=9ap + (1 - a)rjaq + (1 - a)r.

• U3. p -< q,q -< r=9ap+(l-a)r -< q, q -< ßp+(l-ß)r forsome a,ß E (0,1).

Again, if the consequences are multi-attributed, it is sometimes possible to use

additive utility functions, as indicated in the following result. Given a multivariate
probability distribution p, (pI, ... , pn) will represent its marginal distributions.

Proposition 1.5 (Fishburn,1982) Assume there is a utility function u repre-

senting j on Po(C), where C ~ fIi=IC; and all the C; are finite. Then if

there are real funclions Ui on C;, i=l, . .. , n, such that

n n
pjq ~ "LE(u;,pi):s; "LE(u;,qi).
1=1

Results for the cases in which Cis not finite and Po(C) is not only the set of simple
probability measures may be found, for example, in Fishburn (1982).

1.1.3 Comment
This has been a brief review of the foundations of Bayesian decision analysis.
These topics, along with practical ways of building the functions involved, are
fully covered in works such as De Groot (1970), Fishburn (1970, 1982), French
(1986), Rios (1977) or Roberts (1979). This review will facilitate a comparison
with the results we shall give in the next chapter.
These results provide a rational framework in which adecision making prob-
lem can be solved. Any alternative approach has to break this framework in some
direction. We recall the solution suggested by it: if the problem is under certainty,
we model the DM's preferences by a value function v to be maximised; if the prob-
lem is under uncertainty, we model the DM's beliefs by a probability distribution p
and the DM's preferences and risk attitudes by a utility function u whose expected
value has to be maximised. The approach requires building v or p and u. This is a
laborious task of judgemental elicitation requiring precise information. However,
very frequently, the DM is not able to provide it (see e.g. Weber (1987)). More-
over, the assessment procedure assesses the numbers within certain limits (French,
1989) and there is some experimental evidence of the presence of biases and er-
rors in human judgements (see e.g. Kahneman et al. (1982) and the discussions
 6

in Berkeley and Humphreys (1982), Bouyssou (1984) and Jaffray (1989)). Also,
the elicitation process is one of measurement and, as such, may involve some error
(see e.g. Lindley et al. (1979)). Thus, in practice, we have to face cases in which:

• the DM is not able to provide all the information required,

• the DM provides dubious information.

In any case, if, as we shall explain below, we concei ve of the decision making process
as a constructive one, we should have a framework to criticise that information.
This project develops such a framework.
Observe that the first idea we may have is that the Bayesian approach is inad-
equate, considering the cases stated above. We should look at other approaches,
as we do in the following.

1.2 Some views on other decision aids

We shall review some of the more popular alternative decision aids. For each of
them, we give its basic calculation procedure and some notable examples.

1.2.1 Fuzzy-set based decision analysis: Yager's method

A number of fuzzy-set based decision aids have been introduced recently. Zimmer-
mann (1987) gives an introduction to fuzzy sets and their applications to decision
making problems. Many foundational criticisms may be seen in French (1984,
1987). We shall analyse Yager's (1981) method. It is proposed as an example of a
fuzzy-set based decision aid on several surveys on multiobjective decision aids and
Yager introduces it as superior to those of Bellman and Zadeh (1970) and Yager
(1977).

Yager's method

Let S be a linearly ordered finite set, used to indicate preference information. For
instance, Yager illustrates his method with

S = {lowest,very low, low, medium, high, very high, perfect}.

We write S = {bI,"" bp }, where greater subindex indicates more preference. Let

Y = {AI,"" An} be the criteria measuring the performance and A be the set of
alternatives. For each x E A, A;(x) E S indicates the degree to which x satisfies
 7

the criteria specified by Ai. G(A;) = bG(i) ES denotes the importance of the i-th
criterion.
For each x, let Ci(X) = max(bp_G(i)+I, Aj(x)). Reorder the Cj(x)'s, calling them
rj(x), so that i ::; j=}rj(x) ::; rj(x). Then, Vager introduces the relation

x~y ~ r(x) ::;L r(y)

where ::;L is the lexicographic order and r(.) = (rl(.),"" rn (.)). In other words,
the alternative proposed by Vager as optimal is that maximising

D(x) = m}n(max(bp_G(i)+I,Ai(x)))

Eventual ties are broken with the second least Cj(x), and so on. For a justification
see Vager (1981).

Example 1.1

Let S = {O,1,2,3,4,5,6} .

• a. We have four criteria and four alternatives, their scores being

al (1,2,3,4)
az (2,3,4,5)
a3 (2,2,3,4)
a4 (1,5,5,,5)

Suppose that the DM thinks all the criteria are equally important: bG(j) =
bG(j), V i, j. Depending on the choice of that same weight, we get the following
rankings:

bG(i) ranking
0 (al, a2, a3, a4)
2 a4,a2,(aI, a3)
5 a2, a3, a4, al

Observe that

- the importance of the weights is the same in all cases, but there are
r anking reversals ,

- in so me cases, Pareto and weakly Pareto dominated alternatives are

ranked as the alternatives dominating them.
 8

• b. Suppose that the weights are (5,6,6,6). The ranking is a2, a3, a4, al. Even
giving more importance to all the criteria in which a4 is ranked best, this
alternative may not be the best.

• c. Suppose that al = (0,3,2,3) and a2 = (1,2,2,3). If the weights are

(6,6,6,6) then al ~ a2. If S is only an ordinal scale then {(3,3,2,3), (4,2,2,3)}
represent the same alternatives; in this case, a2 ~ al.

• d. Suppose that S has a linguistic meaning as in Yager's example. Imagine

that we want to choose cars and consumption is criterion 1, with weight 6.
Suppose cars A, B consume 9.2, 9.1IjlOOkm, respectively. If the number of
cars is smalI, we may represent consumptions precisely and assign 0 to 9.2
and 1 to 9.1. Suppose that A is represented by (0,2,2,3) and B by (1,1,2,3)
(These are the Ci(X)'S). If the number of cars is big, we may find both
consumptions equally bad so we assign 0 to both of them. Thus, car B is
now performing (0,1,2,3). In the first case, A~B; in the second case, B~A:

we are not respecting the principle of irrelevant alternatives.

These examples point out some flaws of Yager's method that deserve some analysis.

1. The method is a maximin one, thus having their shortcomings (see Milnor,
1954). Particularly, the property in c is not verified by them. Pessimism is
reflected in b.

2. In fact, it is a maximinimax approach. The way in which weights act leads

to the problems mentioned in a.

3. Using poor scales such as that of Vager leads to problems like d.

4. Once the e;(x)'s are defined, we typically have the situation reflected in the
graph, where the number of criteria is two. The Iined part is the set of

Figure 1.1: Alternatives less preferred than x

alternatives less preferred than (Xl, X2). This is astrange way of imposing a
preference order.
 9

Yager presents his method as an improvement on other fuzzy-set based ap-

proaches. We have seen that its application may lead to strange results.

1.2.2 The Analytic Hierarchy Process: a comment

The Analytic Hierarchy Process (AHP) is introduced by Saaty (1980) as an ap-
proach to decision analysis based on the principal eigenvalue method (PEM). Its
relation with additive value theory has been studied recently (Kamenetzky (1982),
Belton (1986)). Yu (1985) uses the PEM as a weight generator for an additive
value function.
French (1986) presents the AHP as follows. Suppose that the DM's preferences
can be modelled with a value function
n

v(a) = ~wivi(ai)
i=l

Assuming vh(aj) =j:. 0 =j:. Wj and calling r~ = vh(a?)/vh(aj), r~ = w;fwj, we

can build the matrices Rh = (r~)ij, Ra = (r~j)ij. The AHP inverts this process:
one first builds the matrices and then deduce the weights and component value
functions. To do this, the r~'s are generated according to a qualitative scale
quantified as a seventeen-point scale with values in {1/j,j}j:l, ... ,9' The w's and
the Vi'S are generated with the PEM, see Saaty (1980).
French (1986) points out that the matrices Rh (h > 0) are not meaningful
under additive value theory. However, Ra is meaningful in that theorYj in fact,
several authors, e.g. Yu (1985), suggest using the PEM to determine the weights
of an additive value function.

Weighting by means of the PEM

Suppose n = 2. Then, the matrix Ra is

where .\ E {j, 1/ j}j=1, ... ,9' Its principal eigenvalue is 2. Its principal normalised
eigenvector is

so
'\1 E {jf(j + 1), 1/(j + 1)L=I,... ,9'
 10

The table Oll the llext page presellts thirty Olle altematives, the (approximate)
lower bOUlld of the ).1'6 that make them optimal aIld the ).1'S gellerated by the
PEM covered by the respective interval.
For example, au will be optimal if ).1 E (.64156, .67742); 2/3 belongs to this
illterval. Observe that all the alternatives are 1l0t cOllvex domillated but many of
the illtervals do 1l0t cover allY Al gellerated by the PEM: many reasollable solutions
Call1lOt be optimal if we adopt this approach. Note that if ). were allowed to vary
freely in (0,00), we might generate aIlY 1l0t convex domillated altemative. "Saaty
illdicates that it is possible to assigll decimal values to respollses ill llumerical
mode, but the philosophy of the scale mitigates agaillst this" , Scholler alld Wedley
(1989).

Figure 1.2: Graph of scores

The moral is: do 1l0t use this approach to generate the weights of aIl additive
value fUllctioll.
 11

1st coor. 2nd coor. weight limit weight gen.

1 465 0 .96774
2 464 30 .93548
3 462 59 .90323
4 459 87 .87097 .875,.888,.9
5 455 114 .83871 .857
6 450 140 .80645 .833
7 444 165 .77419 .8
8 437 189 .74194 .75
9 429 212 .70968
10 420 234 .67742
11 410 255 .64156 .666
12 399 275 .61290
13 387 294 .58065
14 374 312 .54839
15 360 329 .51613
16 345 345 .48387 .5
17 329 360 .45161
18 312 374 .41935
19 294 387 .38710
20 275 399 .35484
21 255 410 .32258 .333
22 234 420 .29032
23 212 429 .25806
24 189 437 .22501 .25
25 165 444 .19355 .2
26 140 450 .16129 .166
27 114 455 .12903 .142
28 87 459 .09677 .125,.111,.1
29 59 462 .06452
30 30 464 .03226
31 0 465 .00000

Table 1.1: Alternative scores and weights generated by PEM

 12

The AHP
This observation leads us to find out whether a shnilar phenomenon occurs with
the ARP. The method outlined above to generate weights is used in the ARP to
determine also the vi{ai)'s, see Belton (1986).
With A E {j, 1/ j};=I, ...,9 we may generate 17 Al weights. To follow the previous
idea, we would need at least 18 alternatives. The computational problems would
be tremendous in this case. Thus, we shall modify the ARP to the case j E {1,2}
and conjecture the result for the actual case.
Then, it is Al E {1/3, 1/2, 2/3}. Consider the matrix B ofpairwise comparisons
of ten alternatives according to the first criterion:

1 1 2 2 2 2 2 2 2 2
2 1 2 2 2 2 2 2
.5 .5 2 2 2 2 2 2 2
.5 1 .5 1 2 2 2 2 2 2
.5 .5 .5 .5 1 2 2 2 2 2
B=
.5 .5 .5 .5 .5 1 2 2 2 2
.5 .5 .5 .5 .5 .5 1 2 2 2
.5 .5 .5 .5 .5 .5 .5 1 2 2
.5 .5 .5 .5 .5 .5 .5 .5 1 2
.5 .5 .5 .5 .5 .5 .5 .5 .5 1

Its principal eigenvalue is 10.539 and its normalised eigenvector is

(.1604, .1487, .1331, .1223, .0999, .0869, .0756, .0658, .0572, .0498)

By reordering conveniently the elements of B, we shall suppose that the scores of

the ten alternatives are
al .1604 .0498
a2 .1487 .0756
aa .1331 .0572
a4 .1223 .0658
as .0999 .0869
a6 .0869 .0999
a7 .0756 .1487
as .0658 .1331
a9 .0572 .1223
a\O .0498 .1604
which we plot in Figure 1.3.
 13

Figure 1.3: Graph of AHP scores

We compute )'la1 + (1 - Ada2 for alternatives a1, a2, a7, a10.

Al= 1/3 Al = 1/2 Al = 2/3

a1 .2600 .2102 .3706
a2 .2999 .2243 .3730
a7 .3730 .2243 .2999
a10 .3706 .2102 .2600

For Al = 1/2, a2 and a7 are optimal. For Al = 1/3, a7 is optimal. For Al = 2/3, a2
is optimal. In consequence, a1 and a10, which are not convex dominated, cannot
be generated as optimal by this modification of the AHP. Neither as nor as, which
are nondominated, may be generated as optimal by the AHP, though this should
cause no worries, provided that we have tested that v is additive.
Thus, for this simplification of the AHP, we face the same shortcoming. Other
issues are raised by Belton (1986) and French (1986).

1.2.3 Outranking methods: Comments on ELECTRE I

Outranking methods (ELECTRE, PROMETHEE, ... ) are based on the concept
of outranking relation:

Definition 1.2 (Roy,1973) Given ai, a; E A it is said that

• ai outranks aj (aiSaj) if we have enough reason to admit that 'ai is at least

as good as a; , and no good reason to refuse it,

• ai does not outrank a; (..,( aiSa;)) if the arguments in favour of the proposition
'ai is at least as good as aj , are considered insufficient.

More details on these relations canbe seen in Roy and Vincke (1984). Outranking
methods proceed modelling the outranking relation and exploiting it. We shall
 14

illustrate thern with a cornrnent on the (chronologically) first of thern: ELECTRE

I (Sussrnan et al, 1967).
Suppose that the DM has assigned positive weights (Ai, i = 1, ... , n, L.\; = 1)
to the objectives, a number c (the concordance threshold) and n numbers d i (the
veto thresholds). Let us consider two alternatives ai, aj. Let Aij be the surn
of weights of the objectives in which ai is at least as good as aj. We say that
(ai, a;) satisfies the concordance filter if Aij ?: c. Let K = {k : aik < ajd and
ek = ajk - aik· (ai, aj) satisfies the discordance filter if ek ::; dk , Vk E K. Then, we
say that aiSaj if (ai, aj) satisfies the concordance and the discordance filters.
Once S is built, one looks for a kernel N of the graph of S, that is, "lai E A \ N,
there is sorne aj E N such that ajSai and if ai,aj E N, ~(aiSail and ~(ajSa;).
Consider the following examples :

Example 1.2

1. The data of the problem are

Objective 1 2 3 4 5 6
Weight 1/8 1/8 1/8 5/24 5/24 5/24
al 1 1/2 0 1 0
a2 0 1 1/2 1 0 1
a3 1/2 0 1 0 1 1
VetoTh. 9/10 9/10 9/10 2 2 2
Suppose that c = 1/2. We get

a 1 Sa2, ~(a2Sad, a2Sa3, ~(a3Sa2), a3Sal, ~(alSa3)'

The graph of S is:

In this case, N # 0. As we see, S need not be acyclic.

2. The data are

Objective 2 3
Weight 1/3 1/3 1/3
al 1 1 0
a2 0 0
a3 2 2 -1
VetoTh.
 15

Suppose that c = 1/2. Imagine, first, that A = {a\,a2}. The graph of S is

a l _a2

and the kernel is N = {ad. Consider now the problem with A = {aJ, a2, a3}.
The graph of S is

and the kernel is N = {a3, a2}. Thus, S is not transitive and ELECTRE I
does not satisfy the principle of irrelevant alternatives.

3. The data are

Ojective 2
Weight 1/2 1/2
a\ 1 0
a2 0
with c = 1/2. Then if

d\ d2 Graph Kernel
<1 > 1 a\ --+ a2 a\
>1 < 1 a\ f-- a2 a2
<1 < 1 a\ a2 {a\,a2}
>1 > 1 a\ +----+ a2 0

Thus, we may think of very unstable situations in which we may pass from an
'all alternatives acceptable' to a 'no response of ELECTRE 1'. This example
may be connected to the following criticism by Arrow and Raynaud (1987)
to ELECTRE: "The consultant tries different choices for the thresholds and
the weights in order to obtain a selected set as wished by the DM".

So, a number of serious flaws can be found for ELECTRE I. Moreover, "if one
possesses the elements necessary to the building of a linear utility function why
should one try a method as sophisticated as ELECTRE" (Arrow and Raynaud,
1987).
 16

1.2.4 Interactive decision aids

Interactive decision aids (IDA's) claim to remedy some of the difficulties encoun-
tered in modelling judgements as demanded in the Bayesian foundations. White
(1982) gives a description of several IDA's. White (1983), French (1986), Marcotte
and Soland (1986) and Daellenbach and Buchanan (1989) analyse them critically.
An interactive decision aid follows this general scheme:

1. Choose a solution x (taking into account the available information).

2. Present it to the DM.

3. If she is satisfied with it, stop. Else, gather information and go to 1.

Distinct aids differ in the ways of choosing solutions and gathering information
and in the information gathered. We consider three particular approaches.
The first one, as presented in French (1986), assurnes an underlying value
function Vj thus, we try to solve max v( x) s. t. x E A. If v were known, we would
have a mathematical pro gram. Typically, we solve it generating a sequence {XI}

such that v(xd > V(XI_l), stopping when some convergence criteria are satisfied.
As we do not know v, we try to emulate the process as folIows:

1. 1<-0, Xo initial solution.

2. If the DM is satisfied with XI, stop.

3. 1<-1 + 1, find XI such that XI-l -< XI. Go to 2.

Different algorithms vary in what is assumed ab out v, the information gathered

to reduce A or the set of acceptable value functions, and the optimisation prob-
lem solved to find a new XI' Geoffrion et al' (1972) provides an example of this
approach.
The second one is based on a satisficing principle. The DM suggests a set of
goals x", for instance, an ideal point. The proposed solution is that of the problem

mind(x,x"),
rEX

where dis some distance (thus, we are using the surrogate value function -d(., x"),
see Rios and Rios Insua, S. (1986)). Typically, the distances used are lpo A number
of interactive methods follow the scheme:
 17

1. Choose distanee and X·.

2. Solve the minimum distance problem.

3. If the DM is satisfied with the solution, stop.

4. Modify the distance and/or x·. Go to 2.

Different ways of modifying the parameters lead to different algorithros. Basically,

we ean group them into aigorithros introducing arbitrary reductions of the param-
eter spaee or the set of alternatives, e.g. Steuer (1986), or a wandering scheme for
modifying x·, e.g. Wierzbicki (1982).
A third approach is based on the iroplicit enumeration of the set of alternatives.
Typically, the approach will consist of

1. Find a pair of alternatives Xi, Xj'

2. Compare them and eliminate the alternatives whieh are known to be worse
than the worst of them.

3. If there is any X E X remaining, take one of them and go to 2. Otherwise,

stop.

The following method is based on that scheme. It is introdueed in Rios Insua

(1987) and developed in Rios Insua (1988).
We assuroe that there is an underlying monotonie continuous value function v,
as in the Wold-Debreu theorem (Fishburn, 1970). Ais a finite set of bi-attributed
alternatives. The basic idea we use is illustrated in the figure. Let ai, aj be efficient
alternatives (that is, there is no other a E A such that ai ::; a, and similarly for
aj).

.ai

Figure 1.4: Elimination proeess in the interaetive method

Supp~se that aj has the best value in the seeond co ordinate and aj -< a •. As
v is monotonie, we ean improve aj by moving in the directions of J{ = R~. Due
 18

to the property of aj, we should only consider a == (1,0) among the directions
of Je Thus, we could determine aj == aj + ßa, ß > 0, such that aj '" a. and
eliminate the alternatives dominated by aj, discarding sorne efficient alternatives
in this operation. We shall call it 'elimination by aj'. Similarly, if aj had had the
best value in the first coordinate then a == (0,1).
Now, if Am designates the available set at iteration m, we shall define

which give us two efficient solutions of Am (and, as Am will be defined, of A) with

the best value in the coordinates. The procedure would be to calculate Yl and Y2
at each step, cornpare thern and eliminate by yj (or y;). This approach is followed
in the algorithrn below. Yj (Yk) designates the best (worst) solution of Yl, Y2. Y is
the set of solutions indifferent to Yj. yj, == Yk + ßa '" Yj, with a defined as above
and ß, sorne positive nurnber.

2. If Yl == Y2, stop: solution <-{Yl} U Y. Else, h<-j.

3. If Yl ~ Y2, Y<-{Yd U Y, elimination by Y2, m<-m + 1, find Y2, go to 2. Else,

go to 4.

4. If Y2 -< Yl, j<-l, k<-2, go to 5. Else, j<-2, k<-l.

5. If j # h, Y<-0.

6. Find Y'k, elimination by Y'k, m<-m + 1.

7. Find Yk, go to 2.

Observe that:

• Condition 5 tests whether the best solution at this iteration coincides with
that of the previous one.

• Condition 2 tests whether Am contains an ideal point.

• 'Elimination by z' rneans Am +1 == Am \ {a: a::; z}.

• To find yZ, we rnay ask the DM directly or proceed by bipartition, in which

the DM is asked for her preferences between a sequence of pairs of options
which are designed to converge interactively to her indifference point.
 19

Clearly, the procedure ends in a finite number of iterations: at each of them

we discard or put in Y at least one eflicient solution and the number of eflicient
solutions is finite. Moreover, if the DM answers consistently, we find the best
solution(s).
The algorithm has been programmed in FORTRAN 77 on a CMS system.
Two kinds of tests were perfo=ed. The first one simulated a DM by means of a
built in value function. We tried to determine the information required from the
DM, namely, the number of questions she would have to answer. As we see, the
DM has to compare alternatives, equivalently, to evaluate and find the larger of
two numbers in the simulation, and find pairs of indifferent alternatives, in the
simulation, solve equations with one unknown. In each run, we generated 100
alternatives, with uniform or normal distribution, and tried linear value functions
with different weights, and multiplicative ones. The following table gives the means
of the number (over 20 random iterations) of pairwise comparisons (CO) and
equation solutions (SO) in the simulation.

Linear Linear Linear Linear Linear Multip.

(.1,.9) (.25,.75) (.5,.5) (.25,.75) (.9,.1)
CO SO CO SO CO SO CO SO CO SO CO SO
Uniform 2.8 2.8 2.6 2.9 2.4 4
Normal 1.8 3.25 7.7 3.55 1.95 7.45

In this simulation, CO=SO. If L were the mean number of questions necessary

to find the indifference point, the mean of the total number of questions would be
CO + L SO. This number is low enough to view this aid as reasonable in terms of
information demand from the DM.
The second type of tests were intended to study the reactions of users to this
approach. The test problem was an imaginary job selection problem. Obviously,
different DM's have different attitudes. Thus, we have built in coherence guides
and indifference checkings in our program, which allow us to decrease the risk of
inconsistent answers. These were of great help for users worried about consistency.
There is, as weil, great difference in the DM's attitudes towards precision in
judgements. The built in facility for determining the indifference points has been
of use in some cases. A general observation is that DM's tend to go for the option
of establishing directly their indifference points, after using that facility for a few
iterations. In general, the reactions were favourable. Questions were not found
diflicult, though remember that the problems solved were bi-attributed. In the
worst cases, the aid has turned out as a good screening procedure: it has helped
 20

to determine a subset of alternatives which include the best ones and between
which the DM finds not much difference, on a first approximation to the problem.
It helped also to let the DM know what Daellenbach and Buchanan call "the
topography of the problem". To sum up, this approach might be of help as a step
prior to attacking the problem according to the lines described later.
From the critical analyses cited above, we can seleet several points that we
should demand from an !DA.

• The approach should be structured according to a(n underlying) value func-

tion. The type of questions posed should be meaningful in the corresponding
theory.

• We should not automatically assurne very strong, e.g. linearity, or hardly

testable properties, e.g. concavity.

• We should be careful with holistic comparisons; built in rules diminishing

the risk of inconsistent responses and sensitivity analyses devices should be
incorporated. It should always be remembered that optimality of the output
is conditional to the consistency of the DM's responses.

• It should provide an efficient solution, even if the DM decides to stop.

Not many interaetive methods seem to follow these rules.

Daellenbach and Buchanan (1989) say that "interactive in MCDM should not
be taken as synonymous with interactive in a computer science context of real-time
response". French (1986) says "The adjective 'interactive' seems somewhat of a
misnomer. 'Sequential data gathering' might be more appropriate".
Anyway, we learn some useful lessons from these approaches, as we explain
below.

1.2.5 A general comment

We have reviewed some of the decision aids proposed as alternative to the Bayesian
ones. By means of examples, we have seen that they may violate some reasonable
principles. All of them were devised for problems under certainty, though some
have their corresponding versions under uncertainty. In general, they lack of a
strong qualitative axiomatic foundation. Considering the examples, it seems hard
to look for an axiomatic system allowing the shortcomings found in some of them.
In few words, we learn little from them, apart from realising the need to use a
strongly founded method, so as to avoid undesirable incoherences.
 21

The problem with some of the IDA's is different. Some of them share the 3)(-

iomatic foundations of Bayesian decision theory, but, in general, they include as

hypothesis very strong assumptions about the DM's preference system, which are
rarely tested. In any case, the goodness of their output is conditional to the con-
sistency of the DM's responses. This is frequently forgotten, which is problematic,
considering the difficulty of the questions often posed to the DM. Perhaps, the
most important lesson learnt from IDA's is their constructive approach to decision
making, which we shall follow below.

1.3 Our problem

The results reviewed in section 1.1 show the role of value, probability and utility
in dealing with decision making problems: any alternative approach has to violate
some of the principles expressed in those axioms.
In practice, some problems may arise while building v or u and p. This process
means that the DM's judgements are encoded by parameters w, the space of pa-
rameters being W, and that an evaluation function iI'( ai, w) = iI'i( w) is associated
with each ai. The 'parameters' may be parameters in the conventional sense when
v, u or p have parametric forms. Alternatively, some of them may be the values
of v, U or p at specific assessment points. We shall treat all problems in paral-
lel, whether they involve certainty or uncertainty and however the parameters are
defined.
The parameters w inherit appropriate uniqueness properties from v, or u and
p. However, it may be difficult to elicit the information to build them exactly.
Hence two, not necessarily mutually exclusive, cases arise:

• The DM is not able to locate w precisely, but she may give some constraints
on it, which we shall represent by wES ~ W. This we shall call decision
making under partial information .

• The DM gives an estimate W of w, but, feeling doubtful about it, she asks for
some help to identify which of her judgements are the critical ones in order
to think about them more elosely. In this case, sensitivity analysis comes to
the fore.

Ideally, the process of decision analysis is iterative:

 22

n=1

Input to the
y
analysis: Sn, W n

More
precision
Output of the Sensitivity or
analysis: a~ analysis different
structure
required?

This conception of decision analysis is similar to that of requisite decision

models, introduced by Phillips (1984).
We shall deal with those two intimately related problems. A brief review of
previous approaches follows.

1.4 Some previous thoughts

We shalllimit the review to those ideas that are related to our particular problem
or can be adapted to it. It leaves aside some related topics such as robust statistics.
Sage (1981) and Berger (1984) provide excellent complementary reviews. Some of
the points will be studied in greater detail in later sections.

1.4.1 The problem

It has been frequently argued that judgemental inputs are vague, see e.g. Savage
(1954) or Berger (1984). Reasons that are suggested include time pressure, lack of
knowledge, fear of committment, ... which leads to the impossibility cf obtaining all
the information required. Moreover, several authors (see Kahneman et al. (1982))
have dwelt on the errors that accompany the information elicitation process.
It is very important to note a fact. Many authors recognise the problems in
specifying the utility function (e.g. Hwang, 1985) or the probabilities (e.g. De
Robertis and Hartigan, 1981). Some authors advert the problems in both types of
information but skip them (e.g. Fishburn et al. , 1968), or despise somehow them
 23

(e.g. Berger, 1985). Very few authors try to deal with the problem in its fuH and
true generality (e.g. Kadane and Chuang, 1978).

1.4.2 Foundations of decision making und er partial infor-

mation
The points mentioned seem to challenge the practical adequacy of the Bayesian
approach to decision making. We have commented on this point. There has been
a lot of research trying to weaken those axioms. Our interest here is in modelling
quasi orders and the related partial orders.
For problems under certainty, the main result for modelling partial orders is
Spilzrajn's lemma, Fishburn (1970). See also the results in Roberts (1979). For
the finite case, there is a number of results on orders representable by additive
value funetions covering a given order (e.g. Krantz et al. (1971) or Adams (1965)).
These results are also valid in probability representation. In this area, there are
several representations for quasi orders with different aims and assumptions, see
e.g. Koopman (1940) or Giron and Rios (1980).
As long as utility is concerned, the two basic representations are the one way for
Archimedean quasi orders (Aumann, 1962) and the one way for non-Archimedean
partial orders (Kannai, 1963).

1.4.3 Ordering the alternatives. Decision making under

partial information
If we are dealing with families of parameters, not a single one, the solutions of
the problem are the nondominated ones. Both Pareto (e.g. Fishburn, 1965) and
weak Pareto (e.g. Hazen, 1986) solutions have been used. Algorithms for finding
these solutions with uncertainty either in the probabilities or in the utilities have
been proposed (e.g. Hazen, 1986). White et al. (1982) give slightly more general
algorithms. See also White (1982).

1.4.4 Additional criteria

TypicaHy, the set of nondominated actions is not a singleton. Many authors sug-
gest additional criteria to seleet one of them such as minimax, maximax, expected
value, Starr's domain criterion, ... , see e.g. Fishburn (1965), Charnetsky and Soland
(1978, 1979), Kmietowicz and Pearman (1981), Schneller and Sphicas (1983).
 24

Their diseussion is related to studies on decision making under uncertaintyl, see

e.g. 11ilnor (1954).

1.4.5 Hierarchical approaches

Hierarehieal approaehes are quite popular in Bayesian statisties, see Good (1980).
The rationale underlying them is that as we have a set of feasible parameters, we
then have a new decision problem, to be treated in the same way as the initial
one. See also the eomments in Berger (1985).

1.4.6 Detection of sensitivity

Sensitivity problems anse when there is no alternative whieh is optimal for every
wES: there are several quasi Bayesian (Rios, 1975), or potentially optimal
(Hazen, 1986) solutions. The identification of all of them is ealled global sensitivity
analysis (Leamer, 1978). This author talks also about loeal sensitivity analysis,
aceording to the derivatives of the optimal estimate with respect to the parameters,
in a statistical context. Another differential approach is that of Ramsay and Novick
(1980). More artisan ways to detect sensitivity problems eonsist of varying one
parameter at a time in discrete steps and observing the output of the analysis
(Edwards and Newman, 1982); or varying the probabilities in a fine lattiee around
the original ones and repeating the same proeedure for utilities (Srnith, 1988).

1.4.7 Sensitivity problems are not important

Some research tries to justify that sensitivity analysis is unnecessary.
One !ine of research is that of flat maxima (De Groot (1970) and Von Win-
terfeldt and Edwards (1987)): decision-theoretical maxima are Hat, taking into
account the eonvexity and boundedness of the maximum expected utility. We
shall refute this argument later.
Another approach is based on correlation analysis (Leung, 1978). This research
is almost only related to weighted additive models under certainty. The idea is
to eompare two models by calculating the correlation of their scores. A corre-
lation of 1 (-1) indicates total (dis)agreement. In general, correlations are high
when attribute intercorrelations are positive (Einhorn and Hogarth, 1975), but in
multi-attribute utility analysis intercorrelations are frequently negative (McClel-
land, 1978).

1 AB is generally understood.
 25

1.4.8 Sensitivity measures

The typical procedure to analyse the sensitivity of an alternative aj is to consider
the interval (minwEs 'l1 j (w),max wEs 'l1 j(w)) (e.g. De Robertis and Hartigan, 1981).
Other approaches include the €-posterior robustness (Berger, 1984) or the relative
expected loss (Von Winterfeldt and Edwards, 1982). Fishburn (1977) gives bounds
for approximations of bi-attribute value functions u(x, y) by functions of the form
v(x, y) = L, f;(x)g;(y).

1.4.9 Detection of competitors

One way of finding the competitors of a current optimal solution is to determine
the potentially optimal alternatives. We may think of ways of detecting competi-
tors when reasonable changes of parameters occur. For example, using distance
minimisation, see French and Rios Insua (1989).

1.4.10 What to do about sensitivity?

Possible approaches are: to get further information or to protect from sensitivity
in some way.
In the first case, as S is the cause of sensitivity, one tries to reduce S. Hierar-
chical approaches could be included here, but their aim is quite different.
In the second case, one tries to find alternatives which protect the DM from
bad results. These include search for S-maximin alternatives, S-maximin regret
alternatives or proceed according to arestricted Bayes approach (Berger, 1985).

1.4.11 Error modelling

Few authors have attempted to model errors in judgemental measurement. Some
references are Eliashberg and Hauser (1986), Stewart (1981) and Lindley (1986).
See also Fischhoff (1980).

1.4.12 Sensitivity analysis in commercial decision aids

Very limited sensitivity analysis tools are available in commercial decision aids. We
comment on four recent and successful aids: ARBORIST, SUPERTREE, HIVIEW
and VISA. The catalogue by Humphreys et al. (1987) gives brief descriptions of
the kind of sensitivity analysis, if any, in other aids.
 26

ARBORIST
ARBORIST, see Samson (1988), is a product of Texas Instruments to construct
and analyse decision trees of up to 1000 nodes. Its rnain strength is its user-
friendliness.
Sensitivity analyses can be performed with respect to one parameter. The user
defines a range of values around the current value of the parameter and trial values
within this range. Expected values are computed for each of these values. Results
are plot ted.

SUPERTREE
SUPERTREE, see McNamee and Celona (1987), is designed by Strategie Decisions
Group to build and analyse decision trees. Some fiexibility in defining the utility
function is available, including a built in exponential utility function.
The Sensitivity option of SUPERTREE al10ws two sorts of one-dimensional
sensitivityanalyses:

• The Probability command plots the expected utility of the alternatives against
changes in the probability at a certain node. If the node has three or more
branches, the middle ones are set to zero probability. If the selected node
has probabilities that depend on another node, this dependence is deleted
and the same procedure is followed .

• The Risk-attitude command plots the expected utility of the alternatives

against changes in the exponential parameter, when the built in exponential
function is chosen.

This is done by multiple tree evaluations.

SENSITIVITY, SUPERTREE's companion program, allows the user to in-
vestigate the relationships between the elements of adecision analysis. For all
important variables, a range of values is established: base case, low and high val-
ues. The model is first evaluated at their base case values. Then, for each of the
values in turn, the model is evaluated at the low value of the variable and then at
the high value (with all the other parameters at their base case) , yielding low and
high values of the model, which are plotted. This gives a measure of how sensitive
the problem is to the uncertainty in that variable. SENSITIVITY automates this
process and includes a joint option to vary two or more variables together, all at
their low (high) values simultaneously.
 27

HIVIEW
HNIEW, see e.g. Barclay (1987), is a product of the LSE Decision Analysis Unit.
It handles, in a very nice way, multi-attribute models under certainty. Alternatives
are ranked according to a weighted additive value function

v(a) = L wivi(ai ).
i

HNIEW permits one-dimensional sensitivity analyses, presented by means of value

paths, where the value of the different alternatives are plotted against the variation
of the parameter (weight). A zoom system permits the user to study in detail
particular subranges of the parameter.

VISA

Belton and Vickers (1989) developed VISA (Visual Interactive Sensitivity Analy-
sis) adecision aid for problems under certainty, based on a simple multi-attribute
value function
v(a) = LWiai.
i
Their system incorporates two modes of sensitivity analysis. The first one is one-
dimensional sensitivity analyses displayed by line graphs as in HIVIEW. The sec-
ond are interactive sensitivity analyses, where the user inputs a new set of weights
and values are recomputed. Bar charts support the presentation.

A general comment

Though in some of them it is done in a very friendly way, only one-dimensional

sensitivity analyses and multiparameter variation by hand are available in the main
commercial decision aids. Moreover, they only deal with one sort of models.
Better and more general sensitivity analysis tools are sorely needed. This
demand has appeared repeatedly in the literature, see e.g. Goicoechea et al. (1982),
French (1986), Von Winterfeldt and Edwards (1987), Watson and Buede (1987)
or Belton and Vickers (1989).

1.5 Comments
The case is made for adopting a Bayesian approach to decision making. However,
new or more general tools are necessary to cope with decision problems under
partial information and sensitivity analysis in decision problems. These tools are
provided in the next chapters.
Chapter 2

Decision Making under Partial

Information: Theory and
Algorithms

The process of assessing v or u and p is essentially the encoding of the DM's judge-
mental inputs in parameters w E W, which inherit the corresponding uniqueness
properties. However, in practice the DM may not be ready or able to provide
the information necessary to locate w precisely. Instead, she may give some con-
straints on her judgements, which we shall represent by wES ~ W. We called
this decision making under partial information. We study it in this chapter.
We start by reviewing some basic results on quasi orders. Then, we give ax-
iomatic foundations for our problem: namely, how should the DM's judgements
be in our case. Briefly, the Bayesian approach to decision making assurnes that
comparative judgements follow a weak order, which leads to encoding judgemental
inputs in parameters w with some uniqueness properties. We shall see that quasi
orders regulating comparative judgements correspond to sets of feasible parame-
ters and, consequently, to constraints of our kind, wES. The problem is studied
for preference and belief modelling. A parametric model is introduced to analyse
problems under certainty and under uncertainty in parallel. This saves some effort
in defining solution concepts, designing algorithms, etc. Algorithms to compute
the solutions are provided for general problems and particularised for the most
common cases in practice: linear and bilinear models.
As the solutions proposed need not be unique, we analyse some ad hoc rules
proposed in the literature to solve the indeterminacy problem. We pay special
attention to hierarchical approaches.
 29

2.1 Some basic results

We give some definitions and results which weshali use repeatedly.
Let (A, R) be a binary relation. We say that:

Definition 2.1

• (A, R) is a quasi order if it is reflexive and transitive.

• (A, R) is a partial order if it is reflexive, transitive and antisymmetrie.

• (A, R) is a strict partial order if it is irreflexive and transitive.

Suppose (A, R) is a quasi order. Define a binary relation E on A by

aEb ~ (aRb and bRa).

Then, Eis an equivalence relation (Roberts, 1979). Let A* be the set of equivalence
classes. Define R* on A* as follows: let a,ß be two equivalence classes and a E a,
bE ßj then, aR*ß iff aRb. (A*, R*) is called reduction of (A, R) and is well-defined
(Roberts, 1979). Moreover,

Lemma 2.1 If (A, R) is a quasi order, (A*, R*) is a partial order.

Definition 2.2 Let (A, R), (A,S) be two bina7-Y relations. 5 is a covering of R
if a, bE A, aRb===}aSb. We write it R ~ S.

From now on, our binary relations shall be written (A, ::S). We shall say that

a -< bif a::sb,-,(b::sa),

a ~ b if a::sb, b::Sa.

2.2 Judgemental quasi orders

The axiomatic foundations of the Bayesian approach to decision making require
precision in judgements (preferences, risk attitudes, beliefs) which imply precision
in the numerical inputs (values, utilities, probabilities) to the analysis. In prae-
tice, however, the decision making process goes through a eyde of information
elicitation / computation / sensitivity analysis with more preeision at eaeh stage
and leading, if successful, to a stage where no more precision is required. In the
intermediate stages, we have to work with a family of values or utilities and prob-
abilities. This seems to dash with the Bayesian foundations: we have to ac count
for imprecision in judgements.
 30

We give results that provide axiomatic foundations for this case. To do this,
we modify slightly the Bayesian principles. Briefly, these require that comparative
judgements follow a weak order (plus some conditions). Our approach requires
that comparative judgements follow a quasi order (plus some similar conditions).
Values, utilities and probabilities are treated in this way. In summary, we suggest
how to model imprecision within a Bayesian context.
We start by modelling general preferences under certainty. Then, we analyse
additive models, which comprise additive preference and probability models. We
conclude by analysing preferences under uncertainty. An introduction to these
results may be seen in Rios !nsua (1989).
Several authors (see Rios (1975) and the references in Gardenfohrs and Sahlin
(1988)) have proposed to work with families of utilities and probabilities to account
for imprecision in judgements. However, there has not been much work from the
axiomatic viewpoint. Some authors, e.g. Fishburn (1970) or Fine (1973), have
provided axioms for related cases. References are given where pertinent.
The same problem with similar motivations has been dealt within the fuzzy set
framework, see e.g. Freeling (1980) or Watson et al. (1979).

2.2.1 General preference quasi orders under certainty

We analyse preference quasi orders under certainty. A will be a finite set of alter-
natives. The relation::; on A models the DM's preferences and is interpreted as
folIows: let ai, aj E A, 'ai::;aj' means 'ai is at most as preferred as a/.
We get a representation of partial orders and then, by reduction, a character-
isation of quasi orders. Roberts (1979) gives a characterisation of strict partial
orders. Spilzrajn's lemma, Fishburn (1970), and a result in Roubens and Vincke
(1985) lead to a representation similar to Roberts'. We could give a representation
of a quasi order by reduction to a strict partial order, but that would lead us to
order alternatives in a weak Pareto sense. Our approach leads to a Pareto order,
which will appear in the rest of the cases we study. Roubens and Vincke (1985)
give a one way representation of quasi orders. IBI will represent the cardinal of
the set B.
 31

Proposition 2.1 Let A be a set such that lAI = m and j arelaiion on A. 11

(A, j) is a partial order, there are m Iunctions 11,"" Im such thai

a. jaj {=} II(a.)::::; II(aj),I= 1, ... ,m

Ior a., aj E A.

Let us write A = {al, . .. , am }. For each a., define

I.(aj) = {1 ifada~
o otherwlse

Suppose that adaj. If ada. then adaj, thus Ik(a.) = Ik(aj). If "'(ada.),
then A(a.) = 0::::; min(O, 1) ::::; Ik(aj). Thus, Ma.) ::::; Ik(aj), Vk.
Suppose ..,(adaj). Then I.(aj) = 0< 1 = I.(a.). Thus, "'(A(a;) ::::; A(aj), Vk).
o

Example 2.1

Consider the partial order whose graph is

The values of the functions are

a c d e f 9 h
a 1 1 1 1 1 0
b 0 1 0 0 1 0 0 0
c 0 0 1 0 0 0 1 0
d 0 0 0 0 0 1 0
e 0 0 0 0 0 0 1 0
f 0 0 0 0 0 1 0 1 0
9 0 0 0 0 0 0 1 1 0
h 0 0 0 0 0 0 0 0
I 0 0 0 0 0 0 0
 32

To model a preference quasi order, we reduce it according to lemma 2.1 and

apply proposition 2.1.

Proposition 2.2 Let A be a finite set and ::::; a relation on A. (A, ::::;) is a quasi
order iff there are r real Junctions VI, .•. ,Vr such that

Jor ai, aj E A, Jor some rEN.

*' Clearly, ::::; is reflexive and transitive.

=} Let (A*,::::;*) be the reduction of (A,::::;). (A*,::::;*) is a partial order and A* is
finite. By proposition 2.1, there are r functions 11, ... , Ir such that

for a, ß E A·. Define the functions Vi on A, i = 1, ... , r, as folIows:

They are well-defined. Let a, bE A and suppose that a E a, bE ß, with a, ß E A*.

Then,

o
Observe that given VI, ... ,V r representing ::::; we have:

Corollary 2.1 FOrai,aj E A

*'* I: Alvl(ai) ~ I: Alvl(aj), \lAI, ... , Ar: AI 2': 0, I: AI = 1

r r r

ai::::; aj
1=1 1=1 1=1

2.2.2 Additive models

We study now two problems that are intimately related: modelling preferences
with families of additive value functions and modelling beliefs with families of
probability distributions. In the first case, we shall consider the possibilities of
imprecision in the weights or imprecision in the component value functions.
We shall relate the representation to a system of linear inequalities and study
its consistency. This technique has been used, among others, by Kraft et al. (1959),
Adams (1965), Scott (1964), Fishburn (1970, 1985) and Krantz et al. (1971). We
use a slightly different kind of systems. Our main condition is a modification of that
 33

of Seott (1964) and implies his and Tversky's (1967) eondition. We eharacterise
additive quasi orders, unifying one line of work (Tversky, Adams, Krantz et al. ,
... ), whieh studies when a relation can be eovered by another admitting an additive
representation, and the other (Seott, Kraft et al., ... ), whieh eharacterises additive
weak orders. Our approach includes the weighted additive ease.

Weighted additive preference modelling

Let A = IT:'=I A;, where IAil < 00. The relation ~ on A models the DM's prefer-
enees and is interpreted as before.

Definition 2.3 (A,~) is a weighted additive weak order (wawo) ij it admits a

representation
x~y {=? I:" A;Vi(X i) ::; I:" '\iVi(yi) (2.1 )
i=l

where x = (Xl, . .. , x"), y = (yl, ... , y") E A, Vi is areal junction on Ai and '\i is
areal number, i=l, ... ,no We shall write v~(x) = L:'\iVi(Xi).

Initially, we look for the representation (2.1). Suppose the DM is able to provide
the component value functions, but she eannot establish the weights A; preciselYj
however, she ean say that ,\ = ('\1"'" ,\") E A. Thus, we are representing ~ on
A as follows:
x~y {=? I:" '\iVi(X i) ::; I:" '\iVi(yi), V'\ E 11. (2.2)
i=l i=l

We provide axioms for this ease.

We assume that the elements of Ai are real numbers and ~ is monotonie in
the sense Xi ::; Yi, Vi===}x~y. Let

X
• = (~C;X X i)"i=l' x. = (rr}li.n x i)"i=l"
We assume that x. -< x'. If it were not, then x. ~ x, Vx E A, and the problem
would be trivial.
Reealling definition 1.1 and proposition 1.2, we shall modify the eondition SC2
there included. First, we give an example:
 34

Example 2.2

In R 2 , consider the sextets

[(2,1), (2, 2), (3,1), (4,1), (1,0), (1, 0)]

[(1,2), (1, 1), (3,0), (4,0), (2, 1), (2, 1)].

Then

[(2,1), (2,2), (3, 1), (4,1), (1, 0), (1, O)]E4+2[(1, 2), (1, 1), (3, 0), (4,0), (2, 1), (2,1)]

Imagine that (1,2)~(2,1), (1,1)~(2,2), (3,O)~(3,1), (4,Ob(4,1). ~ with the

property below would lead to (1,O)~(2, 1).

ky will designate the k-uple (y, y, ... , y). The modification ia:

Definition 2.4 (Property P) (A,~) satisfies property [P] if for all integers
I, k ~ 1 and for all XI, ..• ,X/, XI+I, YI, ••. , yl, YI+I E A such that Xi~Yi, i = 1, ... ,I
and

Proposition 2.3 If (A,~) admits a representation (2.2), it is reflexive, transitive

and satisfies property [P}.

Clearly, (A,~) is reflexive and transitive. Under the hypothesis of property [P],
V)" E A we have
I I
L V~(Xi) + kV~(XI+d = L V~(Yi) + kV~(YI+l)
I I

I I
LV~(Xi)::; LV,),(Y;)
I I

Definition 2.5 (A,~) is a weighted additive quasi order (waqo) if it is a quasi

order and satisfies property [P}.
 35

Proposition 2.4 (A,~) is a waqo iff it satisfies property P and xo~xo for some
Xo E A.

=> Obvious.
<== 'Vx E A, it is (xo,x)E2 (xo,x) and xo~xo, thus x~x and (A,::O is reflexive.
"Ix, y, z E A, (x, y, z)E3 (y, z, x); if x~y, y~z then x~z. Thus, (A,~) is transitive.
o

We shall use the following results and definitions from Rockafellar (1970).

Lemma 2.2 Let S be the system {blx > O,bjx ~ O,btx = o,x E Rn,i E I,j E
J, k E K}, where I, J, K are finite sets. S is consistent iff there are not numbers
J.!., J.!j, J.!~ with i E I,j E J, k E K, J.!" J.!j ~ 0 and at least one J.!' > 0 such that

LJ.!i b• + LJ.!:bj + LJ.!;bk = O.

I J K

Definition 2.6 An inequality böx ~ 0 is a consequenee of the system

{blx ~ 0, i EI}, I finite, if it is satisfied by every x whieh satisfies the system.

Lemma 2.3 (Farkas) An inequality böx ~ 0 is a consequence of the system

{blx ~ 0, i E I}, I finite, iff there exist non-negative real numbers J.!" i E I,
such that
LJ.!ibi = ba.
'EI

The main result of this section is

Theorem 2.1 Let A be as above. Let ~ be a monotonie binary relation on A sueh

that i. -< x·. Then, (A,~) is a waqo iff it is the intersection of (a finite number
of) wawo's.

<== See propositions 2.3 and 2.4.

=> We prove first that there is so me wawo ~' covering ~. The proof is based in
similar results mentioned above. The assertion is true if there are v;'s and Ai'S
such that

Suppose first that Ai = 1, 'Vi. Associate an inequality

with each comparison x~y. Associate the inequality

 36

with the comparison x. ~ x·. Write Ai = {x;, .. . ,x~;} and

Then, we ean rewrite the system as

{ {eb
r ~ O}r } S
eb. > 0

where the b's are vectors whose elements are in {-I, 0, I}. The assertion is true iff
S is consistent. Suppose S is inconsistent, then with K = 0 in lemma 2.2, there
are numbers /Ar ~ 0 and /A. > 0 such that

As the b's are vectors of rational components, the /A 's may be taken to be rationals,
and, consequently, integers. Also, at least one of the /Ar 's has to be positive. As
each inequality is related to a comparison xr :5Y" we deduce that

By property [P], x·:5x., which is a contradiction. So S is consistent: there is a

solution e of S, that is a wawo :5' covering:5. e ma.y be taken as a vector with
rational eomponents, as the components of the b's are rational.
Let us rewrite e = (VI, ... , Vn ). We shall study solutions of the kind

Thus, the original system is rewritten

L AiVi( xi) ~ L AiVi(yi)

L AiVi(X~) < L AiVi(X·i )
or
{ {E Ai Ei'~l ~~IVi(~n ~ O}r} S'
E Ai Ei'~l a~vi(xj) > 0
where a:· E {I, 0, -I}. Observe that the system is in the A's.
Let us call :5~, where A = (Al' ... ' An), to the weak order associated to

such that :5~ covers :5. Let A = P : :5 ~ :5~}. We know that A ::j:. 0 since
A= (1,1, ... ,1) E A.
 37

Let :::;' = n.~EA :::;,1.. :::;' is a waqo such that :::; ~ :::;'. Suppose x, y E A are
such that x #- y, x:::;'y : VA E A, it is x:::;,l.y. Thus, if we write

a.s
I>i f>~yVi(xD ~ 0,
1=1
with a:: E {-1,0, 1}, this inequality is a consequence of the modified system
{ Ü::: Ai Er~l a~'vi( xi) ~ °O}' } S"
E Ai Er~l a~vi(xi) ~
(Observe that S" adds only to S' the equation
"i
~
l..J Ai ~ il i
l..J a, Vi( x,) = O.
1=1

When this happens, v,I.(x,) = V,I.(x'), so V,I.(x) = V,I.(y), Vx,y).

For simplicity, we shall rewrite this saying that

is a consequence of

{ {Ab. ~ O}. } S"

Ab, ~ 0
By Farkas' lemma, there are non-negative numbers JL., JL, such that

As brv #- 0, at least one of the J.I'S has to be positive. As the b's are vectors with
rational components, the J.I'S may be chosen rational. Thus, there are non-negative
integers ß., ß" ßry, with ßrv and at least one of the other ß's positive such that

This implies that there are Xi, Yi E A, i = 1, ... ,.9, with .9 > 0, such that xi:::! Yi, Vi
and
(Xl' ... , x., ßXYY )E,+ß•• (Y1, ... , Y., ßXYX)'
Then, by property P, x:::;y. That is, :::;. ~ :::;.
The :::;,I.'S are weak orders in A, which is finite. Thus, we may choose Al, ... , At
such that
t
:::;=n:::;,l.i.
i=l
o

Given Al , ... , At a.s before, we easily have

 38

Corollary 2.2 Let (A,::s) be as above. Then

Clearly, the representation of ::S need not be unique, but every convex set of
weights determining a family of value functions representing ::S is contained in that
defined by the system 5'. We have the following result, which restates corollary
2.2 and this comment

Corollary 2.3 Let Ai be the set 01 weights defining the wawo ::SA' = ::Si' Let A be
the set 01 weights defining the waqo ::S. II::s = ni ::Si, A :l [U Ai].

From this discussion, we may raise two issues. The first one is how to generate the
representation. To do this, we have to set up the system of inequalities generating
the waqo. Sometimes, it may be helpful to apply some algorithm to generate the
vertices of a polyhedron. A review of this topic may be seen in Matheiss and Rubin
(1980). The second one is that of dimension theory as it is done for strict partial
orders, see Roberts (1979). The objective is to find the minimum nu mb er of wawo's
generating the waqo. The solution of the problem might save some computational
eifort, both in terms of generation and storage of::S. Assurne that we start with
pI, ... , At} generating, via ::SA" ::S. We can use a branch and bound technique to
find a smaller number of ::S.\'s generating ::S. To do this, if M designates the bound
(to the number of ::SA's generating ::S) found up to the moment, the backtracking
rule would be 'go back if the current number is M-1'. A possible approach is:

1. L<--0.

2. If all the extreme points of A have been found, go to 4. Else, find a new Ai.
L<--LUPT

3. If::s = n.\H ::S.\, go to 4. Else, go to 2.

4. Apply the branch and bound procedure to L.

These questions of dimension and generation of ::S have a practical interest and
deserve further study. The problem will be illustrated in the paragraph dedicated
to belief modelling.
Kote that, due to monotonicity, we shall have among our statements VA(X') ::;
VA(X) ::; VA(X')' "Ix. This enables us to assurne that Vi(X:) = O.
The next two paragraphs analyse cases that can be adapted to this one.
 39

Additive preference modelling

A has the same properties as before.

Definition 2.7 (A, j) is an additive weak order (awo) if it admits a representa-

tion
" "
xjy ~ L: Vi(X i ) :::; L: Vi(yi) (2.3)

where x = (x\ ... , x"), y = (y\ ... , y") E A and the Vi'S are functions on Ai.

Our initial rum is to find a representation like (2.3). However, the DM cannot
establish the functions Vi precisely. This case is similar to the previous one.

Definition 2.8 (A,~) is an additive quasi order (aqo) if it admits the represen-
tation
"
x~y ~ L:Vi(Xi):::; L:Vi(yi),VV E V
" (2.4)
i::::1 i=l

where Visa family of functions V = (Vl, V2, •• " v").

Theorem 2.2 Let A be as above. Let ~ be a monotonie relation on A such that

x. -< x'. (A,~) is an aqo iff it is the intersection of (a finite number of) awo's.

{= As in theorem 2.1.
* As in theorem 2.1, we associate a system

{
{cbr ~ O}r} S
cb. > 0

to j and prove the existence of an awo covering ~. Let V be the set of additive
value functions v whose associated order ~v covers ~. Define ~. = nvEV ~v' ~.
is an aqo covering j. Consider x, y E A such that x~'y, x =I y. We associate
an inequality cbxy ~ 0 with it. As in theorem 2.1, we see that this inequality is a
consequence of

Similarly, we prove that xjy. o

Observe that the only difference between both results is that once we have
proved there is a jv covering j, theorem 2.1 uses a system in the '\'s and theorem
2.2 uses a system in the v's (or c). We have also
 40

Corollary 2.4 Let Vi, .•. , VI be additive value functions such that

Then,
I I
x-j,y <=? 'EAiVi(X)-j,'E.\;vi(y),VA: A ~ O,'EAi = 1.
i=l i=l

Scott (1964) gives necessary and suflicient conditions for an awo when n = 2.
Fishburn (1970) gives necessary and sufficient conditions for an awo using -< as
the primitive relation and studies other kind of representations, different to our
vectorial one. Krantz et al. (1971) give sufficient conditions for the existence of
an awo covering an aqo.

Belief modelling

The previous results can be used to model beliefs.

With the rapid growth of interest in expert systems, decision support and ar-
tificial intelligence, much work is being done on the representation of uncertainty.
Some of this builds naturallyon probabilistic models. Some, alas, has looked at
probabilistic models and rejected them on the grounds that no judgemental input
can be as precise as demanded by these: we can only hope at best to delimit
possible values for a DM's subjective probability to a small set. Given this, it is
natural, perhaps, that theories of upper and lower probabilities have developed as
in Koopman (1940), who introduces them as a technical device to define proba-
bilitiesj Smith (1961), who states some initial properties in terms of betting odds,
leading to upper and lower betting odds, represents probabilities by systems of
inequalities and gives properties of upper and lower betting odds; Good (1962),
who states a number of quantitative properties that it seems natural to impose on
upper and lower probabilitiesj Dempster (1967), who gives some statistical appli-
cationsj or Suppes (1974), who gives a qualitative treatment of the problem, using
semiorders. Other references are Walley and Fine (1979) and Shafer (1976). In
general, their emphasis is on properties of lower and upper probabilities. In our
opinion, the emphasis should be on imprecision in beliefs leading us to work with a
family of probabilities. This view is shared in robust Bayesian analysis, see Berger
(1984). That these probabilities have upper and lower probabilities is interesting,
but no more.
We shall justify axiomatically this approach, characterising prob ability quasi
orders by systems of inequalities. Kraft et al. (1959), Scott (1964) and Tversky
(1967) use systems of inequalities to represent probability weak ordersj Fishburn
 41

(1969, 1985) uses them to represent probability se mi orders and interval orders. Our
result weakens Seott's axioms, giving a simple and elegant qualitative treatment
of probability quasi orders. Giron and Rios (1980) provide a eharacterisation in a
more general setting.
Let 8 = {1, ... , k} be the set of states (k ;:: 1), A the algebra of subsets
of 8 and P a probability distribution on (8, A). We assoeiate with eaeh subset
B E A its eharacteristie function XB. The k-uple (xB(i))7=1 is written XB. It
is p( B) = LEB Pi =L PixB. We ean apply the previous results to the case in
whieh the DM eannot establish her beliefs preeisely. Let us first reinterpret the
properties.
The relation will be written :::e and interpreted as in section 1.1. The mono-
tonieity property means here B <;;; C=*B:::eC. The eondition x. --< x' means
°--<e 8 . Property P is read:

Definition 2.9 (Property P) Let 8, A,:::e be as above. (A, :::e) satisfies prop-
erty P if for all integers I, r ;:: 1 and fo1' all Al, BI, ... , A1+I,B1+I E A, such that
Ai:::eBi, i = 1, ... ,1 and

XA , + ... + XA, + rXA'tl = XB, + ... + XB, + rXB'tl

Property P is interpreted in a similar fashion as axiom S3 in proposition 1.3.

We can also give a betting interpretation: suppose the DM has a linear utility of
money and she gets 1 peseta for eaeh Ai or Bi that obtains,i = 1, ... , land r
pesetas for obtaining A1+1 or B 1+1 • Then, if A;:::eB; for i = 1, ... , I it should be
B1+!:::eAI+!.

Definition 2.10 Let 8, A,:::e be as aboue. (A,:::e) is cal/ed a probability quasi

order (pqo) if it satisfies

1.0--<e8,0:::e B ,VBEA.

2. Prope1'ty P.

Proposition 2.5 Let 8, A,:::I be as above. If (A,:::e) is a pqo, it is a monotonie

quasi order. i'vloreover, :::/ satisfies all properties of a compamtive probability except
completeness.

• :::e is reflexive. VB E A. 0:::e0 and X0 + XB = \0 + XB, thus B:::eB.

 42

• -je is transitive. 'rIB, C, D E A it is XB + Xc + XD = Xc + XD + XB. Thus, if

B-jeC, C-jeD then B-jID .

• -je is monotonie. Suppose B ~ C and let D = C \ B. Then X0 + Xc =

XD + XB· As 0-jtD, B-jeC.

For the last part, we only have to prove that if B nD = C nD = 0, then

B-jeC ~ B n D-jtC n D. This is a eonsequenee of XBuD + Xc = XCuD + XB
and property P. o

We, then, have a result similar to theorems 2.1 and 2.2

Theorem 2.3 Let 0, A, -jt be as above. (A, -jl) is a pqo iJJ it is the intersection
0/ (a finite number 0/) probability weak orders.

The proof is analogous to that of theorem 2.2. Observe that now we assoeiate with
B-jeC an inequality

and with 0 -<I ° the inequality

As we told, we ean make PirOl = 0 and, ealling Pi(l) = pi, and p = (PI,"" Pk),
we rewrite the system as

{ {pbr ;:: O}r } S'

pb. > 0
where the br's are vectors of (-I,O,l)'s and b. is a vector of 1'so We ean proceed
now as in theorem 2.2. o

Theorem 2.3 can be rewritten

1 Let 0, A, -jt be as above. Then:

1.0 -<e O,0 -je B,'rIB E A.

2. Property P,

are equivalent to the existence of a (nonempty, finite) family P of probabilities p

such that
B-jeC ~ p(B) ~ p( Cl, 'rIp E P.

Again, we have:
 43

Corollary 2.5 LetP = {pl, ... ,pl},t ~ 1, be afamily ofprobabilities characler-

ising :jt on A. Then
t 1 1
B :jt C {=} L '\ipi(B) ~ L'\ipi(C), \1,\1> ... ,'\1: '\i ~ 0, L'\i = 1.
i=l i::::l i=l

We illustrate section 2.2.2 with the following example:

Example 2.3

Let e = {I, 2, 3, 4}. Consider the two pwo's associated to

pi p2
1 .1 .1
2 .2 .3
3 .3 .4
4 .4 .2

The pwo's and pqo graphs and systems are

1,2,3,4 1,2,3,4
2,3,4 2,3,4
1,3,4 1,2,3
1,2,4 3,4 2,3 1,3,4
1,2,3 2,4 3,4 1,2,4
2,3 1,4 1,3 2,4
1,3 4 2,3 3 1,2
1,2 3 2 1,4
2 4 4
1 1
e e

PI + P3 P4 PI + P4 P2
PI + P2 P3 PI + P2 P3
SI PI + P4 P2 + P3 S2 P2 + P4 PI + P3
Pi ~ 0 Pi ~ 0
LPi l(or > 0) LPi l(or > 0)
 44

P2 2: PI
P4 2: Pt
Pt +P2 P3
Pt +P4
2: P2
S12 Pt +P3 2: P4
P2 +P3 2: Pt +P4
P2 +P4 2: Pt +P3
Pi 2: 0
EPi l(or> 0)
The extreme points of the polyhedron defined by S12 are

a (0,1/3,1/3,1/3)
b (.1, .3, .4, .2)
e = (.1, .2, .3, .4)
d = (1/6,1/6,2/6,2/6)

b and e are the initial solutions. Observe that in this example [U1'il f; 1', where
1',1'i are the sets of probability distributions defining ~,~i'
We can also see how the algorithm proposed to generate ~t would work in this
example:

Trial M ~t generated?
a - no
a,d - no
a,d,e - no
a,d,e,b 4 yes
a,d,b 4 no
a,e 4 no
a,e,b 4 yes
e,b 3 yes
b 2 no

Some interesting solution eoneepts used in modelling beliefs are contained in

1'. For example, if no judgement is gathered about Band C (i.e., ....,(B~tC),
....,(C~tB)), it is sometimes assumed that p(A) = p(B). It is easy to see that there
are P E l' with this property. Note that if A ~t B, there is P E l' such that
p(A) < p(B), which we use in proposition 2.6.
 45

To end this section, we study conditional probabilities. As it happens with

probability weak orders, non unicity may lead to problems while defining proba-
bilities of events conditional on different events, see Fishburn (1986). However, in
our context, it is enough to deal with probabilities conditional on the same event.
BID represents the event B given D.

Proposition 2.6 Let G, A, ~l be as be/ore. Suppose (A, ~l) is a pqo. Assume

• For any B, C, D E A, such that 0 -<'( D, BID~lCID iJJ B n D~lC n D.

Then, there is a /amily P 0/ probabilities such that tor any B, C, D E A, with
o-<'( D,
BID~tCID {::=} p(BID) ~ p(CID), Vp E P : p(D) > 0
where p(BID) = p(B n D)jp(D).

As (A, ~() is a pqo, there is a family P of probabilities representing ~t. Then:

BID~lCID {::=} B n D~lC nD{::=} p(B n D) ~ p( C n D), Vp E P

Thus,
BID~(CID{::=}p(BID) ~ p(ClD), Vp E P : p(D) > o.
o

Therefore, Bayes' theorem holds in the pqo case.

2.2.3 Preference quasi orders under uncertainty

We study now how to model preferences when there is uncertainty involved in
the decision problem. We try to rank the alternatives by their expected utility.
As before, imprecision in the DM's judgements leads us to work with a family of
utility functions.
The space of consequences C will be finite. The set of probability measures
over C will be written Po(C). The DM's preferences over Po(C) are modelled by
the relation ~.. We shall represen t it

P~.q {::=} E(u,p)~E(u,q),VuEU (2.5)

where U is a family of utility functions.

Aumman (1962) considers the axioms:
 46

• AL j . is a quasi order.

• A2. For a E (0, 1),pj.q {=;o ap + (1- a)d.aq + (1- alT.

• A3. ap + (1 - alT -<. s, Va E (0, 1]==}...,(s -<. T).

He obtains the representation

p-<.q==}E(u,p) < E(u,q)

for so me real function u on C. Fishburn (1970, 1982) considers the axioms:

• Fl. -<. is transitive.

• F2. For a E (0, l),p-<.q {=;o ap + (1 - a)r-<.aq + (1 - a)r.

• F3. ap + (1 - a)T-<.aq + (1 - als, Va E (0, 1]==}...,(s -<. T).

He gets the representation

p-<.q==} E( u, p) < E( H, q)

for some real function u on C. \Vhite (1972) gives conditions to represent an order
in Rn bya finite family of linear value f\lnctions, which may bc translated to this
context.
The following simple result establishes the necessity of the main conditions we
shall use to get the representaLon (2.5).

Proposition 2.7 Suppose that (Po(e), j.) admits the repTesentation (2.5). Then,

• B 1. j . is a quasi order.

• B2. For a E (0, l),pj.q {=;o ap + (1 - a)rj.aq + (1 - a)r.

• B3. ap + (1 - a)Tj.aq + (1 - als, Va E (0, 1]==}rj.s.

We need the following results frorn Rockafellar (1970).

Definition 2.11 Given a set B, we call K(B) the minimal convex cone contain-
ing Band R( B) the set of rays geneTated by B.

Lemma 2.4 If B i8 convex, [{(B) =c R(B).

Lemma 2.5 Let C be a convex set of Rn other than Rnitselj. Then, there exists
a closed half-space containing C.
 47

As a first step, we shalilook for a one way utility representation as Aumman's or

Fishburn's. We substitute the continuity eondition (A3 or F3) by a nontriviality
eondition (B4): thus, eontinuity is not essential for a one way utility representation,
provided that we limit it to almost agreement with ~.. Our proof will be based
up to a point on Fishburn's (1982) proof (his results are marked n'), which in turn
develops Aumman's ideas. The proof of results n' is the same of results n, taking
into account the following simple lemma:

Lemma 2.6 B2=*F2.

Theorem 2.4 Let C, 'Po(C),~. be as before. Suppose that ~. satisfies B1, B2.
Then if:

• Br :3p', q' E 'Po(e) : p'-<.q',

there is areal junction u on C such that

Let us eali
D = {p - q : q-<.p}, E = {p - q : q~.p}.

It is E :J D # 0. We first prove that:

1. E is convex.
Let y, z E E. There are p, q, r, S such that q~.p, r~.s, y == p - q, z s - r.
Applying B2, we have

aq + (1 - a)r~.ap + (1 - a)r~.ap + (1 - a)s

for a E (0,1). Thus,

a(p - q) + (1 - a)(s - r) = ay + (1 - a)z E E.

Similarly,
1'. D is convex.
Extending the above argument by indudion we may prove that
2. aj > O,L:aj = 1,qj::S.pj=*L:ajQr:5.L:ajpj.
2'. aj> 0, L: aj = 1, qj-<.Pj=* L: ajqj-<. L: ajPJ.
Then:
3. a] > 0, L aj == 1, qd.pj, j == 1, ... , n - 1, L ajqj = L ajpj=*Pn~.qn'
Because of 2, we only have to prove the result for n = 2. If a E (0,1),
 48

aql + (1 - a)q2 = apl + (1 - a)p2 and qd.Pl we have : aql + (1 - a)P2::;.apl +

(1 - a)p2 = aql + (1 - a)q2, so that P2::;.q2' Similarly,
3'. aj > 0,2: aj = 1, qr-<.pj, j = 1, ... , n - 1,2: ajqj = 2: ajpj==:>pn -<.qn·
Let T = K(D), S = K(E). From 3, we get:
4. P, q E 'Po(C), P - q E S ==:>q::;.p.
We have that

with an > 0, qn::;.Pn' Then,

2)an/Cl:::an + l))qn + (l/(I>n + l))p =

L(an/(L an + l))Pn + (l/(L an + l))q.
As qn::;.Pn, applying 3, we get q::;.p. Hence we may write,
5. P, q E 'Po(C),p - q E E {:::=::} q::;.p.
Similarly,
5'. p,qE'Po(C),p-qED {:::=::} q-<.p.
We then get:
6. D = E\ -E.
H tEE \ -E, t E E==:>t = p - q, q::;.P, and -t rt E==:>"'(p::;.q). That is, q-<.p
and tED. Conversely, ift E D, t = p - q,q-<.P so that p- q E E and q- p rt E.
Hence, tEE \ -E.
As a consequence:
7. T = S\ -So
Then, S ~ R': otherwise, T = 5 \ -5 = 0. But T ~ 0 as D ~ 0. Thus, S is
a convex cone and S ~ R'. We can apply lemma 2.5. There is u ~ 0 such that
x E S==:>utx 2: O. Now, if C = {Cl,"" co}, let us call U(Ci) = Ui. We have:

E(u,q):::; E(u,p).

o
Unfortunately, it is not possible to characterise the cones S in terms of the half
spaces containing them, as can be seen with T = {x : X3 2: 0, Xl > 0, X2 > O}
and S = TU {O}. This causes difficulties in getting the representation (2.5). To
overcome these, we shall add axiom B3. Aumman (1964) provides a related result
of more complicated proof and under stronger conditions.
 49

Theorem 2.5 Let C, Po(C), j, be as before. Suppose that j , satisfies 81-83.

Then, there is a family U of real functions such that

pj,q {=} E(u,p) ~ E(u, q), Vu E U.

Let E be a.s in theorem 2.4. cl(E) is convex (and closed). Let u E cl(E). There is
Uo E E such that
auo+ (1- alu E E,Va E (0, 1J.

Define Un = anuo + (1- an)u, with 1 ::: an 1 o. We have Un = Pn - qn with qnj,Pn.

Then, there are convergent subsequences, say Pn • .!:, p', qn• .!:, q'. Consequently,

that is, u = p' - q'. Hence, as Uo = Po - qo, with qoj.po,

a(po - qo) + (1 - a)(p' - q') E E, Va E (0,1J

therefore
aqo + (1 - a)q'j,apo + (1 - alp', Va E (0, 1J.
By B3, q'j,p' and u E E. Thus, E is closed. Consequently, S is a closed, convex
set (cone) and it can be characterised by means of its support hyperplanes: there
is a family W of vectors such that

Then,

Defining Uw(Ci) = Wi and U = {u w :W E W} we get

qj,p {=} E(u, q) ~ E(u,p), Vu E U.

If S = R"W = 0 and q~, p, Vq,p E Po(C). If j , satisfies B4, W # 0. As in

the previous models,

Corollary 2.6 Let C, Po(C), j , be as before. Let U be a family of real functions

giving the representation (2.5). Then

pj,q {=} E(u,p)~E(u,q),VuE[UJ.

 50

We may think about the possibility of U being finite. In general, it is not

possible to ensure that, but we might reason as follows. Assurne the DM is only
interested, in fact, in ordering a finite number of lotteries p' = {PI, ... ,pt}. We
shall write Xij = Pi - Pj. If ....,(pd.p}), then Xji rf. Sand there is some Wji E W
such that W]iXji < o. These vectors form a finite family W' characterising ~. on
p': Pi~.Pj <==? Wtpi :::; WtPj, Vw E W', whenever Pi,Pj E P'.
To end this section, we summarise results concerning belief and preference
modelling. Imprecision in the DM's beliefs can be modelled with a family P of
probability distributions p. Consequently, we associate a family {p(ai)}pEp of
lotteries with each alternative ai. Alternatives are compared on the basis of the
associated lotteries. We assurne a preference relation ~ on A, represented by
'ai ~ aj' for ai, aj E A and interpreted as 'ai is at most as preferred as aj', given
by

Imprecision in preferences between lotteries leads to imprecision in preferences

between alternatives:

Theorem 2.6 Let A, C, e, A, ~l,~. be as in the previous propositions . Let P, U

be the corresponding /amilies 0/ probabilities and utilities. Let {p( ai)}PEP be the set
o/lotteries associated with each alternative ai. Let ~ be defined as above. Then,

ai ~ aj <==? E(u,p(ai)) :::; E(u,p(aj)), Vu E U, Vp E P.

We have
ai ~ aj <==? p(ai) ~. p(aj), Vp E P
<==? E(u,p(ai)) :::; E(u,p(aj)), Vu E U, Vp E P

The additive case

In most of the problems, consequences are multi-attributed. In this case, the

additive model is popular; the utility function is
n
u(c) = I>j(d).
j=l

We shall use proposition 1.5 to represent ~. by a family of additive utility func-

tions. Remember that given the probability distribution p, (pi, ... , pn) represent
its marginal distributions.
 51

Proposition 2.8 Let C <;;; Cl X ... X Cn , Po(C), j. be as before. Suppose that j.

satisfies E1-E3 and

• ES. For p, q E Po(C), pi == qi, i == 1, ... , n =*p~. q.

Then there is a family U' of additive utility funclions such that:

(2.6)

We know there is a family U of utility functions such that

qj.p {=;> E(u,p)::;E(u,q),VuEU.

Each u in U defines an order ju in Po(C):

qjuP {=;> E(u,p) ::; E(u,q).

Now, p,q E Po(C), pi == qi,i == 1, ... ,n =*p~. q, so

E(u,p) == E(u,q),Vu E U.

Hence, p ~u q, Vu EU. Thus, Vu EU, ju satisfies the hypothesis of proposition

1.5: there is an additive function u' associated to u such that

We get the result calling U' to the set of these functions. o

As before, we have

Corollary 2.7 Under the conditions of proposition 2.8, there is a convex family
U of additiveutility funclions giving the representation (2.6).

2.3 A parametrie model

The previous results show how to model imprecision in judgements by families of
values or utilities and probabilities. Problems under certainty and under uncer-
tainty can be treated in parallel if we introduce a convenient parametric represen-
tation, as we briefly illustratc.
 52

2.3.1 The certainty case

When (A,~) is a quasi order, ~ may be modelIed by

For a given v E V, let Wi = v(ai) and W = (WI, .. "W m ). Let 5 = {w: W =

v(A), v E V}. Typically,5 is defined by constraints of the kind Wi :S w]" We can
define the evaluation of the j-th alternative as Wj(w) = Wj.
When A = rr:~1 A;, under some additional conditions, the v's may be taken
additive or weighted additive. Then, the v's either differ in the component value
functions or the weights:

• For a given v E V, let w) = vi(aj) and W = (w}, ... ,w;n, ... ,w;:'). Let
5 be as before. 5 is now defined by constraints of the kind wj :S w! and
I: w! :S I: w) (one of the inequalities is strict). The evaluation of the j-th
alternative takes the form Wj(w) = c;w, where Cj is a vector of O's and 1's .

• For a given A E A, let Wi = A;, W = (WI,""W n ) and 5 = {w : W =

A,A E A}. 5 is aredefinition of A and is defined by constraints of the kind
I: WiVi( aj) :S I: WiVi( ail (one of the inequalities is strict). The evaluation of
the j-th alternative takes the form Wj(w) = cjw, with Cj = (v,(aj)).
Then, taking into account the results of the previous section, we can represent
~ as follows

2.3.2 The uncertainty case

When (A, ~l) is a probability quasi order, ~l can be modelIed by a family P of
probabilities p:
C~IB ~ p(C) :S p(E), Vp E P.
For a given PEP, let us call Wi = prO;), wlt = (WI,'" ,lOk) and 51 = {w i : wl =
p(G),p E P}. SI is defined by constraints of the kind dw i 2': 0, where the c's are
vectors of -1 's, O's and 1's, and a constraint I:~=I Wi > O.
Imprecision in preferences has been modelIed with a family U of utility func-
tions. For a given u EU, call Wk+1 = u( CI), w2t = (Wk+I, . .. , lOHs) and 52 = {W Z =
u(C),u EU}. Calling w = (w l ,W 2 ), the evaluation of the j-th alternative takes
the form
 53

where B j is a matrix of zeroes and ones. Calling 5 = 51 X 52, we have

as a consequence of theorem 2.6.

A similar result holds in the additive case under uncertainty. Observe that if
either probabilities or utilities are known precisely, we recover the linear case.
As we see, we can give the same parametric treatment, deduced from the
axioms, to problems under certainty and under uncertainty. Note also that, in
both cases, 5 may be assumed convex. We shall use this general parametric model
in the following sections. Many other cases that we have not studied axiomatically
can be subsumed in it. Some illustrations and examples will be given in section
2.6 and chapter 4.
In what follows, t will designate the dimension of the vector W of parameters.

2.4 Orders and solution concepts

The solutions of the problem are the nondominated ones.

Definition 2.13 aj is nondominated if there is no ak E A such that aj -< ak.

The main thrust of the parametric model is to order the alternatives in a Pareto
sense. We recall that

Clearly,

Proposition 2.9 aj dominates ak iff Wk( w) ~ Wj( W), Vw E Sand 3w E S such

that Wk(W) < Wj(w).

Observe that if the W/s are continuous there is a neighbourhood N( wo) of Wo such
that 'l1k(W) < 'l1 j (w), for every W E N(wo) n S.
It may seem appealing to propose as solutions those aj that maximise Wj( w)
for some w E 5. They have received several names in the literature, e. g. quasi-
Bayesian (Rios, 1975, in a context in which there is uncertainty in the probabilities)
and potentially optimal (Hazen, 1986, in an uncertain value or utility functions
context). We adopt the latter term.
 54

Definition 2.14 aj is potentially optimal (p.o.) if 'h(w) ::; 'l!j(w), Vak E A, for
some wES.

In principle, there is no reason to introduce p.o. solutions into adecision analysis

und er partial information. The DM needs only consider the nondominated solu-
tions. However, if we knew w for sure, we should propose those aj maximising
Wj(w); as we know that w E 5, our final solution will be among the potentially
optimal alternatives.
Part of this conflicts with some results in White (1982). He starts with a quasi
order (X,~) and calls a value function v consistent with ~ if it:

• partially agrees with -<, that is, for x, y E X

x -< y===>v(x) < v(y),

and

• almost agrees with ~, that is, for x, y EX

x~y===>v(x) ::; v(y).

Spilzrajn's lemma (Fishburn, 1970) provides the basis for the existence of a con-
sistent value function in White's sense. V will designate the set of all of them.
We only demand the second of the above conditions; proposition 2.2 provides the
basis for the existence of a family V of almost agreeing value functions character-
ising ~. Other results in this chapter provide characterisations of ~ under special
conditions.
Differences between both approaches are not dramatic. The second one extends
Pareto order in Rn and allows characterisations in more cases (e.g. belief quasi
orders or preference quasi orders under uncertainty); however it cannot cope in a
simple way with weak Pareto order in Rn (this is more a technical rather than a
practical shortcoming). The first one can deal with Pareto and weak Pareto orders
very easily, via weighted additive sums.
In any case, the primitive solution set is that of nondominated alternatives,
[(X,~) in White's notation. In his framework, White (1980b) proves that if V is
the set of all consistent value functions (in his sense), this set coincides with

[(X, V) = {x )3y: v(x)::; v(y),Vv E V,v(x) < v(y), for some v E V}

and the set of p.O. alternatives. This does not happen if we work with sets V' C V
and motivates his discussion about the relations and use of those sets, which is
relevant to our case.
 55

Observe that with our representation

[(X,:5) = [(X, V).

Thus, our analysis will be in terms of p.o. and nondominated solutions. We shall
explore some relations between those eoneepts, taking into aeeount our par ametrie
representation.
Neither a p.O. alternative needs to be nondominated nor a nondominated al-
ternative needs to be p.O.

Example 2.4

In R Z , eonsider the example of figure 2.1. We assume a linear value function with
unknown weights and the constraints on the parameters defined by S = {Wl, W2 ~

O,Wl +wz = I}.

Figure 2.1: Potential optimality and dominance: relations

a4 is p.O. but dominated. az is nondominated but not p.O.

As A is finite, it is easy to prove (see White (1980a))

Proposition 2.10 Far each w, there is a nandaminated alternative which max-

imises 'l1 j (w).

The following results will be used in later seetions. We shall assume that the
W/s are continuous.
 56

Proposition 2.11 Let p. be a measure on S such that p.(A) > 0 for every set
A = N(w)nS , where w E S, N(w) is a neighbourhood ofw. If gj = J5 'l1 j(w)p.(dw)
exists (and is finite) for i = j, k and gj ~ g" I a" does not dominate aj.

Suppose a" doroinates aj. There is a set B C S such that p.(B) > 0 and 'l1,,(w) >
'l1 j(w), Vw E B. In Be, it is 'l1,,(w) ~ 'l1 j(w). So

h 'l1"dp. > h 'l1 j dp.

f 'l1 dp. ~ f 'l1 jdp.,

lBe k JBc
and, adding up the inequalities, we get

9" > 9j,

a contradiction. o

Proposition 2.12 Suppose that S is convex and bounded and a" does not domi-
nate aj. Then, there is a measure p. as in proposition 2.11 such that gj ~ 9".

Let f. be the dimension of aff(S). We can reparameterise the problem so that S

is a (bounded, convex) set of R t with nonempty interior. The result is obvious if
aj ~ a". Otherwise, if a" does not doroinate aj, there is an open set B C S such
> 'l1,,(w),Vw E B. Let 9'(C) = Je 'l1,djJ, with jJ the Lebesgue measure,
that 'l1 j(w)
C a measurable set. Then, 9j(B) > 9,,(B). Thus, for some m

Defining
p.(C) = k(m XB + XBc)djJ
for every measurable set CeS, we get the searched measure. o

We may ask whether for every nondoroinated alternative ai there is a measure

as before such that gi ~ 9", Vk. The answer is negative if we limit our attention
to absolutely continuous measures with respect to Lebesgue measure, as shown in
the example.
 57

Example 2.5

Consider a bi-attribute problem under certainty where the Wj'S are linear: Wi( w) =
w(a; - an + a; with w E [0,1]. Suppose there are three alternatives al = (1,0),
az = (0,1), a3 = (1/4,1/4). Let f be the density of}l. We may assume that
f~ f(w)dw = 1. We have:

gl = l wf(w)dw

g2 = l(l- w)f(w)dw = 1- gl
g3 = 1/4
and 0 ::; gl ::; 1. It cannot be

Under the conditions of proposition 2.12, the result is true for alternatives
which are single optimisers.

Proposition 2.13 Suppose that Wj (w) > Wk( w) I Vk f. j, for some wES. Then,
there is a measure }l such that gj 2: gk, Vk.

As above, there is an open set B C S such thaI W](w) > Wk(W),VW E B, Vk f. j

and there are numbers mk such that

Calling m = maxk(mk), we define }l as in proposition 2.12. o

We assume now that int(S) f. 0 and the w's are differentiablc.

Proposition 2.14 Suppose thalw E ml(S) and \V](w) = Wk(W). [f VWj(lV) f.

vih(lV) then ,(ak~aj).

Let g(w) = Wj(w) - Wk(W). If ak~aj, g(w) 2: O. Thus,w is a minimum of g, so

vg(w) = O. This implies vWj(w) = VWk(W), against the hypothesis. 0

We shall give counterexamplcs for thc case w E SS bclow. As a consequence

of the previous propositions, wc have the following relation between p.o. and non-
dominated alternatives.
 58

Corollary 2.8 Let W E 5 and suppose that aj is optimal for that W. Then:

1. If aj is the only optimiser, or

2. If W E int( 5), the \[I's are differentiable and for every k such that \[I j( w) =
\[Ik(W), V\[Ij(w) -# V\[Ik(W),
aj is nondominated.

Part 1 is immediate. To prove part 2, for those ak such that \[Ij(w) = \[Ik(W) apply
proposition 2.14; for thc others, part 1 applies. 0

These results generalise relations between Bayes and admissible solutions in

classical decision theory (see Berger, 1985). Their interest is enhanced by the
following simple observation:

Proposition 2.15 With 5 as in definition 2.12

Suppose that cl(int(5)) = cl(5). We can substitute 5 by int(5) and try to

apply corollary 2.8. When 5 is convex, we can substitute int by rio For instance,
in example 2.4 we can substitute 5 by

ri (5) = {w : Wj, W2 > 0, Wj + W2 = I}.

All the p.O. alternatives are nondominated; the set of nondominated alternatives
coincides with that of the example.
The final result generalises relations between Bayes and cardinal admissible
solutions in classical decision theory. <I> j (w) will designate the vector function
(\[Ii(W))iy!j.

Proposition 2.16 5uppose that 5 is convex and compact and \[Ij - /L<I>j is upper
semicontinuous and quasi concave for every /L E iVf = {/L : Li;ij /Li = 1, /Li :::: O}.
If aJ is not p.o., 3/L E 1',;[ such that

If aj is not p.O. then

thus
 59

Then, under the hypothesis of the proposition (see Stoer and Witzgall, 1970)

so for some J.L

The same proof applies when aj is not a single optimiser for any wES,
substituting ~ for <. Note that J.Lif!j is not always meaningful.
In view of the previous results, the search should be li mi ted to the nondomi-
nated alternatives which are potentially optimal. We adopt this approach in the
rest of the study.

2.5 General algorithms

We provide algorithms to generate those solutions, taking into account the para-
metric representation. We shall assurne that the mathematical programming prob-
lems proposed have solution. Conditions ensuring this, e.g. continuity of the ob-
jective function and compactness of the feasible region, are well-known, see Minoux
(1983). Some of these algorithms have been used in less general contexts; see e.g.
Fishburn (1965), White (1982), White et al. (1982), Hazen (1986) and Malakooti
(1989).

2.5.1 Nondominated alternatives

Determining whether ak ~ aj is equivalent to showing whether there is no wES
such that Wj( w) < Wk( w). Thus, there are several analytic ways available to tackle
the problem .

• Via mathematical programming formulations

Consider the problem

wES

Then, ak ~ aj iff its optimal value is greater than or equal to O. When the
optimal value is greater than 0, we know that ak -< aj. When it is 0, we should
check whether aj ~ ak. In many cases, however, this is not necessary. Several
distinct algorithms may be used to perform the minimisation problem.
 60

• Via eonsisteney- of a system of inequalities

Consider the system

Then, ak ~ aj iff the system is inconsistent. Again several algorithms may

be used to investigate the eonsisteney of that system. Comments similar to
those following [Pkj ] apply here.

As the dominance relation is astriet partial order, we would solve m( m - 1)

problems [hj] or [5 kj ] in the worst ease. Proposition 2.11 allows us some reduction.
Let /.l be a measure as in that result and compute gl,'" ,gm. Relabel the aj's so
that gl ~ ... ~ gm' Then ai cannot be dominated by a j for i < j.
The following forward algorithm generates the set B of nondominated alterna-
tives.

1. ft-O,Bt-0.

2. Set b(j) = 1, Vj E {I, ... , m}.

3. ft-f + 1. If C > m, stop.

4. If b(f) = 0, go to 3.

5. Bt-BU{at},b(C)t-O,st-l.

6. s+-s + 1. If s > m, go to 3.

7. Ifb(s)=0,got06.

8. If ai dominates a, (solving P,l or 5'l)' bIs) = 0.

9. Go to 6.

The algorithm ends in a finite number of steps. If in a eertain iteration we do not

stop at step 3, either we sha11 increase C in the next iteration because of passing
test 4 or we shall mark with 0 at least one alternative and increase e.
The algorithm generates all nondominated alternatives. At the first passage by
4, f = 1, b(C)=l and al is nondominated. We diseard all alternatives dominated
by al. Imagine that at the first k passages by test 4 negatively, we have generated
nondominated alternatives (and have discarded all the ones dominated by thcm).
If we no longer have negative answers to test 4, we have generated all of them.
Otherwise, at the (k+l)-th, al is nondominated. It cannot be dominated by those
 61

already in B, because in that case b(f)=O. It cannot be dominated by those dom-

inated by the on es in B, because dominance is transitive. It eannot be dominated
by the rest of alternatives aj, because for them it is ge 2:: gj. m(m -1)/2 problems
[Pkj ] or [Skj] have to be solved in the worst case.
Alternatively, we may use a backward algorithm:

1. B +-0, f+-O.

2. f+-f + 1.
3. If f > m, stop.

4. If ae is not dominated by any alternative in B +- B U{ ae}.

5. Go to 2.

The correctness of the algorithm is obvious. The number of problems to be solved

in the worst case is m(m - 1)/2.
Observe that we have to compute the g;'s to apply the algorithm. This may be
very time consuming. As an alternative, we eould use just one wES, rank the aj 's
according to Wj( w) and apply any of the algorithms, with the only modifieation
that if Wj( w) = Wk( w) we might have to solve both problems [Pkj ], [PJk ]. We would
solve m( m -1) problems in the worst ease but we can save some effort according to
corollary 2.8. SENSATO, the prototype library for sensitivity analysis described
in chapter 4, uses this approach with g, = W,(w), where w is an estimate of w
provided by the DM.

2.5.2 Potentially optimal alternatives

Two ways of finding whether an alternative aj is p.o. are:

• Via mathematical programmming formulations

Consider the problem

[Pj] mlll Zj

s. t.

Then, aj is p.o. iff the optimal value is S; o.

 62

• Via consistency of a system of inequalities

Consider the system

Then, aj is p.o. iff the system is consistent.

Again, we may use several algorithms to solve these problems.

Problem (Pj] provides the best case for aj. This may be excessive information.
The following problem is solved faster:

[Pi] min Zj
W ES,%j

s. t.

Zj ~ 0
Then, aj is p.o. iff the optimal value is O. For this, we are assuming that aj is not
the only p.o. alternative, the w's are continuous and S is connected.
In principle, we should have to study all these problems for each alternative
with each constraint. Some effort may be saved using the following observations:
• If ak is known to be not p.o., the constraint relative to ak may be dropped
from the problems Pj or Sj.
For example, if we are analysing the consistency of systems, if the system

is consistent, the system

Wh(W) - w;(w) ~ O,Vh f. j wES

is consistent. If it were not, for some w such that Wh (w) - Wj( w) ~ 0 Vh f. k,
it is Wk(W) - Wj(w) > 0, which implies that ak would be p.o., a contradiction.
• We also know that the set of nondominated solutions include maximisers for
every W E 5, see proposition 2.10. Consequently, the study may be limited
to this set, which may be seen in a similar way to the previous point. Notice
that we cannot generate all the maximisers for each W E 5 in this way.

• [Pi] provides two, or more, p.o. solutions.

• Sometimes an wES is available so we can rank the alternatives according

to Wj(w). Obviously, the corresponding optimal alternatives are p.O.
SENSATO uses the above rules in the computation of p.O. solutions, via [PJ
 63

2.6 Some particular cases

All discrete decision making problems under partial information fit into this frame-
work. However, some advantage can be taken if we consider the special features
of some common models.

2.6.1 Linear models

By linear models, we mean problems in which S is defined by linear constraints
and the lV's are linear in the parameters.
Linear programming has been extensively applied to some special cases of this
model before, as illustrated in example 2.6. The structure of the models introduced
in section 2.2 allow us to develop conditions which are efficiently checked. Some of
our results are similar to Hazen's (1986), but we do not have to appeal to Kuhn-
Fourier theorem, due to the different structure of our constraints. Observe also
that Hazen uses weak Pareto orders.
Other authors have developed algorithms for similar problems. The main ref-
erence is Fishburn (1965), who introduces the method of equating coefficients and
conditions for some special types of constraints. See also the references quoted in
White (1982) and Hazen (1986).

Example 2.6

Consider our initial problem given by adecision table. Suppose that the utilities
u(aii) = Uii (of action ai under state Bi) are known precisely but we only know
that probabilities satisfy some constraints of the kind

a S Pil + ... + pir S ß

+ ... + Pi. ~ Pll + ... + P'r
Pil
PI + ... + Pk > 0 (or I: Pi = 1)

We shall write them pEP. We have that IVj(p) = UjlPI + ... + UjkPk. The
problems we have to solve are:

• Is a, j aj?
Consider the LP problem

min IVj(p) -IV,(p) = (uil - Ull)PI + ... + (uik - U'k)Pk

pEP

Then, a, j aj iff its optimal value is greater than or equal to O.

 64

• 1s aj p.o.?
Consider the LP problem

mm Zj
P EP,z,

s. t.

Then, aj is p.O. iff the optimal value is ::; O.

Essential!y, this is the way to deal with linear problems. We consider now the
problems model!ed in section 2.2.

Additive models

We shall analyse the case

Wi( w) = c;w, Vi for the ranking functions

Cw ~ 0, ctw > 0 for the constraints.

We recal! first how it can arise.

Example 2.7

• Additive value functions.

The evaluations are
Wi(W) = Wi1 + ... + Win
There are some constraints due to monotonicity

so me constraints referred to comparisons

Wir + ... + Win ~ Wj1 + ... + Wjn

with a strict one to avoid triviality
 65

• Weighted additive value functions.

The evaluations are

There are some constraints referred to comparisons

with a strict one to avoid triviality

• Known utilities, unknown probabilities.

The evaluations are

There are some constraints referred to comparisons

Wil + ... + Wir 2: Wjl + ... + Wj.

and
Wl + ... + Wk > o.
• Known probabilities, unknown utilities (additive or general).
The evaluations are

There are some constraints referred to comparisons

with a strict one

These examples fit into the model we are considering, which we analyse now.
 66

• Is a/ j aj?
We have to find if

(Cj - c,)tw < 0

Cw ~ 0
ctw > 0
is consistent. We can weaken the last inequality:

- if imprecision is referred to probabilities, this means only adding the

solution Wi = 0, 'Vi, so that (Cj - c,)t w = 0,
- if imprecision is referred to preferences, this implies that the worst and
the best possible alternatives are equal in value, but then all alternatives
are equal in value. In particular, (Cj - c,)t w = O.

Thus, equivalently, we can solve the problem

B.t.

a/ j aj iff the optimal value is ~ o. The feasible set is conical: the optimal
value can be 0 or -00. Since w = 0 is a feasible solution, a/ j aj iff w =0
is an optimal solution .

• Is aj p.o.?
We have to find if the system

(Cj-C;)t w ~ O,i-j.j
Cw ~ 0
ctw > 0
is consistent. This can be studied solving the LP problem

s.t.
(Cj -Ci)t w ~ O,i-j.j
Cw ~ 0

Again, it is an LP problem with conical feasible region: its optimal value can
be 0 or 00. aj is p.o. iff w =0 is not optimal.
 67

Some general results

The following results relate dominance and potential optimality. They are partic-
ular instances of results from seetion 2.4.

Proposition 2.17 5uppose that w E int(5), Wi(W) = Wj(w) and Wi(W) 1= Wj(w)
for some W E 5. Then, ~(ai~aj), ~(ada;).

Suppose that ai~aj. Then, w is a minimum of Wj-Wi in 5. Thus, V(W j - Wi)( w) =

0, so that Ci = Cj and Wi(W) = Wj(W) Vw E 5, against the hypothesis. 0

Corollary 2.9 lf w E int(5) and WJ( w) ~ Wk( w), Vk, aj is nondominated.

We cannot ensure the same result if w E 85:

Example 2.8

Let Wj(w) = W, W2 (w) = 1 - w, 5 {(W .5::: W ::: 1}. We have that

Wj(.5) = Wz(.5) and az -< aj.

As we know, aj may be nondominated but not p.o. We have the following

corollary of proposition 2.16.

Corollary 2.10 Assume that 5 is defined as in the model and aj is not p.o. Then
there are Ai ~ 0 such that == Ai = 1 and

cjw ::: (L AiC;)W, Vw E 5.

i#j

2.6.2 Bilinear models

In bilinear models, the evaluation functions are bilinear and S is defined by lin-
ear constraints. Typically, they will correspond to cases of decision making nnder
uncertainty with imprecision in probabilities and utilities. White et al. (1982) de-
scribe how nondominated alternatives can be determined via linear programming,
under certain conditions.

Example 2.9

In adecision problem under uncertainty, Wj, W2 designate utilities and probabili-

ties, respectively. The evaluation functions are
 68

and the constraints are of the kind

C1Wl ?: a
C;Wl > 0
C2W2 ?: ß
c1 w2 > 0
See sections 2.3 and 2.6.1.

We analyse the case in which a = ß = O. Let us call W = (Wl,W2) and rewrite

the first and third groups of constraints as

CW?: o.
• Is al ::; aj?

We have to check whether the system

wi(Bj - BI)W2 < 0

Cw ?: 0
C;Wl > 0
C;W2 > 0

is consistent. As in the linear case, we can weaken the last inequalities. We

then solve

S.t.
Cw 2: 0
c; Wl ?: 0
c1w2 ?: 0

As W =0 is a feasible solution, 0 is an upper bound of the optimal value.

Thus, we have that al ::; aj iff W =0 is an optimal solution of the problem .

• Is aj p.o.?
We have to determine if the system

w;(Bk - Bj )W2 ::; 0, Ifk I- j

Cw ?: 0
C;Wl > 0
C~W2 > 0
 69

is consistent. We can solve

S.t.
w;(Bk - B j )W2 :S 0, Vk -# j
Cw 2: 0
C;WI 2: 0
C~W2 2: 0

o is a lower bound of the optimal value since w =0 is a feasible solution of

the problem. Thus, aj is p.o. iff w =0 is not optimal.

2.1 Additional criteria

The set of nondominated potentially optimal alternatives need not be a single-
ton, see the examples in chapter 4. Several authors (Fishburn (1965), Charnetski
and Soland (1978, 1979), Kmietowicz and Pearman (1981), ... ) propose additional
criteria to choose a single alternative: maximax, maximin, expected value, hy-
pervolume, ... All of them are discussed in relation to linear problems but they
can be easily extended to more general contexts. In fact, they are extensions of
classical criteria for decision making under uncertainty. So we could discuss them
partly along the lines of Milnor (1954), Schneller and Sphicas (1983) or French
(1986). We shall analyse their compatibility with the information provided by the
DM; White (1982) presents some discussion in this direction.
Alternatively, we can perform a hierarchical analysis as in Bayesian statistics,
see Good (1980) or Berger (1985). Its discussion will shed some light on the other
criteria.

2.7.1 Some additional criteria

Criteria used in decision making under uncertainty to choose a single alternative
may be extended to our context. <P(aj) will designate the evaluation of the j-th
alternative according to the corresponding criteria:
 70

• Maximin ~(aj) = min,.,eswj(w).

• Maximax ~(aj) = maxweS Wj(w).

• Maximin return ~(aj) = minwes(Wj(w) - maxi W;(W)).

• Laplace ~(aj) = fs Wj(w)dw.

• Domain ~(aj) = vol(Sj), where Sj = {W ES: Wj(w) :::: Wk(W), Vjp.

We assume the appropriate conditions ensuring the existence of the ~(aj)'s (e.g.,
continuity of the evaluation functions and compactness and convexity of the pa-
rameter set). Note also that the W;'s have to correspond to an interval scale and
that intervals be interpretable if we want the Laplace and maximin return crite-
ria to be meaningful, whereas for the other criteria we only require that the Wj 's
correspond to an ordinal scale. Schneller and Sphicas (1983) define the criteria
in their original context. Their example shows that they may lead to different
answers.
None of these solutions are suggested by the proposed axioms: these only imply
that we may associate a certain parametric model to the problem, the solutions be-
ing the nondominated ones; potentially optimal alternatives seem also acceptable,
both on intuitive grounds and because of their relations with the nondominated
ones. It is natural, then, to study the compatibility of the above criteria with
this framework. Proofs and counterexamples will not be provided, since they are
simple.

• Compatibility with ~. We quest ion whether

This corresponds to OUf not ion of consistent value function, see the discussion
in section 2.4. Clearly, all the criteria are compatible with ~.

• Compatibility with the original solution concepts. The correspond-

ing solutions should be p.o. nondominated. Clearly, Laplace and domain
solutions are nondominated and there are nondominated solutions among
the minimax, the maximax and the maximin return solutions. Only the
maximax and the domain solutions are p.o.

lThese sets are studied in chapter 3.

 71

• Compatibility with the parametric model. We question if there is

wES such that

Only the Laplace solution satisfies this criteria, under certain conditions (sec
2.7.2; if this is the case, Laplace solutions are potentially optimal).

According to the information provided, the DM cannot find any of the p.O. non-
dominated alternatives better than the others. If a single alternative has to be
chosen with the available information, any of them would do as weil. None of
those rules are the right one; pretending the contrary implies assuming informa-
tion that might be incompatible with that available. In short, these criteria are
ad hoc.

2.7.2 Hierarchical approaches

Hierarchical approaches are used in Bayesian statistics to resolve uncertainty in
the elicitation of the prior distribution. The rationale underlying them is that as
there is a set of potential parameters, we have a new problem, to be treated in the
same way as the primary one.
Specifically, each aj has an associated evaluation function Wj. Since we know
that wES, there is uncertainty about w. To carry out a Bayesian approach, we
assurne a (possibly second order) random variable ( over S. Thus, we associate a
lottery, with distribution ((.) ovcr the prizes Wj(.), with each aj; we compare the
alternatives via their expected utility: we have to build a (possibly second order)
utility U such that

where -:::.' is a preference order on A, such that -:::. c:; -:::". Obviously, we have to
be careful in the elicitation of U and (, since the choice depends on them. This
points out the main problem. We are assuming that the DM is unable to provide
more information on w in the primary problem. Observe that :

• ( and U are defined over presumably quite abstract objects for the DM,

• the assessments are now continuous.

In summary, this does not seem the best way to help the D?vI think more carefully
about her problem. It is very likely that we end up with imprccision in thc second
 72

stage, that is, knowing only that ( E l:, U E l' and

and the starting of an infinite regress.

In spite of that, we shall assume that the DM is able to provide ( and U and
explore the consequences. Specifically, we study the compatibility of this approach
with the parametric model, as in the previous section. We consider several cases.

• Uncertainty in probabilities. We have a distribution (( W2) over the prob-

abilities W2 with which prizes Cj are received. This is equivalent to receiving
prizes Cj with probabilities E(W2). If S is convex, W2 = E(W2) is a probabil-
ity distribution in S: precision in the secondary problem implies precision in
the primary one.

• Uncertainty in utilities_ Prizes w; B j are received with probabilities p.

There is a distribution ((wd over Wl. Suppose the second order utility is
positive affine, that is U(z) = az + b, with a > 0; then, this is equivalent
to receiving prizes aE( wd' B j + b with probabilities p; if S is convex, Wl =
E( Wl) E Sand we have precision implied in the primary problem. A similar
reasoning applies to the case of uncertainty in value weights.

• Uncertainty in probabilities and utilities. We assign to each aj the lot-

tery with distribution (( Wl, W2) over prizes wi Bjw2. Assuming that the dis-
tributions over Wl and W2 are independent and that the second order utility
function is affine positive, we have

So, if S = SI X S2 is convex, E(wd E SI, E(W2) E S2 and we have precision

implied in the primary problem.

• General case. Assume that the secondary utility is increasing. Let tV =

E(w); if S is convex, wES. Try to approximate U(Wj(w)) by its first order
Taylor's expansion around w:

Discarding the error terms, we get

 73

As U is increasing, ranking the alternatives by E(U(Wj)) is equivalent, to a

first order approximation, to ranking them by Wj( tU): we get an approximate
implication of precision in the primary problem. Bounds can be given for
the approximation.

To sum up, at least to a certain approximation and under some conditions, a

hierarchical approach implies precision in the primary problem, where we assumed
imprecision. This suggests that hierarchical approaches are ad hoc. Their use
demands certain caution.
Other issues will be raised in the next chapter.

2.8 Comments
We have studied how to deal with decision making problems under partial infor-
mation. The same principle allows us to model preferences under certainty, beliefs
and preferences under uncertainty. All problems can be treated with the same
parametric model, whieh associates a function Wj( w) with each alternative aj and
includes some constraints wES on the parameters to represent the partial in-
formation. The solutions suggested by the model are the nondominated ones, in
the Pareto sense. Potential optimality is a reasonable concept whieh allows us
to reduce the final pool of solutions. Algorithms to compute those solutions are
provided. In summary, we have a more robust theory of decision making based
on a weaker set ofaxioms, but embodying coherence, since it essentially implies
carrying out a family of coherent decision analyses.
Very frequently, the set of solutions is not a singleton. To achieve further
reduction, the DM has to provide more information. This rules out the automatie
use of a number of ad hoc criteria. A hierarchical approach is a way of eliciting more
information; however, given the circumstances originating that sort of analyses, we
have to view them as ad hoc, as well.
The results of this chapter will be used in the next one to identify the possible
competitors of a current optimal solution, on a first phase of sensitivity analysis.
Illustrations of the previous ideas are given in chapter 4.
Chapter 3

Sensitivity Analysis in
Multi-objective Decision Making

The basic data in chapter 2 were the evaluation functions Wj and the constraints
wES. In addition, we now have an estimate W of w. We rank the alternatives
according to Wj(w). The current best alternative will be called a. and its evaluation
W.(w). However, we must be ready to criticise that ranking: sensitivity analysis
comes to the fore.
The first section introduces technical results about the maximum ranking func-
tion and the subsets of optimality. The need for sensitivity analysis tools is then
justified. An initial approach is to proceed according to a DPI problem and find
the possible competitors of a•. Certainly, sensitivity analyses are necessary if a.
has competitors, but several arguments try to justify their irrelevance, even in this
case. We present the most important of them, the Flat Maxima Principle, and
refute it on technical, practical and conceptual grounds. The message we convey is
that sensitivity analysis is a constructive way to clarify the DM's mind suggesting
to her the judgements she should consider more carefully. As a result, we mention
some properties that a sensitivity analysis tool should have.
Several concepts of adjacent potential optimality allow us to reduce further the
set of competitors of a •. Then, we introduce and analyse so me sensitivity analysis
tools based on gauge functions. Remarks on errors in judgemental measurement
lead to same other tools.
 75

3.1 Some basic results

We give some results about the maximum ranking function and the subsets of
optimality, which will support the later discussion. The 'i'j's are assumed contin-
uous. Topological results refer to the relative topology of S. [Bl will designate the
convex hull of B.

3.1.1 The maximum ranking function

We shall study properties of the maximum ranking function

'i'(W) = max'i'j(w).
]

As the 'i' /s are continuous, 'i' is continuous.

It is well-known (see e.g. De Groot (1970)) that if the 'i'j's are linear, 'i' is
convex (and piecewise linear). The following example shows that the property is
not true in more general cases.

Example 3.1

Suppose we have a bi-attribute problem under uncertainty, the utility function

being weighted additive. There is imprecision in probabilities (p) and weights (A)
with p,). E [0,1].
Alternative al is defined by the lottery

We have

Alternative a2 is defined by the lottery

We have

'i'(p, A) is not convex.

 76

11<

Figure 3.1: Graph of W

The example refers to abilinear model. We study these using the notation
W = (Wt,W2) and Wj(w) = Wj(Wt,W2) = W:BjW2. Thus, W(w) = W(Wt,W2). The
following result is easy:

Proposition 3.1 W(Wt, W2) is a piecewise linear convex function 0/ Wl (W2), for
a fixed W2 (wt}.

We can give abound, similar to that of convex functions, for this case. Its proof
is simple.

Proposition 3.2 Suppose that Wj(Wt, W2) = W:BjW2, Vj and J.L E [0,1]. Then

The following result contains several bounds for the differences of W.

Proposition 3.3 If the Wj 's are Cl in a compact set K such that S c K, W is

Lipschitzian.

Let w, Wo E S. For some j, k we have

so that

and

By the mean value theorem

 77

The result can be repeated for every pair (w, w') E Sand we get the bound

1'Ir(w) -1lI(w')1 ~ sup max 1'i7ll1 j (w")llw - w'l·

w"eS J

Calling J.I. = SUP","ESmaxj 1'i7'Ir j (w")I, which exists by hypothesis, we have

IIlI(w) - 'Ir(w')1 ~ J.l.lw - w'l·

The hypothesis about K may be strong. As 'Ir is subsmooth, see Clarke (1975),
we can use the following result:

Proposition 3.4 (Clarke,1975) 111 is su.bsmooth on an open set U C Rn, I is

locally Lipschitzian on U.

In any case, we can apply Rademacher's theorem,

Proposition 3.5 A locally Lipschitzian function I on an open set U C Rn is

differentiable except at a set 01 measu.re zero.

So 'Ir is differentiable except at a set of measure zero. For a given w, let J(w) =
{j: 'Irj(w)::: llI(w)}. Then, 111 is differentiable in w iff 'i7'Irj (w) = 'i7'Ir k (w),Vj,k E
J(w).
The theory of generalised subgradients, see Rockafellar (1981) or Clarke (1983),
gives a general concept of differentiability of 'Ir which will be useful. We need the
following results and definitions:

Definition 3.1 Let I be areal function defined on some vector space X. The
directional derivative 01 I at x in the direction y is

!,(x,y) = lim(f(x + ty) - I(x»ft,

110
if the limit exists.

Definition 3.2 Let I be a locally Lipschitzian reallu.nction defined on some vector

space X. The generalised directional derivative 01 I at x in the direction y ia

li (x,y) = limsup (f(x' + ty) - I(x'»ft.

t!O,xl-tr
 78

Definition 3.3 Let f be a locally Lipschitzian real function defined on some vector
space X. The generalised subgradient of f at x is

5f(x) = {z: Ji (x,y) 2: yt z , Vy}.

Proposition 3.6 (Danskin,1967) Let f = sUP'\EA <P,\, where A is a compact
topological space and each <P,\ is a differentiable function such that <P,\ (x) and
V<P,\(x) depend continuously on (A, x). Let J(x) = p : <P,\(x) = f(x)}. Then the
directional derivative of f at x in the direction y exists for all y and is given by
the formula
f'(x,y) = max ytV<P,\(x).
'\EJ(x)

Proposition 3.7 (Rockafellar,1981) If fis the function of Danskin's theorem,

then
fi(x,y) = f'(x,y), Vy,

and
5f(x) = [{V<P,\(X),A E J(x)}].
When f is eonvex, it is z E 5f(x)~f(x') 2: f(x) + (x'- x)tz,Vx', whieh is the
c1assieal definition of a subgradient. If f is a 'max' function

z E 5f(x) ~ f(x') 2: f(x) + (x' - x)t z + o(lx' - xl), Vx'.

Let us apply the previous results to our problem. Results 1 ean be used in
loeal directional approximations; results 3 ean be used to give bounds for the loss
of optimality; results 2 are prior to results 3:

• Linear case

1. The directional derivatives satisfy

2. The subgradients are

51I'(w) = [{cJ : jE J(w)}].

3. As II' is eonvex,

lI'(w') 2: lI'(w) + JEJ(w)

max (w' - w)t Cj •
 79

• Bilinear case

1. The directional derivatives satisfy

2. The subgradients are

3. The subgradients satisfy

W(w;,w~) 2: W(Wj,W2)+
((w; - wd, (w~ - W2))'( I: {Lj(BjW2, wiBj)) + o(lw - w'I),
jEJ(w)
with {Lj 2: 0, L {Lj = 1. Observe that, in this case, we can deduce
another kind of inequalities, taking into account the partial convexity
of W:
W(w;, w~) 2: W(w;, W2) + (w~ - W2)'02 2:
W(Wj, W2) + (w~ - W2)'02 + (w; - wdoj,
where 02 is a sllbgradient of the convex fllnction

Ww;(z) = max(w;'BJ)z
J

at W2 and Oj is a subgradient of the convex fllnction

at Wj. In fact, we can write

W(w;, w~) 2:

W(Wj, W2) + )EJ(w;,w,)

. max w;tBj(w~ - W2) + JEJ(Wl,W,)
. max (w; - wdBjW2'

• General case
We assume that 'VW)(w) is continllous Vj.

1. The dircctional derivatives satisfy

W'(w,y) = max yt'VWj(w).

JEJ(w)

2. The sllbgraclients are

OW(w) = [{'VWj(w): jE J(w)}].

3. The subgraclients z satisfy

W(w') 2: W(w) + (w' - w)t z + o(lw - w'I).

 80

3.1.2 The optimality subsets

The optimality subsets will play an important role in the theory, specially in
connection with concepts of adjacent potential optimality. We shall assurne that
S is convex.

Definition 3.4 The optimality subset Sj associated with aj is

The stnct optimulity subset Cj ussociated with aj is

Clearly, if aj is not p.o., Sj =0 and if aj is dominated, Sj C Sb for some k.

We shall assurne that we are working with p.o. nondominated alternatives (for
example, after having applied the results of chapter 2).

Proposition 3.8 Sj is closed.

Let wES and suppose Wn->W, with {w n } C Sj. Then, Ilij(w n ) 2: Ilik(w n ), Vn, Vk.
As the Ilik's are continuous, Ilij{w) 2: Ilik(w),Vk. So w E Sj. 0

Proposition 3.9 Cj C int(Sj).

This example shows that the inverse indusion is not true in general.

Example 3.2

Under the conditions of example 3.1, suppose there are two alternatives:

BI B2
a1 (1,1 ) (0,1)
a2 (1,0) (0,2)

The graph of SI and 52 is

 81

Figure 3.2: Graph of SI and S2

The thickest part belongs to SI nS2' In this case, we have int(S2) = {(p,'\ ):
p< 1/2}. So, int(S2) 1. C2.

The linear case

Now, Sj = {w ES: c;w ~ 4w, Vk}. Clearly, Sj is convex and, consequently,

connected. As only nondominated alternatives are considered:

Proposition 3.10 int(Sj) = Cj} i/Cj f. 0.

Suppose that w E int(Sj)/Cj . For some I f. j, c;w = clw ~ 4w, Vk. If aj does
not dominate al, there is w/ E S such that c;w/ < clw/. Then, [w/,w) C Sand
Vw" E [w/,w), c;w" < clw". This is against the hypothesis. 0

The general case (with bilinear examples)

Convexity and connectivity properties of Sj no longer hold in the general case.

Example 3.3

Und er the conditions of example 3.1, suppose there are three alternatives

BI B2
al (1,0) (0,1 )
a2 (25/51,25/51) (25/51,25/51)
a3 (2,-98) (-98,-198)

Then

Wl (p,,\) 1- p -,\ + 2p'\

W2(P, ,\) 25/51
W3 (p,'\) 100p + 100'\ - 198
 82

s,

Figure 3.3: Graph of the subsets of optimality.

The figure shows the graph of the subsets of optimality. The Sj's need be
neither convex nor connected.

Convexity requires that the segment linking any two points of Sj is in Sj. We
shall be interested in another property: is there a segment in Sj whose extremes
are a certain point of Sj and any point of Sj nSI, for some I? Example 3.3 allows
us to answer it negatively. It is easy to see that [(.5,0), w] ct. SI, Vw E SI nS3'
Under some conditions, there is a stronger result (proposition 3.11). We need
some previous resuIts and definitions.

Definition 3.5 h is a tangent to S at x if there are sequences {x n} C Sand

{An} C R+ such that Xn-4X and An(X n - x)-4h.

The set of tangents to S at x will be called T (S, x).

Lemma 3.1 Let S = {x : f(x) = O} with Ja CI function. Then T(S,x) = {d:

dlVf(x) = O}.
Proposition 3.11 Suppose that the 1l' j 's are CI. Let w', wES such that [w, w') C
Cj and 1l'j(w') = 1l'i(W') > 1l'k(w'),Vk ,;, j, i. Suppose that (w' - w)t(V1l'j(w')-
V1l'i(W')) ,;, O. Then there is a closed ball B(w,r) such that [w,w') C Cj,
Vw E B(w,r)nS.
 83

Suppose it is not true: there is a sequence {w n } C S convergent to w, such

that \In there are kn cf- j and wn E [wn,w') such that Il1k n(W n) = Il1 j(w n). As
alternatives are finite, we assurne that k n is the same, say kn = k \In. Let us write
wn = W' + fln(W n - w'), where fln E (0,1]. By compactness, this sequence has an
accumulation point w" E [w,w'] such that Il1 k(w") = Il1 j(w"). As [w,w') c Cj, it
must be w" = w', thus k = i. So, we have a sequence 1 {w n}, with wn cf- w', such
that

We would have a contradiction if w' were an isolated point of {w : ll1 i ( w) = Wj( w)}.
If it were not, since in addition

l/I-ln(w n - w') = lOn - w'-t u...' - w',

w - w' would be a tangent to {w: Wi(W) = Il1 j(w)} at W'j therefore

(w' -w)\'VWj(w') - 'Vll1i(W')) = 0,

which is against the hypothesis. o

3.2 Do we need sensitivity analysis? (Yes)

a. is optimal for w = w. However, the DM doubts about w: it has to be con-
templated as a tentative value, S being her set of uncertainty. The first question
which we have to answer is whether (and when) sensitivity analysis is necessary.

3.2.1 Strong optimality

A first possibility is to proceed according to a DPI problem with uncertainty set S
and find the nondominated alternatives. If a. is the only solution, further analysis
seems unnecessary. This motivates:

Definition 3.6 a. is strongly optimal (5.0.) if it is optimal \lw ES.

Eqllivalently, a. is s.o. if it dominates or is equivalent to any other alternative.

White (1982) calls those solutions perfeet. We could solve (m - 1) problems [p).]
to find if a. dominates all the aj's.

llt is a subsequence of {w"} but wo shall write it il$ this one.

 84

Alternatively, we may solve the nonsmoooth problem

min(W.(w) - maxWj(w)).
wES J

Let u' be its optimal value. Then, a. is s.o. iff u' ~ o.

Example 3.4

For the special models of section 2.6,

• In the linear case, we solve the problem

min(
w
c: W - max cjw)
J

S.t.
Cw~ 0

a. is s.o. iff w = 0 is optimal.

• In the bilinear case, we solve the problem

min(wiB.wz - maxw;Bjwz)
W J

S.t.
Cw ~ 0
CiWl ~ 0
c~wz ~ 0

a. is s.o. iff w = 0 is optimal.

In other cases, we could try several recent algorithms for nonsmooth optimisa-
tion, see Polak et al. (1983).
u' provides useful information: it is the (negative of the) maximum loss of
optimality due to choosing a. incorrectly. Let w be the optimal solution of the
problem; for some j it is

We may be tempted to use this as a sensitivity measure: if -u' is smalI, losses are
not big when a. is not the optimal alternative. The term close call (Bell et al.,
1988) defines that idea: two alternatives are a elose call if their utilities are elose.
This definition lacks rigour. 1s u' meaningful? Since it is a difference of evalu-
ations, it is meaningful if the w;'s correspond to a measurable value function (see
 85

French (1986) for this concept and its relation with various concepts of meaning-
fulness). Then, u' gets its meaning by comparison with differences in the same
value function, that is by comparisons of the kind

for some I, k. This leads us to define a relative measure. One possibility is

-u' j(lJIj(w) - ~n lJIi(w)),

Ir]

which would tell us, informally, how a. would perform if wwere the true parame-
ters. That quantity is invariant under affine transformations. We could study the
function
g(w) = (lII(w) -1II.(w))/(III(w) - m.in lII i (w)).
I

However, gis not very manageable. Von Winterfeldt and Edwards (1987) propose
studying a related function, the relative expected loss, but still this one has to be
stupied numerically.
These sorts of functions or measures have some shortcomings: it may not be
possible to relate their values from one phase of the analysis (when the parameter
set is Sn) to the following (when it is Sn+1); moreover, they do not suggest how
further analysis should be carried out.
We can think of two other approaches to try to explain the implications of u'
to the DM:

• Since we have identified aj as a competitor of a., we can present them to

the DM, iI;S a warning of a possible loss. Similarly, we can translate that loss
into differences in one of the attributes .

• We can use relative errors as in physical measurement, see Pugh and Winslow
(1966). Effectively, we are measuring the value/utility of the optimal alter-
native. lu'l represents the worst (absolute) error. We should consider the
relative error
lu·I/III(w).
We shall use an informal version of this approach, as illustrated in chapter 4: we
announce which is the maximum loss and where do the ratings of the incumbent
alternatives lie. This is far from rigorous but used consistently may have a didactic
effect on DM's.
This topic of relating closeness of alternatives to the preference structure is
very important and deserves much study. Bell et al. (1988) and White (1989)
rruse this issue, which is studied to some extent in Rios and Rios Insua, S. (1986).
 86

As we see, we can develop some 'quick' sensitivity measures for a •. However,

they give no suggestions of how furt her analysis should be carried out. We need
more constructive tools.

3.2.2 A Flat Maxima Principle?

We have seen how to determine wh ether a. has competitors, given the current
information wES. If this is the case, we shall argue that we need a constructive
way to explore the DM's mind, which is sensitivity analysis.
Some authors have suggested that, even in this case, sensitivity analyses are not
necessary. We have al ready mentioned one of the arguments, based on correlational
analysis. We shall analyse the most popular one: the Flat Maxima Principle
(FMP) (Von Winterfeldt and Edwards, 1982, 1987 and De Groot, 1970). Brießy,
the FMP says that in decision making problems under uncertainty with imprecision
in probabilities2, the convexity and continuity of the maximum expected utility
constrains severely the loss of optimality due to an incorrect choice of probabilities,
if we limit the analysis to nondominated alternatives.
Specifically, consider a problem in which 'lIj(w) = cjw. We know that the
maximum function 'lI is convex (and piecewise linear if the number of alternatives
is finite).

Example 3.5

Suppose S = {Wl,W2 2: 0 : Wl + W2 = 1}, which can be identified with [O,lJ.

The graph represents two ex am pies of W, the first one with a finite number of
alternatives, the second one with an infinite number.

--
-~------

Figure 3.4: Two examples of W

2The argument would apply to problems with imprecision in weights of an additive utilityj
value function
 87

If w were the value of the parameter, the DM would choose a •. If the actual
value were w, the loss of optimality would be

h(w) = W(w) - W.(w).

The FMP tries to justify that h( w) cannot be big under certain conditions. As-
suming that a. is nondominated, W.(w) is tangent to W(w) at w. If W is Cl we
have
(3.1 )

where ii; = (iiW j iiw;) Iw. The expression on the right is the first order Taylor
approximation to W(w) centered at w. So W.(w) approximates W(w) near w, the
error of the approximation being given by

h(w) = 1j2(w -w)tH(w -w),

where H is the hessian of Wevaluated at some w' E [w, w] (We assurne that W is
C2 ). If w is near wand W is smooth, that quantity will be small. A reasonable
conclusion is that sensitivity analyses do not seem to be of major importance in
practical decision analysis: the DM should choose a. without further study.
We raise several points.
Suppose W is strictly convex: H will be positive definite. If A. is the minimum
eigenvalue, we shall have

h(w) 21j2A.llw-wll.

Assurne that the Iw; - wd are smalI, but they are bigger than a certain L Then,

h(w) 2 1j2A.Eyn.

Then, as the number of parameters increases, the loss of optimality may be big
even if w is near wand W is smooth.
Moreover, W need not be smooth. For example, if therc is a finite number of
alternatives and none of them is s.o., W may be nondifferentiable at some set (of
zero measure). In that case, the argument leading to equation (3.1) is generalised
to the corner points, using directional derivatives. However, the loss of optimality
may be big if w is near a corner point.
Also, in our model, the expression 'w near w' is [ar from convincing: S repre-
sents our 'sure knowledge', so the study of h has to be done not only near w but
in all S.
But, perhaps, in our claim for a general approach to sensitivity analysis, the
most important point is that we should consider sensitivity both with respect to
 88

utilities and probabilities. Then, 'lI need not be convex, as we have seen in example
3.1. Thus, one of the basic arguments no longer holds. In connection with this,
suppose the 'lI/s are bilinear, say 'lIj(w) = wiBjw2. Let us study the variations of
g(w) = 'lIj(w) - 'lI.(w) = wi(Bj - B.)W2 =wiDjW2, calling Dj = Bj - B•. If we
limit the analysis to one type of parameter, as assumed by the FMP, and we vary
Wl to Wl + tl we have

Results are similar, if we limit the analysis to the second type of parameters.
However, making a general analysis we get

The variation may be bigger: if we make general analyses, greater sensitivity is

detected. Incidentally, traditional one dimensional sensitivity analyses miss the
effects of dimensionality and the interaction of different types of parameters.
The Flat Maxima Principle is intended to be a justification of empirical ob-
servations that decision making problems seem remarkably robust to reasonable
variations in the parameters of the problem, see Von Winterfeldt and Edwards
(1987). The examples in chapter 4, several of them from real applications, show
that this may not be true. See also the comments in Barron and Schmidt (1988a)
and White (1979).
Finally, turning to more conceptual matters, we should remember the aim of
decision analysis: to help the DM to think closely about her problem. At the
present stage in which we have the functions 'lI/s, the set S, the estimate wand
the current optimal alternative a. we need a means of suggesting ways to improve
the current status to the DM, easing her in her discomfort with w. We believe
that the DM's understanding is so enhanced by sensitivity analysis that eventual
empirical robustness becomes a side-issue.

3.2.3 Demands of a sensitivity analysis tool

As stated in the schema of chapter 1, we conceive of the decision analysis process as
an iterative cycle of stages of judgemental elicitation, computation and sensitivity
analysis. When we reach a sensitivity analysis phase, we assurne that the DM
cannot provide furt her information in the elicitation phase without further insight
and reflection. Sensitivity analysis may be viewed as a means of stimulating her to
pursue further thoughts on her problem. Thus, several properties that we should
demand of a sensitivity analysis tool are that it:
 89

• provides a sensitivity measure, which helps to identify sensitivity issues and

suggests ways to improve the current status,

• helps to identify critical judgements,

• helps to identify competitors of a.,

• uses only the current information (avoiding extraneous elements such as €-

neighbourhoods of w),

• should be easy to implement.

According to this, we should agree that the work introduced before fits par-
tially into this framework, but, certainly, the sensitivity measures were not very
constructive. It seems also that they only provide us with an uninformative list of
those alternatives to which the DM should confine her attention.
More constructive tools are explored in the next sections.

3.3 Adjacent potentially optimal solutions

We have introduced some theory and algorithms to find the good alternatives:
the p.o. nondominated ones. However, we are not taking into account information
concerning w or the optimality of a•. We shall introduce several concepts that allow
us to reduce the set of competitors to the current optimal alternative according
to the idea that its optimality has to be criticised via 'reasonable' perturbations
of w. The first concept is inspired by that of adjacent efficiency from vector linear
programming (see e.g. Yu (1985)). The others follow from it.

3.3.1 Adjacent potential optimality

We try to convey the idea that in order to criticise the optimality of a., we should
consider those aj's sharing optimality with a •.

Definition 3.7 aj is adjacent potentially optimal (a.p.o.) to a. if Sj ns. "# 0.

Observe that Sj has to be nonempty: aj has to be p.o.

Example 3.6

Consider the example of the figure with four bi-attributed alternatives. We suppose
that the value function is linear with unknown weights and the constraints on the
parameters defined by S = {Wj, Wz ~ 0, Wj + Wz = 1}.
 90

Figure 3.5: Potential optimality and dominance: relations

Suppose that a4 is the current optimal solution. Then, a2 is a.p.o. to a4, a, is

p.O. but not a.p.o. to a4 and a3 is not p.o.

Two ways of finding whether an alt~rnative aj is a.p.o. to a. are:

• Via mathematical programmming formulations

Consider the problem

[p;J max 'lIj(w) - 'lI.(w)

wES

S. t.
'lI;(W) - 'lI.(w):::; O,Vi

Then, aj is a.p.o. to a. iff the optimal value is O.

• Via consistency of a system of inequalities

Consider the system

[5;] 'lIj(w) - 'lI.(w) = 0

'lI;(w) - 'lI.(w) :::; O,i #j
wES

Then, aj is a.p.o. to a. iff the system is consistent.

Example 3.7

• Linear case. We have tc find if

 91

(Cj - C.)'w 0
(Ci - C.)'W ::; O,i -# j
Cw ~ 0
C'W > 0

is consistent. We can solve the problem

maxc'w
S.t.
(Cj - c.)'w = 0
(Ci - c.)'w ::; 0, i -# j
Cw ~ 0

Then, aj is a.p.o. 1.0 a. iff w =0 is not an opt.imal solution of the problem .

• Bilinear case. We havc 1.0 check whether the system

wi(Bj - B.)w2 0
wi(Bi - B.)W2 ::; 0, i -# j
Cw > 0
ciwI > 0
C;W2 > 0

is consistent. We can solve the problem

5.1.

wi(Bj - B.)w2 =0
wirB, - B.)w2::; O,i-#j
Cw ~ 0

ciWI ~ °
C~W2 ~ 0

Then, aj is a.p.o. 1.0 a. iff w =0 is not an optimal solution of the problem.

 92

3.3.2 Adjacent potential optimality according to w

As a motivation for the second concept, remember that S. need not be connected.
Therefore, we should consider alternatives sharing optimality with a. for some w
in the connected component of S. containing w.

Definition 3.8 Let T. be the connected component of S. such that w E T •. aj is

adjacent potentially optimal to a. according to w (apow) if Sj nT. i- 0.
This procedure characterises the apow alternatives to a •. Let w E T., y be such
that w + y E S. For each aj find

[P~j] min,\ = ,\j

s.t.
1l1 j (w + '\y) = 1l1.(w + '\y)
w+'\y ES
'\;:::0

Say ,\j = 00 if the problem is infeasible. Let ,\! = min'\j and suppose ,\! < 00.
The minimum is achieved at some J. Then, if S is convex, aJ is apow to a •.
The proof follows the same pattern of those in section 3.4, to which we refer.
Conversely, if aJ is apow to a., ,\! = ,\J for every w E T. nSJ.

3.3.3 Straight adjacent potential optimality

For the third concept, we perturb w in a fixed direction. As a motivation, if the
DM feels doubtful about w she may wish to explore sensitivity in the direction
{w + '\y,'\ ;::: O}, finding the alternatives shanng optimality with a. for some w in
that set.

Definition 3.9 aj is straight adjacent potentially optimal (sapo) to a. if w E

Sj ns., or if for some y there is ,\ > 0 such that [w, w + '\y) C C. and w + '\y E
SjnS•.

We can use problems [P!j] (see section 3.3.2) to find the sapo alternatives to a •.
More manageable procedures will be introduced in the next section, where this
concept is justified via perturbations of w within convex sets. Hansen et al. (1989)
have used problems similar to [P!j] in sensitivity analysis in multi-objective linear
programming.
 93

3.3.4 Relations between the concepts

We condude this section studying the relations between the concepts discussed
above. Let J., J J. denote the sets of apo, apow and sapo alternatives to
IM , a.,
respectively.

Proposition 3.12 J. C J C J•. If the problem is linear J.

IM = J = J•.
IM

To prove the first part just apply the definitions. For the second part, J. = J",
comes from the connectedness of S. and J. = J comes from its convexity.
IM 0

In summary, if the problem is linear the concepts are identical. The example
shows that this is not true in general.

Example 3.8

Consider the example 3.3 with a fourth alternative

Then, 1I'4(P,'\ ) = -200p + 200'\ - 198. We sketch the sub sets of optimality.

Figure 3.6: Subsets of optimality.

Suppose that w = (1,0). a. = a2, J. = {al,a4},J", = {ad. Suppose that

w = (.5,0). a. = al, JIM = {a2,a3},J, = {ad.

3.4 Some sensitivity tools

We introduce a dass of sensitivity analysis tools which comprises many of the
previous approaches. The study is carried out along the lines proposed in 3.2.3
and paralleIs that of Rios Insua and French (1990) for distance based tools. The
dass is then restricted to gauge function based tools with several advantages.
 94

3.4.1 A dass of sensitivity tools

It seems natural to base sensitivity tools on the differences w - w. Tools of this
sort are used in Isaacs (1963), Fishburn et al. (1968), Evans (1984), Schneller and
Sphicas (1985) (Euclidean distances, in problems with no constraints in probabil-
ities), Pierce and Folks (1969) (general distances, in problems with no constraints
in probabilities), Barron and Schmidt (1988 a, b) (entropy based tools with con-
straints in the weights of an additive value function) and French and Rios Insua
(1989) (general distances with constraints in any of the parameters).
Consider a dass of nonempty sets [( = {I(,}aE[O,oo) such that [(0 = {O},
5 C [(a for some er and int([(a) = Ua'<a [(a" The index er can be used to express
deviation from w, if we solve the problem

w E 5,0

s. t.
w -w E [(a

This gives us a Wj, the solution of problem [PKj ], for which aj and a. are indifferent
and which minimises er.
[( is associated to a function 9K : [(00---> R, whose minimum is attained at O.
We can solve the equivalent problem to [P Kj ]:

min 9K(W - w)
wES

s. t.

For this method to be of use, we have to restrict the dass [(. We shall consider
sets [(a which, in addition, are star-shaped with 0 a point of common visibility,
that is, if w E [(a, [w,O] C !{a (see Stoer and Witzgall, 1970). Then

Proposition 3.13 Assume that 5 is convex and the 'Ir j 's are continuous. Let erj
be the optimal value oJ[PKJ !f erj > 0, 'Irj(w) < 'Ir.(w), Vw E (w + [(a) n5, for
a < aj.

Suppose that wE (w + [(a)n5, a < a j and 'Ir j (lV) :::: 'Ir.(w). Then, as 5
is convex, [w,w] C 5 and 'Irj(w) < 'Ir.(w), there must be w' E [w,w) such that
'Ir}( w') = 'Ir. (w') and w' E (w + [(a) n5, against the definition of a]' 0
 95

Let p = minj aj. The solution of the problem is achieved at a certain J. Then:

Proposition 3.14 Assume that S is convex and the Wj 's are continuous. aJ is
sapo to a •.

If p = 0, the result is obvious. If p > 0, then by proposition 3.13 and

the properties of I<, [w,wJ) C C•. It is wJ(wJ) = w.(wJ) and, by continuity,
w.(wJ)::::: Wj(wJ),Vj. 0

We have that a ::; pimplies W.(w) ::::: Wj( w), Vj, Vw E (w+ I<",) nS. Therefore,
p is an absolute measure of the insensitivity of a •. Let us solve the problem

max a
w E S,a

s.t.

and call its optimal value 8 and its optimal solution w". Note that I<p is the
biggest subset in I< such that (w + I<p) nSeS. and I<. is the smallest subset in
I< such that Sc (w + I<.). So

R = vol(I<p)/vol(I<,)
may be seen as a relative measure of the insensitivity of a. to changes in w. It is
a surrogate of the measure vol(S.)/vol(S). As R may be difficult to compute, we
approximate it by
r=p/8.
We have:

• If ,. = 1, a. is completely insensitivc,

• If r = 0, a. is completely sensitive,

• For intermediate values, r monotonically increases with p.

r provides some guidance about how furt her analysis could bc carricd out. To
increase it, we may either increase p or decrease 8. To decrease 8, we have to
reduce S; we have a direction in which we might try the reduction: [w",w], which
gives 8. To increase p, we have to compare (holistically) aJ and a.; if the DM finds
a. superior, we may rule out aJ, thus incrcasing S. and, possibly, p. We should
be careful, however, with such holistic comparisons. Note that both actions have
 96

influence on the other effect: if we reduce S, we might eliminate SJ thus implicitly

discarding aJ; if we rule out aJ, we introduce the constraint WJ( w) - W.( w) :S 0,
thus reducing S. In any case, and perhaps this is the most important point, observe
that both actions imply further information elicitation. As a result of it, we might
arrive at a new wand a new a •.
Moreover, the approach helps us identify some critical judgements through WJ.

If a :S p, w.(w) 2 Wj(w) Vj. Thus, we do not have to worry about the optimality
of a., Vw E (w +K p) nS, but moving outside (w + K nS means that we need
p)

to worry. WJ marks, in asense, the point at which we need to begin to take care
and can be used as apointer to sensitivity issues.
This approach is very flexible: choices for K or gK are multiple. For example,
we can choose the system of balls of a distance or, equivalently, the distance as gK
as in French and Rios Insua (1989); if W is naturally decomposed as (Wj, wz), we
can choose systems K j , K z for the first and second subsets of parameters and use
the system K = K j X K z; if W refers to probabilities we can use relative information
measures, such as entropies.

3.4.2 Gauge function based sensitivity tools

We shall choose a particular dass based on gauge functions. Some definitions on
these functions follow. See Rockafellar (1970) for further details.
Let Ce R t be a convex set:

Definition 3.10 The gauge /unciion 0/ C is the /unciion ,(.IC) : R t ---> R+ de-
fined by ,(xIC) = inf{oo,'\ 2 0: X E '\C}.

Lemma 3.2 ,(.IC) is convex. [fC is closed,

{x: ,(xlCl :S'\} = '\C,if'\ -I- 0,

{x: ,(xlCl = O} = {y: X +'\y E C, '1,\ 2 0, xE C}.

We shall use a specific kind of gauge functions. To motivate the choice, remember
that we had to solve the problems [PZj ] to generate the sapo alternatives; the
DM may not find it easy to specify y. Problems [PKj ] provide the possibility
of solving simultaneously a family of those problems, as weil as to investigate
the perturbation of w within K. As we know, they lead to sapo alternatives
(proposition 3.14). We may wonder if the reciprocal result is true. The dass
we shall choose provides a positive answer under certain conditions. ~Ioreover,
 97

it leads to a simple formulation of problems, has some convenient computational

properties and uni fies several conventional approaches to sensitivity analyis.
In R t , consider a closed, convex, bounded polyhedron P such that 0 E int(P).
We can write P [{ql, ... ,qT}], where r::: t + 1. Then, we have, if we write
gp(x) = ,(xIP):
{x: gp(x) = O} = {O},
{x: gp(x) ::; A} = AP.
The system {APh~o plays the role of 1<, which we shall rename P. We shall see
first the simple formulation of problems in this case .

• gp(x) is computed solving

mm p=gp(x)
s.t.
x = p L \qi
Ai::: O,I:\ = l,p::: 0

So gp(x) can be easily defined. [Pp,] is formulated

mm gp(w-w)
wES

s.t.

We can rewrite it as

w E 5,>.,g)

s.t.

WJ(w) = W.(w)
Ai ::: 0, I: Ai = 1.

It is immediate to dcterminc if gj = O. If this is not the case, we can define

hJ = 1/gj and solve:
 98

max hJ
w E S,>.,h,

S.t.
hj(w - w ) = L )..,qi
Wj(W) = W.(w)
).., 2:: O,LAi = l,h j 2:: 0

which is simpler. Observe that gj is the minimum A such that the image
of the polyhedron w + P by the application z >---> W + A(Z - w) intersects
{w: Wj(w) = W.(w)}.
Similarly, for the maximum problem we solve

max 9
w E S,>.,g

S.t.
W -w =gLA iqi
A;2::0,LA;=I,g2:0

or, if 9 i= 0, defining h = l/g we solve:

min h
w E S,>',h

S.t.
h( W - w)= L Aiqi
A; 2:: O,LA; = l,h 2: 0

• pP and 8P play the role of [{p and [{., respectively. In this case, R can be
computed explicitly. P can be partitioned into r (t +1)-edra Pi, with a com-
mon ver tex O. Then val(P) = L val(P;), and val(P;) = l/t!ldet (Pt,··· ,pDI
where Pt, ... ,p~ are the nonzero vertices of Pi. We then have:

val(pP) = L val (pP;) = l/t! L Idet (pp:, . .. , ppDI =

pt/t!Lldet (pL ... ,pDI = /Lval(Pi ) = ptval(P).

Similarly, va/(8P) = 8tval(P). Thus, R = (p/W

 99

'Ne can provide two reciprocal results to proposition 3.14 with the aid of this
dass.

Proposition 3.15 Suppose that [w, w') C C., w' E Sj ns., the W's are CI and
w' - w is not a tangent 10 {w : Wk(W) = W.(w)} al w', Vk E J(w'). Then there is
a P such that aJ = aj.
Under the above conditions, there is r > 0 such that [w, w') C C. Vw E B(w, r) nS
(Proposition 3.11). Choose WI,.'" Wt E B(w, r) such that P +w = [WI"'" Wt, w']
is a (dosed, convex, bounded) polyhedron with 0 E int(P) and ((P + w) \
{w'} ) nSeC•. Thus, gj = 1 and for the rest of ak's gk 2: 1. 0

The next result is simple, taking into account the properties of convex sets.

Proposition 3.16 Suppose that aj is sapo to a. and S. is convex. Then, there is

P such that aJ = aj.

Thus, this dass has the interesting feature of allowing the characterisation of
sapo alternatives, under certain conditions.
It may be argued that the search should be limited to a certain region. As we
said, we should only use the available information, i.e. wES. It is expected that
through sensitivity analyses, the search might be limited to certain regions in the
next stages. Observe, however, that the solution is P-dependent.

Example 3.9

Suppose that S = {w E R3 : Wi ::: 0, LWi = I}, w = (.25, .5, .25) and there are

three p.o. alternatives al, a2 and a3. a. = a3 for w. S is partitioned into S2, S., SI
as explained below and illustrated in the flgure.
The line separating S, and S. is

W2 .8, tE [0, .2]

W3 .2 - t

Consider now a weighted Tchebycheff distance 3 : d( w, w) maXi(i'i 1 Wi - Wi I)

where Ei> o,Lei = 1. For w E S2nS.,

d(w,w) = max((.25 - t)e l , .3e2 , (.05 + t)e3 ).

3We see below that bath Tchebycheff and city-block distances are part of the class of functions
that we are considering.
 100

(0,0,1)

Figure 3,7: Partition of Sinto S2, S., SI'

Let us make f l = 1,f2 = f 3 = 0 (it is no longer a distance). Then, mind(w,w) =

min(.25 - t), t E [0, .2] = .05. Thus, if lIT1, l210, ldO, L: f; = 1 and d(ll, f 2, ( 3 )
represents the minimum distance, d(f l , f 2 , ( 3 ) -+ .05 . Similarly, if fllO, f 211, f 3 10,
L:f; = 1 then d(f l ,f2 ,f3 ) -+.3.
The li ne separating SI and S. is

W2 .25(1 - t), tE [0,1]

W3 .75(1 - t)

As before, d(f l ,f2,f3) -+ 0, if lI1I,l2l0,f3 10,L:f; = 1, and d(fl ,f2,f3) -+ .35, if

lllO,f211,l310, L:f; = l.
Thus, for some P's, al is the alternative chosen and for so me others, a2 is the
chosen one.

One way of mitigating against this is by using several polyhedra (PI,"" p.).
Much may be learnt in practice using several gauge functions to find the corre-
sponding a.p.o. alternatives, see French and Rios Insua (1989) and the examples
in chapter 4, specially section 4.2. However, it is interesting to carry efficiency
ideas one stage further, as follows: for each aj find (gfl, ... ,gf') = (gJ, ... ,gJ)
and determine the efficient alternatives of the problem

min(gJ, ... , gJ).

',EA

We have:

Proposition 3.17 Assume th,:: 5 i.s convex and the llIj's are continuous. An al-
ternative aj generated by the aJore procedure is either nondominated or dominated
by an alternative ak such thai g; = gi, i = 1, ... , s.
 101

Suppose that aj is dominated by ak. Then, if wj design at es the Wj for the i-th
polyhedron we have IJij(wj) = 1Ji.(wj) ::; IJik(wj). Thus, gj 2: gL Vi, but aj is
efficient for the problem, so gj = gL Vi. o

A possible approach is to choose a particular polyhedron P

and different sets of weights ).I"",).m > 0, L:).i = 1, such that we can use the
polyhedron PA1 ,... ,A m = [).IPI'''',).mPm]' This opens the problem of choosing P
and the weights.
There are at least four ways of choosing P. They are related to conventional
sensitivity analysis approaches. However, it seems much more constructive to use
them in our way. Notice also that they can be combined, fer example, if any of
the procedures proposed does not permit us to generate enough vertices to build
P.
1. Parameters ware generated randomly and the result of substituting w by w
in the output is observed. Alternatively, we propose to generate the vertices
of P randomly and proceed as above.

2. The DM suggests parameters Wi instead of w or directions Yi to perturb

w. Alternatively, we propose using the polyhedra P = [{pi(Wi - w)}] or
P = [{piY;}], where the Pi'S are positive numbers chosen to associate a
ver tex to each parameter or direction suggested by the DM, and proceed as
above.

3. w is perturbed in the steepest direction of lJi at wand possible changes in

the output are observed. When lJi is not differentiable at w, the analogous
procedure would be to perturb w in the direction -I, where / is the minimum
norm direction in 81lr(w) (see Clarke, 1983). In a case like that of the graph
the procedure would be useless: perturbations in the suggested diredion
lead to determine no change in the optimal alternative, though the problem
is very sensi ti ve.

---
 102

As W(w) = W.(w), it seems more useful to use the steepest direction of

at w. Or, if we label the j's as jl, ... ,jm-l so that Wj.(w) ~ Wj,(w) if
k S; I, we can use the steepest direction of W- Wj, , ... , W- Wj, to generate a
polyhedron P and proceed as above. If Wj, (w) = ... = Wj'+k (w) and i +k S; I
we do not have to worry about the possible nondifferentiability of W - Wj,'
When i + k > I, i S; 1 we can use the I - i steepest directions with sm aller
norm from the functions {W -Wj,},=i .... ,i+k.

4. The last approach looks for simple formulations. These are, perhaps, achieved
with Tchebycheff distance (whose polyhedron is P = [{(ai)l=I}], where ai E
{±1}) or city block distance (whose polyhedron is P = [{ ei, !i, i = 1, .. . , t}],
where eij = Oij, !ij = -Oi;' with Oij referring to Kronecker's delta).

We study in more detail the last approach, first seeing that, effectively, the prob-
lems have simpler formulation .

• For the weighted Tchebycheff distance, problem [Pp,] is reformulated

mlll U

w E S,U

S.t.
Wi-U/).i S;Wi

Wi + u/,\; ~ Wi

• For the weighted city block distances, problem [Pp,] is reformulated

mlll 'L(Ui + Vi)

w E S,u,V

S.t.

Ui,Vi ~ 0
Wj(W) = W.(w)

Observe, however that the P's corresponding to Tchebycheff and city-block

distanees are symmetrie. Thus, they cannot be used to generate all the sapo
alternatives, as shown in the example:
 103

Example 3.10

Suppose that 5 = {w E R~ : Wl +W2 = l},w = (.75, .25), 51 n5. = {(.9, .ln =

{y}, 5 2 n5. = {(.I,.9n = {x}. Then, for any distance d, d(w,x) > d(w,y), due
to symmetry.
x

w
y

Undoubtedly, this points to a shortcoming of using these distances only. How-

ever, the computational eifort may be less, which is an advantage. Moreover, since
the parameters Wj are likely to correspond cognitively to sensible judgemental en-
tities, e.g. the likelihood of a well-defined event, the ability of distances to focus
on each of the parameters in turn may be valuable.
There is also the problem of weights. Observe that the bigger the weight, the
smaller we shall try to make the corresponding deviation. If we are especially
concemed about some parameter we may try to allow it to behave as 'freely' as
possible, giving it a small weight. Also, if we are going to use several polyhedra
we can vary weights in a lattice.
In summary, we have seen some sensitivity analysis tools which give very useful
information for our task, as demanded in section 3.2.3. The power of these ideas
can be seen in chapter 4, where we describe SENSATO. The approach adopted
there is to let the DM choose between different lp-distances.

3.5 Error modelling in decision analysis

We have seen how to model adecision making problem with the data

obtaining a current optimal solution a., how to find the possible competitors of a.
and how to identify sensitivity issues using the information conveyed by sensitivity
measures.
We mentioned that one of the objectives of sensitivity analysis is to detect
errors in modelling. We have not dealt with this problem explicitly. However, we
 104

have studied it indirectly since the identified critical judgements may be used as
pointers to possible errors. The eventual robustness of the model may increase
our confidence in it. We briefly address the problem now. As a consequence, we
develop additional sensitivity tools. Section 4.6 explains how to use the previous
ideas to validate a specific model.
Three main topics in error analysis can be identified: sources, detection and
correction of errors. They can be referred to the three elements of our problem:
the functions iflj, the set Sand the estimate w. References to sources of errors are
Kahneman et al. (1982) and those quoted in Farquhar (1984).
The detection and correction of errors will depend on the elements under study.
Given the generality of our approach, problems concerning ifl j seem less impor-
tant (in our finite state, finite consequence spaces context) since we can assign a
parameter to each value (weight, utility, probability, value, ... ) appearing in the
analysis 3 j in later stages, we can reduce the number of parameters, when there is
more confidence in the model.
With respect to S, the problem is essentially one of studying the consistency of
a system of equations for which there is some theory (see Rockafellar (1970) and
Kohlberg (1983)) and algorithms (see Schnabel (1982)), though it is addressed
mainly to the convex case. Pratt and Meyer (1968) and Hazen (1986) analyse
the problem of consistency of preferences and Lindley et a1. (1979) the problem of
consistency of beliefs. Let us mention that mathematical progra.mming libraries
warn when a mathematical program is inconsistent and include devices to deter-
mine, so to speak, the 'nastiest' constraints. It may help to check consistency after
each constraint is added. If inconsistencies are detected then, probably, the only
remedy is to revise the constraints. We shall not study, however, these problems.
Useful references are Sage and White (1984) and White et a1. (1984).
The last problem refers to w. We test whether w rt S by substitution. When
this happens, we can appeal to some sort of projection, e.g. via weighted least
squares, or we can follow a Bayesian approach. Even when wES, we should view
it as a tentative value. We shall discuss these points.

3.5.1 Errors in judgemental modelling

It is sometimes suggested that value, utility and probability estimation are uncer-
tain. The elicitation process is one of measurement and, as such, involves an error:
we get an estimate w of the parameter w, distorted by an error (see the discussions

30f course, after having accepted a Bayesian approach.

 105

in Lindley et al. (1979) and in Wallsten and Budescu (1983)). Consequently, we

might be using an erroneous wand choosing a suboptimal alternative. We have
developed tools to attack this problem. We shall try to deal with it explicitly.
Several authors have studied the problem of errors in judgemental modelling
in decision analysis. Some central ideas are:
1. Stewart (1981, 1984, 1986) studies inconsistencies in the DM's preference
responses in problems under certainty and, mainly, with linear value functions
w(x,w) = Ew.x •. w is not known precisely.
In a Bayesian approach (1981), assuming that <Pr is the prior over w at stage
r and the DM states a. -< aj at the (r + l)-th, the beliefs about ware updated
according to Bayes' formula

Introducing convenient express ions for <pr(w) and P(aj -< a;), he manages to pre-
serve normality approximately at every stage. He ranks alternatives by E~,(wj).
In classical approaches (1984,1986), he develops a formula for the likelihood <I>(w)
of a statement {all -< al" ... ,al, -< al'+l} and estimates wby the maximum like-
lihood method (MLM). He either ranks the alternatives by 'l1 j (w) or defines the
set
<I> = {w: rp(w)f<P(w) 2: Rcr •• }
for some Her •• , finding the potentially optimal alternatives when w E <I>. He
incorporates these ideas into several interactive methods.
Chaterjee (1986) uses these approaches but his expressions for P(a. -< aj) and
<I>r( w) appear somewhat more natural.
2. The paper by Eliashberg and Hauser (1985) may be seen as classical utility
estimation. They assurne a utility function known up to some parameters u(.,w).
Due to errors, w is modelled as a random variable whose density f is known
up to several parameters'\. ~ is estimated by MLM. Once with f(w, ~), they
compute the distribution of u(x, w), of the expected utility and the prob ability of
choice of an alternative. They do this essentially for a uniparametrie univariate
logarithmic or potential utility function, where w is distributed either normally or
exponen tiall y.
3. Lindley (1986) deals with this problem from a Bayesian viewpoint. Suppose
that a DM has to choose between aj,a2' She does not know their utilities but
has beliefs about them measured in p( Uj , U2 ). If she performs adecision analysis
and VI, V2 are the assessed utilities, her beliefs over the U's are updated via Bayes'
 106

theorem

where P(Vl, V2!Ul, U2) is the distribution of the result of the analysis, before it is
done, given that the true utilities are U1 , U2 • The proposed action is that max-
imising E(U;lvl' V2). He also uses the variance of the assessed expected utility to
perform some simple sensitivity analyses. His study is limited to the uniattributed
case, and, essentially, to the normal case.
4. Laskey and Fischer (1987) assurne that preference assessments can be rep·
resented as

where rit, Sill eit are, respectively, the response, systematic part and error term at
time t associated to stimulus i. They use a least squares procedure to fit an additive
value function and report some experiments. A similar decomposition is used in
Hershey et al. (1982), to compare utility assessment methods, and Wallsten and
Budescu (1983), to compare prob ability assessment methods. Dyer and Ravinder
(1986), Ravinder et al. (1988) and Yoon (1989) use it to analyse or estimate the
variance of the assessed additive values, probabilities through decomposition and
multiplicative values, respectively. This is the typical error model and can be
seen in the framework of location parameter estimation models (see Diaconis and
Freedman, 1986) and hierarchicallinear models (see French, 1978). We shall be
interested in models where the systematic part is constrained; some of the few
references to this case are Casella and Strawderman (1981) and Berger (1980,
1985).

3.5.2 An error model

We shall use an error model adapted to our conception of the decision making
process: it considers the cyclic nature of decision analysis; it allows dependence
between errors and parameters; it considers the constraints on w; it is multivariate;
it is not meant to determine choice, just to support the analysis. To do this, we
make the decomposition w = w + v, where wES and v represents the error. We
assess the distributions of w and v, find the distribution of wlw and use it to test
the optimality of a•. The model is extended to several cycles of decision analysis.
Finally, we make some ad hoc suggestions about the distributions and derive the
corresponding results.
 107

One cycle model

Given that the estimate W of the parameter w is subject to error, our purpose is to
find a distribution on w, after having measured w. Let us decompose w = w + v,
where w is the parameter and v the measurement error. Prior to the elicitation,
the DM has beliefs ab out wES, measured by a density 7ro(w), and beliefs about
v. Possibly, the distribution of v depends on w; we call its density fl(vlw). Then,
the joint density of v, w is

f(v,w) = 7ro(w)fl(vlw).

Standard computations give

Example 3.11

Consider a problem in which

7 -4w
5+2w
2+7w

and w E [1/6,5/6]. All actions are p.o. nondorninated. Suppose that w has a
uniform distribution over S, i.e. w ~ U(S).

1/6 5/6

Figure 3.8: Graph of Sand the 'It /s

Suppose we measure w with result w = 19/30 and error v ~ N(O,l/4). a3 is

the current optimal solution, the ratings being

The updated density is

7ro(wI19/30) cx exp{ -8{w - 19/30)2)I[1/6.5/6J{W).

 108

Wh at do we do now?

We have the functions 'IIj(w), w, a distribution over wand a current optimal

solution a •. What can we do with this information?
One possibility is to follow a hierarchical approach. We have associated a
lottery, with density ?ro(.lw) over prizes 'IIj(w), with each alternative aj. This is a
similar case to that of hierarchical approaches. We should be careful in building
the second order utility. Lindley (1986) and Stewart (1981) use this approach,
with the identity as the secondary utility. Another possibility is to estimate w,
e.g. by the posterior mode or mean, and rank the alternatives by '11 j( w).
We believe that these approaches are somewhat misled in our case. If we use
them, we are forgetting that a. is current optimal: this is the information we want
to criticise. We think that the natural way to do this is via Bayesian hypothesis
testing of:

• Ho : a. is the optimal solution

• H1 : a. is not the optimal solution

Or, equivalently,

• Ho: wES.

For example, if the loss is 0-1, we reject Ho if

{ ?ro(wlw)dw < 1/2.

is.
We shall combine this approach with that developed in section 3.4. There,
we found a competitor iJ.J to a. with some 'optimality property' (e.g., minimum
distance). We suggested there that a way to improve the current status is via
comparison of a. and aJ. Given the current information wES, it is neither
aJ -( a. nor a. -( aJ, but we could test which of these two cases holds for the
parameter w. Thus, we test:

• Ho: aJ ~ a.

We should elicit the loss function to perform the test. This would imply that
the DM would need to provide further information. However, we shall use the
test in a purely exploratory way: since the result of the analysis does not depend
 109

directly on the test, the choice of the loss function does not seem so important.
The approach we propose is to carry out several tests. Then, we face a problem
similar to that of multiplicity in classical hypothesis testing, see Diaconis (1985):
if enough tests are computed, some of them will lead to accept Ho and others to
accept H j • Diaconis suggests several solutions but only that of replication seems
to apply in our case: rejecting Ho in one of the tests suggests that the analysis
is not deep enough and that the DM must think more profoundly; if there is no
rejection, we can use them as a support to the current status, in the event that
the DM does not want to provide further information.
Let us see the expressions of some tests:

• 0-1 1055 The criteria is: reject Ho if

or, equivalently,

• Loss of optimality Suppose

'iJJ(W) - 'iJ.(w)
'iJ.(w) - 'iJJ(w).

Then, we reject Ho if

or

• Regret loss Suppose

L(w, a.) = max('iJ.(w), 'iJ J(w)) - 'iJ.(w)

0 if 'iJ.(w) 2: wJ(w)
{
= wJ(w) - 'iJ.(w) otherwise
Similarly for L(w,aJ). We reject Ho if

E(L(w,a.)) > E(L(w,aJ))

or, after simple computations, if

This is the same result as before: the hierarchical criteria with the identity
as the second order utility.
 110

• Hierarchical loss of optimality If U is the seeond order utility, defining

L(w,a.) U(IltJ(w)) - U(Ilt.(w))

L(w,aJ) U(Ilt.(w)) - U(IltJ(w))

the eriteria is: reject Ho if

E(U(Ilt J )) > E(U(Ilt.))

the general hierarehical eriteria.

Given the above eomments, we shall earry out the tests with 0-1 and regret
losses, due to their easily interpretable expression.

Example 3.12

In the previous example, a2 is a.p.o. to a3' We have:

P(a3 = a.) = P(a2 :j a3) = 1 1ro(wlw)dw =

3/5
5/6
.45

Thus, with 0-1 loss we reject the current optimality of a3' With regret loss:

E(1lt 3 ) = f5/6(2 + 7w)1ro(wlw)dw <X 3.908

A/6

E(1lt 2) = 1 1/6
5/6
(5 + 2w)1ro(wlw)dw <X 4.467

Again, we reject the hypothesis that a3 is optimal. The sensitivity index r is 0.071.
All this demands furt her analysis.

Several cycles model

As we eonceive of the decision making process cyclically, we may modify the inputs
w n, Sn to the analysis. We need to extend the previous approach to several cycles.
We assume first that S remains unchanged, and then give a solution when S
ehanges.
At the seeond eyde, the prior distribution over w is

We measure w' = w + v', where v' is the error. The distribution over v' depends
possibly on v and wand we represent it with a density h{v'lv,w). Then we find
that
1r1(wlw') <X 1r1{w)h{w' - wlw - w,w).
 111

The formula is extended to several cycles.

Wh at happens if 5 changes from one cycle to the next? We shall make the
hypothesis that Sn+l C Sn, when we move from one cycle to the following. Sn
represents the DM's sure knowledge about w; later cycles of analysis should in-
crease her knowledge about w, thus reducing Sn. In this case, it is a consequence
of Bayes' theorem that if f was the density over w after the n-th cycle and we
know that the (n + l)-th support set is Sn+l' our prior density for this cycle is
proportional to

Summing up, if 7I"i_l represents the prior, Si-l the support, Wi the measurement
and fi(.lw,vl'" . ,Vi-tl the error density on the i-th cycle, i = 1, ... ,n + 1, we
have

and

It may be reasonable to assurne a sort of Markovian property for the errors such
as

and the pertinent changes are made in the formulas. Even the vis may be judged
independent given w, so that

We shall make this hypothesis from now on. Sometimes we can assurne that the
Vi 's are independent of w

The classic error model assurnes that the vis are i.i.d. and S R. Then, the
formula is

See Diaconis and Freedman (1986) for more details about it.
 112

Example 3.13

We determine now that w E [.5, .75]. Only a2 and a3 remain in the analysis. a3

is the current optimal solution and a2 is a.p.o. to a3. w = 5/8 is measured with
error v ~ N(O, 1/8). We get that:

<X exp( -40(w - 99/150)2)I[.5,.75J(W).

Then:

P(a3 = a.) <X 0.46

P(a3 -; a.) <X 0.27
E(W 3) 4.52
E(W 2 ) 4.43

We do not reject the current optimality of a3. The sensitivity index is 0.2. This
might all help to reassure the DM that the result of the analysis was not spurious.
The approximate graphs showing the changes in the distribution over ware

1/6 1/2 3/45/6

Figure 3.9: Densities in the three stages

3.5.3 Some thoughts on the elements of the problem

We shall consider some particular cases. We should devote some time to the proper
assessments in each analysis, although the following results may be of help in this
difficult problem, especially if we think about automated solutions. It is important
to note that we are not facing the same case as in hierarchical approaches, as the
problem is now different to the primary one.
 113

The W distribution

We only know that wES. It would be very strong to assurne that w is normally
distributed, as in Lindley (1986).
We justify the adoption of a uniform distribution over S. Apart from in-
variance arguments (see Villegas, 1977), the typical approach is that one chooses
the most intuitively reasonable parameterisation for the problem and a lack of
prior knowledge should correspond to a constant density in this parameterisation
(Berger, 1985). The counterargument (see O'Hagan, 1988) appeals to the use of
transformations invalidating the constancy of the density over S.
In our case, wES and we know nothing else so, assuming vo/(S) > 0, we put
as a density
7ro(w) = l/vol(S)Is(w).
Observe that our parameterisation is not only reasonable but also deduced from the
axioms introduced. Let us rehearse the transformation counterargument. When
the error distribution is independent of w, w is a location parameter of the dis-
tribution of w (see definitions in Berger, 1985). Then, the dass of admissible
tr ansformations is
T = {sc: sc( w) = w + C, Yw}.
This is so far as probability arguments are concerned. Our parameters also have
a meaning from the point of view of evaluation; to be valid, Sc has to verify

Consider a linear problem, with Wi(W) = clw. Suppose there are nonempty sets
E.,E,Ej eS such that
(c. - Cj )t w > 0, w E E.
(c. - c,)t w = 0, w E E

(c. - cdw < 0, w E E j

(This happens, e.g., if ~(ar~a.), ~(a.~aJ) and 5 is connected). Clearly, only those
admissible transformations verifying

are valid. In this way, we can generate a full rank system {(Cj-c.)t c = OL, possibly
introducing hypothetical alternatives with associated evaluation Wj(w) = c}w.
(This is valid since the DM's judgements apply to hypothetical alternatives). The
only solution of the system is c = 0: the only valid transformation is the idcntity.
 114

A similar argument applies when the iVj's are bilinear. In the general ease, we ean
apply it to the first order Taylor approximations.
When the error distribution is not independent of w, or more gene rally when
w is not a loeation parameter of w, we eannot limit the study to T. Then, we
ean appeal to the uniqueness properties of the probabilities, utilities and values.
For example, if w represents probabilities, w is unique so we eannot talk about
transformations of w apart from the identity; if the w's refer to utilities they are
affine unique but then the transformation eounterargument is not valid, as the
density of the new parameterisation is uniform as well. Similarly if the w's refer
to weights or additive values. The only ease whieh would not be eovered is that
of general values, whieh are unique up to a monotonie transformation. Then, we
have to appeal to the initial argument.
Onee we have 'justified' a uniform prior over S = So, the eorresponding formu-
las are

1l"n(w) (X IsJw)II;,;Ni(wi - wlw)

1l"n(wlwn) (X ls.(w)II?=di(wi - wlw)

The error distribution

We eonsider now the error distribution. The arguments given here are mueh
weaker. For each model, we eompute the posterior distribution and give the eor-
responding tests.

The uni dimensional case

A normal model Continuing with the physieal tradition, we shall ass urne
normal errors. We might think that due to a big number of sourees of errors (see
e.g. Kahneman et al. (1982)), the error distribution may be assumed normal. This
argument should be taken with eare.
We shall use the following standard results (see De Groot (1970) and Johnson
and Kotz (1972)):

Lemma 3.3 Suppose XIO ~ N(O,u) and 0 ~ N(Q,p). Then

 115

Lemma 3.4

117:1 exp( -.5( w - ßi)2 / Cii) rx exp( -.5p( w - L)ßd Cii)/ p)2)
with p = L l/ Ci i '
Definition 3.11 A random variable X has a normal distribution with parameters
(J1, (7) truncated at [a, b] if its density is

f(x) = _1_ exp( -.5( -p) 2) I[a,bJ(x)

y'2;17 17 <I>(k) - <I>(h)
where <I> is the distribution function of the standard normal distribution and h =
(a -p)/I7, k = (b - J1)/I7.
Its distribution function is

~ 1~O(("
ifx:S: a
F(,) - p)/.) - O(hll/(O(') - .(")) if a:S: x :s: b
ifx> b
Its mean is
Pt = P + d<j>(h) - <j>(k))/(<I>(k) - <I>(h))
where <j> is the density of the standard normal distribution.
Its variance is

We shall assume that Vi ~ N(Pi,17i). Beliefs about Pi are modelled in such a

way that the error distribution is dependent on w; it is natural to assume normality
of its distribution, say
Pi ~ N(<pi(w),pi).5

In the first stage, we apply the formula

7r(wlwd rx f(w,wd = Jf(w,wI,pddpI

rx Jf(wl - WlpJ}7r(' llw)IsJw)dpl
1 = 1 (w) Jf(wl - wlpd 7r (pllw)dpl'
51

More generally,

7r n (wlw n ) rx I sn (w)11i:1 Jf(Wi - wlpi)7r(pd w)dpi.

If we call <p(z) = z - w, applying lemma 3.3, we get

A number of models follow:

50f course, a more general approach should consider distributions on 0';, typically gamma
(see, e.g., Giron (1989)). For simplicity, we shall assume that the O';'s are known.
 116

• When the /1,'S are known precisely, we can make 'P,( w) = /1, and p,->O. Then

1I"n(wlwn ) oe ISn(w)IIi::1 exp(-.5(w, - w - /1,)2jO})

= ISn(w)IIi=1 exp( -.5(w - (w, - /1,))2 ja?).

Applying lemma 3.4

with

m n is a weighted mean of the measurements w, corrected by /1,.

• When the /1,'S are known very imprecisely, we can make p,->OO so that

This is an interesting example in which we use the noninformative priors

11"(/1,) = 1. Observe that w, and ware not independent given /1" but they
are independent!!!

• When the /1, ~ N(ß" Pi), we have

Applying lemma 3.4 it is

with

- Tn = Tn_1 + 1j(a~ + p~), Ta = 0,

- m n = (rn_1 mn_1 + (W n - ßn)j(a~ + p~))jrn, ma = O.
• Suppose that 'P,(w) can be weil approximated linearly in S, around Z; ES;:

'P,(w) ::: 'P,(z,) + 'P;(z,)(w - z,).

Then, we approximate the posterior by

ir (wlw)
n n
I ()II~
oe Sn W .=1 exp
(_ 5(
. w
_ Wj - 'P,(z,) + 'P:(Z,)Zi) 2j
1 + 'P:(Z;) (1
a;+'P:(z,))2
+ p; )
and applying lemma 3.4

with
 117

mo=O .

• When the linear approximations are not valid, we can expand the squares
and apply lemma 3.4 to get

1r n(W\Wn) oe Is.(w) exp( -.5rn(w - m .. )2) exp( -.51J'P~(w)/(o} + p~)))

exp(- 2:('Pi(W)(Wi - w)/(ol + p~)))
with

- rn = rn_l + 1/(<T~ + p~), ro = 0,

- mn = (rn-lmn-l + Wn/(<T~ + p~))/rn, mo = O.

We shall study the cases leading to a truncated normal distribution. The rest
have to be tackled with numerical integration. Typically, in the unidimensional
case, Sn = [en,!n] and ~j(w) = djw + Ij. Suppose that ~.(wn) :::: ~J(Wn) and
~.(Jn):::: ~J(Jn)' Let w = (I. -IJ)/(dJ - d.) so that ~.(w) = ~J(w).

Figure 3.10: illustration of our case.

We have that

Then:

• with a regret loss, we reject the optimality of a. if

E(d.w+l.) < E(dJw+lJ)

 118

or

Substituting the expected value (definition 3.11) we reject the optimality of

a. if
(4)(e n ) - 4>(J"))/(<p(J") - <p(e n)) < Fn(w - m n)

with en == (e n - mn)y'r;;" and f" == (fn - mn)y'r;;".

• with a 0-1 loss, we reject the optimality of a. if

<P(J") + <p(e n) ::; 2<P((w - mn)Fn)·

A robust approach Robust approaches have been earried out in problems

where the prior distribution is not known precisely. We shall use a dass of error
distributions and compute the posterior distributions and tests for that dass. We
shall also give some ideas on how to use that information.
A plausible dass of error densities is

C == {g: 9 == h(lv - Ool),h decreasing}.

These distributions are unimodal and symmetrie about 00 , the most likely error.
Berger and Berliner (1986) use them as an E-eontamination dass of the prior. They
point out that this is the dass of mixtures of distributions in

Cf == {U[00 - a,Oo + a],a ~ O}.

C' will be suffieient for the inferenees we shall make. We start with a one cyde
model. The posterior distribution in this case is

with

• eo == max(eo, (w - 00 ) - a)

• fö == min(fo, (w - 00 ) + a)
We assurne that [e o, föl 1= 0. Suppose first that the loss is 0-1. Let wbe as before.
Then:
 119

We do not reject the optimality of a. if

or

We have:

fo+ eo if a ~ max(w - 80 - eo,fo - (w - 80))

• •
fo + eo = 1 fo + (w - 80 ) - a
(w - 80 ) + a + eo
if fo - (w - 80 ) :5 a :5 w - 80
if w - 80 - eo :5 a < fo - (w - 80 )
- eo

2(w - 80 ) if a :5 min(w - 80 - eo, fo - (w - 80 ))

1. If (w - 80 ) ~ (eo + fo)/2, the graph of fa + e~ as a function of a is

2(w - 80 ) 1------...

~fo+eo

Figure 3.11: Graph of fa + eo, case 1

• If w > w - 80 , we reject whatever the value of ais.

• If w < (eo + fo)/2, we never reject.
• Otherwise, we reject if a > fo + (w - 80 ) - 2w.
2. Similarly, if (w - 80 ) < (eo + fo)/2,
• If w < w - 80 , we never reject.
• If w > (eo + fo)/2, we always reject.
• Otherwise, we reject if a < 2w - (eo +w - 80 ),

Let us study the test with the regret 10ss. The quantities involved are:
/;
E('l1i) =
J{••o (diw + li)dw = u; - e~)(di(e~ + !;l/2 + li)'

We reject the optimality of a. if

 120

or, operating, if

that is, the same test as before.

Remember that C was the dass of mixtures of C. Suppose for simplicity that
80 = o. If f(a) were the density weighting the distributions, the error density
would be
{'>O IJ(2a)I[_a,a](v)f(a)da = [00 f(a)f(2a)da = g(v).
k A~
f(a) may be reinterpreted as the density representing the beliefs of the DM about
the range of v. With this interpretation, and particularly via F(a), the DM may
find relatively easy (and certainly easier than in other parametrie models, see
Diaconis and Freedman, 1986) to provide the information necessary to assess the
distribution of v and act consequently. The posterior distribution would be:

1I"(wJw) cx 1 00 I[eo,/o](w)IJ(2a)I[_a,a](W - w)f(a)da

= I[e.,/o](W) [00 f(a)J(2a)da.

A,,-wl
As an example, if f(a) = -2ah'(a) and h(a)-+a-+oo 0 it is

1I"(wJw) cx I[eo,/o] (w)h(Jw - wll·

Then, we could compute the exact tests. Observe that tests computed with uniform
errors are conservative as indicated in the following result, which says that the
acceptance region is wider for the exact test.

Proposition 3.18 Let a be Buch that we reject the optimality 0/ a. ilf a ~ a.

Let 9 E C with support in [-a,a]. Then, p"'(w. ~ WJ) ~ p"'(W' < WJ) and
E".(w.) ~ E,...(w J), where '/[g is the posterior associated to g.

fO W W

Figure 3.12: illustration of proposition 3.18

 121

We can decompose [e~, f;] in [e~, w] U[w, 2w - ea]U[2w - ~,f;]. Then,

(due to the properties of lrg). Similarly,

+ ( . . . ('l1. - 'l1 J )dlrg

}[2W-.o.Jo1

~ ( ('l1.-'l1J)dlrg~O
}[2w-.ö.J;l
o

Similar results hold in the other cases. Dependence of the distribution of v

on w can be modelIed by a weighting density function f(alw) instead of f(a). Of
course, Bo plays the same role than J.l in the normal model and similar criticisms
can be posed here about its elicitation.
To summarise, three new elements appear in the problem: Bo, f (.) and a. Dif-
ferent schemes on how to introduce these ideas can be outlined. One of them is
(where a' represents the semirange of the error distribution).
 122

For more than one cyde, the approach is similar. If the error on the i-th cyde
follows a distribution U[O; - Ci, 0; + Ci], the posterior distribution is

with

and e:' j = -00, I:j = 00. The tests have similar expressions. The weighting
functions have now the form I;(c;jw).

The multidimensional case

Let us give first the expressions of the tests when the pertinent random variable
is ~. Let J.L = E(w) and V = var(w).
When we use the regret loss:

• The linear case. We reject the optimality of a. if

(CJ - c.)tJ.L > o.

• The bilinear case. Suppose w = (Wj,W2), J.L = (J.L1,J.L2) and

with BI = (bL), Wj = (w;), W2 = (Wj). We reject the optimality of a. when

• The general case. Try approximate Wj( w) by its second order Taylor ex-
pansion around J.L

where Hj(J.L) is the Hessian of Wj(w), evaluated at W = J.L. We reject the

optimality of a. (according to the approximation) if

When we use the 0-1 loss:

 123

• The linear case. We reject the optimality of a. if

• The bilinear case. We reject the optimality of a. when

• The general case. We reject the optimality of a. when

A normal model The analysis is based on these results:

Lemma 3.5 Suppose X\II ~ N(II, E) and 11 ~ N(p, Eo). Then

Lemma 3.6

with

where 0 is a matrix 0/ zeroes.

The results for the unidimensional case are translated to this case with slightly
more complicated formulas. For example, if we assume that Vi ~ N(Pi' Ei), then

= I Sn (w)rr::"l exp( -.5(w - (Wi - Pi))tEi1(w - (Wi - Pi)))

cx I sn (w)exp(-.5(w - Z;;lTn)tZn(w - Z;;lTn))

with Zn and Tn as in lemma 3.6, with (w n - f.ln) in place of f.ln. Similar results
hold in the rest of the cases.
The main problem here is the integration. We shall study the ease in which

7r n(w\Wn) cx Isn ( w) exp( -.5( w - ;1 SE;;-l( W - ;1 n))

 124

where En is diagonal and Sn == n::1[e;, f;]. The hypothesis on En is comparable

to those made in Bayesian estimation of normal means, see Berger (1985). This
is not meant as a justification, however, as probably some of the parameters are
correlated. Moreover, the constraints are very simple. In this case, the exact
density is

( I ) -- () /( )1/2 1 exp(-.5((w; - (.l ;)/(7;)2)

'lC'n W W n /Sn W 1 2'1C' n;:l(7;(<I>((J; _ (.l ;)/(7;) _ <I>((e; _ (.l ;)/(7;))

the mean vector is

with
h. _ tP((ei - (.l ;)/(7i) - tP((J; - (.l ;)/(7;)
, - <I>((f; - (.l ;)/(7;) - tP((ei - (.l ;)/(7i)
and the matrix V of variances is diagonal with diagonal elements given as in
definition 3.11.
Then, if we use the regret loss:

• The linear case. We reject the optimality of a. if

• The bilinear case. We reject the optimality of a. if

• The general case. We reject the optimality of a. if

(WJ(P) - w.(P)) + .5E(h1(p.) - h:'(P.))v;; > O.

If we use a 0-1 loss, we have to apply numerical integration.

A number of results for truncated normal distributions can be seen in Johnson
and Kotz (1972), but in the end we have to rely on numerical integration. Berger
(1985) considers some ways of incorporating partial information of the form {Bw ==
d} to modify his robust estimators, different to our truncation approach.

A robust approach We may try a robust approach as in the unidimensional

case. For simplicity, we shall assume that 00 = O. The dass of error distributions
we use is
[' = {U[-a,a],a 2: O}.
 125

When the DM expresses his beliefs over the range of the error distribution, say via
a density fra), the error distribution is:

J-t-1r-a,aj(v)f(a)da
1 == 1 ... 1 fra) dal ... dat
00 00
~II
2 IIa, 2 Ivd lVII ai

For a one cyde problem, the posterior density would be

7r(wlw) c< JIs(w)2t~ai Ir-a,aj(w - w)f(a)da

c<Is(w) 1Iw,-wd ... 100IWI-WII -2fra)
00
I I dal.·.dat
t ai

We then compute the exact tests, possibly via numerical integration.

Suppose the DM does not want to provide fra). We can compute the tests for
each of the densities of [,l We have, again for a one stage model,

We can give explicit formulas for the tests in some special cases. Suppose that
S == IIl=l[ei,J;]. Then

with

• f: == min(fi, w + a;)
Thus, if we use the regret [oss:

• The linear case. We reject the optimality of a. if

2)ef - eiltet + J;) > O.

• The bilinear case. We reject the optimality of a. if

• The general case. Approximating by Taylor around the centroid i5 of

IIl=l[ei, f:l, we reject the optimality of a. if
 126

When we use the 0-1 loss, we can give general formulas only in the linear case.
We know that
P(c'w :::; c'ö) = .5
Therefore, we reject the optimality of a. if

(c. - C])'ö < O.

In the other cases, we have to apply numerical integration. These results are easily
extended to more than one stage.

3.6 Comments
We have argued that sensitivity analysis tools for decision aids are essential as
a means of exploring the DM's mind to identify the most critical judgements
in determining choice. We find first whether the current optimal alternative has
competitors. If this is the case, sensitivity analyses are necessary. Several concepts
of adjacent potential optimality help us to reduce the set of competitors to the
current optimal alternative. A dass of easily implementable sensitivity analysis
tools, based on gauge functions, helps us to identify sensitivity issues in a very
informative way. The problem of error modelling in judgemental measurement
leads to so me other sensitivity tools based on Bayesian hypothesis testing. Chapter
4 illustrates these ideas in various examples.
Chapter 4

Computational experience

Is this framework computationally feasible? This chapter gives a positive answer

describing some computational experience in the implementation of some of the
previous ideas. First, we introduce SENSATO, a prototype library for a general
purpose sensitivity analysis package for decision aids. This tool is used to anal-
yse: a flood-management problem, based on a linear model; a portfolio selection
problem, based on abilinear model; a technology-purchasing problem, based on a
general (tree) model; a road evaluation problem, based on a general model; and
a linear problem, aimed at giving various counterexamples. For each of them, we
present the problem and the corresponding model, the input to SENSATO, its
output and a comment on the results.
As a byproduct, these examples show that sensitivity issues arise more fre-
quently than suggested, for example, by the Flat Maxima Principle. Thus, this
chapter also completes the discussion of section 3.2.

4.1 An introduction to SENSATO

We describe SENSATO, a prototype library for sensitivity analysis of Bayesian
decision aids, which follows the theory developed in chapters 2 and 3. It is written
in FORTRAN 77 on an AMDAHL 5860, running under VM/CMS. The compiler is
VS FORTRAN 2.3.0. SENSATO calls NAG FORTRAN Mark 13 and GHOST-80
Release 7 libraries. It comprises three routines, according to the ideas we have
introduced:
 128

• SENSLIN, for linear problems, that is 'itj(w) = CjW and 5 is defined by linear
constraints.

• SENSBIL, for bilinear problems, that is 'itj(w) = W\BjW2 and 5 is defined

by linear constraints.

• SENSGEN, for general problems, that is the 'itj's and 5 are general.

The routines have the same general structure:

1. Read data.

2. Rate and rank alternatives and reorder data.

3. Find the nondominated alternatives.

4. Find the (nondominated) p.o. alternatives.

5. Find the a.p.o. alternatives.

6. Choose a p for an lp distance.

7. Find p.

8. Find 8.

9. Present results (distances, solutions, graphs and ways of improving the cur-
rent status).

10. If the DM wants to use another distance, go to 6. Otherwise, stop.

They are designed to use NAG library as efficiently as possible. Thus, problems
referred to finding nondominated, p.o., and a.p.o. alternatives are solved with the
optimisation option. The only exception is the computation of a.p.o. alternatives
in the linear case, which is done with the consistency of a system of inequalities
option: the system of linear inequalities is set up faster than the linear program.
We use the forward algorithm of section 2.5, with gi = 'iti(w), to find the non-
dominated alternatives. Steps 3, 4 and 5 filter the alternatives of interest to the
DM. Any alternative discarded in any of these steps is eliminated from the rest of
the analysis. For instance, if aj is dominated we do not look for the alternatives
dominated by aj; we do not check whether aj is p.o. or a.p.o.; we drop, in fact, the
constraints relative to aj in a.p.o. and p.o. computations, by putting their bounds
as -00,00; and we do not compute dj .
As optimisation problems are solved with NAG, distance problems have to be
formulated as smooth:
 129

• For the Euclidean distance, we solve

wES

s.t.
W.(W) - Wj(w) = O.

Thus, the Euclidean distance d = J2D

j j + L,w? Solving

[Pd] min -.5 I>1 + l:WjWj = ~

wES

gives maximum distance as fJ = J-2~ + L,wl.

• For the Tchebycheff distance, we solve the problem

[Pd j ] min dj
w E 5,dj

s. t.
W.(W) -Wj(w) = 0

Wj + dj ~ Wj.

To find the maximum distance, we solve the problems

maxWj =!l
wES

and

minwi = fl
wES

giving

.
fJ = ~x(fl- Wj,Wj - In .
 130

• For the rest of Ip problems, we solve

w E S,.,t

s. t.

Si, ti 2: o.

To find the maximum distance, we solve the problem

[Pd] max 2)Si + ti)P = 8P

w E S,.t,t

S. t.

Siti = 0
Si,ti 2: o.

The following results are presented on the screen and, with some more detail, are
also written on a file (see the appendix for a sampie):

• rating and ranking of alternatives according to ll1 i (w).

• list of nondominated alternatives,

• list of p.o. alternatives and their best case,

• list of a.p.o. alternatives and parameters for which they share optimality
with a.,
• for each aj remaining in the analysis, its dj , parameters for which aj outranks
a., and graphs,

• p, 8, parameters giving the maximum distance and their graph, and the
sensitivity index,

• possibilities available to the DM to improve the current status.

A graphical representation of this information seems necessary to convey its

meaning to the DM. The w's to be represented need to be considered relative to
 131

w so we consider the representation of the w - w's. In SENSATO, the DM can

choose between bar charts and modified star graphs. Conventional star graphs
(see Chambers et al., 1983) do not allow the representation of points with negative
cooordinates. A possibility is to add a positive constant to each coordinate. It
is important, however, to represent negative coordinates as such. Modified star
graphs overcome this problem. In R 2 , we represent t lines intersecting in (0,0)
capturing angles of :Ir ft rad among them. Let us designate them, consecutively,
rl, ... , rl' If Xi ~ 0, we mark a segment of length Xi in rj n{y : Y2 ~ O}j if Xi < 0,
we mark a segment of length -Xi in rj n{Y : Y2 < O}.

- f~

Figure 4.1: A sketch of modified star graphs

Negative and positive changes are easily appreciated. The use of colours might
enhance this feature. In our opinion, these graphs provide a faster view of the
changes in the parameters. This is due, perhaps, to a distorting effect of the axis
in bar charts, but this has to be evaluated from a psychological point of view. Both
types of graphs are plotted with the aid of GROST-80. SENSATO also provides
tables indicating patterns of the changes in the parameters leading to outranking
a. by aJ and the greatest changes in S. Graphs and tables are illustrated in section
4.2.3.
 132

The following table summarises the NAG routines used in the different parts
of SENSATO routines:

SENSLIN SENSBIL SENSGEN

DOMINANCE E04MBF E04NAF E04VCF
P.O. E04MBF E04VCF E04VCF
A.P.O. E04MBF E04VCF E04VCF
L1MIN E04MBF E04VCF E04VCF
L1MAX E04MBF E04VCF E04VCF
L2MIN E04NAF E04VCF E04VCF
L2MAX E04NAF E04NAF E04VCF
LPMIN E04VCF E04VCF E04VCF
LPMAX E04VCF E04VCF E04VCF
CHMIN E04MBF E04VCF E04VCF
CHMAX E04MBF E04MBF E04VCF

The input to the routines is described in sections 4.2, 4.3 and 4.4.

4.2 Linear models: Flood-plain management

This example is based on Goicoechea et al. (1982) who take it from Novoa and
Halff (1977). It refers to flood-plain management in Dallas, Texas. Each of the
eight alternatives a, is described by a lO-attributed performance as summed up in
the table.

a1 a 2 a3 a4 a5 a6 a7 a8 a9 a 10
al 6 6 8 8 8 4.3 3.3 1 4 6
a2 5 5 7 7 7 5.9 4 1 4 7
a3 8 2 3 2 4 7.4 5.6 5.3 2 4
a4 7 8 6 6 6 2.7 1 1 6 1
a5 4 4 4 5 5 6.6 4.7 1 3 5
as 6 7 5 4 2 8 2.8 8 7 2
a7 4 3 3 3 4 7.7 2.7 8 1 8
as 1 3 1 1 1 1 8 6 8 2

We shall call this matrix C. The function proposed to rank the alternatives is
10
v{a) = Ea'w"
i=l
 133

with weights Wi fixed as

(10,7,3,5,6,8,9,4,2,1).

From the information contained in the text, the only constraints are Wi ~ 0 and
E~~l wi=55 (this is the addition of the initial parameters). To add some fiavour to
the problem, we consider Bome constraints on the w's (which are typica.l of what
might arise in practice). These are:

• we assume the following lower and upper bounds on the parameters,

Par LB UB
Wl 5 20
Wl 4 10
W3 0 15
W4 5 10
Ws 0 10
W6 5 10
W1 5 15
Ws 3 5
W9 0 3
W10 0 5

• we assume the constraints

o :::; W6 + W1 + Ws - W9 - W10
10
L;Wi = 55
1;::1

4.2.1 The model

Our uncertainty in this problem will be in the weights of the additive value func-
tion. Following our comments, we should include in the analysis the component
values, but we prefer to start with a simple illustration of a linear model. More-
over, if great sensitivity is detected, there will be greater sensitivity in the 'true'
problem. This is shown in section 4.5, where we present a more complex analysis
of this problem.
 134

Thus, in this case, we have

where c; = (Clj,"" ClQj). For exa.mple,

The constraints can be written in the form

Dw::;d.

So we have a linear model.

Finally, the estimate W of the weights is

W = (10,7,3,5,6,8,9,4,2,1).

4.2.2 Input to SENSLIN

The input to SENSLIN in this case is:

• Number of parameters 10.

• Number of alternatives 8.

• Number of linear constraints 4.

• Matrices C, D.

• w.

• Upper and lower bounds of parameters and constraints.

The rest of the matrices, dimensions, bounds and constraints are defined/created
by the routine.

4.2.3 Results
We comment in some detail on the results. The rating and ranking of the alter-
natives is:
 135

a\ 296.1
a6 293.2
a2 285.2
a3 275.8
a4 257.6
as 245.1
a7 236.9
aB 167.0

a\ is the current optimal solution (and the implemented one). Nondominated

alternatives are found solving problems [Pii]' The table shows their optimal values.

a\ -68.5 -14.5 -73 -13.3 3 -11 23.5

a6 -81 -46.9 -25.5 - -45 -
a2 -77.5 -38 - -22 -
a3 -89.6 - -22 -
a4 - -82.5 -
a6 a2 a3 a4 as a7 aB

For example, the optimal value of [P2d is -14.5 so that al does not dominate
a2; the value of [Psd is 3 and al dominates as. A dash means that the problem
need not be solved. As we see, al dominates as and aB. The set of nondominated
alternati ves is

The p.O. nondominated alternatives are found solving problems [Pi]' Their solu-
tions are shown in the table, where the second column presents the optimal value
of the corresponding problem:

a6 -59.07
a2 -12.41
a3 -46.90
a4 -3.16
a7 1.11

For example, a6 is p.o. and for some wES its value is 59.07 higher than the
best of the rest of the alternatives. a7 is not p.O. as the optimal value is greater
than O. Thus, the p.O. nondominated solutions are
 136

That should be the solution of the decision making problem under the partial
information wES. More analysis is required as al is not strongly optimal.
A.p.o. alternatives are then computed studying the consistency of [SJ] with the
results

aS Consistent
a2 Consistent
a3 Consistent
a4 Consistent

Thus, the alternatives which are a.p.o. to al are

These alternatives enter into the distance analysis phase. Results are reported
for lp distances, with p=1,2,3,4,00 (rounded to two decimals). The table gives
the d/s, the maximum distance p and the sensitivity index r for each of the lp
distances:

ds d2 d3 d4 p r
1 0.44 9.70 4.71 20.20 32.02 0.01
2 0.25 3.87 2.04 8.35 14.90 0.02
3 0.20 2.83 1.45 6.57 12.70 0.02
4 0.20 2.41 1.23 6.00 12.18 0.02
00 0.10 1.49 0.76 5.17 12.00 0.01

In all cases, as gives the minimum distance. Observe that r is very small; this
suggests that we have to be very careful in the elicitation of weights. Very small
perturbations of the weights lead as to outrank al. For example, for

w = (10.00,7.10,2.89,5.00,5.89,8.10,8.89,4.10,2.10,0.89),

obtained with Tchebycheff distance, q,s(w) = q,1(W). Looking at the table of

problems [Pij ], we see that the value of [PSI] is -68.5. Thus, if as were optimal,
the loss of optimality due to choosing al might be as large as 68.5. This is large
compared with the values of al and asl. To improve the current status, we can
either compare as and al or reduce S, as indicated by SENSLIN.
Having determined the need for a more careful elicitation of weights, we can
analyse in further detail the data provided by SENSLIN. We look first at graphs
and tables.
I Remember the comments of section 3.2.1 on the loss of optimality
 137

The trends tables give the sort of changes in the parameters producing the
outranking of a. by aJ and giving the maximum distance. A 0 indicates no change,
a -1 a reduction and a 1 an increase in the parameter value (with respect to wo).

p J Wj Wz W3 W4 Ws W6 W7 Ws Wg WjO

1 6 -1 -1 -1 -1 -1 1 -1 1 1 -1
2 6 -1 1 -1 0 -1 1 -1 1 1 -1
3 6 -1 1 -1 0 -1 1 -1 1 1 -1
4 6 -1 1 1 0 -1 1 -1 1 1 -1
00 6 -1 1 -1 0 -1 1 -1 1 1 -1

Table 4.1: Trends in changes leading to outranking of a. by aJ with respect to

various distances.

We see that J = 6 for all the distances used. The pattern is that changes occur
when Wz, W6, Ws, W9 are increased and Wj, W3, Ws, W7, WjQ are decreased. There need
be little or no change in W4' This information would clarify in the DM's mind the
least changes in her judgements that would lead to a change in the suggested
course of action.

p Wj Wz W3 W4 Ws W6 W7 Ws Wg WjQ

1 -1 -1 1 0 1 -1 -1 -1 1 1
2 -1 -1 1 0 1 -1 -1 -1 1 -1
3 -1 -1 1 1 1 -1 -1 -1 -1 -1
4 -1 -1 1 1 1 -1 -1 1 -1 -1
00 -1 1 1 0 -1 -1 -1 1 -1 1

Table 4.2: Trends in greatest changes in 5 with respect to various distances.

This table gives the patterns leading to the greatest changes possible in S.
There should be a reduction in Wj, Wz, W6, W7 and an increase in W3, W4, Ws. There
is no pattern for Ws, Wg, WjO' This information would clarify in the DM's mind
the greatest possible changes in her judgements, given the current information
expressed in S.
The DM would probably ask for some more quantitative information. Graphs
seem to be the next step. We give below bar charts and star graphs of the results
when the Euclidean distance is used.
 138

,1.000
....

-1.490

-1"

1.000 '.000

0.154
1.0OCQ..5f1

0.000

-0.49(1
~ O.~29
-0.289
·0.269 '0

.0. 002
.0.1198

0.529 O./'S4 0.59' '.000

0.000

I' I' i' I' '0

-O,2H-O.2l19
-D.6"..o,'J'02 -0.6"
 139

,
1..19 LOOG Lm "14,' LOQ
, . o.l,.,o.07~.900

... ...
0.0lI01 0.919

... '" ...... .....'41

..o.oca'" .0.0'"
..

1.'\0
UD'

.
I
1.107

0,'140.17' .~..," 0., a.DOCI

r--r--r--,---l--1--.o...--- ' " I••SO 1.,7&

.1.101

.. I. Jot
_1.101 _I, XII _1.301 .I.JOI
_1.111

.1.,.,

tut 1"0551"( tfWtC(5 I" 'NI~TtJtS 1ft 5

4.000

-5. 000 -- 0.000

'.000

.1.000 .1,000
·,m
.,,,,,
.1,000

.'5.~'
 140

In this ca.se, minute changes lead to as outranking a\. Compare especia.lly the
star graph associated with as with the others. Changes in favour of a4 seem, on
the contrary, very big. Fina.lly, we observe the possibility of very big variations
in Wb W2, W3, Ws, Ws and W7 within 5. Similar thoughts arise when we look at the
graphs relative to other distances.
The DM may ask for more quantitative information. We can use the numerical
output of SENSLIN; see appendix. The two points of concern are how big are the
least changes leading to as and a4 outranking a\.
Looking at the list of critica.l parameters, we see that deviations from the
origina.l ones do not need to be greater than 0.2 in any of the parameters to cause
the outranking of al by as. In fact, except for Tchebycheff distance only changes
in Ws, Ws and WIO have to be greater than 0.1 to cause that outranking. Changes
in Wl and W4 are very sma.ll. By now, the DM should be sufficiently motivated to
think more carefully about the problem.
More sensitivity issues arise. The biggest difference in rating between a4 and
a\ is only of 13.3 which is not too much. Moreover, the biggest difference in rating
between a4 and the rest of the alternatives is 3.16. And the d4 's seem very big.
Other features which appear in the ana.lysis of the data are the very big changes
possible in parameters Wl, W2, W3, Ws, Ws, W7 confirming the impression acquired by
looking at the graphs.
Sometimes it is not possible to identify sensitivity issues so easily: some help
can be found using tools from multivariate exploratory data ana.lysis. We can
proceed as follows 2 •
If t is the number of parameters, the procedures we have introduced identify
points W of R t (parameters representing least changes for the outranking of a. or
greatest changes in 5). We ca.n reduce the dimension using principal components
analysis. The first two principal components may be regarded as the plane of
best fit (in the least squares sense) to the initial observations: we transform, by
projection into aplane, the points Wj - w into a set of points Cj whose relative
positions and Euclidean distances are approximations of those of the Wj - w.
This idea is ca.rried out with the results from lp-distances (p = 1, ... ,9,00).
We represent the scores of the data relative to Tchebycheff distance in the two
first principal components after an orthogona.l rotation.

lThis part is not incorporated in our library. Computations were done with SPSS PC+.
 141
I.

-I.

-I I !
Figure 4.2: Scores of parameters leading to outranking of aj by aj, i = 2,3,4,6
and greatest changes in S(l) for Tchebycheff distance, in the first two principal
components.

Being projections, the representation may be confusing to the user. Alterna-

tively, we can use the directions of the components and represent in this plane, in
the direction of the projection of Wj - w, a bar of length d(w,w).

Figure 4.3: Modification of the scores, as suggested.

This represents the elements of the Tchebycheff distance problem. We can also
display the minimisers for all the distances for the different alternatives and the
maximisers. The following graph was built with the aid of a cluster analysis and
represents these data. Note that the scales are quite distorted.
We get a picture of so me of the issues we have identified: as is the most likely
alternative to be the competitor of aj; very small changes in the parameters lead as
to outrank aj; a4 does not seem to be a competitor of a\; and very big reductions
in S seem possible.
 142

1.1
3

1.05

.. 6

.11
-.,11
-.n
-.141
-.n
-.17\
-.14
_.,,\
~~
'"
01
1
.11
.UlS

.'"

.,1
..
.,~
."
.11

•• 01

I.H

,
-1.2
,, I 1
".
1.1
I
•I•
-.,~ -.. ~ -.ll -.' 1
-.14 -.n -.' -.n
1

-, 1
 143

More work needs to be done to explore further the possibility of using these
tools. Observe also that we have used them in a conn.rmatory sense. They should
be more useful in an exploratory phase.
The approach seems quite powerful as it allows to identify sensitivity issues in
a very constructive way. It should help DM's to think more carefully about their
problems.

4.3 Bilinear models: Portfolio selection

This example is adapted from French (1986). A broker has to decide arnong six
investments {al, a2, a3, a4, as, a6}. Their next year payoffs depend on the future
n.nancial market which rnight have very low (BI), low (B 2 ), medium (B 3) or high
(B 4 ) activity. Each investment pays off after one year. Payoffs are given in the
table. The broker must choose one and only one of the investments.

BI B2 B3 B4
al 800 900 1000 1100
a2 600 1000 1000 1000
a3 400 800 1000 1200
a4 600 800 800 1200
as 400 600 1100 1200
a6 800 800 1100 1100

Alternatives are ranked by their expected utility

4
E(u(a),p) = 2>(B;)u(a,Bj).
i=l

Let us call:

Wj :::: p(Bj),i:::: 1, ... ,4

Ws u(1200)
W6 u(1100)
W7 u(lOOO)
WB u(900)
W9 u(800)
WIO u(600)
W11 u(400)
 144

The estimates and bounds of the parameters are:

Par LB Wj UB
W1 .20 .30 .50
W2 .20 .30 .40
W3 .10 .20 .40
W4 .10 .20 .60
Ws 1.0 1.0 1.0
Ws .85 .90 .95
Wr .80 .85 .90
Ws .75 .80 .85
Wg .60 .70 .75
WJO .30 .50 .60
Wu .00 .00 .00

There are also some constraints in the probabilities

i=l
(for example, the first one means that the broker believes that a very low to low
market is at least as likely as a medium to high market). To ensure monotonicity
in the utilities, we add the constraints

W10 :5 Wg

Ws :5 W7
wr:5 ws·

4.3.1 The model

The uncertainty in this problem is both in the probabilities and the utilities. We
have
4
Wj{w) = EWjWjj,
j=l
 145

where Wij = U(aij) E {W5,"" Wll}. For example,

which can be written

with

w 2 == (ws, W6,··., Wll)

and BI the matrix

(:o : : : : : :1
000 0 0
For the rest of the alternatives

with the Bj's defined in a similar fashion. The constraints can be written

Dw::;d.

So we have abilinear model.

Finally, our estimate W of the probabilities and utilities is

W = (.3, .3, .2, .2, .1, .9, .85, .8, .7, .5, 0).

4.3.2 Input to SENSBIL

The input to SENSBIL in this case is:

• Dimension of w l 4.

• Dimension of w 2 7.

• Number of alternatives 6.

• Number of linear constraints 7.

• Matrices D, BJ •

• w.

• Upper and lower bounds of parameters and constraints.

Thc rest of the matrices, dimensions, bounds and constraints are defined/ created
by the routine.
 146

4.3.3 Results
We comment on the results in less detail. The rating and ranking of the alternatives
IS:

al .800
a6 .780
a2 .745
a4 .700
a3 .580
a5 .530

al is the current optimal alternative. There are four non domina ted alternatives

The results of solving [Pj 1 are

a6 -.060
a2 -.060
a4 -.035

Thus, the p.o. nondominated solutions are

A.p.o. alternatives are then computed solving [PJJ. The a.p.o. alternatives to

These alternatives enter in the distance analysis phase. Results are reported
for lp distances, with p=1,2,3,4,00:

d6 d2 d4 p r
1 0.06 0.15 0.47 0.92 0.07
2 0.04 0.07 0.18 0.35 0.11
3 0.03 0.06 0.14 0.27 0.12
4 0.02 0.05 0.12 0.24 0.11
00 0.01 0.03 0.08 0.20 0.08
 147

with the Euclidean distance we find out that for

w = (0.3,0.29,0.2,0.2,1.,0.91,0.83,0.78, 0.72,0.5,0.0),

1l1 1 (w) = 1l1 6 (w). In this case, the maximum loss due to choosing al instead of ae
is 0.06. This is not very large compared with the values of the alternatives, which
are around 0.8.

4.4 General models: Technology-purchasing de-

cis ion
This example is taken from French (1986). An airline has to decide whether or not
to buy a new airliner. The airliner may be very reliable (vr), moderately reliable
(mr) or very unreliable (ur). The directors can ask for areport to a private
consultant which may be favourable (f) or unfavourable (u). Courses of action are
descri bed in the decision tree.

ottO.(,Mt-'O"'-J

0.000.10 01 ..... ....,

/'--- 0.110.101" - .......

O.:UO.IO"'O-"·"J

0000.10 01 0-' ••••.1

._,'0 , 000

00"

,.....-- .. ".
Figure 4.4: Decision tree for the airliner problem (French, 1986).

Alternatives are: al(N, N, N), a2 (N, Y, Y), a3 (Y, Y, Y), a4 (Y, Y, N),
as (Y, N, Y), aa (Y, N, N), where the first Y-N means (asking or not for the
survey), the second means (buying or not, if the report is favourable) and the third
 148

means (buying or not, if the report is unfavourable). Some of the alternatives are
obviously bad: SENSATO will discard them. The directors are asked to provide
their probabilities for the result of the airliner and for the result of the consultant 's
report conditional to the result of the airliner. The utility of the airline depends
on the annual operating profit x!, the annual number of passengers carried X2, the
proportion of flights delayed X3 and the cost of the survey X4' It is established
that
U(Xj, X2, X3, X4) = g(Xj, X2, X3) - kX4.

Let us call

Wj p(vr)
W2 p(mr)
W3 p(ur)
W4 pUlvr)
Ws pUlmr)
W6 pUlur)
W7 p(ulvr)
Ws p(ulmr)
W9 p(ulur)
WjO g(vr)
Wu g(mr)
Wj2 g(nb)
Wj3 g(ur)
Wj4 k
WjS X4

where p(E) is the probability measuring the directors' beliefs in event E and g(B)
is the component utility associated to results (Xj, X2, X3) when B is the decision.
The directors provide the following constraints:

• for coherence of probabilities

Wj + W2 + W3

Ws + Ws
 149

• for the likelihood of the results of the airliner

• to ensure monotonicity of the utility function

• to give bounds for the probabilities of unfavourable and favourable reports

.3::; + W2W5 + W3WS ::;.6

WIW4

.5::; WIW7 + W2WB + W3W9 ::;.8

They also provide the following assessments and bounds for the parameters:

Par LB Wi UB
Wl .10 .20 .40
W2 .20 .30 .60
W3 .30 .50 .70
W4 .85 .90 .95
W5 .40 .60 .70
Ws .05 .10 .15
W7 .05 .10 .15
WB .35 .40 .45
W9 .85 .90 .95
WIQ .90 1.0 1.1
Wll .25 .34 .50
W,2 .10 .17 .30
Wl3 .00 .01 .05
WH .05 .10 .15
Wl5 .05 .10 .15

4.4.1 The model

The uncertainty in this problem is both in the probabilities and the utilities. We
compute (symbolically) the expected utilities, using Bayes' theorem, and simplify
 150

them

W1(W) WI2

W2(W)

W3(W)
W4(W) WIW4WIO + W2WSWll + W3 W S W I3 +
(WIW1 + W2WS + W3W9)W12 - W14 W IS

WIW7WIO + W2WSWll + W3 W 9 W I3 +

(WIW4 + W2WS + W3WS)WI2 - W14 W IS

WS(W) = W12 - W14WI5

Linear constraints can be written as

Dw S. d

There are two nonlinear constraints. So we have a general model.

Finally, our estimate W of the probabilities and utilities is

W = (.2, .3, .5, .9, .6, .1, .1,.4, .9, 1., .34, .17, .01, .1, .1).

4.4.2 Input to SENS GEN

The input to SENSGEN in this case is:

• Number of parameters 15.

• Number of alternatives 6.

• Number of linear constraints 7.

• Number of nonlinear constraints 2.

• Matrix of linear constraints D.

• w.

• Upper and lower bounds of parameters and constraints.

• A routine with the evaluation functions.

• A routine with their derivatives.

• A routine with nonlinear constraints and their derivatives.

The rest of the matrices, dimensions, bounds and constraints are defined/created
by the routine.
 151

4.4.3 Results
The rating and ranking of the alternatives is:

a4 .332
a2 .307
a3 .292
al .170
as .160
as .125

So a4 is the current optimal alternative. There are only two nondominated

alternatives
{a2, a4}.
And a2 is potentially optimal with advantage 0.1192. Obviously, a2 is a.p.o. to a4.
We have the following distance analysis.

P d2 P r

1 0.04 1.13 0.03

2 0.03 0.31 0.10
3 0.02 0.22 0.11
4 0.02 0.20 0.12
00 0.01 0.20 0.06

Again, we have to be extremely careful with the elicitation of utilities and

probabilities. For example, we detect with city-block distance that for

W = (.2, .3, .5, .9, .6, .1, .1,.4, .9, 1., .34, .12763, .01, .1, .1)

(that is, only a change in W12 of 0.0424) W2(W) = W4(W). The loss of optimality
due to choosing a4 instead of a2 may be as bad as 0.1192, which is a lot, since
values are around .3.

4.5 Dallas' problem revisited

Once the general models have been illustrated, we analyse a more complex version
of Dallas' problem (section 4.2). We observed that it was very sensitive to changes
in weights Wl and W4. Note as weil that values on first and fourth attributes were
based on ranks. We shall briefly analyse sensitivity with respect to weights and
values in those attributes.
 152

(Wl, W2, .•• , WIO) will have the same meaning and will satisfy the same con-
straints of section 4.2. We can fix two of the values of the first and fourth scales,
say at 1 and 8. The remaining four values in the first scale are represented as
(Wll, W12, W13, Wl4); the remaining six values in the fourth scale are represented as
(W15, W16, W17, W18, W19, W20). They are assessed and bounded as folIows:

Par LB Wi VB
Wll 1 4 8
W12 1 5 8
W13 1 6 8
W14 1 7 8
W15 1 2 8
W16 1 3 8
W17 1 4 8
W18 1 5 8
W19 1 6 8
W20 1 7 8

To ensure monotonicity of scales, we include the constraints

4.5.1 The model

The uncertainty here is in the weights and the values of the first and fourth scales.
Substituting the values of the parameters, we get the evaluation functions. For
example

Linear constraints can be written as

Dw:S d.

There are no nonlinear constraints in this case. We have a general model. The
estimates of the parameters are

W = (10,7,3,5,6,8,9,4,2,1,4,5,6,7,2,3,4,5,6,7).
 153

The input to SENS GEN is similar to that of the previous model, so we do

not repeat it. The only point worth mentioning is that, due to the structure of
the program, we have to say that there is one nonlinear constraint and provide a
dummy subroutine computing its value and derivatives.

4.5.2 Results
The rating and ranking of the alternatives is as in section 4.2.3. There are now
seven nondominated alternatives:

The results of solving [Pj ] are

a2 -27.5
a3 -156.9
a4 -96.8
as -13.5
as -88.5
a7 -23.7

Thus, all nondominated solutions are p.o. They are all a.p.o. to al and enter
into the distance analysis. As an illustration, with 14 distance, the results are

ds d2 d3 d4 ds d7 p r
Analysis 4.5 0.15 0.52 0.81 1.47 2.69 2.16 12.77 0.01
Analysis 4.2 0.20 2.41 1.23 6.00 - - 12.18 0.01

As indicated in 4.2.1, greater sensitivity is detected in this more complex analy-

sis: more alternatives pass the different stages of the analysis; losses of optimality
are biggerj minimum distances are smaller; maximum distances are bigger and,
consequently, sensitivity indices are smaller. In short, the problem is even more
sensitive than it seemed to be after the analysis in 4.2.

4.6 Questioning a model: Road selection

This example is taken from Gomez Orea et al. (1989), who use it to illustrate
the performance of adecision support system for environment al impact analysis
of road designs. Seven different road designs {al, a2, a3, a4, as, as, a7} are studied.
 154

Their performance is measured with fourteen attributes. Weights are assessed and
alternatives are ranked by

Their approach can only be called pseudo-Bayesian, due to the way of assessing
the Wj'S and the Vj's. To do this, they use the scale:

Level Impact
9 V.High+
8 High+
7 Medium+
6 Low+
5 Negligible
4 V.Low-
3 Low-
2 Medium-
1 High-
0 V.High-

The w;'s and the Vj'S are assessed independently on each scale without consid-
ering compensation. We think we cannot appeal to uniqueness properties, apart
horn monotonicity.
Component values are assessed as folIows:

Xl X2 X3 X4 X5 X6 X7 Xs Xg XIO Xn Xl2 Xl3 X14

al 1 1 2 2 1 9 3 2 3 5 0 0 2 2
a2 2 3 3 3 1 9 3 2 3 5 0 4 3 2
a3 1 2 2 2 1 7 5 2 2 3 2 2 2 1
a4 1 1 2 2 1 8 5 2 2 2 2 2 2 0
a5 2 2 1 1 3 8 5 3 0 1 1 0 1 1
a6 3 2 0 1 3 8 5 3 1 1 1 0 1 2
a7 2 2 3 3 3 8 5 3 1 0 1 0 3 2

Weights are fixed as:

(8,8,3,9,9,9,5,2,7,8,7,9,3,8)

4.6.1 Analysis 1
Following our comments, we assign a parameter to each value on the scale and
use only monotonicity properties. We can fix Wo = 0; also, W6 does not appear in
 155

the study so we do not use it. Finally, to bound the space of parameters we make
Wg = 9. Recoding W7, WB as Ws, W7, our estimates and bounds are:

Par LB w. UB
Wl 0 1 9
W2 0 2 9
W3 0 3 9
W4 0 4 9
Ws 0 5 9
W6 0 7 9
W7 0 8 9

The monotonicity constraints imposed are:

Given our feeling that the model is inadequate in general, we study whether
it is inadequate in this particular example. The question to answer here is: Are
there scales conveying the same information which result in a different ranking?

4.6.2 The model (Analysis 1)

The uncertainty here is in the values of the scale. Substituting the values of the
parameters and simplifying, we get the evaluation functions:

9Wl + W2(W2 + 2W3 + W7 + 9) + W3(WS + ws) + w7(2wj + ws) + 81

9Wl + W2(W2 + 2W7) + W3(2w3 + Ws + Ws + W7 + W9) +

WSW7 + 9W4 + 81

1}3(W) 9Wl + W2(W2 + 2W3 + Ws + W7 + 18) + w~ + w7(2wj + W3) + 9ws

I} 4( w) W2(W2 + 2W3 + 2ws + W7 + 18) + w~ + W7(2wl + 9) + 9wj
'*'s(w) Wj(2W3 + Ws + 2W7 + wg) + W2(2w7 + W3) + 9W7 + w~ + 9W3
1}6(W) Wl(W3 + 2ws + W7 + 9) + W2(2w7 + W3) + w; + W7(3w2 + 9)
1}7(W) 2WIWS + W3(W2 + 2W3 + 18) + W7(3w2 + 9) + w;.
Linear constraints can be written as

Dw::;d.

There are no non linear constraints in this case. We have a general model. The
estimate of the scale values are

w = (1,2,3,4,5,7,8).
 156

4.6.3 Results
The rating and ranking of the alternatives is:

a2 307
ar 237
a3 233
al 232
a6 220
a4 218
as 200

a2 is the current optimal alternative. It was also the implemented one 3 . There
are two nondominated alternatives

For some scales under the stated conditions, a3 is better than a2. Strictly speak-
ing, this is enough information to answer the questions posed. However, it is
informative to continue the analysis.
a3 is potentially optimal with advantage 81 and is a.p.o. to a2. The distance
analysis is:

P d3 p r
1 4.00 33.00 0.12
2 2.20 13.96 0.16
3 1.78 10.83 0.16
4 1.59 9.65 0.16
CXJ 1.23 8.00 0.15

Again, the problem seems to be sensitive. This means that there are scales
relatively elose to the original one for which w3 (w) == w2 (w). The loss due to
choosing a2 instead of a3 may be as bad as 81 (for values around 300).
The model may not be adequate in this case.

4.6.4 Analysis 2

Suppose now that the analysis has been conducted properly: the Wj'S and the v;'s
have been assessed using adequate methods.

3Its implementation did not depend on this selection process. The example is retrospective.
 157

As in analysis 1, seven parameters are assigned to the scale values: we fix those
corresponding to Wo = 0 and W9 = 9; as Ws does not appear, we recode W7, WB as
Ws, W7. There are also six different weights, but we can fix one of them, say to 9.
So we have five more parameters Ws to W\2' Estimates and bounds are assessed as
follows:

Par LB W· . UB
W\ 0 1 9
W2 0 2 9
W3 0 3 9
W4 0 4 9
Ws 0 5 9
Ws 0 7 9
W7 0 8 9
Ws 1 2 9
W9 1 3 9
W\O 1 5 9
W11 1 7 9
W\2 1 8 9

Some monotonicity constraints are imposed

The following normalisation constraint, satisfied by the initial weights, is in-

cluded:
WB + 2W9 + WIO + 2W11 + 4W12 = 59.

4.6.5 The model (Analysis 2)

The uncertainty is both in the weights and the values. Substituting the parameters
and simplifying we get the evaluation functions:

'lt1(W) = w\(9 + 2W12) + W2(WS + 2W9 + W12 + 9) +

W3(WlO + WIl) + WSW12 + 81

9w\ + W2(WS + 2wd + W3(2w9 + WlO + WIl + W12 + 7) +

WSW12 + 9W4 + 81
 158

W3WI2 + WSWIO + 9ws

WI(2w12+ 9) + W2(WS + 2wg + 2Wll + WI2 + 18) + WSWIO + 9W7
wI(2wg + Wll + 2W12 + 9) + 2W2W12 + W3(W9 + 9) + 9W7 + WIOWS

WI(Wg + 2Wll + W12 + 9) + 2W2W12 + 9W7 +

WSWIO + W3(WS + W12 + 9)

>I'7(W) = 2WIWll + 3W2WI2 + W3(WS + 2wg + 18) + WSWIO + 9W7.

Constraints are expressed as in analysis 1. We have a general model. The estimate

IS

w = (1,2,3,4,5,7,8,2,3,5,7,8).

4.6.6 Results(Analysis 2)
The rating and ranking are as in analysis 1. Again, there are two nondominated
alternatives {a2,a3}. a3 is p.o. with advantage 81 and a.p.o. to a2.

The distance analysis is:

P d3 p r

1 4.33 43.90 0.09

2 2.26 14.49 0.15
3 1.76 11.09 0.15
4 1.55 9.74 0.16
00 1.09 8.00 0.13

The problem is slightly more sensitive with this analysis. Numerieal eomments
are similar to those of analysis 1. However, the eonclusion in this ease would be
that we have to be very careful in the elicitation of the wis and the vis.

4.7 Some counterexamples

This example is adapted from Schneller and Sphicas (1983), where apart of it
is used to show the performance of different criteria for decision making under
uncertainty4. We shall use it to give examples of nondominated not p.o.; p.o.
dominated; p.o. nondominated not a.p.o. alternatives, and distance analyses with
differen t met ries leading to different alternatives.

<In the classical sense

 159

Consider adecision making problem under uncertaintyS with three states and
seven alternatives. Let the payoff matrix be

01 O2 03
al 5 5 5
a2 10 -3 6
a3 -2 11 4
a4 1 -1 12
as -6 12 0
a6 3.5 3 6.75
a7 -2 -2 12

We shall call this matrix C and eta) the payoffs associated to a. Alternatives
are ranked by their expected payoff
3
E(e(a),p) = LP(Oi)Ci(a).
i=l

Let us call p(O;) = Wi, i = 1,2,3. We have the following constraints:

3
L
i=l
W i=l

Wi ;::: 0, i = 1,2,3.

4.7.1 The model

The uncertainty is in the probabilities. The evaluation functions have the form

Constraints can be written

Dw::; d.

We have a linear model.

5 In our sense
 160

4.7.2 Results
Suppose that
w = (0.23,0.24,0.53).

The rating and ranking of the alternatives is:

a4 6.3500
a7 5.4200
aB 5.1025
aj 5.0000
a2 4.7600
a3 4.3000
as 1.5000

a4 is the current optimal solution. The set of nondominated alternatives is

The p.O. nondominated solutions are

(Note that a7 is optimal for w = (0,0,1)). The a.p.o. alternatives to a4 are

These alternatives enter into the distance analysis. For 11 ,1 00 , a3 is the alter-

dj d2 d3 P T

1 0.2077 0.2120 0.2050 1.5205 0.1348

2 0.1364 0.1447 0.1393 0.9651 0.1413
3 0.1202 0.1286 0.1231 0.8434 0.1425
00 0.1038 0.1060 0.1025 0.7700 0.1331

native chosen; for 12 ,13 , al is the chosen one.

The following table summarises the results and explains the counterexamples
given. x (-) means that the corresponding alternative has (not) the correspond-
ing property. MDl, MD2 refers to the minimisation of city-block and Euclidean
distances, respectively. Thus,
 161

• a7 is p.o. domina ted,

• a6 is nondominated not p.o.,

• as is p.o. nondominated not a.p.o.,

• a3 gives the minimum distance in the city-block distance analysis,

• al gives the minimum distance in the Euclidean distance analysis.

NDOM PO APO MD1 MD2

al x x x - x
a2 x x x - -
a3 x x x x -
a4 x x x
as x x - - -
a6 x - - - -
a7 X - - -

4.8 Comments
In view of the previous description, we believe that this approach is computation-
ally feasible. The following table provides the details and CPU times in microsec-
onds of some of the problems solved.
The first block rows describe the problems: type of evaluation functions and
number of alternatives, parameters, linear and nonlinear constraints. The last
block gives the total time spent in the analysis. The intermediate blocks give
times for the different steps of the analysis: rating, ranking of alternatives and
reordering of data; computation of nondominated alternatives; computation of
p.o. alternatives; computation of a.p.o. alternatives; distance analysis with city-
block, Euclidean, 13 , 14 and Tchebycheff distances. The first row indicates the total
time spent in that step; the second, the number of problems solved6 ; the third,
the average time. For example, in Dallas' problem (section 4.2), we spent 73485
J1.sec in computing the p.o. alternatives; we solved 6 (optimisation) problems; on
average, we spent 73485/6=12247 J1.sec on each of them. Response seems to be in
real time. However, remember that our problems are not very big.

6For distance analyses, this number is Napo+ 1.

 162

Dallas Portfolio Tree Roadl Road2 Sphicas

Type Linear Bilin. Gener. Gener. Gener. Linear
Nalt 8 6 6 7 7 7
Npar 10 11 15 7 12 3
Nlin 4 7 7 6 11 1
Nolin 0 0 2 0 0 0
Eval 9243 5224 4518 4702 4495 9286
Aveval 1155 870 753 671 642 1326
Domin 124619 162863 202480 787561 805479 40133
Ndom 17 8 7 7 7 16
Avedom 7330 20357 28925 112508 115068 2508
Pop 73485 823536 192931 79197 122424 25539
Npop 6 3 1 1 1 5
Avepop 12247 274512 192931 79197 122424 5107
Apo 27661 134715 72042 22546 59168 14299
Napo 4 3 1 1 1 4
Aveapo 6915 49185 71848 48854 99782 3574
Cblock 1192409 2500362 1508586 306405 781477 111343
Avcbl 238482 625090 754293 153202 390738 22268
Euc 61511 194270 105237 65010 164169 20830
Aveuc 12302 48567 52618 32505 82084 4166
L3 1100476 775022 2526018 302938 1366545 170032
AvL3 220095 193755 1263009 151469 683272 34006
L4 1417776 1624434 5491236 448011 1474053 746770
AvL4 283555 406108 2745618 224005 737026 149354
Tch 194658 500117 1037503 265335 1089999 32546
AvTch 38931 125029 518751 132667 544999 6509
Total 4201838 6720543 11140551 2334331 5949037 1170778

It is important to remark that the codes are adapted to NAG environment.

This means solving smooth optimisation problems, which implies reformulation
of some of our problems leading to ones with sparse matrices. This may cause
difficulties when the number of parameters is big as NAG is not meant to deal
with big sparse problems. Moreover, there might be monotonicity constraints as
weil. Even in our examples, we had to face some cases of ill-conditioning which
led us to introduce the corresponding corrections (now automatically included in
the routines) following NAG manual instructions.
 163

As we have to deal with small numbers in the distance problems, underflow

(standardly corrected) has been detected in some of the routines internally called
by NAG. Its only consequence is a huge amount of output.
A small theoretical error has been detected in some cases: d; is a decreasing
function of p (the exponent of the distance) and a few of our examples are in conflict
with this point. This has an explanation: the optimisation problems solved find
tI; with aprecision f, say; we then find d; with precision flIp. Due to the structure
of the problem, f cannot be very small so flIp is even bigger and the errors are
greater than expected.
Another problem is that we find local optima. In some cases, this has led to
a small p and, consequently, to a big r. We overcame it in an ad hoc fashion,
changing the initial solution. This is far from satisfactory.
The formulation of problems for SENSLIN and SENSBIL is straightforward.
It is a bit cumbersome for SENSGEN, as derivatives for the rating functions have
to be provided. A checking device, based on NAG's, helps to find where errors in
derivatives have been made. Also, other error messages tell us in which phase of
the routine there is an error and which error it is.
The amount of information generated by the analysis is very exhaustive and,
we believe, very useful to an analyst helping the DM to think more carefullyabout
her problem.
Of course we claim no panacea; some directions of improvement for the pro-
grams will be given in the next chapter. But we insist: the approach is feasible
(and successful).
Chapter 5

Conclusions

This chapter provides a summary of the ideas introduced and suggests some areas
of potential development.

5.1 Summary
The case was made for adopting a Bayesian approach to decision making. How-
ever, problems arising in practice led us to study decision making under partial
information and sensitivity analysis in decision making.
Decision making problems under partial information are characterised by the
data {{Wj(w)}j,w ES}, where the Wj's are evaluation functions and S are con-
straints on the parameters w modelling judgements. The three questions to ans wer
are:

• what are the axiomatic foundations of this model?

• what are the corresponding solution concepts?

• how can we compute these solutions?

As far as the first quest ion is concerned, if the Bayesian approach requires a weak
order regulating judgements (plus some conditions), the DPI model requires a
quasi order regulating judgements (plus some similar conditions). This applies
to values, utilities and probabilities. Consequently, alternatives are ordered in a
Pareto sense:

Then, the solutions are the nondominated ones. Potentially optimal alternatives
also seem acceptable, both from an intuitive point of view and because of their
relation with the nondominated ones. Algorithms are provided to compute those
 165

solutions in general problems and in some particular cases (linear and bilinear).
Other solution concepts are criticised on the grounds of being ad hoc. In summary,
we have a more robust theory of decision making based on a weaker set ofaxioms,
but embodying coherence, since it essentially implies carrying out a family of
coherent decision analyses.
When we have the data { { {'l1 j( w)} j, wES}, w}, we can rank the alternatives
by 'l1 j (w). However, we have to be ready to criticise this ranking. We argue
that sensitivity analysis is the means to help the DM explore her thoughts in
a constructive way to find which of her judgements are the most influential in
determining the decision. This inc1udes the criticism of the Flat Maxima Principle.
Several concepts of adjacent potential optimality help us to reduce the competitors
of the current optimal alternative. A dass of easily implementable sensitivity tools
based on gauge functions is then introduced which allows us to:

• identify critical judgements,

• identify competitors of the current optimal alternative,

• identify sensitivity issues according to certain sensitivity indices.

We then explore some questions on error modelling in judgemental measurement,

providing some other tools to deepen the analysis.
The above framework is implemented in SENSATO, a prototype library for a
sensitivity analysis package for decision aids. Some computational experience with
it is described, proving the feasibility and success of our approach.

5.2 Tüpics für furt her research

We shall mention a number of areas for further research. Some of them are mere
conjectures; für üthers, a more detailed descriptiün is given.

5.2.1 Decision making under partial information

Theory for finite problems

Für finite problems, the main point remaining is the possibility of incorpürating
bounds for parameters in additive models. Specifically, there is a fairly extensive
literature on upper and lower probabilities (see Fine, 1988); however, this litera-
ture tends to put the emphasis on quantitative aspects. Suppes (1974) treats the
behavioural side of the problem but he uses semiorders. See also Fishburn (1986).
 166

The aim would be to provide a quasi order representation as in 2.2, but including
upper and lower bounds on the probabilities.
Multiplicative and multilinear quasi order representations might also be con-
sidered.

Algorithms for finite problems

Our framework requires the solution of a large number of problems, classifiable in

a small number of categories. Constraints vary from one category to another. It is
possible to adopt a different approach reformulating the problems as nonsmooth,
according to the following steps.
First, we determine if we need sensitivity analysis. We solve

min (1l1.(w) - II.J.ax ll1;(w))

wES
J"'.
a. has competitors iff the optimal value is < 0. H wished, we can eliminate the
alternatives dominated by a. solving problems [Pj .] , as indicated in chapter 2.
Then, we look for the p.o. alternatives. For each aj, we solve

max (ll1 j (w) - T,ffll1 k (w))

wES

aj is p.o. iff the optimal value is ~ 0. To compute the a.p.o. alternatives we solve

min (max((ll1 j (w) -1l1.(W))2,O, Il1 j (w) -1l1.(w)))

wES

aj is a.p.o. to a. iff its optimal value is 0. Finally, remember that the distance
problems were nonsmooth (except in the Euclidean case).
Some advantages of this approach are the smaller number of problems to solve,
the smaller number of constraints per problem and the same structure in con-
straints and objective functions. However, the theory and practice of nonsmooth
optimisation problems is not yet as developed as that of smooth ones.
We have not studied multi-stage decision problems, though one of them served
for illustration purposes of SENSGEN. As Bayes' theorem is still valid, we can
see that backward dynamic programming extends to the computation of nondom-
inated and p.o. alternatives under the usual conditions. White and White (1989)
and White and EI Deib (1983, 1986) provide relevant references in this area.
 167

Theory and algorithms for infinite problems

Our theory is lirnited to the finite case. There is same theory for the infinite
case: for general preference quasi orders we can use Spilzrajn's lemma but, as we
discussed in section 2.4, this would lead to a representation different to ours; the
procedure in section 2.2 extends to the countable case; Giron and Rios (1980) give
a characterisation of general probability quasi orders; Aumann (1962) and White
(1972) give results for linear value functions. The problem with utility functions
seems more difficult (see Kannai (1963)) but perhaps can be treated as Giron and
Rios treat probabilities. Nondominated and potentially optimal alternatives are
still the solution concepts. As algorithms are concerned, we can appeal to vector
optirnisation results (Yu, 1985, The Luc, 1989). We can also discretise the problem
and apply algorithms for finite problems. It seems useful to apply Hazen's (1988)
differential results to special forms of value functions.

5.2.2 Sensitivity analysis

Other approaches to sensitivity analysis can be devised. For example, we have
concentrated on instability because we tried to find the nearest parameters to w
leading to a change in the suggested action; we could invert the procedure: solve
[P.] to find the parameters for which a. is best and present them to the DM to
determine if she finds them acceptable.
We have touched implicitly the topic of sensitivity of a parameter. We deal with
it giving a range of the parameter in which there is no change in the suggested
course of action. That is what we do with Tchebycheff distance problems: for
Wi E (Wi - P,Wi + p) there is no change in the optimal solution.
We have not dealt much with differential approaches. They seem more useful
in infinite problems. These problems require the development of procedures to find
fast bounds for the loss of optimality. \Ve can use results from stability theory for
systems of inequalities: the optimality of a. for w, implies that W is a solution of
the system

Several results study the stability of these systems. For example, when the in-
equalities are strict for w, if we use Tchebycheff's distance in Robinson's results
(1975, 1976) we get bounds of the kind

diam( S) .:; o(loss of optimality)

 168

where Cl is some constant. This can be used to know how small S has to be if the
DM is ready to accept some loss of optimality due to choosing a. wrongly.
Another point is that a.p.o. concepts are no longer useful, in general, in infinite
problems. Alternatively, we can use t-a.p.o. alternatives to a. in a similar vein as
t- Bayes actions are used in statistical decision theory, see Ferguson (1967).
More precise meaning has to be given to the concept of elose call, see Bell et
al. (1988).
Our emphasis has been on the need for more precision. We have dealt briefly
with some topics on errors in modelling, though we have seen how to use SENSATO
to quest ion the validity of a model. Several questions in error modelling were posed
in section 3.5. We insist on the issue of consistency of S; some ideas can be seen
in White et al. (1984). Some points in our error model need further exploration
such as the use of other distributions or improvements in computations in the
multivariate case.
We can also develop estimators for bounded normal means that should be
compared with others in similar settings such as those of Casella and Strawderman
(1981) or Berger (1985).

5.2.3 Computation
Though SENSATO is based on NAG, it is easily modifiable if another optimisation
library or paekage is finally used. Other routines can also be added by simple
modification of the existing ones: for example, it would be immediate to prepare a
routine not requiring the derivatives of the evaluation functions, based on NAG's
E04UCF.
Due to the advantages mentioned above, we should consider the design of a
nonsmooth version. Moreover, the use of a smooth code led to so me ill-conditioncd
problems, see 4.8. The points raised in that section should be considered, espe-
cially those referred to global optimisation. In bilinear models, we use quadratic
programming heavily; recent developments in bilinear programming might prove
useful, see Thieu (1988).
Some work is required in the formulation of problems. Some possibilities are:
built-in facilities to generate some types of constraints (monotonie, eonvex, ... ) on
the parameters; devices to build and derive symbolically the evaluation functions,
The inclusion of helps to choose the appropriate routines should be useful.
Mueh work is needed to find how to present the results. Apart from the graphs
we have used, there are other possibilities in multivariate data analysis graphie
teehniques, see Chambers el. al. (1983). The use of eolours might enhanee the pre-
 169

sentation. When comparing aJ and a. holistically, a graph of {(Ilt J( w), Ilt.( w)) }WES
should be of help. More work should be done to determine the usefulness of the
exploratory data analysis techniques used in section 4.2, even building a small
expert system to interpret the results.
Another possibility is displaying the results via semantic representations. We
have mentioned the process of modelling the judgemental inputs of the DM, which
consists of an iterative protocol of questioningJ answeringJ encoding and leads to
w. Through sensitivity analyses, we get parameters w leading to different optimal
alternatives. Thus, we could think of inverting the protocol, finding the questions
and the answers which would lead to these parameters, pose them to the DM and
use her comments to modify, if necessary, the estimate w andJor the set S. For
this, we could draw upon naturallanguage interfaces currently under development,
as in the COMMUNAL project (Fawcett, 1989).
A possible interface is given below (this would link to the last part of SEN-
SATO's output, see the appendix, which announces that the DM can either reduce
S along some direction or compare holistically aJ and a., to improve the current
status):
 170

Before restarting the analysis, the inclusion of a stage of Bayesian elicitation

is compulsory. Some thought is required to determine how to store the new con-
straints (in the bilinear and general cases) and how to correct the estimates when
they are infeasible (above we wrote 'project w' in general, without specifying any
method). There is also work to do on the tests: the results in chapter 3 are mere
first steps. Quite probably, we would have to use Monte Carlo integration.
Since the approach has been successful on a mainframe, we believe it is timely
to port this system to more available computers, viz PC's. One possibility is to
write interfaces between SENSATO and the main commercial decision aids.

5.2.4 Group decision making

Our results are useful in group decision making in two directions. First, the results
of the DPI part can be used to model agreement betweeen DM's: suppose that n
decision makers have beliefs on A regulated by ~j, i = 1, ... , n. Then, agreement
is defined via ~ as follows:

B ~ C {=} B jj C,Vi = 1, ... ,n,

with B, C E A. Similarly for preferences. The results of chapter 2 apply to j.

Second, as mentioned in French (1986, 1989), sensitivity analyses can be used in
group decision making to avoid sterile discussions and focusing on the crucial issues
for the problem. See also Phillips (1984).

5.2.5 Imprecision in expert systems

An important topic in expert systems is that of imprecision in probabilities, see
Lauritzen and Spiegelhalter (1988). We have seen that our framework can cope
with that imprecision in a coherent way. Some indications on this topic can be
seen in Rios Insua (1989). We would have to develop fast procedures for updating
(families of) probabilities. Note that we can use most of Lauritzen and Spiegelhal-
ter's graphical structures. Parallel processing might be particularly useful in this
area.

5.2.6 Restructuring
Restructuring concepts, as in Shien ehen and Yu (1989), are of use in our problems.
In our context they mean: find nontrivial transformations of 5 such that a. is
s.o. or such that the 1055 of optimality is limited by some quantity. Then, these
 171

transformations could be presented to the DM to determine if she finds them

reasonable and, consequently, modify W or S. In asense, this is what we do with
the sensitivity tools introduced in chapter 3.

5.2.7 Integration with other methodologies

A point to consider is the integration with interactive methods, such as those in-
troduced in section 1.3. After a sensitivity analysis the DM should have a better
picture of her problem so she might be able to make some holistic comparisons.
This is partly acknowledged in the schema of section 5.2.3. A run with an interac-
tive method would also let her know the 'topography of the problem' (Daellenbach
and Buchanan, 1989), therefore serving as a stimulus to think more about it. So
we can think of a sequence of stages interactive method- (Bayesian) information
elicitation- sensitivity analysis. Of course, the interactive method should at least
possess the properties announced in 1.3.4. For example, in a first phase we could
use the method of 1.3.4, and in later phases some modification of it.
This would correspond to a horizontal approach. In many problems, objectives
are structured according to a hierarchy. A vertical approach would consider as-
cending in the hierarchy when more information is required (see e.g. Rios Insua,
S. (1981)). Our opinion is that decision methodologies should strive to integrate
vertical and horizontal approaches within the Bayesian framework. This, along
with a progressive introduction of information technology and improved methods
of information elicitation, should keep us busy for some time.
Bibliography

[1] Adams, E. (1965) Elements of a theory of inexact measurement, Philos. Sei.,

32, 205-228.

[2] Arrow, K. and Raynaud, H. (1987) Soeial Choiee and Multieriteria Deeision
Making, MIT Press.

[3] Aumann, R. (1962) Utility theory without the completeness axiom, Econo-
metriea, 30, 3, 445-462; (1964) 32, 1,210-212.

[4] BarcJay, S. (1987) A user's manual to HIVIEW, Deeision Analysis Unit,

LSE.

[5] Barron, H. and Schmidt, C. P. (1988a) Sensitivity analysis of additive mul-

tiattribute value models, Opns. Res., 36, 1, 122-127.

[6] Barron, H. and Sehmidt, C. P. (1988b) Entropy based selection with multiple
objectives, Naval Research Logistics Quarterly, 35, 643-654.

[7] Bazaraa, M. and Shetty, C. (1979) Nonlinear Programming, Wiley.

[8) Bell, D., Raiffa, H. and Tversky, A. (1988) Decision Making, Cambridge U.P.

[9] Bellman, R. and Zadeh, L. (1970) Decision making in a fuzzy environment,

Mgt. Scienee, 17, 141-164.

[10] Belton, V. (1986) A comparison of the AHP and a simple multi-attribute

value function, EJOR, 26, 7-21.

[U] Belton, V. and Vickers, S. (1989) VISA - VIM for MCDA, in Lockett (ed),
Improving Deeision Making in Organisations, LNEMS, Springer Verlag.

[12J Berger, J. (1980) A robust generalized Bayes estimator and confidence region
for a multivariate normal mean, Annals 0/ Statisties, 8, 716-761.
 173

[13] Berger, J. (1984) The robust Bayesian viewpoint, in Kadane (ed), Robustness
of Bayesian Analyses, North Holland.

[14] Berger, J. (1985) Statistical Decision Theory and Bayesian Analysis,

Springer Verlag.

[15] Berger, J. and Berliner, M. (1986) Robust Bayes and empirical Bayes analysis
with e-contarninated priors, Annals of Stat., 14,461-486.

[16] Berkeley, D. and Humphreys, P. (1982) Structuring decision problems and

the 'bias' heuristic, Acta Psychologica, 50, 201-252.

[17] Bouyssou, D. (1984) Decision aid and expected utility theory: a critical
survey, in Hagen and Wenstop (eds), Progress in Utility and Risk Theory,
Reidel.

[18] Casella, G. and Strawderman, W. (1981) Estimating a bounded normal

mean, Annals of Statistics, 9, 870-878.

[19] Chambers, J. M., Cleveland, W. S., Kleiner, B. and Tukey, P. A. (1983)

Graphical Methods for Data Analysis, Duxbury Press.

[20] Chankong, V. and Haimes, Y. (1982) Multiobjective Decision Making, North

Holland.

[21] Charnetski, J. and Soland, R. (1978) Multiple attribute decision making with
partial information: the comparative hypervolume criterion, Nav. Res. Log.
Quart., 25, 279-288.

[22] Charnetski, J. and Soland, R. (1979) Multiple attribute decision making with
partial information: the expected value criterion, Nav. Res. Log. Quart., 26,
249-256.

[23] Chatterjee, H. K. (1986) Multi-objective lnteractive Programming, Ph. D.

Thesis, U. Manchester.

[24] Chuang, D. T. (1984) Further theory of stable distributions, in Kadane (ed),

Robustness of Bayesian A nalyses, North Holland.

[25] Clarke, F. H. (1975) Generalized gradients and its applications, Trans. Amer.
Math. Soc., 205, 247-262.

[26] Clarke, F. H. (1983) Optimisation and Nonsmooth Analysis, Wiley.

 174

[27] Cyert, R. M. and De Groot, M. (1975) Adaptive utility, in Day (ed) Adaptive
Economics, Prentice Hall.

[28] Daellenbach, H. and Buchanan, J. (1989) Some desirable properties of in·

teractive multi·objective programming methods, in Lockett (ed) Improving
Decision Making in Organisations, LNEMS, Springer Verlag.

[29] Danskin, J. (1967) The Theory 0/ Max·min and its Applications to Weapon
Allocation Problems, Springer Verlag.

[30] De Groot, M. (1970) Optimal Statistical Decisions, McGraw Hill.

[31] De Robertis, L. and Hartigan, J. A. (1981) Bayesian inference using intervals

of measures, Annals 0/ Statistics, 9, 235·244.
[32] Dempster, A. P. (1967) Upper and lower probabilities induced by a multi·
valued mapping, Anns. Math. Stats., 38, 325-339.

[33] Diaconis, P. (1985) Theories of data analysis: from magical thinking through
classical statistics, in Hoaglin, Mosteller , Tukey (eds) Exploring Data Tables,
Trends, and Shapes, Wiley.

[34] Diaconis, P. and Freedman, D. (1986) On the consistency of Bayes estimates,

Annals 0/ Stats., 1, 1-26.
[35] Dickey, J. M. (1976) Approximate posterior distributions, JASA, 71, 680·
689.

[36] Dyer, J. and Ravinder, H. (1986) A rationale for additive decomposition

in multiattribute utility assessment, in Sawaragi, Inoue, Nakayama (eds)
Towards Interactive and Intelligent Decision Support Systems, LNEMS,
Springer Verlag.

[37] Edwards, W. and Newman, J. R. (1982) Multi-attribute Evaluation, Sage.

[38] Einhorn, H. and Hogarth, R. (1975) Unit weighting schemes for decision
making, Org. Beh. Hum. PerJ., 13, 171·192.

[39] Eliashberg, J. and Hauser, J.(1985) A measurement error approach for mod·
elling consumer risk preference, Mgt. Science, 31, 1-25.

[40] Evans, J. R. (1984) Sensitivity analysis in decision theory, Decision Sciences,

15,239·247.
 175

[41J Farquhar, P. H. (1984) Utility assessment methods, Mgt. Sc., 30, 1283-1300.

[42J Fawcett, R. P. (1989) Reporl on the COMMUNAL Project January - July,

1989, COMMUNAL, Rep. 25, Comp. Ling. Unit., UWCC.

[43] Fine, T. L. (1973) Theories of Probability, Ac. Press.

[44] Fine, T. L. (1988) Lower prob ability models for uncertainty and nondeter-
ministic processes, Jour. Stat. Plan. InJ., 20, 389-411.

[45) Fischhoff, B. (1980) Clinical decision analysis, Opns. Res., 28, 28-43.

[46) Fishburn, P. C. (1965) Analysis of decisions with incomplete knowledge of

probabilities, Opns. Res., 13, 217-237.

[47) Fishburn, P. C. (1969) Weak qualitative probability on finite sets, Annals

Math. Stats., 40, 2118-2126.

[48] Fishburn, P. C. (1970) Utility Theory for Decision Making, Wiley.

[49) Fishburn, P. C. (1971) One way expected utility with finite consequence
space, Annals of Mathematical Statistics, 42, 572-577.

[50] Fishburn, P. C. (1977) Approximations of two attribute utility functions,

Maths. of OR, 2, 30-44.

[51J Fishburn, P. C. (1982) The Foundations of Expected Utility, D. Reidel.

[52J Fishburn, P. C. (1985) Interval Orders and Inte1"/}al Graphs, Wiley.

[53] Fishburn, P. C. (1986) The axioms of subjective prob ability, Statistical Sci-
ence, 1, 335-358.

[54J Fishburn, P., Murphy, A. and Isaacs, H. (1968) Sensitivity of decisions to

probability estimation errors: a reexamination, Opns. Res., 16, 254-267.

[55] Freeling, A. (1980) Fuzzy sets and decision analysis, IEEE Trans., SMC-10,
341-354.

[56] French, S. (1978) A Bayesian three-stage model in cristallography, Acta

Cryst., 34, 728-738.

[57J French, S. (1982) On the axiomatisation of subjective probabilities, Theory

and Decision, 14, 19-23.
 176

[58] French, S. (1984) Fuzzy decision analysis: some criticisms, in Zimmermann,

Zadeh and Gaines (eds), Fv.zzy Sets and Decision Analysis, North Holland.

[59] French, S. (1986) Decision Theory, EIlis Horwood.

[60] French, S. (1987) Fuzzy sets: the unanswered questions, Res. Rep., Stat.
Lab., U. Manchester.

(61] French, S. (1989) Readings in Decision Analysis, Chapman and Hall.

(62] French, S. and Rios !nsua, D. (1989) Partial information and sensitivity anal-
ysis in multi-objective decision making, in Lockett (ed), lmproving Decision
Making in Organisations, LNEMS, Springer Verlag.

[63] Gardenfohrs, N. and Sahlin, P. (1988) Decision, Probability and Utility, Cam-
bridge U. P.

(64] Geoffrion, A., Dyer, J. and Feinberg, S. (1972) An interactive approach to

multicriteria optimization with an application to the operation of an aca-
demic department, Mgt. Sc., 19, 357-368.

(65] Gill, P., Murray, W. and Wright, M. (1981) Practical Optimization, Prentice
Hall.

[66] Giron, F. (1989) Mixturas de distribuciones normales con aplicaciones a

problemas estadisticos complejos, presented at the Real Academia de Cien-
das, Madrid.

(67] Giron, F. and Rios, S. (1980) Quasi-Bayesian behaviour: a more realistic

approach to decision making?, in Bernardo, De Groot, Lindley and Smith
(eds), Bayesian Statisties, Valencia U.P.

(68] Goicoechea, A., Hansen, D. and Duckstein, L. (1982) Multi-objeetive Deci-

sion Analysis, Wiley.

[69] Gomez Orea, D., Quintana, A., Aguado, M. and Villarino, M. (1989) Modelo
para La Evaluacion del Impacto Ambiental de Carreteras, ETSIA, Madrid.

[70] Good, 1. J. (1962) Subjective probability as a measure of a nonmeasurable

set, in Nagel, Suppes and Tarski (eds) Logie, Methodology and Philosophy of
Seienee, Stanford U. P.
 177

[71] Good, 1. J. (1980) Some history of the hierarchical Bayesian methodology, in

Bernardo, De Groot, Lindley and Smith (eds), Bayesian Statistics, Valencia
U. P.

[72J Hannon, E. L. (1981) Obtaining nondominated priority vectors for multiple

objective decision making problems: problems with different combinations
of cardinal and ordinal information, IEEE Tmns., SMC-11, 538-543.

[73J Hansen, P., Labbe, M. and WendelI, R. (1989) Sensitivity analysis in multiple
objective linear programming: the tolerance approach, EJOR, 38, 63-69.

[74] Hazen, G. B. (1986) Partial information, dominance and potential optimality

in multi-attribute utility theory, Oprs. Res.,34, 296-310.

[75] Hazen, G. B. (1988) Characterizations of nonconical dominance in multiple

objective decision-making, Maths. of OR, 13, 174-189.

[76] Hershey, J., Kunreuther, H. and Schoemaker, P. (1982) Sources of bias in

assessment procedures of utility functions, Mgt. Sc., 28, 936-954.

[77J Humphreys, P. and Wisudha, A. (1987) Methods and tools for structuring
and analysing decision problems: vols. 1, 2, Tech. Rep.) Decision Analysis
Unit, LSE.

[78] Hwang, J. T. (1985) Universal domination and stochastic domination. Deci-

sion theory simultaneously under a broad dass of loss functions, Annals of
Statistics, 13, 295-314.

[79J Is aacs , H. (1963) Sensitivity of decisions to prob ability estimation errors,

Opns. Res., 11, 536-552.

[80] Jaffray, J. Y. (1989) Some experimental findings in decision making and their
implication, EJOR, 38, 301-306.

[81J Johnson, N. and Kotz, S. (1972) Distributions in Statistics: Multivariate

Distributions, Wiley.

[82J Kadane, J. B. (1984) Robustness of Bayesian Analyses, North Holland.

[83] Kadane, J. B. and Chuang, R. (1978) Stable decision problems, Annals of

Statistics, 6, 1095-1110.

[8.4] Kahneman, D., Slovic, P. and Tversky, A. (1982) Judgement under Uncer-
tainty: Heuristics and Biases, Cambridge U. P.
 178

[85] Kamenetzky, R. D. (1982) The relationship between the analytic hierarchy

process and the additive value function, Decision Sciences, 13, 702-713.

[86] Kannai, Y. (1963) Existence of a utility in infinite dimensional partially

ordered spaces, Israel Journal of Mathematics, 1,229-234.

[87] Keefer, D. and Pollock, S. (1980) Approximation and sensitivity in multiob-

jective resource allocation, Opns. Res., 28, 114-128.

[88J Keeney, R. and Raiffa, H. (1976) Decisions with Multiple Objectives: Pref-
erences and Value Tradeoffs, Wiley.

[89J Kirkwood, C. (1982) A case history of nuclear power plant selection, JORS,
33, 353-363.

[90J Kirkwood, C. and Sarin, R. (1985) Ranking with partial information: a

method and an application, Oper. Res., 33, 38-48.

[91J Krnietowicz, Z. and Pearman, A. (1981) Decision Theory and Incomplete

Knowledge, Gower.

[92J Kohlberg, E. (1983) Consistency of finite systems of convex inequalities,

Math. Prog., 25, 359-362.

[93J Koopman, B. O. (1940) The bases of probability, Bull. Am. Math. Soc., 41,
269-292.

[94] Kraft, C., Pratt, J. and Seidenberg, A. (1959) Intuitive probability on finite
sets, Ann. Math. Stats., 30, 408-419.

[95J Krantz, D., Luce, D., Suppes, P. and Tversky, A. (1971) Foundations of
Measurement, Acadernic Press.

[96J Krzysztofowicz, R. (1983) Strength of preference and risk attitude in utility

measurement, Org. Beh. Hum. Perf., 31, 88-113.

[97J Laskey, K. and Fischer, G. (1987) Estimating utility functions in the presence
of error, Mgt. Science, 33, 8,965-980.

[98J Lauritzen, S. and Spiegelhalter, D. (1988) Local computations with prob-

abilities on graphicaI structures and their applications to expert systems,
JRSS B, 50, 157-224.
 179

[99] La Valle, 1. (1978) Fundamentals of DecÜlion Analysis, Holt, Reinhart and

Winston.

[100] Leamer, E. E. (1978) Specijication Searches, Wiley.

[101] Leung, P. (1978) Sensitivity analysis of the effect of variations in the form
and parameters of a multi-attribute utility model: a survey, Behav. Sc., 23,
478-485.

[102] Levi, 1. (1974) On indeterminate probabilities, Journal of Philosophy, 71,

391-418.

[103] Lindley, D. V. (1985) Making DecÜlions, Wiley.

[104] Lindley, D. V. (1986) The reconciliation of decision analyses, Oper. Resch,

34, 289-295.

[105] Lindley, D. V., Tversky, A. and Brown, R. V. (1979) On the reconciliation

of probability assessments, JRSS A, 142, 146-180.

[106) Malakooti, B. (1985) Assessment through strength of preference, Large Scale

Systems, 8, 169-182.

[107] Malakooti, B. (1989) Identifying nondominated alternatives with partial in-

formation for multiple objective discrete and linear programming problems,
IEEE Trans., SMC-19, 95-107.

[108) Marcotte, O. and Soland, R. (1986) An interactive branch and bound algo-
rithm for multicriteria optimization, Mgt. Science, 32, 61-75.

[109) Matheiss, T. H. and Rubin, D. S. (1980) A survey and comparison of methods

for finding all vertices of convex polyhedral sets, Maths. of OR, 5, 167-185.

[110] McNamee, P. and Celona, J. (1987) DecÜlion AnalysÜl for the Professional
with Supertree, Scientific Press.

[111] McClelland, G. (1978) Equal versus differential weighting for multiattribute

decisions, Rep. 207, University of Colorado, Institute of Behavioral Sciences.

[112] Milnor, J. (1954) Games against nature, in Thrall, Coombs and Davis (eds)
Decision Processes, Wiley.

[113) Minoux, M.(1983) Programation Mathematique, Dunod.

 180

[114J NAG FORTRAN Library Reference Manual (Mark 13) (1988) Numerical
Algorithms Ltd., Oxford.

[115] Novoa, J. 1. and Halff, A. H. (1977) Management of flooding in a fully

developed low cost housing neighbourhood, Water Resources Bulletin, 13,
6, 1237-1252.

[116] O'Hagan, A. (1988) Probability: Methods and Measurement, Chapman Hall.

[117] Phillips, 1. D. (1984) A theory of requisite decision models, Acta Psycholog-

ica, 56, 29-48.

[118] Pierce, D. and Folks, J. L.(1969) Sensitivity of Bayes procedures to the prior
distribution, Opns. Res., 17, 344-350.

[119] Polak, E., Mayne, J. Q. and Ward, Y. (1983) On the extension of constrained
optimization algorithms from differentiable to nondifferentiable problems,
SIAM Jour. Gon. Opt., 21, 179-203.

[120] Pratt, J. W. and Meyer, R. F. (1968) The consistent assessment and fairing
of preference functions, IEEE Trans., SSC-4, 270-278.

[121] Prior, W. J. (1970) Ghost-SO User Manual, Release 7, UKAEA Culham Lab.

[122] Pugh, E. and Winslow, G. (1966) The Analysis 0/ Physical Measurements,

Add. Wesley.

[123] Ramsay, J. O. and Novick, M. R. (1980) PLU robust Bayesian decision

theory: point estimation, JASA, 75, 901-907.

[124] Ravinder, H. V., Kleinrnuntz, D. and Dyer, J. (1988) The reliability of sub-
jective probabilities obtained through decomposition, Mgt. Sc., 34, 186-199.

[125J Rios, S. (1975) Ordre quasi-Bayesien des regles de decision, Bull. ISI, 46,
694-698.

[126] Rios, S. (1977) Analisis de Decisiones, lCE.

[127] Rios, S. and Rios lnsua, S. (1986) Vector value function and vector dis-
tance methods in multicriteria optimization, presented at Int. Gon/. Vector
Optimization, Darmstadt.

[128] Rios, S., Rios lnsua, S. and Rios lnsua, M. J. (1988) Procesos de Decision
Multicriterio, EUDEMA.
 181

[129] Rios Insua, S. (1980) Decisiones con Multicriterios, Ph.D. Thesis, UCM.

[130] Rios Insua, D. (1987) Optimizacion Vectorial en el Tiempo, Tesina, UCM.

[131 J Rios Insua, D. (1987) Sobre soluciones optimas en problemas de optimizacion

multiobjetivo, Trabajos de Investigacion Operativa, 2, 1, 49-67.

[132J Rios Insua, D. (1988) An interactive approach for solving multi-objective

optimisation problems in discrete sets, Proceedings XVII SEIO Meeting, Al-
icante U.P.

[133J Rios Insua, D.(1988) Modelling Imprecision, presented at the Workshop on

Choice and Aggregation of Imprecise Opinions, Brussels.

[134] Rios Insua, D. (1989) Imprecision in probabilities, presented at the Forum

on Expert Systems and Statistics, Edinburgh.

[135J Rios Insua, D. and French, S. (1989) A framework for sensitivity analysis in
discrete multiobjective decision making, to appear in European Journal of
Operational Research.

[136] Roberts, F. (1979) l'vfeasurement Theory, Addison Wesley.

[137] Robinson, S. M. (1975) Stability theory for systems of inequalities. Part I:

Linear systems, SIAJ'vJ Jour. Num. Anal., 12,5, 754-769.

[138J Robinson, S. M. (1976) Stability theory for systems of inequalities. Part II:
Differentiable nonlinear systems, SIAM Jour. Num. Anal., 13, 4, 497-513.

[139J Rockafellar, T. (1970) Convex Analysis, Princeton U.P.

[140J Rockafellar, T. (1981) The Theory of Subgradients and ils Applications 10

Problems of Optimization, Helderman Verlag.

[141] Roubens, M. and Vincke, Ph. (198.5) Preference i'vfodelling, LNEMS, Springer
Verlag.

[142] Roy, B. (1973) How outranking relation helps multiple criteria decision-
making, in Cochrane, Zeleny (eds) 111ultiple Criteria Decision kJaking, South
Carolina UP.

[143] Roy, B. and Vincke, Ph. (1984) Relational systems of preferences with one
or more pseudo criteria: so me new concepts and results, Mgt. Sc., 30, 1323-
1355.
 182

[144] Saaty, T. L. (1980) The Analytic Hiemrchy Process, McGraw Hill.

[145] Sage, A. P. (1981) On sensitivity analysis in systems for planninng and

decision support, Journal 0/ the Fmnklin Institute, 312, 265-291.

[146] Sage, A. P. and White, C. C. (1984) ARIADNE: A knowledge-based in-

teractive system for planning and decision support, IEEE Tmns., SMC-14,
35-47.

[147] Samson, D. (1988) Managerial Decision Analysis, Irwin.

[148] Sarin, R. K. (1977) Screening of multiattribute alternatives, OMEGA, 5,

481-489.

[149] Sarin, R. K. (1977) Interactive evaluation and bound procedure for selecting
multi-attributed alternatives, TIMS Studies Mgt. Sc., 6, 211-214.

[150] Savage, L. J. (1954) The Foundations 0/ Statistics, Wiley.

[151] Schnabel, R. (1982) Determining feasibility of a set of nonlinear inequality
constraints, Math. Prog. Study, 16, 137-148.

[152] Schneller, G. o. and Sphicas, G. P. (1983) Decision making under uncer-

tainty: Starr's domain criterion, Theory and Decision, 15, 321-336.

[153] Schneller, G. o. and Sphicas, G. P. (1985) On sensitivity analys;s in decision

theory, Decision Sciences, 16, 399-408.

[154] Schoner, B. and Wedley, W. (1989) Alternative scales in AHP, in Lockett

(ed) Improving Decision Making in Organisations, LNEMS, Springer Verlag.

[155] Scott, D. (1964) Measurement structures and linear inequalities, Journal 0/

Mathematical Psychology, 1, 233-247.

[156] Shafer, G. (1976) A Mathematical Theory 0/ Evidence, Princeton UP.

[157] Shien Chien, I. and Yu, P. L. (1989) MCDM decision support using restruc-
turing concept, in Lockett (ed) Improving Decision Making in Organisations,
LNEMS, Springer Verlag.

[158] Smith, C. A. B. (1961) Consistency in statistical inference and decision,

JRSS A, 128, 469-499.

[159] Smith, J. Q. (1988) Bayesian Decision Analysis, Chapman and Hall.

 183

[160] Steuer, R. (1986) Multiple Criteria Optimization, Wiley.

[161] Stewart, T. J. (1981) An interactive approach to multiple criteria decision

making based on statistical inference, IEEE Trans., SMC-ll, 733-740.

[162] Stewart, T. J. (1984) Inferring preference in multiple criteria decision anal-

ysis using a logistic regresssion model, Mgt. Sc., 30, 1067-1077.

[163] Stewart, T. J. (1986) A combined logistic regression and Zionts-Wallenius

methodology for multiple criteria linear programming, EJOR, 24, 295-304.

[164] Stoer, J. and Witzgall, C. (1970) Convezity and Optimization in Finite Di-
mension,. I, Springer Verlag.

[165] Suppes, P. (1974) The measurement of belief, JRSS B, 36, 160-175.

[166) Sussman, B., Gremy, J. P., Mare, M. and Buffet, P. (1967) Peut on choisir
en tenant compte des criteres multiples? Une methode (ELECTRE) et trois
applications, METRA, 6, 283-316.

[167] The Luc, D. (1989) The Theory 0/ Vector Optimization, LNEMS, Springer
Verlag.

[168] Thieu, T. V. (1988) A note on the solution of bilinear programming problems

by reduction to concave minimization, Math. Prog., 41, 245-260.

[169) Tversky, A. (1967) Additivity, l1tility and subjective probability, Jour. Math.
Psychol., 4, 175-201.

[170) Villegas, C. (1977) On the representation of ignorance, JASA, 72, 651-654.

[171) Von Winterfeldt, D. and Edwards, W. (1982) Costs and payoffs in perceptual
research, Psych. Bull., 19, 609-622.

[172) Von Winterfeldt, D. and Edwards, W. (1987) Decision Analysis and Be-
havioural Research, Cambridge U.P.

[173) Walley, P. and Fine, T. (1979) Varieties of modal and comparative proba-
bility, Synthese, 41, 321-374.

[174) Wallsten, T. and Budescu, D. (1983) Encoding subjective probabilities: a

psychological and psychometrie review, Mgt. Sc., 29, 151-174.

[175) Watson, S. R. and Brown, R. V. (1978) The valuation of decision analysis,

JRSS A, 141, 69-78.
 184

[176] Watson, S. R. and Buede D. M. (1987) Decision Synthesis, Cambridge V.P.

[177] Watson, S. R., Weiss, J. W. and Donnell, M. L. (1979) Fuzzy decision anal-
ysis, IEEE Trans., SMC-9, 1-9.

[178] Weber, M. (1987) Decision making with incomplete information, EJOR, 28,
44-57.

[179] White, C. C. (1979) Multi-parametric sensitivity in decision-making under

uncertainty, Computers and Biomedical Research, 12, 125-130.

[180] White, C. C., Dozono, S. and Scherer, W. (1983) An interactive procedure

for aiding multiattribute alternative selection, OMEGA, 11, 212-214.

[181] White, C. C. and EI Deib, H. (1983) Multi-stage decision making with im-
precise utilities, in Hansen (ed) Essays and Surveys on Multiple Criteria
Decision Making, LNEMS, Springer Verlag.

[182] White, C. C. and EI Deib, H. (1986) Parameter imprecision in finite state,

finite action dynamic programs, Oper. Res., 34, 120-129.

[183] White, C. C., Sage, A. and Dozono, S. (1984) A model of multi-attribute

decision making and tradeoff weight determination under uncertainty, IEEE
Trans., SMC-14, 223-229.

[184] White, C. C., Sage, A. and Scherer, W. (1982) Decision support with par-
tially identified parameters, Large Scale Systems, 3, 177-189.

[185] White, C. C. and White, D. J. (1989) Markov decision processes, EJOR, 39,
1-16.

[186] White, D. J. (1972) Vncertain value functions, Mgt. Sc., 19,31-41.

[187] White, D. J. (1980a) Optimality and Efficiency I, EJOR, 4, 346-356.

[188] White, D. J. (1980b) Optimality and Efficiency II, EJOR, 4, 426-427.

[189] White, D. J. (1982) Optimality and EjJiciency, Wiley.

[190] White, D. J. (1983) The foundations of multi-objective interactive program-

ming: some questions, in Hansen (ed) Essays and Surveys on Multiple Cri-
teria Decision Making, LNEMS, Springer Verlag.

[191] White, D. J. (1989) Vsing weighting vectors to solve multiple attribute prob-
lems: a framework, EJOR, 293-304.
 185

[192] Wierzbicki, A. (1982) Multiobjective trajectory optimization and model

semiregularization, in Grauer, Lewandowski and Wierzbicki (eds) Multiob-
jective and Stochastic Optimization, IIASA.

[193] Wolfenson, M. and Fine, T. L. (1982) Bayes-like decision making with upper
and lower probabilities, JASA, 77, 80-88.

[194] Yager, R. (1977) Multiple objective decision making using fuzzy sets, Int.
Jour. Man-Machine Studies, 9, 375-382.

[195] Yager, R. (1981) A new methodology for ordinal multiobjective decisions

based on fuzzy sets, Decision Sciences, 12, 589-600.

[196] Yoon, K. (1989) The propagation of errors in multi at tribute decision analy-
sis, JORS, 40, 681-686.

[197] Yu, P. L. (1985) Multiple Criteria Decision Making, Plenum.

[198] Zimmermann, H. J. (1987) Fuzzy Sets, Decision Making and Expert Systems,
Kluwer.
Appendix

A SampIe of SENSATO output

This appendix provides the output of the analysis of Dallas' problem with SEN-
SATO. See seetions 4.1 and 4.2 for comments and details on the results.
 187

CURRENT OPTIMAL SOLUTION IS A 1 WITH RATING 296.10000

THE RANKING or ALTERNATIVES ACCORDING TO RATING IS :
ALTERNATIVE RATING
1 296.1000
6 293.2000
2 285.2000
3 275.8000
4 257.6000
5 245.1000
7 236.9000
8 167.0000

THERE ARE 6 NONDOMINATED ALTERNATIVES:

ALTERNATIVE 1
ALTERNATIVE 6
ALTERNATIVE 2
ALTERNATIVE 3
ALTERNATIVE 4
ALTERNATIVE 7
•• ******************* ••••• ****

ALTERNATIVE 6 IS POTENTIALLY OPTIMAL WITR ADVANTAGE-59.0714

ALTERNATIVE 2 IS POTENTIALLY OPTIMAL WITR ADVANTAGE-12.4100
ALTERNATIVE 3 IS POTENTIALLY OPTIMAL WITH ADVANTAGE-46.9000
ALTERNATIVE 4 IS POTENTIALLY OPTIMAL WITR ADVANTAGE -3.1667
THERE ARE 5 POTENTIALLY OPTIMAL NONDOMINATED ALTERNATIVES:
ALTERNATIVE 1
ALTERNATIVE 6
ALTERNATIVE 2
ALTERNATIVE 3
ALTERNATIVE 4

THE CURRENT OPTIMAL NONDOMINATED ALTERNATIVE IS A 1

ALTERNATIVE A 6 IS ADJACENT POTENTIALLY OPTIMAL

PARAMETERS rOR WHICH OPTIMALITY IS SHARED ARE:
5.45098 10.00000 0.00000 5.00000 6.31373
10.00000 8.23529 5.00000 0.00000 5.00000
ALTERNATIVE A 2 IS ADJACENT POTENTIALLY OPTIMAL
PARAMETERS rOR WHICH OPTIMALITY IS SHARED ARE:
5.45098 10.00000 0.00000 5.00000 6.31373
10.00000 8.23529 5.00000 0.00000 5.00000
ALTERNATIVE A 3 IS ADJACENT POTENTIALLY OPTIMAL
PARAMETERS rOR WH ICH OPTIHALITY IS SHARED ARE:
9.26961 6.72689 0.00000 5.00000 5.76821
10.00000 8.23529 5.00000 0.00000 5.0.0.000
 188

ALTERNATIVE A 4 IS AnJACENT POTENTIALLY OPTIMAL

PARAMETERS FOR WHleH OPTIMALITY IS SHARED ARE:

20.00000 10.00000 0.00000 5.00000 5.70588

5.00000 5.00000 3.83824 0.45588 0.00000

THERE ARE APO ALTERNATIVES TO A 1

ALTERNATIVE 6
ALTERNATIVE 2
ALTERNATIVE 3
ALTERNATIVE 4

PIlS
THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING
OF THE CURRENT OPTIMUM BY ALTERNATIVE 6
IS 0.4462

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

10.00000 7.00000 3.00000 5.00000 5.77692

8.00000 9.00000 4.22308 2.00000 1.00000

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF TRE CURRENT OPTIMUM BY ALTERNATIVE 2
IS 9.7000

TAE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

9.00000 6.00000 2.00000 5.00000 4.15000
10.00000 9.00000 4.00000 2.00000 3.85000

TRE MINIMUM DISTANCE TO PRODUCE TRE OUTRANKING

OF TRE CURRENT OPTIMUM BY ALTERNATIVE 3
IS 4.7160

TRE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

10.00000 7.00000 0.64198 5.00000 6.00000
9.35802 9.00000 5.00000 2.00000 1.00000

TRE MINIMUM DISTANCE TO PRODUCE TRE OUTRANKING

OF TAE CURRENT OPTIMUM BY ALTERNATIVE 4
IS 20.2000

TAE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

16.10000 10.00000 3.00000 5.00000 0.90000
8.00000 5.00000 4.00000 3.00000 0.00000

ALTERNATIVE 6 GIVES TRE MINIMUM DISTANCE

WHICH IS 0.446
THE MAXIMUM DISTANCE IS 32.0196
THE SENSITIVITY INDEX IS 0.01393

TO IMPROVE THE CURRENT STATUS YOU HAVE TO:

COMPARE HOLISTICALLY ALTERNATIVES 6 AND 1

WHERE TAE CRITICAL PARAMETERS ARE:
 189

10.00000
7.00000
3.00000
5.00000
5.77692
8.00000
9.00000
4.22308
2.00000
1. 00000

OR REDUCE TRE FEASIBLE SET OF PARAMETERS, ALCNG TRE DIRECTION

10.0000 5.0000
7.0000 4.0000
3.0000 10.1579
5.0000 5.0000
6.0000 9.9953
8.0000 5.0000
9.0000 5.0000
4.0000 3.0000
2.0000 2.8559
1.0000 4.9908

***** ••• *.****.***.*******.*.*

PI IS 2
TRE MINIMUM DISTANCE TO PRODUCE TRE OUTRANKING
OF TRE CURRENT OPTIMUM BY ALTERNATIVE 6
IS 0.2507

TRE LEAST CRANGES ARE ATTAINED AT PARAMETERS:

9.99711 7.01879 2.93208 5.00000 5.86705

8.07732 8.98627 4.14885 2.06214 0.91040

TRE MINIMUM DISTANCE TO PRODUCE TRE OUTRANKING

OF TRE CURRENT OPTIMUM BY ALTERNATIVE 2
1S 3.8747

TRE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

8.69868 5.69868 1. 69868 5.00000 4.69868
10.00000 10.20709 4.17422 2.17422 2.64975

TRE MINIMUM DISTANCE TO PRODUCE TRE OUTRANKING

OF TRE CURRENT OPTIMUM BY ALTERNATIVE 3
IS 2.0375

TRE LEAST CRANGES ARE ATTAINED AT PARAMETERS:

10.52942 6.30244 2.09795 5.00000 5.30244
8.75436 9.59076 4.99975 1.71143 0.71143

TRE MINIMUM DISTANCE TO PRODUCE TRE OUTRANKING

OF TRE CURRENT OPTIMUM BY ALTERNATIVE 4
IS 8.3472

THE LEAST CRANGES ARE ATTAINED AT PARAMETERS:

15.68914 10.00000 0.50955 5.00000 3.50955
6.60016 5.69159 5.00000 3.00000 0.00000
 190

ALTERNATIVE 6 GIVES THE MINIMUM DISTANCE

WHICH IS 0.251
THE MAXIMUM DISTANCE IS 14.8997
THE SENSITIVITY INDEX IS 0.01683
TO IMPROVE THE CURRENT STATUS YOU HAVE TO:

COMPARE HOLISTICALL1 ALTERNATIVES 6 AND 1

WHERE THE CRITICAL PARAMETERS ARE:
9.99711
7.01879
2.93208
5.00000
5.86705
8.07732
8.98627
4.14885
2.06214
0.91040

OR REDUCE THE FEASIBLE SET OF PARAMETERS, ALONG THE DIRECTION

10.0000 5.0000
7.0000 4.0000
3.0000 15.0000
5.0000 5.0000
6.0000 10.0000
8.0000 5.0000
9.0000 5.0000
4.0000 3.0000
2.0000 3.0000
1.0000 0.0000

PI IS 3

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM B1 ALTERNATIVE 6
IS 0.1998

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

10.00000 7.01032 2.96458 5.00000 5.85333
8.07688 9.00000 4.13613 2.07158 0.88718

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM B1 ALTERNATIVE 2
IS 2.8294

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

8.61983 5.61745 1. 61745 5.00000 4.59881

9.86810 10.28766 4.46373 2.H373 2.46323

THE MINIMUM D1STANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 3
1S 1. 4564

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

10.65183 6.30768 2.20390 5.00000 5.31418
8.77433 9.69782 4.88863 1. 58536 0.57626
 191

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 4
IS 6.5739
THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

15.65784 10.00000 0.33516 5.00000 3.25064

6.93343 5.82293 5.00000 3.00000 0.00000

ALTERNATIVE 6 GIVES THE MINIMUM DISTANCE

WHICH IS 0.200
THE MAXIMUM DISTANCE IS 12.6912
THE SENSITIVITY INDEX IS 0.01575

TO IMPROVE THE CURRENT STATUS YOU HAVE TO:

COMPARE HOLISTICALLY ALTERNATIVES 6 AND 1

WHERE THE CRITICAL PARAMETERS ARE:

10.00000
7.01032
2.96458
5.00000
5.85333
8.07688
9.00000
4.13613
2.07158
0.88718
OR REDUCE THE FEASIBLE SET OF PARAMETERS, ALONG THE DIRECTION

10.0000 5.0000
7.0000 4.0000
3.0000 15.0000
5.0000 6.0365
6.0000 10.0000
8.0000 5.0000
9.0000 5.0000
4.0000 4.0000
2.0000 0.0000
1. 0000 0.9635
***** ••• **************.*******

PI IS 4

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 6
IS 0.2038
THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

10.00000 7.01161 3.00000 5.00000 5.83167

8.05557 8.99415 4.12796 2.09862 0.88042
THE MINIMUM DISTANCE Ta PRODUCE THE OUTRANKING
OF THE CURRENT OPTIMUM BY ALTERNATIVE 2
IS 2.4124

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

8.57857 5.57610 1. 56430 5.00000 4.56344
 192

9.72740 10.34681 4.57472 2.59285 2.47581

THE MINIMUM DISTANCE TO PRODUCE THE OOTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 3
IS 1.2298

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

10.70332 6.31343 2.23761 5.00000 5.31208

8.78150 9.72546 4.84849 1. 53740 0.54070

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 4
15 6.0027

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

15.48425 10.00000 0.00000 5.00000 3.13071

7.94682 5.43822 5.00000 3.00000 0.00000

ALTERNATIVE 6 GIVES THE MINIMUM DISTANCE

WHICH IS 0.204
THE MAXIMUM DISTANCE 15 12.1859
THE 5ENSITIVITY INDEX IS 0.01673
TO IMPROVE THE CORRENT STATUS YOU HAVE TO:

COMPARE HOLISTICALLY ALTERNATIVES 6 AND 1

WHERE THE CRITICAL PARAMETERS ARE:
10.00000
7.01161
3.00000
5.00000
5.83167
8.05557
8.99415
4.12796
2.09862
0.88042

OR REDUCE THE FEASIBLE SET OF PARAMETERS, ALONG THE DIRECTION

10.0000 5.0000
7.0000 4.0000
3.0000 15.0000
5.0000 5.7534
6.0000 10.0000
8.0000 5.0000
9.0000 5.0000
4.0000 4.3355
2.0000 0.0000
1.0000 0.9111

*****************************.

PlIS 210

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 6
IS 0.1028

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

 193

10.00000 7.10284 2.89116 5.00000 5.89116

8.10284 8.89716 4.10284 2.10284 0.89716

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM Br ALTERNATIVE 2
IS 1.4932
THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

8.50685 5.50685 1. 50685 5.00000 4.50685

9.49315 10.49315 4.49315 3.00000 2.49315
THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING
OF THE CURRENT OPTIMUM Br ALTERNATIVE 3
IS 0.7603
THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:
10.76030 6.23970 2.23970 5.00000 5.23970
8.76030 9.76030 4.76030 2.00000 0.23970

THE MINIMUM DISTANCE TO PRODUCE THE OUTRANKING

OF THE CURRENT OPTIMUM BY ALTERNATIVE 4
IS 5.1667

THE LEAST CHANGES ARE ATTAINED AT PARAMETERS:

15.16667 10.00000 0.00000 5.00000 1.83333
10.00000 5.00000 5.00000 3.00000 0.00000
ALTERNATIVE 6 GIVES THE MINIMUM DISTANCE
WHICH IS 0.103
THE MAXIMUM D1STANCE 1S 12.0000
THE SENSITIVITY INDEX IS 0.00857
TO 1MPROVE THE CURRENT STATUS rou HAVE TO:

COMPARE HOL1ST1CALLY ALTERNATIVES 6 AND 1

WHERE THE CRITlCAL PARAMETERS ARE:
10.0eooo
7.10284
2.89716
5.00000
5.89716
8.10284
8.89716
4.10284
2.10284
0.89716
OR REDUCE THE FEASIBLE SET OF PARAMETERS, ALONG THE DIRECTION
10.0000 5.0000
7.0000 8.0000
3.0000 15.0000
5.0000 5.0000
6.0000 1. 0000
8.0000 5.0000
9.0000 5.0000
4.0000 5.0000
2.0000 1.0000
1. 0000 5.0000