report no.

4/06

Analysis of the CAFE

cost benefit analysis

Prepared by the CONCAWE Air Quality Management Group’s Special Task Force
on cost benefit analysis (STF-66):

L. Gephart
J. Lewis
P. Roberts
E. Salter
L. White

L. Post (Technical Coordinator)

Reproduction permitted with due acknowledgement

© CONCAWE
Brussels
June 2006

I
 report no. 4/06

ABSTRACT

This report discusses a few aspects of the Cost Benefits Analysis that was
developed within the context of the Clean Air For Europe (CAFE) programme of the
European Commission. The aspects considered are the monetisation of impacts of
air pollution on human health and the results of a statistical uncertainty analysis.

The CAFE programme has resulted in the publication of the Thematic Strategy on
Air Pollution (TSAP) and this TSAP is also considered in the report.

KEYWORDS

Clean Air For Europe, CAFE, Cost Benefit Analysis, Thematic Strategy, VSL, Value
of a Statistical Life, VOLY, Value of One Life Year

INTERNET

This report is available as an Adobe pdf file on the CONCAWE website

(www.concawe.org).

NOTE
Considerable efforts have been made to assure the accuracy and reliability of the information
contained in this publication. However, neither CONCAWE nor any company participating in
CONCAWE can accept liability for any loss, damage or injury whatsoever resulting from the use
of this information.

This report does not necessarily represent the views of any company participating in CONCAWE.

II
 report no. 4/06

CONTENTS Page

SUMMARY IV

1. INTRODUCTION 1
1.1. HEALTH BENEFITS IN CAFE 1
1.2. CAFE SCENARIOS 1
1.3. OVERALL STRUCTURE OF CAFE COST BENEFIT
ANALYSIS 2

2. VALUATION OF HEALTH IMPACTS 4

2.1. VALUATION OF MORTALITY 4
2.1.1. Stated preference studies for eliciting VOLY 6
2.1.2. Life insurance approach 9
2.1.2.1. Methodology and interpretation 9
2.1.2.2. Results 10
2.2. VALUATION OF MORBIDITY 11

3. UNCERTAINTY ANALYSIS OF THE CAFE CBA 13

3.1. FOLLOWED APPROACH 13
3.2. CENTRAL VALUES OF CAFE CBA BASED ON MEDIAN
VOLY 13
3.3. ASSUMPTIONS USED IN THE CAFE CBA UNCERTAINTY
ANALYSIS 14
3.4. RESULTS UNCERTAINTY ANALYSIS WITH FULL VOLY
DISTRIBUTION 15
3.4.1. Reference case 15
3.4.2. Impact of morbidity 17
3.4.3. Defra VOLY variation 17
3.4.4. Variations of Defra VOLY with alternative PM chronic
mortality dose-response function 19
3.5. RESULTS UNCERTAINTY ANALYSIS WITH
REPRESENTATIVE VOLY DISTRIBUTION 21
3.5.1. Defra average means as representative value 21
3.5.2. Defra average means as representative value with 2% CRF
slope 23
3.5.3. Defra average medians as representative value 25
3.5.4. Combining several variations into a ‘reasonable’ case 26

4. CONCLUSIONS AND RECOMMENDATIONS 28

5. LIST OF ABBREVIATIONS AND ACRONYMS 29

6. REFERENCES 30

APPENDIX 1 OVERVIEW OF STATED PREFERENCE STUDIES ELICITING

VOLY 32

III
 report no. 4/06

SUMMARY

This report discusses a few aspects of the Cost Benefits Analysis that was
developed within the context of the Clean Air For Europe (CAFE) programme of the
European Commission. The aspects considered are the monetisation of impacts of
air pollution on human health and the results of a statistical uncertainty analysis.

The CAFE programme has resulted in the publication of the Thematic Strategy on
Air Pollution (TSAP) and this TSAP is also considered in the report.

The main conclusions of the study are:

1. The CAFE CBA methodology suffers from fundamental shortcomings in terms

of the CBA metric and the lack of using a marginal analysis. This leads to
some wrong conclusions in terms of scenario justification.

2. In this study we argue that within the air pollution context of CAFE the relevant
monetisation metric is Value of One Life Year (VOLY) rather than the Value of
a Statistical Life (VSL) metric that has been used in the CAFE CBA
methodology.

3. Values for VOLY as obtained by the Commission’s NewExt study are very
much higher than those found in other studies.

4. Defra1 commissioned a study which is a high quality study directly eliciting

VOLY and giving quite reasonable VOLY results.

5. An alternative approach, based on life insurance policies, is discussed in this

report. This approach gives an interesting different angle to the subject of
mortality valuation leading to much lower estimates. However, a direct
interpretation of these results is not clear.

6. The economic justification of the CAFE TSAP ambition level is not very robust,
it is strongly affected by reasonable variations of relevant parameters.

The CAFE CBA methodology could be considerably improved by taking the

comments above into account, especially by using a proper marginal analysis
(comment 1) and using VOLY rather than VSL (comment 2). Using a more balanced
VOLY value, also accounting for the outcomes of the Defra study, is also strongly
recommended (comments 3 and 4). This would also improve the robustness of the
TSAP when varying relevant parameters (comment 6).

1
UK Department for Environment, Food and Rural Affairs

IV
 report no. 4/06

1. INTRODUCTION

1.1. HEALTH BENEFITS IN CAFE

This report discusses two aspects of the Cost Benefit Analysis (CBA) that was
developed within the context of the Clean Air For Europe (CAFE) programme of the
European Commission, namely the valuation of health impacts and the results of an
uncertainty analysis.

The health benefits considered in the CAFE CBA fall in two categories: mortality and
morbidity. The two main pollutants considered for human health impacts are
Particulate Matter (PM) and ozone. For PM the focus is especially on the PM2.5
fraction: PM with an aerodynamic diameter smaller than 2.5 micrometer. This
fraction is considered to be more harmful to human health than PM10.

The following overview of monetised health impacts, based on the CAFE CBA
results, will give the reader an idea on the distribution of the monetised impacts over
the different health endpoints. For the PM morbidity category, a further subdivision is
given as well.

Sub category Contribution to

Health impact Sub-category contribution to total monetised
total health impact health impact

PM2.5 Chronic
68%
mortality

PM2.5 Morbidity 29%

Chronic bronchitis
13%
(27+)

Restricted Activity
10%
Days RAD

Lower Respiratory
6%
Symptoms

Ozone Acute Mortality

3%
plus Morbidity

It is very clear from these data that according to the CAFE CBA methodology the
health impacts of PM (mortality plus morbidity) are very dominant, with PM Chronic
mortality being 68% of the total monetised impacts.

1.2. CAFE SCENARIOS

Within the CAFE programme the analysis was done using so-named scenarios.
Each scenario is defined by a certain given improvement in environmental and
health impacts, or by what is often referred to as the ambition level. Then using an
optimisation technique a set of abatement measures is determined at country level

1
 report no. 4/06

which will give the most cost-effective way, calculated at the aggregated European
level, to obtain this prescribed ambition level. The time horizon for the scenarios is
the year 2020. The year 2000 is the baseline year.

The range of the ambition levels is determined by two endpoints. The lower ambition
level is given by the so-named CLE 2020 scenario which implies that Currently
agreed Legislation is Enacted upon. The high end of the ambition spectrum is the
so-named MTFR scenario where the Maximum Technically Feasible Reductions are
being applied. Unlike the other CAFE scenarios these two endpoint scenarios are
not optimised and are not driven by a given ambition level.

Ideally speaking a set of intermediate scenarios should be selected covering the

whole spectrum in an even way in terms of closing the gap between CLE 2020 and
MTFR. In the later stages of the CAFE programme several sets of scenarios were
used. Each set contained three scenarios called A, B and C, where the A scenario
represented the lowest ambition level. It must be noted that this A scenario already
implied a 50% closure of the gap between CLE and MTFR in terms of health
impacts. Thus the A scenario already implied a considerable ambition in terms of
impacts and thus in costs.

The CAFE programme culminated in the adoption of the so-named Thematic

Strategy on Air Pollution, which will be important for future ambition levels in air
quality legislation. The ambition level of the TSAP is between A and B.

1.3. OVERALL STRUCTURE OF CAFE COST BENEFIT ANALYSIS

The Cost Benefit Analysis methodology as used for CAFE is described in three
volumes [6, 7, 8].

A complete human health Cost Benefit Analysis with the CAFE programme consists
of the following steps:

1. Starting from the total dataset of European emission inventories, as available

in the EMEP programme, a dispersion calculation is made with the EMEP
model using a given set of meteorological conditions (for a specific year or for
average conditions). This results in a set of European wide concentrations
fields for all relevant pollutants. The information is available on a grid covering
Europe.

2. Using these concentration fields as input the RAINS model calculates the
population-weighted impacts of the pollutants. In other words, the exposure of
the European population to the calculated pollutant concentrations is
calculated. For every scenario run with RAINS the costs per country for the
selected set of measures is also calculated.

3. Using concentration response functions, it is possible to estimate the health

impacts of the relevant pollutants per EU country in terms of numbers of cases
(both for mortality and morbidity). Some of these health impacts are estimated
directly within RAINS (e.g. PM chronic mortality and ozone acute mortality), but
the full health impact estimation is done separately using the CAFE CBA
methodology as developed by AEAT [6, 7, 8].

4. The health impacts are then expressed in monetary terms using the valuation
parameters as developed within the CAFE CBA. These valuated health

2
 report no. 4/06

impacts are in fact always ‘damages’ because they are a valuation of the
negative impacts of pollution on human health.

5. When comparing different CAFE scenarios, the improvement in health impacts

(thus the lowering of the negative health effects) going from one scenario to
the next can then be compared directly with the increase in costs between the
two scenarios. In other words, for two scenarios one can calculate the marginal
increase in ‘benefits’ (valuation of lower health impacts) and compare those
with the increase in marginal costs. The net marginal benefits are then simply
calculated as the difference between the marginal benefits and the marginal
costs. This difference is the correct metric to be used in any CBA as opposed
to the ratio of the benefits over the costs. See [21, 22] for further discussion.

Following up on point 5 it should be emphasised that unfortunately the CAFE CBA

methodology does not consistently use a full marginal analysis. A marginal analysis
was shown amongst other results in [14], but not in the most recent report on TSAP
[15]. For the MTFR scenario an incorrect application of CBA economics by the CAFE
CBA team leads to the wrong conclusion that MTFR has a positive net benefit.
Moreover in both reports the benefit cost ratio is used as a metric, rather than using
the difference.

We recommend using a consistent marginal approach with the correct metric of

marginal benefits minus marginal costs, as this would improve the quality of the
analysis.

It will be clear that uncertainties are present in each step of the analysis and these
will influence the resulting reliability of the final metric: the difference between the
marginal benefits and the marginal costs. For example, in the first step the quality of
the emission inventories as well as the choice of the meteorological conditions will
have an impact on the values found for the benefits.

The uncertainty analysis presented in this report considers the uncertainties in the
valuation of the health impacts rather then non-health impacts, because within the
CAFE CBA methodology certain choices have been made which can be replaced by
equally reasonable, plausible alternatives. Using the uncertainty analysis presented
here the influence of these choices on the CBA will be clarified.

The most important choices within the CAFE CBA methodology are:

• The figure used for the Value of One Life Year VOLY, as the few available
studies to elicit VOLY directly give quite different results compared to the
NewExt study as will be discussed in detail in section 2.1.1.
• The slope of the concentration response function used for PM2.5 all cause
mortality and for the morbidity endpoints chronic bronchitis and restricted
activity days (RADs). The assumption that there is no concentration threshold
below which both mortality and morbidity occur is also a very critical
assumption. See sections 2.2 and 3.4.4 for more details.
• The economic value assigned to chronic bronchitis.

Of course there are several other sources of uncertainty which have not been
considered here, e.g. in emission estimates and in air quality modelling. We have
focussed on the effects of uncertainties in the health impacts monetisation because
these can be quantified and they prove to have a major impact on the results of the
Cost Benefit Analysis.

3
 report no. 4/06

2. VALUATION OF HEALTH IMPACTS

In order to be able to quantify the human health impact of certain air quality
scenarios for use in a formal Cost Benefit Analysis (CBA), one needs to assign a
monetary value to these impacts. There are two main categories of human health
impact: mortality and morbidity.

2.1. VALUATION OF MORTALITY

A crucial part of any formal Cost Benefit Analysis (CBA) methodology, including the
one as developed for the CAFE programme, is the quantification of health benefits
related to mortality.

In this context two metrics are often used: the Value of a Statistical Life VSL
(sometimes called the Value of a Prevented Fatality VPF) and the Value of One Life
Year or Value of One Year Lost (VOLY or VOYL).

The VSL is the amount of money that a community of people are willing to pay to
lower the risk of an anonymous instantaneous premature death within that
community (e.g. by certain traffic safety measures). Thus VSL is a limited amount,
whereas of course to save a specific individual in danger, usually no means are
spared. VSL is calculated by dividing the amount people are willing to pay by the
change in mortality risk. VSL is the correct metric in the context of observable deaths
such as in traffic accidents.

The VOLY is defined as the amount of money that people are willing to pay for one
year of additional life expectancy.

There is an ongoing discussion in the literature as to what metric is appropriate in

what context. The fact that there are many studies trying to elicit VSL values, but
only a handful of studies doing the same for VOLY sometimes is a reason for
researchers to use VSL. VSL is used quite often in the USA, see e.g. [22] for more
information.

In this report however, we conform to the position taken by the researchers of the
ExternE project supported by the European Commission DG research. See [16,
page 140 and Table 7.3] and also [5] for an extensive discussion and overview of the
pros and cons of using VSL and VOLY in different contexts. This position is also
shared by the research team of the more recent NewExt study [10, p. 12].

The ExternE researchers state that in the context of air pollution VOLY is a more
appropriate and defensible metric than VSL. This is because for air pollution the
impact is not instantaneous, but it is the cumulative result of years of exposure (so-
named chronic mortality), so that the number of deaths is not observable. As a
result, it is impossible to tell whether a given exposure has resulted in a small
number of people losing a large amount of life expectancy or in a lot of people losing
a small amount of life expectancy. Only the average number of years of life lost can
be calculated and so it is clear that indeed VOLY is indeed the only correct metric in
this context.

As an additional remark it should be noted that considering changes in remaining life

expectancy is fully consistent with long term epidemiological studies (so-named
cohort studies) which allow the calculation of total population life expectancy losses,

4
 report no. 4/06

but not the number of deaths attributable to air pollution [5]. In other words, using
epidemiological evidence as is done within the CAFE programme, again dictates that
VOLY, rather than VSL, be adopted.

In this report we fully support this reasoning and therefore use VOLY rather than
VSL to monetise health impacts. Using VSL based quantifications next to VOLY as a
‘sensitivity case’ as done by the CAFE CBA team [14, 15] is in our opinion
methodologically wrong and cannot therefore provide valuable information, even as
sensitivity study.

The actual numerical value of VOLY (or VSL for that matter) can be determined in
different ways. There are two main approaches: so-named stated preference
methods and revealed preference methods.

In the first approach survey respondents are asked to explicitly state monetary
values for a hypothetical change in risk or in life expectancy. The results are then
scaled back to the value for VSL or VOLY. To measure VOLY directly the survey
should try to elicit the value of a change in life expectancy directly and not a change
in risk of dying as this requires a non-trivial conversion from VSL to VOLY. There is a
very large number of studies eliciting VSL and deriving VOLY from that, but there are
not many studies directly eliciting VOLY.

In a well-known variant of the second approach, the so-named hedonic pricing

method, one tries to obtain the implicit value of an attribute by comparing prices and
attribute levels. A concrete example is looking at wages for jobs with different risk
levels: by accepting jobs with higher risks for a higher wage, people implicitly use a
value for VOLY or VSL. Other factors influencing the job choice should be filtered out
using statistical methods. Another variant of the revealed preference method looks at
risk averting behaviour of people e.g. the amount of money that people spend on
smoke detectors or the use of seat belts by motorists. From this information one can
again try to elicit a number for VOLY/VSL.

Within the CAFE CBA framework a choice was made to use a stated preference
method, a so-named Willingness To Pay (WTP) approach using so-named
contingent valuation techniques. Using a survey technique, respondents are being
asked what they would be willing to pay for a small benefit, typically a small
reduction in their risk of dying or a change in life expectancy of one month. Then the
answers are scaled to give a value for the VOLY which is of course the monetary
value associated with one year of life expectancy change.

As a survey technique is being used, the answers of the respondents, and thus the
value of the derived VOLY, unavoidably show a certain range which can be quite
large. Also, different surveys, applying different questions to the respondents, will
unavoidably give different results.

The CAFE CBA team have selected the NewExt [10] study results for VOLY as their
choice of central value in the CAFE CBA methodology. In the next section we will
compare the NewExt results with those of other comparable studies and we will
demonstrate that the NewExt values are much higher than those found with these
other studies.

In section 2.1.2 we will also discuss a quite different approach to value mortality
using a large life insurance policy database.

5
 report no. 4/06

2.1.1. Stated preference studies for eliciting VOLY

The NewExt study results are described in [2] and [10]. The study is based on a
survey that was conducted in three countries, the UK, France and Italy. The NewExt
standard report [2] gives the pooled results of these surveys and these have been
used for the CAFE CBA as well. The NewExt values for VOLY derived from the
pooled results are 52,000 € based on the median of the full sample distribution and
118,000 € based on the mean.

NewExt results are described both in terms of VSL and VOLY. There are many
studies that try to elicit VSL values as compared to VOLY (which is not the most
appropriate metric within an air pollution context as was discussed in detail in section
2.1), however the NewExt authors state that they were at the time aware of only two
other studies that have employed stated preference techniques for placing a value
on life expectancy changes directly trying to elicit VOLY values [2, page 22]. This is
confirmed in [10]. It should be noted that the NewExt study itself also elicits VSL and
then derives VOLY values from the VSL data.

The two studies directly eliciting VOLY as mentioned by the NewExt team are the
study by Johannesson and Johansson [4], which is based on a Swedish survey and
the study of Morris and Hammitt [9], which is based on two US surveys, one phrased
in terms of a risk reduction of annual mortality and one in terms of a direct change in
life expectancy. For more details on both these studies see Appendix 1.

It is important to note that Morris and Hammitt [9] report that in a survey people find
it much harder to value a change in risk reduction than a direct change in life
expectancy expressed in e.g. months.

The NewExt authors clearly state that the two studies [4, 9] seem to imply much
lower VOLY figures than those produced by their NewExt study [2, page 22].

In the meantime a new study by Chilton et al. [3] and commissioned by the UK
Department for Environment, Food and Rural Affairs (Defra) has become available.
We consider this to be a high quality study, with a well thought-through methodology
and led by a research team which is very experienced in this field. The Defra study is
phrased in terms of a direct change in life expectancy (in months) and not in terms of
mortality risk changes. This is a very strong advantage of this study, as people
understand a direct life expectancy change much better, as is confirmed by
Desaigues et al in [11].

The Defra study asks three separate sets of UK respondents for WTP in terms of a
life expectancy increase of 1, 3 or 6 months respectively, both in normal and in poor
health, so in total six datasets for three different population samples are available.
For more details see Appendix 1.

A summary of the results of the Defra study is given in Table 1. The WTP results of
the survey are in terms of a life expectancy increase of 1, 3 or 6 months and they
need to be normalised to Value of One Life Year (VOLY) by multiplying with a factor
12, 4 and 2 respectively. As the WTP amount asked for was phrased in terms of a
yearly payment for the rest of the life of the respondent, the WTP given is also
multiplied by 78, which is the current life expectancy in the UK. The final results of
this scaling are given in Table 1. The VOLY values given in the table are based on
the mean of the survey sample distribution or on the median.

6
 report no. 4/06

Looking at the results in Table 1 (for Normal and Poor heath separately) we note that
although the direct WTP results from the survey increase with the life expectancy
(LE) asked for (1, 3 or 6 months), the scaling to one year with a factor 12, 4 and 2
respectively causes the VOLY to decrease with the LE asked for.

As said, in general for a larger LE change, the mean and the median Defra WTP
values itself, which are the figures coming directly from the survey, are indeed higher
than for a shorter LE change. This is not self-evident, because the WTP values for
the three different LE values are given by three different groups of respondents.
However the increase in the WTP values is not fully proportional to the increase in
LE and, after scaling, this causes the decrease in VOLY for larger LE values as
shown in the table.

Table 1 Summary of results from Defra VOLY study

(N: normal health, P: poor health)

Based on: 1 month N 1 month P 3 months N 3 months P 6 months N 6 months N

Mean € 45,298 € 12,421 € 22,771 € 5,693 € 12,705 € 2,168

Median € 15,360 € 1,783 € 2,218 €0 € 2,663 € 395

As a second observation we note that most Defra values are much lower than the
NewExt values. This could be caused by the fact that this is a UK-only study. From
the NewExt study, which looks at the UK, Italy and France, it is clear that WTP
values are different per country. The UK median WTP is lowest whereas Italy WTP
is highest being 188% of the UK median WTP. But even if we would consider
median VOLY figures that would be twice as high as the Defra median VOLY values,
we would still find VOLY figures that are still significantly larger than the NewExt
median VOLY of 52,000 €, namely 30,720; 3,566; 4,436; 0; 5,326 and 790 € (using
Table 1, second row). Thus the conclusion remains valid that the Defra VOLY figures
are lower than those of NewExt.

When asking for a LE increase in Normal health (N) one can assume that the
valuation then implicitly includes morbidity. In other words, using the Normal health
values, one should not add a separate morbidity valuation as this would be double
counting.

For the NewExt and the Defra studies, sufficient data is available to obtain the full
probability distributions of the WTP answer sets.

Curve fits using a Weibull two-parameter distribution as they were used and
discussed by the NewExt authors themselves have been used by us as well. We
have generated the relevant Weibull fits for the Defra study using the full data set as
made available to CONCAWE by the authors. For [4], we have requested the
authors for the Weibull parameters as they were not given in the original paper and
out of the four available options we have used the distribution giving the largest
range.

The other study [9], gives a range VOLY values based on a set of four very specific
curve fits as developed by the authors. Analysing the raw data using Weibull curve
fits resulted in a smaller VOLY range, which has not been used here, instead the
published wider range has been used.

7
 report no. 4/06

The VOLY distributions as found from the surveys are always very non-symmetrical
ones, strongly skewed towards zero. This is demonstrated in Figure 1 where the
VOLY distribution as found by the NewExt authors (pooled data, 5:1000 risk
reduction case) is shown. The mean value is indicated and the median is the split
between the blue and red areas of the curve.

Figure 1 Probability distribution for NewExt VOLY [10]

Forecast: Newext VOLY Pooled 5:1000

14,200 Trials Frequency Chart 13,036 Displayed
.081 1157

.061 867.7

.041 578.5

.020 289.2

Mean = 118,230.14
.000 0
0.00 87,500.00 175,000.00 262,500.00 350,000.00
Certaintyis 49.93%from-Infinityto 52,000.00 Euro

Please note that Figure 1 clearly shows that a certain amount of people declare a
WTP value of zero (0). This is true for all surveys discussed here. The respondents
take the full survey, but select a WTP of zero, these are called the ‘non-protest’
zeros. Very often when performing the survey there is also a significant group of
respondents who refuse to assign a WTP e.g. because they are of the opinion that
the government should pay for the benefits asked for. These respondents are called
‘protest-zeros’ and they are always ignored in the survey WTP analysis.

As a general remark it can be stated that the mean VOLY of a certain study is very
sensitive to the curve fitting technique being used, as the mean is very sensitive to
the precise description of the right-hand tail of the probability distribution
representing the high VOLY values. The median is defined as the mid-point of a
distribution, with half of the sample less than or equal to the median, and half of the
sample greater than or equal to the median value. One can expect that the median
value is less sensitive to a limited number of extreme high values than the mean and
it is indeed true that to characterise strongly skewed distributions the median is
usually a much more robust statistic than the mean, see also [1].

The next figure compares the range of VOLY values found from all four studies.
Where a full distribution was available the 20% to 80% percentile range is shown, for
the other study [9] the whole range of the mean VOLY as given in the paper is being
used here.

For the studies with the largest ranges mean and median values are also indicated.

It is very clear from the figure that the NewExt study not only has the widest range,
but also has a strong bias towards very high mean and median values. It definitely is
not a good representation of what has been found for VOLY in quite comparable
studies.

8
 report no. 4/06

Figure 2 Range of VOLY values found in the different studies.

Overview VOLY distributions (20% - 80% percentiles)

New ext pooled 5:1000

DEFRA Average Good Health

Name of study plus subset

DEFRA Average Poor Health

DEFRA Overall Average

Johannesson & Johanson : mean value of distribution

: median value of distribution

Morris & Hammitt

00

00

00

00

00

00

00

00

00
0

0
00

00

00

00

00

00

00

00

00

00
€

,0

,0

,0

,0

,0

,0

,0

,0

,0

0,

0,

0,

0,

0,

0,

0,

0,

0,

0,
10

20

30

40

50

60

70

80

90

10

11

12

13

14

15

16

17

18

19
€

€
VOLY range

In Table 2 an overview of the VOLY ranges as well as the values for mean and
median are given.

Table 2 Numerical VOLY data of the different WTP studies

Study 20% 80% Range Median Mean Ratio median

NewExt pooled 5:1000 9,700 183,000 173,000 52,000 118,000 -
Defra average good health 4,200 40,000 36,000 14,000 27,000 4
Defra average poor health 400 7,800 7,400 1,900 6,800 27
Defra overall average 3,800 24,800 21,000 10,200 16,800 5
Johannesson et al. 400 1,500 1,100
Morris et al. 140 2,300 2,200 700 1500 75

The column ‘Ratio median’ gives the ratio of the NewExt median VOLY value and
the median VOLY of the different studies. These numbers reinforce the conclusion
that the NewExt VOLY values are much higher than those found in the other eliciting
studies.

2.1.2. Life insurance approach

2.1.2.1. Methodology and interpretation

As a totally different way of looking at effects of life expectancy increases,
CONCAWE has conducted a study which involves data on life insurance policies for
three different countries (United Kingdom, France and Italy).

The basic approach is as follows. For every country a very large database with life
insurance policies plus relevant details is available. Also per country we have

9
 report no. 4/06

detailed information on mortality rates (or equivalent life expectancy) in age bands
and for both sexes.

By adjusting the known mortality rates we can now increase the life expectancy at
birth for both sexes by 1 year (365 days). Assuming the total sum of premiums paid
over the whole policy running period remains constant, we can then consider the
corresponding change in annual premium that reflects the change in life expectancy.
In other words, based on a constant total premium, increasing the life expectancy
will decrease the annual premium to be paid and we consider this premium
differential for a one year increase in life expectancy at birth.

The total monetary increase associated with an additional year of life expectancy is
then found by multiplying the annual premium differential by an assumed average
life expectancy of 72 years.

It is clear that this method can be classified as a revealed preference method as it is

clearly based on actual (premium) payments being paid by people.

The interpretation of the resulting figures is less clear. The derived premium
differential is not related to what people are willing to pay for a life expectancy
increase or a mortality risk decrease because a life insurance is taken to give a
certain cover after the policy holder has actually died. But the life insurance itself is
intended to cover part of the loss that occurs after death.

Interesting aspects of this work are the enormous amount of observations (life
insurance policies) on which it is based (about 22 million in both UK and France and
11 million in Italy) and the fact that this is about payments that have actually been
made by people as opposed to an expressed willingness to pay.

2.1.2.2. Results
When using a life expectancy increase of 1 year at birth the results for the total
benefit are as follows:

Table 3 Total monetary increase associated with a 1 year life expectancy

increase at birth based on life insurance premiums

Country Total benefit (€)

United Kingdom 3,067

France 3,714

Italy 5,805

The United Kingdom figure does not include Death In Service (DIS) pensions,
although this type of life insurance is quite common in the UK. Including the DIS
benefits the UK figure for the total benefits would be 13,611 €.

It is clear that differences between countries can be quite significant. The UK has the
lowest figure, the result for France is 121% of this and for Italy the result is 189%. It
is remarkable that the NewExt study, trying to elicit Willingness To Pay figures for
certain reductions in mortality risks, finds the same kind of differences between
these three countries. Again taking the UK result as a reference, from [10, table 3] it

10
 report no. 4/06

can be seen that the figure for France is 124% and the figure for Italy is 188% of the
UK figure. The explanation for this is not clear, the finding could be fortuitous.

As a variation the premium differential was also calculated using a one month and
one week life extension compared to the above-mentioned one year. Per country the
total benefits are almost directly proportional with the life extension period, a result
which is to be expected.

The final interpretation of Table 3 is not clear due to reasons mentioned above.
There do not appear to be any studies in the published literature that used similar
methods and data sources with which to compare these findings. As such, additional
research using other data sources and similar methods to replicate the results would
be informative. However, the values in Table 3 are notably similar to findings
reported in stated preference surveys by Morris and Hammitt [9], Johannesson and
Johansson [4], and the Defra [3] study – see Figure 2, Table 2 and Appendix 1.

2.2. VALUATION OF MORBIDITY

The valuation of morbidity is even more uncertain than that of mortality. However,
within the CAFE CBA methodology this uncertainty is not properly addressed.

For the bronchitis valuation the following problems can be mentioned:

• The concentration response function used to derive monetary estimates is

based on a single older U.S. study [17]. Thus U.S. background disease rates
are used and these are implicitly assumed to be the same for Europe. This
study made no attempt to determine whether or not a threshold existed for the
effect, yet a no threshold approach is used in the CBA. This problem is
exacerbated by the fact that the levels of pollution in the study were very high
compared to those prevailing in Europe today.
• The findings were not statistically significant at the 5% level, raising the
question of whether they are real or not.
• The findings are likely confounded by smoking, by far the largest factor in the
etiology of bronchitis and presumed to not occur in the study population. The
high lung cancer rates in males versus females appear to indicate smoking did
occur in this population.
• The exposure data are based on estimates for total suspended particulates
(TSP), which are large coarse size PM. These estimates then need to be
converted to PM10, and finally PM2.5. Airport visibility data were also used to
estimate fine PM levels. The methods used for these conversions are unclear
and add significant uncertainty.
• Attributing bronchitis to fine rather than coarse PM is clinically implausible, as
chronic bronchitis is a disease of the upper airways where coarse PM is
deposited.
• The value used to monetise bronchitis events was based on a single Canadian-
based study [20] without consideration for differences in baseline rates or costs
of health care between Europe and Canada.

11
 report no. 4/06

For the Restricted Activity Days (RADs) the following issues were identified:

• This concentration response function is again based on single old U.S. study
[18, 19] in locations/time where photochemical air pollution was high, raising
concern for confounding/double counting with other pollutants in particular
ozone.
• RAD incidence is highly subject to confounding by socioeconomic and other
factors not controlled in the study. These include time spent outdoors, health
practices including how RADs are recorded, age, race, education, income,
marital status, employment conditions/rates and smoking rates. In the study
used, high city-to-city differences were observed. Even greater differences are
expected when extrapolating results to Europe

We conclude that the concentration response functions and economic values used in
the morbidity valuation were highly uncertain.

12
 report no. 4/06

3. UNCERTAINTY ANALYSIS OF THE CAFE CBA

3.1. FOLLOWED APPROACH

Given the RAINS results as input, the CAFE CBA methodology gives a calculation
procedure that in the end produces monetised health impacts for PM and ozone for
every RAINS scenario. There are two health impact categories: mortality and
morbidity.

Within the CAFE CBA methodology certain parameter choices have been made
which have a direct impact on the results of the health impact valuation. One of the
most important parameters is the VOLY (Value Of a Life Year). The calculated
mortality valuation is directly proportional to VOLY. As was demonstrated earlier in
this report (section 2.1.1) the VOLY is determined using surveys and therefore
inherently has a certain statistical distribution. In the standard CAFE CBA approach
two ‘single point’ calculations are made: one using the median of the VOLY and one
using the mean. This results in two ‘single point’ estimates of the health benefits. Of
course one loses a lot of information in this way and it would be much better to use
the full statistical distribution of the VOLY to get a better idea of the inherent spread
in the health benefits resulting from the spread in the VOLY. In this section two ways
to take this inherent distribution into account will be explored.

In fact, it is possible to do an analysis using the full statistical distribution. The

approach is simply to perform a very large number of ‘single point’ calculations, each
time selecting a value for VOLY in a random process where the chance of picking a
certain VOLY-value is given by the statistical distribution of the VOLY. After
performing say 20,000 to 30,000 of these ‘single point’ calculations it is clear that we
will also get a distribution of values for the health benefits. This distribution then
gives us a good ‘picture’ of the spread in the health benefits caused by the spread in
VOLY. The mathematical term for such an approach is Monte Carlo analysis and
there is commercial software available to perform these analyses from within a
spreadsheet.

The software tool Crystal Ball has been used for the current analysis. Using this tool
it is also possible to look at the impact of the statistical spread of a number of
(independent) parameters simultaneously.

3.2. CENTRAL VALUES OF CAFE CBA BASED ON MEDIAN VOLY

Before we start looking at the impact on the CAFE CBA of different kinds of
uncertainty we first present the ‘central’ estimates of benefits and costs as presented
in [14, 15]. In the table below the VOLY median values are presented. In [14, 15]
also VOLY mean, VSL median and VSL mean results are given, but these are
presented here because, as discussed above, we do not advocate the use of VSL or
mean values, see section 2.1 and section 2.1.1.

In the table below all relevant CAFE scenarios are shown to give the complete
picture. The annualised ‘damages’ are the sum of PM mortality, PM morbidity, ozone
mortality, ozone morbidity and non-health impacts (crops, materials). Compared to
CLE 2020 the other scenarios show lower damages and the difference are called
‘benefits’. The marginal benefits given in a certain column are the difference in
damages between the scenario of that column and the previous scenario. The same
is true for the marginal costs. Annualised costs are all with respect to CLE 2020, the

13
 report no. 4/06

costs of CLE 2020 itself are 65,862 million € (M€) per year. The costs of the other
scenarios include the so-called Euro 5/6 standards for the transport sector.

Table 4 Overview ‘central’ values CAFE CBA based on VOLY median values, all
values in million € (M€) per year.

CLE 2020 A TSAP B C MTFR

Annualised ‘damages’ 191,100 153,535 148,861 145,237 141,093 134,198

Marginal Annual Benefits 37,565 4,674 3,624 4,144 6,895

Annualised Costs 5,923 7,100 10,679 14,852 39,720

Marginal Costs 5,923 1,177 3,579 4,173 24,868

Marginal Benefits minus Marginal Costs 31,642 3,497 45 -29 -17,973

3.3. ASSUMPTIONS USED IN THE CAFE CBA UNCERTAINTY ANALYSIS

Using the Monte Carlo approach as discussed in the previous section, the adopted
Thematic Strategy for Air Pollution (TSAP) scenario will be put in perspective with
two other scenarios, A and B from the so-named (final) D23 set as well as with the
MTFR scenario. See section 1.2 for more information on the CAFE scenarios.
Compared to the TSAP the A and B scenario have a lower and higher ambition level
respectively, although in absolute terms even the A scenario already has a high
ambition, see section 1.2 and further discussion below.

The actual numerical results of applying the CAFE CBA methodology to the different
CAFE scenarios and the TSAP are given in two reports [14, 15].

For the statistical analysis calculations the following assumptions have been made:

1. For the statistical distribution of the VOLY two philosophies have been used.
a. In the first part of our analysis we look at the impact of the underlying
VOLY distribution for one particular study. Here we directly mimic the
behaviour of the respondents in a kind of ‘voting’ process as we look at
the survey sample distribution of the VOLY value directly. See section
3.4.1 for a further clarification of this voting process. The used single-
study VOLY distribution is the so-named Weibull distribution as
mentioned by the NewExt authors themselves [2]. The Weibull
probability is a flexible distribution often used in reliability engineering
and also often used to describe VOLY distributions. In its usual form it
has two parameters (a shape and a scale parameter), but sometimes a
location parameter is added. Following [2] the NewExt VOLY has a
scale parameter of 503.64 and a shape parameter of 0.67427. The
location parameter is not used. As a variation we have looked at the
VOLY as elicited by the Defra study [3].

14
 report no. 4/06

b. In the second part of the analysis we use a more common approach for
the VOLY parameter using now a distribution around a representative
value. In this report a normal distribution is being used with several
variations for the average (or central or representative) value and a
given standard deviation of 30%.
2. In the second part of the analysis where a representative VOLY distribution is
used, the following distribution has been used for the (annualised) costs of
each scenario: a normal distribution with a standard deviation equal to 20% of
the mean value µ. This implies that the 99% confidence interval of the annual
costs distribution is (0.5·µ, 1.5·µ) which is a realistic assumption for the
uncertainties in the costs data. The mean values µ used for the annualised
costs are of course exactly the values used in the CAFE CBA and reported in
[14, 15] for the different scenarios.
3. For the morbidity valuation the default is the standard CAFE CBA figures, but
as a variation we have also looked at cases where morbidity has not been
valuated either because of the highly uncertain nature of these numbers or
because the Defra Normal Health VOLY values already implicitly contain a
morbidity contribution anyway.

3.4. RESULTS UNCERTAINTY ANALYSIS WITH FULL VOLY DISTRIBUTION

3.4.1. Reference case

This is the standard CAFE CBA result where the NewExt full (sample) distribution is
used. Results are shown for the A, TSAP, B and MTFR scenario.

The figures below show the full statistical distribution of the proper CBA metric
‘difference between marginal benefits and marginal costs’.

In these and all following figures the horizontal scale gives the net benefit in M€
(million €) and the vertical scale gives the probability. The horizontal scale endpoints
are -30,000 M€ to 60,000 M€ in this chapter and -30,000 M€ to 30,000 M€ in the
next chapter.

The figures can be interpreted as an analysis of the ‘voting’ behaviour of the

respondents: whenever a VOLY value taken from the full VOLY distribution would
imply a marginal cost that is greater than the marginal benefit, a ‘No’ vote is given,
represented by the colour red (net benefit less than zero). The colour blue is used to
indicate ‘Yes’ votes where marginal benefits exceed marginal costs. So the figures
represent the impact of the distribution of VOLY values across the survey sample
and thus, assuming the NewExt study is a correct representation, the range of VOLY
across the population.

15
 report no. 4/06

Figure 3 Standard CAFE CBA results, full NewExt VOLY distribution

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 13,669 Displayed
.035 704

.026 528

.018 352

.009 176

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 0.00% from -Infinity to 0 MEuro

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 19,863 Displayed
.157 3146

.118

.079

.039 786.5

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 0.00% from -Infinity to 0 MEuro

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 19,954 Displayed
.233 4650

.174

.116

.058

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 49.69% from -Infinity to 0 MEuro

16
 report no. 4/06

Forecast: Scenario MTFR - Benefits minus Costs

20,000 Trials Frequency Chart 19,841 Displayed
.146 2910

.109

.073

.036 727.5

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 86.56% from -Infinity to 0 MEuro

From the figures it is very clear that even assuming that the NewExt VOLY
distribution is a representative one, which is in our opinion not the case, the MTFR
scenario can never be justified. Not less than 86% of the respondents would favour a
VOLY value that implies marginal costs that exceed marginal benefits and they
would therefore see MTFR as not being justified economically. Please note again
that statements made by the CAFE CBA authors that MTFR is justified [14, 15] are
not based on marginal costs and benefits, but on the absolute ones and this is
simply a wrong way of using cost benefit analysis.

For the B scenario there is a 49% ‘No’ vote. For the lower ambition scenarios A and
TSAP there are only ‘Yes’ votes.

3.4.2. Impact of morbidity

As a variation we have run the Reference case of section 3.4.1, but now without
accounting for morbidity. This simply results in a shift of the graphs to the left. The
net effect is that the TSAP scenario would get about 29% ‘No’ votes.

As the shapes of the distributions do not change very much the corresponding
figures are not shown here.

3.4.3. Defra VOLY variation

An interesting variation of the reference case is replacing the NewExt full distribution
by the Defra full distribution, which is much less extreme in its values than NewExt,
see Figure 2 and the discussion in section 2.1.1. As there are three independent
data sets in the Defra study (corresponding to 1, 3 and 6 months of life expectancy
increase), we have taken the average of the three full VOLY Weibull distributions for
Normal Health, an exercise which is very easy to perform in Crystal Ball although the
resulting distribution is of course a complex one and no longer an exact Weibull
distribution.

As we have used the VOLY distributions for Normal Health, we also have set the
morbidity contribution to zero, because the Normal Health values will already include
an implicit valuation of morbidity and we do want to avoid double-counting.

The results are given below.

17
 report no. 4/06

Figure 4 Variation of reference case with full average Defra distribution

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 19,524 Displayed
.058 1169

.044 876.7

.029 584.5

.015 292.2

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 40.18% from -Infinity to 0 MEuro

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.370 7395

.277

.185

.092

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 57.67% from -Infinity to 0 MEuro

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.324 6477

.243

.162

.081

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 91.97% from -Infinity to 0 MEuro

18
 report no. 4/06

Forecast: Scenario MTFR - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.277 5542

.208

.139

.069

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 99.68% from -Infinity to 0 MEuro

Note that in this case the TSAP scenario would get almost 60% of ‘No’ votes. This
means that, using what is in our opinion a more acceptable VOLY distribution, the
TSAP ambition level seems not very well justified.

The TSAP net benefits distribution has a mean of about 500 M€ and a median of -
300 M€.

The MTFR scenario is now completely out of the picture: more than 99% of the votes
would call this scenario not justified.

3.4.4. Variations of Defra VOLY with alternative PM chronic mortality dose-

response function

To estimate the impact of PM2.5 on chronic mortality one needs a concentration

response function CRF which enables a quantification of the change in chronic
mortality with a change in PM2.5 concentrations. The concentration response function
is assumed to be linear without a threshold so the ratio of change in mortality with
the concentration change is constant and equal to the slope of the CRF. Information
on this slope can be obtained from epidemiological studies although this is a
complicated exercise.

For the CAFE CBA the CRF used implies a 6% all cause mortality change per
10 µg/m3 PM2.5 change. However, other analyses of the data give rise to a different
CRF slope. Here we will consider as an alternative a slope of 2% as per Krewski et
al. [12] in which study the PM mortality was adjusted for the confounding effects of
SO2 in the atmosphere. This is a reasonable alternate case to consider since in
Europe today, the SO2 levels are quite low relative to those in the U.S. during the
study period. Further, there is a substantial toxicology and human clinical literature
clearly indicating that exposure to particulate matter with SO2 produces a much
higher degree of respiratory effects than exposure to PM alone. A second alternative
would be a 4% slope value as per Pope et al. [13] which considers a long time lag
for PM effects as per the WHO Global Burden of Disease Study.

19
 report no. 4/06

Only the 4% case is shown here, because this already leads to a situation where
even the A scenario would get a vote of less than 50%, in fact there is a 52%
probability of benefits minus costs being less than zero. The TSAP benefits minus
costs distribution is largely below zero: there are 72% ‘No’ votes. So even with an
intermediate value for the CRF slope the TSAP is seen to be strongly suboptimal in
terms of benefits minus costs.

The results for the MTFR scenario are not shown here: as above it is fully unjustified.

Of course using the slope of 2% would shift the balance even more. Calculations
show that the A scenario would have 73% of ‘No’ votes in this case and the TSAP
would even have as much as 87% of ‘No’ votes.

Figure 5 Variation of reference case with full average Defra distribution

and alternative exposure-response function slope

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 19,842 Displayed
.088 1763

.066

.044 881.5

.022 440.7

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 52.23% from -Infinity to 0 MEuro

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.486 9728

.365

.243

.122

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 70.59% from -Infinity to 0 MEuro

20
 report no. 4/06

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.431 8623

.323

.216

.108

.000 0
-30,000 -7,500 15,000 37,500 60,000
Certainty is 96.67% from -Infinity to 0 MEuro

3.5. RESULTS UNCERTAINTY ANALYSIS WITH REPRESENTATIVE VOLY

DISTRIBUTION

The following results are no longer using the ‘voting’ concept but use the more
conventional approach of characterising the VOLY statistical spread by using a
central value (average, mean) and a standard deviation. In all cases the standard
deviation is taken to be 30%. We also assume that the VOLY has a normal
distribution.

For all the results presented in this chapter for the costs a normal distribution was
used as discussed in section 3.3 , bullet 3.

3.5.1. Defra average means as representative value

This case again uses the Defra study results as we judge the NewExt results to be
too extreme. As a central VOLY value we take the average of the three Defra mean
values for Normal Health, again assuming no separate morbidity valuation. This
means that the VOLY has a central value of 26,925 €. The standard CRF slope of
6% is used.

Results for the A, TSAP, B and MTFR scenario are given here in terms of marginal
benefits minus marginal costs.

21
 report no. 4/06

Figure 6 Representative VOLY; average of three Defra means

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.038 760

.029 570

.019 380

.010 190

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 2.36% from -Infinity to 0 MEuro

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.083 1653

.062

.041 826.5

.021 413.2

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 38.06% from -Infinity to 0 MEuro

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.064 1275

.048 956.2

.032 637.5

.016 318.7

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 79.38% from -Infinity to 0 MEuro

22
 report no. 4/06

Forecast: Scenario MTFR - Benefits minus Costs

20,000 Trials Frequency Chart 16,408 Displayed
.020 407

.015 305.2

.010 203.5

.005 101.7

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 99.49% from -Infinity to 0 MEuro

Note that the shape of the curves is now quite different than when using the full
(Weibull) distributions and is much closer to a normal distribution. Also note that
there are two parameters with a normal distribution: the VOLY and the costs figure.

The interpretation of the colours is quite simple: the areas of the curves where the
net marginal benefits (marginal benefits minus marginal costs) are less than zero are
coloured red. E.g. for the TSAP scenario there is now a 39% probability that the net
benefits are negative, that is that marginal costs exceed the marginal benefits. Of
course for a robust scenario selection one would like this probability to be small, at
least smaller than 50%. Clearly the B and MTFR scenario have a high chance of
‘regret investment’ meaning that there is a high probability that costs will exceed
benefits.

Again, this spread in the net marginal benefits is caused by the inherent statistical
spread in the two parameters VOLY and ‘annualised costs’.

3.5.2. Defra average means as representative value with 2% CRF slope

Building on the previous case, as an interesting parameter variation the slope of the
CRF was set to what we consider its lowest plausible value of 2%. The 4% value will
be discussed later on. Again the separate morbidity valuation is set to zero as we
assume this included in the VOLY Normal Health results.

Results are given below, leaving out the MTFR graph which shows an almost 100%
chance of regret investment.

23
 report no. 4/06

Figure 7 Representative VOLY; Defra average means, CRF slope of 2%

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.090 1806

.068

.045 903

.023 451.5

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 62.83% from -Infinity to 0 MEuro

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.086 1714

.064

.043 857

.021 428.5

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 62.60% from -Infinity to 0 MEuro

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.064 1282

.048 961.5

.032 641

.016 320.5

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 87.74% from -Infinity to 0 MEuro

The A and TSAP scenario show a 62-63% chance of regret investments.

24
 report no. 4/06

3.5.3. Defra average medians as representative value

As was already noted in section 2.1.1, when dealing with such skewed parameter
distributions as we have for VOLY in this CBA context, looking for a representative
central parameter value the median is in general a much more robust choice than
the mean in terms of characterising the typical value. See also the discussion in [1].

Therefore we have also analysed the case where the representative value is
calculated as the average of the three Defra medians for Normal Health (no explicit
morbidity). This results in a central value of 6,747 €. It is clear that compared to the
mean the median value is much less influenced by the long but low right-hand side
tail of the VOLY Weibull distribution.

The used CRF slope is the CAFE CBA value of 6%.

Figure 8 Representative VOLY; Defra average medians, CRF slope of 6%

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.103 2067

.078

.052

.026 516.7

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 86.30% from -Infinity to 0 MEuro

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.086 1727

.065

.043 863.5

.022 431.7

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 66.05% from -Infinity to 0 MEuro

25
 report no. 4/06

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.064 1276

.048 957

.032 638

.016 319

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 87.87% from -Infinity to 0 MEuro

The impact of this variation is large: the A scenario now has a chance of generating
negative net benefits of 87% and for the TSAP this number is 66%.

3.5.4. Combining several variations into a ‘reasonable’ case

Looking at all the calculated results above, we can try to define a case which is
reasonable and uses values that are not at the lowest or highest point of a range.
This means that rather than NewExt we will use the Defra study. Also instead of
using a CRF slope of 2% or 6% it seems more reasonable to use the intermediate
value of 4%. And although using the median as a typical value is without doubt the
most robust approach, we will not use the average of the Defra medians, which
gives a rather low representative VOLY value, but instead the maximum of the three
Defra Normal Health median numbers. This leads to a representative value of
15,360 € which is, taking all considerations as discussed in previous sections into
account, a very reasonable number and definitely not too low. As before, using the
Normal Health numbers implies that we use no additional explicit morbidity valuation.

The results for this quite ‘reasonable’ case are shown in the figures below.

Figure 9 ‘Reasonable’ case; Defra maximum median, CRF slope of 4%

Forecast: Scenario A - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.081 1625

.061

.041 812.5

.020 406.2

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 49.01% from -Infinity to 0 MEuro

26
 report no. 4/06

Forecast: Scenario TSAP - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.088 1752

.066

.044 876

.022 438

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 61.07% from -Infinity to 0 MEuro

Forecast: Scenario B - Benefits minus Costs

20,000 Trials Frequency Chart 20,000 Displayed
.064 1271

.048 953.2

.032 635.5

.016 317.7

.000 0
-30,000 -15,000 0 15,000 30,000
Certainty is 87.09% from -Infinity to 0 MEuro

The conclusions for this case are clear: the A scenario looks to be precisely at the
point where net benefits are zero (almost equal chance of either negative or positive
net benefits), but the TSAP is well beyond that point in the sense that chances of
receiving negative net benefits are fairly high (61%).

Of course, if one would have additional scenarios covering the gap between CLE
2020 and the A scenario, it could well be that the true optimum would be below the A
ambition level as we are doing a marginal analysis here and a better description of
the change in net benefits could modify our conclusions.

Also, for a robust scenario selection, one may argue that the ‘50-50’ distribution of
the A scenario for this case is not good enough, certainly taking the results of the
previous section into account where a much lower but still defendable VOLY central
value was used.

27
 report no. 4/06

4. CONCLUSIONS AND RECOMMENDATIONS

1. The CAFE CBA methodology has two fundamental shortcomings:

- It uses the wrong CBA metric

- It does not use a proper marginal analysis.

This leads to some wrong conclusions in terms of scenario justification.

2. In the air pollution context of CAFE the relevant metric is VOLY rather than
VSL.

3. NewExt VOLY numerical results are very much higher than those found in
other studies.

4. Defra commissioned a high quality study directly eliciting VOLY. The results of
this study have not been given sufficient attention in the CAFE CBA.

5. The life insurance approach gives an alternative to the subject of mortality

valuation. The results suggest much lower estimates, although a direct
interpretation of these results is not clear. Efforts to replicate the findings in
different populations by different investigators would be informative, although it
is worth noting that the findings are similar to several previous stated
preference surveys.

6. The CAFE TSAP ambition level is not very robust when looking at the impact
of reasonable parameter variations such as:

• VOLY distribution (full sample distribution and distribution around a

representative value)

• PM concentration response functions (2, 4 or 6%)

• Morbidity (with or without)

The CAFE CBA methodology could be considerably improved by taking the

comments above into account, especially by using a proper marginal analysis
(comment 1) and using VOLY rather than VSL (comment 2). Using a more balanced
VOLY value, also accounting for the outcomes of the Defra study, is also strongly
recommended (comments 3 and 4). This would also improve the robustness of the
TSAP when varying relevant parameters (comment 6).

28
 report no. 4/06

5. LIST OF ABBREVIATIONS AND ACRONYMS

A Scenario from CAFE programme

B Scenario from CAFE programme

C Scenario from CAFE programme

CAFE Clean Air For Europe, EU air quality programme

CBA Cost Benefit Analysis

CLE CAFE scenario Current Legislation Enacted

CRF Concentration Response Function

EMEP UN-ECE’s cooperative programme for monitoring and evaluation

of the long-range transmission of air pollutants in Europe

LE Life Expectancy

MTFR CAFE scenario Maximum Technically Feasible Reductions

PM Particulate Matter

PMx PM with a nominal mean aerodynamic diameter less than

or equal to x micrometer (x is 10 or 2.5 in this context)

RR Risk reduction

TSAP Thematic Strategy on Air Pollution

RAD Restricted Activity Day

RAINS Mathematical model used for CAFE

(Regional Air Pollution Information and Simulation)

VOLY Value of One Life Year

VPF Value of a Prevented Fatality

VSL Value of a Statistical Life

WHO World Health Organization

WTP Willingness To Pay

29
 report no. 4/06

6. REFERENCES

1. Alberini, A. (2004) Robustness of VSL values from contingent valuation surveys.

Working Paper 135.04. Milano: Fondazione Eni Enrico Mattei

2. Alberini, A. et al (2004) Willingness to pay to reduce mortality risks: evidence from a

three-country contingent valuation study. Working Paper 111.04. Milano: Fondazione
Eni Enrico Mattei

3. Chilton, S. et al (2004) Valuation of health benefits associated with reductions in air

pollution. Defra Publication PB 9413. London: Department for Environment, Food
and Rural Affairs

4. Johannesson, M. and Johansson, P.-O. (1996) To be, or not to be, that is the
question: an empirical study of the WTP for an increased life expectancy at an
advanced age. J Risk and Uncertainty 13, 163-174

5. Rabl, A. (2003) Interpretation of air pollution mortality: number of deaths or years of

life lost? J Air & Waste Manage Assoc 53, 1, 41-50

6. AEAT (2005) Methodology for the cost-benefit analysis for CAFE: Volume 1:
Overview of methodology. AEAT/ED51014/Methodology Paper Issue 4. Oxfordshire:
AEA Technology Environment
(http://www.cafe-cba.org)

7. AEAT (2005) Methodology for the cost-benefit analysis for CAFE: Volume 2: Health
impact assessment. AEAT/ED51014/Methodology Volume 2 Issue 1. Oxfordshire:
AEA Technology Environment
(http://www.cafe-cba.org)

8. AEAT (2005) Methodology for the cost-benefit analysis for CAFE: Volume 3:
Uncertainty in the CAFE CBA: methods and first analysis.
AEAT/ED51014/Methodology Volume 3 Issue 1. Oxfordshire: AEA Technology
Environment
(http://www.cafe-cba.org)

9. Morris, J. and Hammitt, J.K. (2001) Using life expectancy to communicate benefits of
health care programs in contingent valuation studies. Medical Decision Making 21,
468-478

10. IER (2004) New elements for the assessment of external costs from energy
technologies (NewExt). Final report to the European Commission, DG RTD.
Stuttgart: Institute for Energy Economics and the Rational Use of Energy.
(http://www.ier.uni-stuttgart.de/forschung/projektwebsites/newext/)

11. Desaigues, B. et al (2004) Monetary valuation of air pollution mortality: current

practice, research needs and lessons from a contingent valuation. Document de
Travail No. 2004-39. Aix-Marseille: GREQUAM

12. Krewski, D. et al (2000) Reanalysis of the Harvard Six Cities study and the American
Cancer Society study of particulate air pollution and mortality. Part 2: Sensitivity
analysis. Special Report. Cambridge MA: Health Effects Institute

30
 report no. 4/06

13. Pope, C.A. III et al (2002) Lung cancer, cardiopulmonary mortality, and long-term
exposure to fine particulate air pollution. JAMA 287, 1132-1141

14. AEAT (2005) Cost-benefit analysis of policy option scenarios for the Clean Air for
Europe programme. AEAT/ED48763001/ABC Scenarios, Issue 2. Oxfordshire: AEA
Technology Environment

15. AEAT (2005) Cost-benefit analysis of the thematic strategy on air pollution.
AEAT/ED48763001/Thematic Strategy, Issue 1. Oxfordshire: AEA Technology
Environment

16. EU (2005) ExternE - Externalities of energy - methodology 2005 update. EUR

21951. Luxembourg: Office for Official Publications of the European Communities

17. Abbey, D.E. et al (1995) Estimated long-term ambient concentrations of PM10 and
development of respiratory symptoms in a non-smoking population - particulate
matter less than 10 microns in diameter. Arch Environ Health 50, 2, 139-152

18. Ostro, B.D. (1987) Air pollution and morbidity revisited: a specification test. J of
Environ Economics Management 14, 87-98

19. Ostro, B.D. and Rothschild S. (1989) Air pollution and acute respiratory morbidity: an
observational study of multiple pollutants. Environ Res 50, 2, 238-247

20. Krupnick, A.J. et al (2002) Age, health, and the willingness to pay for mortality risk
reductions: a contingent valuation survey of Ontario residents. J Risk and
Uncertainty 24, 161-186

21. Haddix, A.C. et al (2002) Prevention effectiveness - a guide to decision analysis and
economic evaluation. Oxford: Oxford University Press

22. US OMB (2003) Regulatory analysis. OMB Circular A-4, September 17, 2003.
Washington DC: The Office of Management and Budget
(http://www.whitehouse.gov/omb/circulars/a004/a-4.html)

31
 report no. 4/06

APPENDIX 1 OVERVIEW OF STATED PREFERENCE STUDIES ELICITING

VOLY

The NewExt reference paper

Reference

Alberini, A., Hunt, A., Markandya, A., 2004, Willingness to Pay to reduce Mortality Risks:
Evidence from a Three-Country Contingent Valuation Study, Fondazione Eni Enrico Mattei Note
di Lavorro Series, Nota di Lavorro 111.2004, September 2004

Target group

Persons aged between 40 and 75. Respondents from UK, France and Italy.

Type of WTP asked for

Reduction in risk of dying, metric is VSL

Specific questions

WTP for a risk reduction of 5 in 1000 to be experienced over the next 10 years (beginning
immediately)

Same, but for 1 in 1000 risk reduction (no results reported here)

WTP for a risk reduction of 5 in 1000 to be experienced over the 10 years but beginning at age 70
(no results reported here)

Survey technique

Self-administered (computer), dichotomous choice questions, dichotomous choice follow-ups

Main conclusions

VSL are within and at the low end of the range recommended by DG Environment

No evidence that WTP (VSL) is lower for older persons

WTP responses combined with life expectancy implied by the 5 in 1000 risk reduction is used to
estimate VOLY

VOLY for pooled results 52,000 € (median) and 118,000 € (mean)

32
 report no. 4/06

Morris & Hammitt

Reference

Morris, J., Hammitt, J.K., 2001, Using Life Expectancy to Communicate Benefits of Health Care
Programs in Contingent Valuation Studies, Medical Decision Making, Nov-Dec 2001, pp. 468-
478, 2001.

Target group

US citizens, all younger than 60 (70)

Type of WTP asked for

Life expectancy increase, reduction in risk of dying, metric is VOLY

Specific questions

Half of the respondents were asked for WTP for a vaccine benefit expressed as a life expectancy
gain

For the other half the equivalent benefit was expressed as a reduction in average annual chance
of death

Each half was further split (in total 4 subsamples): two samples were asked to give WTP if the
vaccine was given at age 60, the other two samples for a vaccine given at age 70

Survey technique

National random-digit-dial phone interviews followed by a follow-up phone interview after

respondents had received a mailed packet of information

Double-bounded dichotomous WTP question

Main conclusions

Response rate 75%, of those 30% would not consider getting the vaccine

Life Expectancy method has greater validity than Risk Reduction method (scope test)

Life Expectancy also values higher than Risk Reduction

VOLY between $492 and $698

33
 report no. 4/06

Johannesson & Johansson

Reference

Johannesson, M., Johansson, P-O., 1996, To Be, or Not to Be, That Is the Question: An
Empirical Study of the WTP for an Increased Life Expectancy at an Advanced Age, J. of Risk and
Uncertainty, volume 13, pp. 163-174, 1996.

Target group

Swedish citizens, aged 18-69

Type of WTP asked for

LE, metric is VOLY (directly measured)

Specific questions

Life expectancy increase of 1 year (10 instead of 11 years) at age 75 due to medical treatment,
conditional on having survived until the age of 75

Survey technique

Binary WTP question (only one WTP value is mentioned, answer yes or no, if yes: ‘are you
sure?’, no more questions), telephone interview.

Main conclusions

71% of the answers is ‘no’: many individuals seem to have a zero WTP

Average VOLY is $400 to $1500 depending on statistical estimation technique used

34
 report no. 4/06

Defra commissioned study

Reference

Chilton, S., Covey, J., Jones-Lee, M., Loomes, G., Metcalf, H., 2004, Valuation of health benefits
associated with reductions in air pollution, Defra publication PB 9413, May 2004.

Target group

UK citizens

Type of WTP asked for

Life Expectancy, metric is VOLY (directly measured)

Specific questions

WTP was asked for a life expectancy increase of 1, 3 or 6 months (three subsamples). For each
sample: 1/3/6 months extra life in Normal and Poor health.

All respondents were also asked WTP for Avoiding Hospital Admissions & Avoiding Breathing
Discomfort.

Survey technique

Ordering of cards with WTP values

Main conclusions

VOLY range:

Based on: 1 month N 1 month P 3 months N 3 months P 6 months N 6 months N

Mean € 45,298 € 12,421 € 22,771 € 5,693 € 12,705 € 2,168

Median € 15,360 € 1,783 € 2,218 €0 € 2,663 € 395

35