Partenariato Pubblico Privato
Partenariato Pubblico Privato
Partenariato Pubblico Privato
Background Paper
About Ecorys
At Ecorys we aim to deliver real benefit to society through the work we do. We offer research,
consultancy and project management, specialising in economic, social and spatial development.
Focusing on complex market, policy and management issues we provide our clients in the public,
private and not-for-profit sectors worldwide with a unique perspective and high-value solutions.
Ecorys remarkable history spans more than 80 years. Our expertise covers economy and
competitiveness; regions, cities and real estate; energy and water; transport and mobility; social
policy, education, health and governance. We value our independence, integrity and partnerships.
Our staff are dedicated experts from academia and consultancy, who share best practices both
within our company and with our partners internationally.
Ecorys Netherlands has an active CSR policy and is ISO14001 certified (the international standard
for environmental management systems). Our sustainability goals translate into our company policy
and practical measures for people, planet and profit, such as using a 100% green electricity tariff,
purchasing carbon offsets for all our flights, incentivising staff to use public transport and printing on
FSC or PEFC certified paper. Our actions have reduced our carbon footprint by an estimated 80%
since 2007.
ECORYS Nederland BV
Watermanweg 44
3067 GG Rotterdam
P.O. Box 4175
3006 AD Rotterdam
The Netherlands
T +31 (0)10 453 88 00
F +31 (0)10 453 07 68
E [email protected]
Registration no. 24316726
W www.ecorys.nl
NL02 26598
Table of contents
Summary
Introduction
11
3.1
Introduction
11
3.2
11
3.3
22
3.4
26
3.5
29
3.6
29
31
4.1
Introduction
31
4.2
Result equation
32
4.3
38
4.4
40
4.5
42
44
5.1
Introduction
44
5.2
44
5.3
47
5.4
50
53
53
53
53
53
55
Summary
In preparation of the ESF 2014-2020 multi-annual framework, the European Commission organised
a learning seminar on setting and adjusting targets for ESF Operational Programmes. To account
for expenditures, indicators on outputs (numbers of participants, entities) and results (numbers of
participants finding employment, gaining a qualification etc.) are essential. This background paper
discusses methodologies for setting and adjusting quantified cumulative employment targets.
According to a survey of the EC, two commonly applied methods in the 2007-2013 programming
period were extrapolation from the past and a combined use of budget allocation and unit cost.
Common challenges were accounting for external factors and lack of data.
This paper first discusses the combined use of budget allocation and unit cost per participant. This
approach allows for an integrated setting of output and result targets to maximise the expected
achievement of an objective. An objective could be to maximize the number of participants in
employment (the result target). Another objective could be to maximize the total number in
employment from both participants and non-participants. This may result in different output targets
and higher employment levels overall (overall effectiveness) despite lower result targets (gross
effectiveness). Another approach is a SWOT analysis, which allows an integrated approach of topdown setting of objectives and bottom-up allocation of budgets given unit costs. Methods to allocate
budgets proportionally to target groups and by extrapolation are easier to apply but have the
drawback that they are not result oriented.
When calculating unit costs from historical data, it is also preferable to use actual historical costs
instead of historically committed costs because the actual number of participants may be different
from the initially targeted number and this could result in distorted unit costs and output targets for
the next programming period. In the case of on-going interventions, it is still preferable to use data
on actual costs incurred and to add estimates of the remaining cost to be incurred, over using
initially committed budgets for the whole period. Unit costs are also preferably calculated at a
disaggregate level to calculate more accurate output targets. When gaps in disaggregate data exist,
or perhaps when costs are influenced by factors that change in the future, a cost equation that
relates unit costs to other factors can be used to estimate unit costs. However normative issues
also play a role: are efficiency gains possible to reduce unit costs, if some projects achieve similar
results for similar target groups at lower cost? Finally, in the complete absence of any data on unit
costs, Eurostat LMP data can be used, but own judgement will still be needed.
The setting of result targets is as easy or as difficult as applying success rates to the output targets.
More than unit costs, success rates depend on a large number of factors beyond the influence of an
ESF Operational Programme, and a result equation can be used to predict future success rates. To
predict the future success rates this way, the development of the other factors need to be predicted
as well, or need to be indicated in scenarios of the future. Result equations could be tailor-made to
specific interventions, but it requires advanced econometrics to estimate such equations. A Bartik
(2009) type of model is linear and uses micro-data from individual participants and is less sensitive
to assumptions underlying the model. As for unit costs, own judgement is needed as well to assess
whether better success rates could be achieved in the future. Simply picking the project with the
highest success rate is not a sound method, because this project may have been targeted at easier
groups, where participants would have found a job without intervention as well. Randomized
experiments and matching are methods that help compare success rates for similar groups and
frontier analysis further helps choosing projects with higher result rates that should be achievable
for other projects. In the end, allocating more budget to such projects make it more likely that
(higher) result targets are achieved.
Finally, the result equation can also be used to adjust targets if at a certain moment during the
programming period, the forecasts of the factors influencing the success rates change. In this case
one needs to add the predicted results for the remainder of the programming period to the results
that could have been expected up to that moment based on the previously set targets.
Introduction
In preparation of the ESF 2014-2020 programming period, the European Commission organised a
learning seminar on setting and adjusting targets for ESF Operational Programmes. To account for
expenditures, indicators on outputs (numbers of participants, entities) and results (numbers of
participants finding employment, gaining a qualification etc.) are essential. For a limited number of
output and result indicators, quantified cumulative targets for the whole programming period need
to be set. This background paper presents and discusses methodologies for setting and adjusting
quantified cumulative employment targets.
Targets at Operational Programme level
In the ESF, Member States set national targets for outputs and results for the whole period 20142022 (up to two years beyond the ESF programming period). The results of an Operational
Programme are the sum of the results of its investment priorities and ultimately the individual
projects, as illustrated in Figure 1.1. If for example a combined total of 10,000 people participate in
projects conducted in the framework of one investment priority, and out of them 30% on average
are in employment upon, different projects under the Operational Programme may well have
varying success rates. In the example of Figure 1.1, the success rates vary from 20% to 50%.
Likewise, a target at Operational Programme level does not imply proportionate targets for each
project.
Figure 1.1
Source: Ecorys, prepared for the learning seminar on targeting of 14 March 2013.
Targets at the level of investment priorities are not only different from targets at project level, they
are not targets at the macroeconomic level either. According to a survey of the EC discussed in the
next chapter, macroeconomic performance indicators such as the employment rate and the
unemployment rate are influenced by external factors beyond the control of ESF Operational
Programmes, such as the business cycle and changes in national legislation. In addition, only part
of the target population participates in ESF co-funded projects. The wider impact of ESF
Operational Programmes on for example the unemployment rate are therefore hard to estimate.
However, the output and result indicators do not relate to wider impacts, and are not discussed in
this background paper.
List of common indicators
The draft ESF regulation contains a provisional list of common indicators to be used for the period
2014-20201, which is also given in annex A in this paper.
The two types of indicators to be used are:
Regulation of the European Parliament and of the Council on the European Social Fund and repealing Council Regulation
EC No 1081/2006; Proposal of 14.3.2012,
http://ec.europa.eu/regional_policy/sources/docoffic/official/regulation/pdf/2014/proposals/regulation/esf/esf_proposal_en.pdf
2
Ecorys (2011), Performance Targets for ESF Operational Programmes. http://www.coprbm.eu/?q=node/494
At the start of 2013, the European Commission conducted a survey on targeting methodologies
used in the previous 2007-2013 programming period, in preparation of the learning seminar. From
this survey, it became clear that the Member States applied different methodologies. Extrapolation
of historical data was the most commonly applied method, by 10 out of 17 responding countries,
see Table 2.1. However this method was always combined with other methods. Another frequently
applied method was to calculate output targets from budgets and unit costs (7 countries). However,
new Member States had less data from the past to extrapolate from and some newer Member
States started from national strategic documents (5). Some countries explicitly indicated they
adjusted targets from the past according to the new economic situation (2) or estimated social and
labour market trends (3). Ex ante evaluations (3) and expert judgement (6) were also applied and
one country explicitly mentioned the use of benchmarking.
Table 2.1
Number of countries
10
Expert judgement
Ex ante evaluation
Benchmarking
17
As Member States used different methodologies, they also faced different challenges. The most
frequently mentioned challenge was the long time horizon of the ESF period, mentioned by 7
countries (Table 2.2). In the 2007-2013 programming period, external factors such as changes in
the demand for labour, demographic changes and even changes in national legislation required
adjustments of priorities and the associated targets.
Table 2.2
Number of countries
Realistic estimations
Number of countries
17
A lack of historical data is another frequently mentioned challenge (5 countries), and indicates that
the most often applied methodology of regression / extrapolation from the past can be far from
straightforward. Lack of data is mentioned in particular by newer Member States, but also by other
countries, for new types of interventions.
Developing macroeconomic impact indicators was another challenge faced in the 2007-2013
programming period (4 countries). However, the list of common indicators for the ESF 2014-2020
period does not include macroeconomic impact indicators. Moreover, as for programme-specific
indicators, the Commission does not recommend to use impact indicators for ESF programme
monitoring. A more specific but nevertheless important question is whether to formulate output
targets in percentages (percentage of the population) or in absolute numbers (2 countries).
A reality check was another challenge mentioned specifically by a few countries (3) how realistic
will it be to deliver the targets? Quantifying targets was also a challenge for interventions that
consist of a mix of activities, in the situation of strong disparities between regions or target groups
(2 countries each).
Two countries that extrapolated data from the past, mention the challenge to ensure coherence with
national policies. Two countries question the application or even usefulness of targets in result
based management. Related issues raised by countries are whether to formulate few or many
targets, the coherence between indicators (if interventions have poor results prioritizing other
interventions to achieve better results implies that output targets for the original interventions are
not met). Also, other targets besides ESF indicators may be important for specific interventions.
The example given in the survey is getting closer to the labour market. It can be noted in this
respect that the list of common indicators for the 2014-2020 period includes participants with an
improved labour market situation 6 months after leaving.
Two specific methodological issues mentioned by countries concern the formulation of the
baselines and the harmonization of methodologies with EC guidance. In this background paper the
formulation of baselines with limited data is discussed in the concluding sections of the next two
chapters (Sections 3.6 and 4.5). Harmonization of methodologies with EC guidance is mentioned
by one country, but not further elaborated, nor in this background paper.
10
3.1
Introduction
Annex 1 of the ESF Regulation for the programming period 2014-2020 distinguishes between two
types of output indicators:
This chapter discusses setting output targets for participants. The main target groups are set out in
the provisional list of common indicators (see Annex A). In the previous ESF programming period,
objectives were to be set in National Action Plans (top-down approach), and unit costs were applied
to calculate what outputs were feasible given the budget (bottom-up approach).
Ideally output targets are set in such a way that the result target (gross effect of the programme), or
even better the number of persons in employment (overall effectiveness) is maximized and this
method is discussed first. A SWOT analysis that the French government applied in the previous
programming period combined both the top-down and bottom-up approach coherently. This method
determined the budget allocation. Other countries applied different methods to allocate budgets.
After determining the budget allocation, unit costs were applied to calculate output targets. All these
methods are discussed in Section 3.2.
Section 3.3 discusses the calculation of unit costs which play such an important role in the setting
of output targets. Output targets are set most accurately with disaggregate unit costs, but this
requires accurate disaggregate data. To overcome some of the data requirements, Section 3.4
discusses how a cost equation can be estimated, which relates unit costs to characteristics of the
target groups, interventions or regions. However, normative issues which are discussed in Section
3.5 also play a role: is it possible to reduce the unit costs in the future? Finally, a fall-back option for
estimates of unit costs in the complete absence of data is discussed in Section 3.6.
3.2
The calculation of the output target therefore depends on how budgets are allocated and on the
estimate of the unit cost which is discussed in the next section.
The allocation of the budget is an important issue, because the need for interventions and the
effectiveness of interventions may change over time, in different ways for different regions and
target groups. For example, when labour demand increases, job counselling is likely to be more
effective. In times of crisis, occupational mobility tends to increase3 and the same applies to
3
Moscarini, G. and F. Vella (2008), Occupational mobility and the business cycle, NBER Working Paper,
www.econ.yale.edu/~gm76/cps_nber_wp.pdf (based on USA data)
11
sectoral mobility,4 and different measures may further support this mobility such as perhaps
retraining workers.
For allocating budgets, the following methodologies are applied in Member States:
1. Ex ante analysis;
2. SWOT analysis;
3. Proportionally to the distribution of the population;
4. Extrapolation of trends.
1.
Ex ante analysis
The ideal ex ante analysis compares the situation without intervention with the situation with
interventions. This may result in a different solution compared to maximizing result targets.
Maximizing result targets optimizes the result achieved with the interventions (gross effectiveness)
whereas maximizing the total number in employment (among participants and non-participants)
optimizes the result achieved through the interventions (overall effectiveness).5
The following hypothetical example illustrates the difference between gross and overall
effectiveness. Suppose out of two interventions, intervention A has unit cost 10,000 per
participant and results in 70% of the participants in employment, and intervention B has unit cost
5,000 and results in 40% of the participants in employment.6 Without intervention, 20% of the
unemployed would find employment. The target population is 20 and the budget is 100,000. Table
3.1 illustrates the results of investing all in intervention A, intervention B or nothing:
A.
If the budget is invested in A, it allows for 10 participants, and the remaining 10 persons of
the target group do not participate. With success rates of 70% and 20% respectively, 7
participants and 2 non-participants find employment, or 9 in total.
B.
If the budget is invested in B, the budget allows for 20 participants, which equals the whole
target group. With a success rate of 40%, this means 8 participants find employment
which is also the total number in employment.
C.
Table 3.1
Number of participants
Intervention A
Intervention B
(maximizes numbers
(maximizes the
in employment)
result target)
No intervention
10
20
Number of non-participants
10
20
Target population
20
20
20
(= output target)
Participants in employment
(= gross effectiveness)
(= result target)
Non-participants in employment
Total in employment
(= overall effectiveness)
4
5
6
12
Scheele, D., R. van Gaalen en J. van Rooijen (2008), Werk en inkomsten na massaontslag: De zekerheid is niet van de
baan. Verkenning van CBS en WRR. (Work and income after mass layoff: Certainty is not in the job. Study of CBS and
WRR), Amsterdam University Press.
At the 14 March seminar the exercises maximized gross effectiveness because this can be done step by step.
Methods to obtain such data from partial data or to estimate these data without historical data, are discussed in later
paragraphs of this chapter and the next chapter.
As can be seen from the above table, intervention B maximizes the output and result targets, with 8
participants in employment compared to 7 for intervention A. Nevertheless, intervention A results in
the highest number of persons in employment: 9 compared to 8 for intervention B. In the above
example the high success rate of intervention A offsets the lower reach because of the high costs,
because a sufficiently high share of non-participants find employment, but this would not have been
the case if the success rate of intervention B had been 60% instead of 40%.
Finally, one needs to compare the benefit of the additional number in employment through the
intervention compared to no intervention, which is (9 4 = 5) for intervention A, with the cost. The
cost per additional person in employment is 100,000 / 5 = 20,000 for intervention A, and 100 / 4 =
25,000 for intervention B. For comparison to judge these costs, an order of magnitude of 50,000
can be given of an annual benefit of 7,000, discounted 10 years at 5%.7
Whether one maximizes the result target or the total number of persons in employment in an ex
ante analysis, the advantages and disadvantages are:
Advantages:
-
Disadvantages:
-
Needed:
-
Second best: survey data for a sample of participants / persons in the target group and
aggregate statistics per intervention;
Expertise to estimate success rates with (and ideally also without) intervention, and to value
one additional person finding work.
The example to illustrate this method is taken from the first exercise of 14 March. 10,000 workers
are threatened with unemployment, of which 50% would find a job without any intervention. The
value per job-finder is 50,000 and the budget is 10 m. The data for three interventions are
summarized in the table below. The last column reflects capacity constraints: maximum numbers of
participants that can be reached through an intervention. However, the approaches illustrated
below also work with other constraints, for example minimum numbers per region.
Alternative
Success
rate
Capacity
Job counselling
1,000
55%
5,000
Mobility scheme
4,000
60%
10,000
16,000
90%
500
Retraining
Unit cost
per
participant
If the annual benefit 7,000 and a job, if any, is found after 1 year, then the value of reduced expenditures in year 1-10
discounted at 5% per year is: 7,000/(1+5%) + 7,000/(1+5%)2 + + 7,000/(1+5%)9 = 49,755.
13
A:
Unit cost per
participant
Alternative
1. Retraining
B:
Difference in
success ratea)
C:
Unit value per
participantb)
D:
Unit value-cost
differencec)
16,000
+40%
20,000
+4,000
2. Job counselling
1,000
+5%
2,500
+1,500
3. Mobility scheme
4,000
+10%
5,000
+1,000
a)
b)
c)
The budget of 10 million can then be allocated in two rounds as follows. First, budget is allocated
to the best intervention until the budget or capacity limit is exhausted, as illustrated in the table
below.
Alternative
A:
B:
unit cost Budget limit for
participants a)
1. Retraining
16,000
2. Job
counselling
1,000
5,000
3. Mobility
scheme
4,000
10,000
Total
14
625
C:
Capacity
limit
500
D:
Participants b)
= min[B ; C]
E:
Budget c)
500
8,000,000
500
8,000,000
a)
For the first intervention, the budget limit is 10 million divided by the unit cost (column A)
b)
The lower value of the numbers from the budget limit and capacity limit (column B and C)
c)
In the second round, the remaining budget is allocated to the second-best intervention until
budget or capacity is exhausted, as shown in the table below.
Alternative
A:
B:
unit cost Budget limit for
participants a)
C:
Capacity
limit
D:
Participants b)
E:
Budget c)
1. Retraining
16,000
625
500
500
8,000,000
2. Job
counselling
1,000
2,000
5,000
2,000
2,000,000
3. Mobility
scheme
4,000
10,000
2,500 10,000,000
Total
a)
For the second intervention, the budget limit is the remaining budget of 2 million divided by the
unit cost (column A)
b)
c)
The result targets are finally calculated by multiplying the number of participants (the output) with
the success rates, in the table below. Retraining results in 90% in employment for 500 participants,
or 450 persons and job counselling results in 55% in employment for 2,000 participants, or 1,100
persons. The total result target is then 1,550. This method maximizes the cost effectiveness per
participant because step for step, budget is allocated to first the alternative with the highest cost
effectiveness per participant, then the alternative with second-highest cost effectiveness per
participant and so on. In the training, the exercise was built around maximizing the cost
effectiveness per participant because it can be done step by step in Excel.
Alternative
1. Retraining
A:
Output
B:
Success rate
C:
Result target a)
500
90%
450
2. Job counselling
2,000
55%
1,100
3. Mobility scheme
60%
2,500
Total
a)
1,550
15
To find the optimum number in employment, one needs to solve a so-called linear programming
problem under budget and capacity restrictions (or political restrictions of minimum numbers) and
the population restriction which specifies that the sum of participants and non-participants is equal
to the number of persons in the target group:
Maximize in employment = result%[1] * participants[1] + + result%[n] * participants[n]
Subject to:
-
This linear programming problem can be solved with an Excel add-in.8 One feeds the data for the
linear programming problem in an Excel sheet, and the add-in solves the optimal numbers of
participants per intervention that maximize the number of persons in employment within the
available budget. It is a black box approach which is illustrated step-for-step in Annex 2; but the
add-in is easy to apply once the user is familiar with it. The result after solving the linear
programming problem as in Annex 2 is given below.
Alternative
Non-participants
A:
Budget
B:
% jobfinders
C:
Output a)
D:
Result b)
---
50%
4,688
2,344
Job counselling
5,000,000
55%
5,000
2,750
Mobility scheme
60%
5,000,000
90%
313
281
5,313
3,031
Retraining
10,000,000
Target
Total in employment
5,375
a)
The output is calculated by the black box linear programming solver add-in.
b)
The result target is also calculated by the black box linear programming solver add-in, but the results
can be verified by multiplying columns B and C.
The total number of employment of 5,375 is higher than achieved by maximizing the costeffectiveness per participant as in the first approach described above. In this approach, the number
16
http://it.usu.edu/plugins/work/sitemaps/107/files/Linear_Programming_Using_Excel.pdf
of participants is 2,500 and therefore the number of non-participants is 7,500. The total number in
employment is therefore as given below.
The black box method of linear programming finds the maximum number in employment that can
be achieved with the available budget, but does not indicate whether the value of the mix of
interventions in terms of reduced benefit expenditures outweighs the cost. To do so, one can
calculate the cost per additional person in employment compared to the situation with no
intervention. In the above example, in the case of no intervention 50% of the population of 10,000
would find a job, or equivalently 5,000 persons. The above mix of interventions therefore results in
(5,375 5,000) = 375 additional persons in employment. The cost of 10 million euro to achieve this
result is then equivalent to 26,667 per participant, which is less than the value of 50,000
associated with each job-finder in terms of reduced benefit expenditures.
Alternative
Budget
Non-participants
Output
% jobfinders
Result
---
7,500
50%
3,750
Job counselling
2,000,000
2,000
55%
1,100
Mobility scheme
60%
8,000,000
500
90%
450
10,000,000
2,500
Target
1,550
5,300
Total in employment
The table below compares the outcomes between the output and result targets calculated after
applying the above two methods. The first method maximizes the success rate per participant and
the second method maximizes the number of employed with the same budget.
Table 3.2
b. Maximizing numbers in
Output (a)
2,500
5,313
Result (b)
1,550
3,031
62%
57%
7,500
4,688
Non-participants (c)
Success% per non-participant
50%
50%
Non-participants in employment
3,750
2,344
5,300
5,375
17
While the strength of the ex-ante analysis is that setting output and result targets happens in an
integrated process as illustrated above, the weakness of this approach is that it requires quite a lot
of information:
In reality, the success rate can vary over the business cycle. For the next programming period, the
success rates of participants starting in 2014-2020 can be different for each year. How to set
targets for the whole programming period, was illustrated on 14 March by the third exercise. In this
exercise, only one intervention is considered. This exercise is elaborated in Chapter 5 on adjusting
targets (for the business cycle).
2.
SWOT analysis
A SWOT analysis compares strengths and weaknesses of interventions, and opportunities and
threats (at the macro level) for these interventions.
It is a logical analysis that can be used to rank interventions qualitatively rather than quantitatively
as in a full-blown ex ante analysis.
Advantages:
-
Disadvantages:
-
Needed:
-
A mix of quantitative / qualitative data per intervention to identify strengths and weaknesses;
An example of a SWOT analysis can be found in a document of the French government that is
available on the internet.9 It first identified at the macroeconomic level (for France)10:
Strengths such as: a highly qualified workforce and regional business dynamics;
Weaknesses such as: the gender pay gap and difficult target groups;
Opportunities such as: jobs in personal services and jobs with durable perspectives;
After elaborating the SWOT elements, a number of policy orientations were identified (page 18-20
of the document):
9
10
18
http://www.fse.gouv.fr/IMG/pdf/valuation_ex_ante_du_PO_FSE_version_definitive-2.pdf
Page 12 and 13 of the document for the full list of SWOT elements.
Chapter 2 of the document elaborated how each priority axis of the ESF Operational Programme is
expected to contribute to objectives falling under the above policy orientations. Chapter 3 of the
document described the principles behind allocating budgets to regions, which depends on a mix of
diagnostics as at the national level and budget negotiations. Another document explains how in
France historical ESF (and Cohesion Fund) budgets were adjusted according to changes in SWOT
elements in each region.11
Shift-share models can be used to adjust historical budgets while taking into account regional shifts
in underlying variables; this model is usually applied in analysis of regional growth12 but can be
applied more widely.
3.
In this approach, the distribution of the population between regions and target groups is applied to
budget.
Advantages:
-
Disadvantages:
-
Needed: Population data or reliable survey data (e.g. Labour Force Survey)
A number of countries apply this method to allocate budgets between regions, even before target
setting in countries where the Operational Programmes were already regionalised (for example,
Italy).
The method of applying the distribution of the population is however also very useful as a reality
check: can the outputs be delivered or is the targeted number of participants larger than the target
group population in a region? To apply this method, the future population and its distribution have to
be predicted.
Assuming that the ESF intends to support three different types of interventions helping school
leavers (depending on the skill levels of school leavers) Table 3.3 informs which share of money
should go to which type of intervention in the identified regions (also assuming that regions and
interventions are equally important). If the total available budget were 100 million euro for
interventions, this would imply that 4 million euro would be allocated to interventions for high
educated school leavers in the North region.
Table 3.3
Target group
Total
North
Central
South
10%
10%
10%
30%
16%
18%
16%
50%
Low educated
school leavers
Medium educated
school leavers
11
12
The actual budgeting of ESF is integrated with the budgeting of the Cohesion Policy, see
http://www.fse.gouv.fr/IMG/pdf/Cadre_de_Reference_Strategique_National-2.pdf
E.g. Zaccomer, G. and P. Mason (2011), A new spatial shift-share decomposition for the regional growth analysis: a local
study of the employment based on Italian Business Statistical Register.
19
Region
Target group
Total
North
Central
South
4%
12%
4%
20%
30%
40%
30%
100%
High educated
school leavers
Total per region
If the population is predicted for the whole programming period 2014-2020, a different set-up of the
table is more convenient, as illustrated in the table below.
Table 3.4
Region
2014
2015
Low
10%
8%
Medium
16%
16%
High
4%
6%
Low
10%
8%
Medium
18%
18%
High
12%
14%
Low
10%
8%
Medium
16%
16%
Target
2020
group
North
Central
South
4%
6%
Total
100%
100%
100%
100%
N school leavers
1 mln
1.02 mln
1.12 mln
High
In the above example, one may allocate proportionally higher budget (in prices of 2014) to years for
which a larger total target group population is predicted.
To set output targets, the allocated budget for each region and target group is divided by the
average unit costs, preferably the average unit cost per region and target group.
4.
Trend extrapolation
Trend extrapolation is another approach to allocate budgets. Trend extrapolation to set output
targets consists of two steps: trend extrapolation per target group / region or type of intervention,
and an adjustment to fit the allocation within the available budget.
20
Advantages:
-
Disadvantages:
-
Needed: Population data or reliable survey data (e.g. Labour Force Survey)
For example if the available budget remains the same and three target group populations increase
by +5%, +7.5% and +10% respectively, an adjustment will be needed as the populations increase
more than the budget. Table 3.5 illustrates this principle, where the total budget for the next
programming period remains the same. As before, the shaded cells indicate already available data.
Table 3.5
B: Population
increase
C: Preliminary
budget
D: Final budget
Intervention 1
20 million
+5%
21 million
19.5 million
Intervention 2
40 million
+7.5%
43 million
39.8 million
Intervention 3
40 million
+10%
44 million
40.7 million
108 million
100 million
100 million
b)
a)
a)
b)
The preliminary budget (column C) adjusted to the total budget (100 million).
In the first step, a preliminary budget is calculated per intervention by increasing the budget with the
same percentage as the increase of the population. In the above example, historically 20 million
euro was spent on the first intervention, and the target group of the first intervention increases by
5%. In a preliminary budget allocation, this results in 21 million euro (20 million + 5%). The
preliminary budgets for interventions 2 and 3 are calculated in the same way.
In the second step to allocate the budget, the preliminary budgets are adjusted to the total budget.
In Table 3.5, the preliminary budgets add up to 108 million euro, while the total available budget is
only 100 million euro. Multiplying all preliminary budgets by
suffices to adjust the preliminary
budgets to the total budget of 100 million euro.
As in all other methods discussed above, output targets are calculated as the budget divided by the
unit cost per participant. The following section discusses the calculation of the unit costs which play
such an important role in setting output targets.
3.3
21
historical unit cost is the total budget of 22 million euro divided by the total number of 4,000
participants, or 5,500, as shown in the table below.
Budget
Intervention A
Intervention B
Total
10,000,000
12,000,000
22,000,000
10,000
4,000
5,500
1,000
3,000
4,000
Unit cost
Participants
Now suppose that in the next programming period, the total budget remains the same but 6 million
euro is allocated to intervention A and 16 million to intervention B. If one uses for each intervention
the historical average unit cost across all interventions ( 5,500), then the output target changes
proportionally to the total budget, regardless of the budget allocation.
In the example, if the number of participants of both interventions A and B is calculated by dividing
the new budget by the historical average across both interventions, this results in 6 million / 5,500 =
1,091 participants in intervention A and 16 million / 5,500 = 2,909 in intervention B, adding up to
4,000. The number of participants is the same as the historical number, because the total budget
has remained the same in the example.
Intervention A
Intervention B
Total
6,000,000
16,000,000
22,000,000
Unit cost
5,500
5,500
Participants
1,091
2,909
Budget
4,000
However if the historical unit costs per intervention is used, this results in 6 million / 10,000 = 600
participants in intervention A and 16 million / 4,000 = 4,000 participants in intervention B, adding up
to 4,600 participants, see the table below. The reason that more participants can be reached with
the same budget, is that more budget is allocated to an intervention with lower unit costs. By
allocating more budget to lower-cost interventions, the average unit costs across interventions
decreases, from 5,500 to 22 million / 4,600 = 4,783 in this example.
Budget
Unit cost
Participants
Intervention A
Intervention B
Total
6,000,000
16,000,000
22,000,000
10,000
4,000
600
4000
4,600
The advantage of using disaggregate unit costs is that the output targets can be set more
accurately, but the data requirements are large and certain disaggregate data may be missing. How
to deal with a situation of partially missing disaggregate data, is discussed in the Section 3.4.
Estimating costs of still running interventions
If an intervention is fully completed, one can simply divide the actual expenditures by the total
number of participants. However, if an intervention is still running, one may assume that all
expenditures have been incurred for those participants who have left the programme, but not for
those who are still in the programme. Figure 3.1 illustrates this situation, where 9 euros have been
spent, and the number of participants is 5. Naively dividing the spent budget of 9 by the number of
participants would result in an estimated unit cost of 1.8 per participant. However, recognizing that
one euro still needs to be spent for those participants who are still in the program, the unit cost is
calculated by dividing 10 by 5, resulting in a unit cost of 2.
22
Figure 3.1
The unit cost in the situation of running intervention can be calculated by assuming that for those
participants still in the program, only a percentage p of the unit cost need to be incurred. This can
be done by solving the equation:
The resulting formula for the calculation of unit cost of still running interventions is then:
) (
This generalization of the unit cost calculation reduces to the standard formula when the
intervention is completed and 100% of the costs are incurred, because then p=1.
Actual versus committed cost
A pragmatic solution for still running interventions might seem to calculate the unit cost from
historically committed budgets and historically targeted numbers of participants. However, this is
not preferred because the actual number of participants could be quite different from the targeted
number whereas the actual cost is likely to be closer to the committed costs because some costs
are fixed, such as start-up costs and overhead.
Figure 3.2 illustrates this situation, where 8 million euro is committed for a targeted number of 4,000
participants. Based on this committed budget and output the unit cost would be 2,000. However if
actually 6 million euro is actually spent on an actual number of 2,000 participants, the unit cost is in
reality 3,000.
23
Figure 3.2
Whether the historically committed or the historical actual unit costs are used for setting output
targets, matters substantially in the above example. In the first case, the budget allocated for the
next programming period would be divided by 2,000 and in the second case it would be divided
by 3,000. The method to estimate the costs that still need to be incurred for those participants
who are still in the program results in more reliable estimates of the unit cost because it uses more
actual data.
Price inflation
The average unit costs of the 2007-2013 ESF programme need to be inflated to prices of 2014 and
beyond for the 2014-2020 ESF programme. In formula, if expenditure[t] is the expenditure of year t
and pi[t] is the price inflation in year t, then the average unit costs in prices of 2014 is calculated as
follows:
[
Unit cost[2014] =
]
[
For example if prices increased by 2% each year in 2008-2012 except in 2009 when prices
remained constant, and for 2013 and 2014 an inflation of 2% is expected, the unit cost in prices can
be calculated as illustrated in Table 3.6.
Table 3.6
Expenditures
2007
2008
2009
2010
2011
2012
2013
2007-13
24
100
2008
2009
2010
2011
2012
2013
2014
102.0
102.0
104.0
106.1
108.2
110.4
112.6
102
102.0
104.0
106.1
108.2
110.4
112.6
104
106.1
108.2
110.4
112.6
114.8
106
108.1
110.3
112.5
114.7
108
110.2
112.4
114.6
110
112.2
114.4
112
114.2
799.1
In Table 3.6, the shaded cells indicate the actually incurred expenditures, in prices of the going
year. Expenditures were only 100 in 2007 compared to 112 in 2013. However when the inflation of
2008-2014 is accumulated for the expenditures of 2007, the expenditures of 100 in 2007 amount to
112.6 in expected prices of 2014. The expected expenditures of 112 in 2013 needs to be inflated by
only the expected price index of 2014 (1.02) which results in 114.2 in prices of 2014. In prices of
2014, the total expenditures of 2007-2013 is therefore not
100 + + 112 = 742 (sum of shaded cells in Table 3.6) but
112.6 + + 114,2 = 799.1 (sum of last column in Table 3.6).
If the total number of participants were 100 over the period 2007-2013, then the unit cost in prices
of 2014 should in the above example be calculated as 7.991 instead of 7.420.
For the period 2014-2020, future price inflations should be forecasted and applied to both receipts
of ESF funds and expenditures, but this matter of financing goes beyond the scope of this paper.
3.4
In this formula, the value a0 reflects fixed costs such as start-up costs and overhead that are
incurred even when there are no participants. The unit cost per participant can be related to the
13
However, when the type of future target groups and interventions are not known, one still needs to make assumptions
about future target groups and interventions, perhaps for different scenarios.
25
number of participants if economies of scale exist: projects with more participants may have lower
unit costs, although unit costs tend to increase again for very large projects. In the above formula,
the unit cost is assumed proportional to the number of participant, with value a1.
The formula further explained:
The variable: Region2 takes on the value 1 if the project is active in Region2, and otherwise
the value 0
The values b2, b3 indicate euro mark-ups for regions compared to a reference region
(Region1 in this example) which must not be included in the equation.
The values c2, c3, are euro mark-ups for target groups targeted by the project, compared to
the reference target group
The values d2, d3, are euro mark-ups for the activities of the project.
The last term is the prediction error of the formula, where 2 indicates the variance of the difference
between the actual unit cost (left hand side of the equation) and the predicted unit cost (right hand
side of the equation).
If there are 30 regions and 10 target groups, this means that 40 values must be estimated to relate
unit costs to these factors. However, often many values need not to be used to calculate unit costs:
1. Values that are virtually zero;
2. Values that are statistically not reliable;
3. Values in a close range.
The principle of Ockhams razor in econometrics is to remove variables with zero, unreliable or
similar values, to greatly reduce the number of variables. For example, consider a cost equation for
unit costs per project, with the following estimated values, with the error margin given below the
estimate (the example abstracts from different activities)14:
Then the value for the second region may be neglected because the mark-up of 3 is very small.
The value for the third region may be neglected because the error margin ( 100) is larger than
the value of the percentage mark-up ( 50). This implies that regions 1, 2 and 3 can be grouped
together as the reference region. The euro mark-ups for regions 4, 5 and 6 may all be assumed to
be 300, because for all three regions, 300 is within the error margin of the estimated values.
The unit cost in the last region is 500 lower than for equally large projects for the same target
groups in the reference region. After applying Ockhams razor the equation becomes (ignoring the
error margins of the estimated values):
14
26
Usually the error margin is determined as the 95% range of values that would be estimated for random selections of the
population which can be calculated as 2 times the standard deviation of the estimate.
Table 3.7 illustrates how with the previous formula, the unit cost of one region (R3) with missing
data for some projects can be calculated. The example is simplified by assuming that all projects in
one region have the same target group, but similar calculations apply for real life data:
Table 3.7
Region
Number
Historical
Historical unit
Estimated unit
projects
participants
cost
cost
per project
R1
20
50
7500
5500
R2
20
100
4000
5100
R3
20
200
4200
R4
20
50
8000
5800
R5
2&3
20
100
5000
5600
R6
20
200
3500
4500
R7
20
400
2000
2000
120
18,000
63,500,000
66,700,000
20
4,000
+4000 x ?
16,800,000
140
22,000
83,500,000
Total excl. R3
R3
Total incl. R3
The cost equation provides an objective estimate for projects with missing data, but can also be
used for projects that have a mix of target groups, as in region 5 in the above example, or for new
target groups in new regions (not illustrated in the above example). Without estimating the cost
equation, arbitrary choices would have to be made. For example:
However, the power of the cost equation lies not in correctly predicting the unit cost of each and
every project. In the above example, the estimated unit cost is too low for projects in region 1 and
too high for projects in region 2. The power of the cost equation lies in the ability to correctly
estimate the average unit cost over all projects, taking account of differences between regions,
target groups and activities, economies of scale and whatever else is relevant. In the above
example, the total cost exclusive region 3 are estimated at 66.7 million, which is 5% higher than the
actual total cost of 63.5 million; a difference that is generally considered acceptable.
What makes the above model easy to apply, is that a linear cost function is assumed. One
drawback of a linear cost function is that projects with very high unit costs (outliers) have a
disproportionate impact on the estimated cost of projects with no data (historical or future): they
elevate the average. To overcome the disproportionate effect of outliers, one option is to identify
them and remove them from the data, and another is to scale the unit cost, for example by taking
logarithms. However if the cost equation is specified in logarithms:
Log(unit cost) = a + b*x + N(0,2)
One needs to be aware that then the expected cost is not:
exp(a + b*x) but
exp(a + b*x + 2).
27
Non-linear functions for unit costs have in turn the drawback that estimates are very sensitive to the
exact function that is used, and the expected cost can be complex to calculate. For practical
reasons, the linear model, with possibly removing outliers from the data, and applying Ockhams
razor to keep the model transparent and accessible, should be considered first.
3.5
Ratio historical to
estimated unit cost
R1
7500
5500
1.36
R2
4000
5100
0.78
R3
4200
R4
8000
5800
1.38
R5
5000
5600
0.89
R6
3500
4500
0.78
R7
2000
2000
1.00
Both regions have in common that projects are relatively small: only 50 participants per project
compared to 100 and above in other regions. One might then consider procuring projects with 100
or more participants only. According to the cost equation, the unit cost should then be 500 euros
less, because the cost decreases by 10 euro for every extra participant from 50 to 100.
However, whether to actually require lower unit costs remains a normative choice, because lean
and mean projects are not necessarily the best; comparing historical and estimated unit costs is a
tool to identify potential cost inefficiencies, but not more than that.
3.6
28
Number of participants per Member State per intervention: entrants and average stock per year.
The unit cost can be calculated by dividing the expenditures by the number of participants. For the
number of participants, the larger value of the number of entrants and the average stock per year
can be used. Figure 3.3 illustrates the costs of training calculated with the above Eurostat
database: typically, the costs of training are in the 3,000 - 8,000 range.
Figure 3.3
The database can help to estimate unit costs in the absence of any data on ESF projects. However
if a Member States experiments with more effective and expensive interventions, or a lighter than
average intervention, one still needs judgement in the estimate of the cost for the new type of
intervention.
15
Ecorys and IZA (2012), Analysis of costs and benefits of active compared to passive measures, Final report, Rotterdam.
29
4.1
Introduction
Setting result targets is as easy or as difficult as applying success rates to the output targets. The
previous Chapter 3 discussed methods for setting the output targets, ideally integrated with setting
result targets.
The success rate can in principle be calculated as the percentage of participants leaving
successfully, in particular the participants leaving and finding employment. However, the success
rates depend on many factors which are likely to change in the future, including:
Because all of the factors which influence the success rates are likely to change in the future,
applying historical success rates to output targets does not necessarily result in achievable result
targets. A result equation relates the success rates to such underlying factors. Once these factors
are forecasted, the result equation can be used to predict future success rates, as discussed in
Section 4.2.
Another aspect is whether higher success rates would be achievable if best practices were
adopted in all projects, where the best practice should be defined as the practice that results in
the highest success rates, highest cost efficiency or ideally the highest overall effectiveness.
Benchmarking is the general method to identify best practices. However, a project aimed at a more
difficult target group may have a larger overall effect than a project aimed at persons who would
have found a job without intervention as well. Benchmarking is only useful after controlling for
differences in the composition of the target group and other relevant effects. Techniques for
controlling for these effects include, with references to empirical literature:
Randomized experiments16;
Matching17.
Randomized experiments have to be set up before the intervention, and consist of randomly
(blindly) assigning some persons to the one or the other intervention, or no intervention at all: the
random assignment ensures that effects of other factors average out, and the intervention with the
highest success rate can arguably be attributed to the intervention itself. Matching consists of
comparing the success rates of one group of participants in the one intervention with a similar
group in the other intervention (or with a similar group of non-participants). The project with the
highest success rate for that group, is arguably the best practice for that group.
16
17
Gautier, P., P. Muller, B. van der Klaauw, M. Rosholm and M. Svarer (2012), Estimating equilibrium effects of job search
assistance, working paper, http://www.oecd.org/els/emp/DanishExperiment.pdf.
Ecorys and IZA (2012), Analysis of costs and benefits of active compared to passive measures, Annex F provides
references to studies based on matching techniques in Austria, Germany, the Netherlands, Poland and the UK.
31
The first method requires large efforts in setting up interventions, whereas the second method
requires detailed data on individual participants. Still, both instruments are strongest in providing ex
post information on the net-effect of an intervention in the past.
Instead, Section 4.4 discusses the frontier analysis, which can be applied after controlling for other
effects, whether through the result equation, randomized experiments or matching. Section 4.5
concludes the chapter with discussing practical methods in the absence of data on historical
success rates for particular groups.
4.2
Result equation
The result equation relates observed results to relevant variables such as target groups, regions
and the phase of the business cycle (impact variables) and a variable that captures the effect of the
participation. In its most abstract form, the result equation reads:
The value a would be the average result if all other values b and c were zero, and in general is
needed to ensure the average result for average values of the other factors. The value b controls
for the impact of relevant factors such as the business cycle, composition of the target group
etcetera. The value of real interest in this equation is the third value c. If the value c is greater
than zero, than the programme is effective towards the result.
Advantages:
-
Method is direct;
Observations from all programmes are used to estimate likely results in specific situations;
Knowing what the result would have been in specific situations in the past enables setting
attainable result targets for such a situation in the future.
Disadvantages:
-
The target setter still needs to judge whether better results in the future compared to the
past are realistic.
Needed:
-
Second best: survey data for a sample of participants / persons in the target group and
aggregate statistics per intervention;
With regard to the disadvantage that the result equation does not take into account wider impacts
beyond the results, it should be noted that the estimates of wider impacts are not part of the target
setting procedures for the ESF 2014-2020 period. Macroeconomic models could in theory be used
to explore wider impacts, including:
Displacement effect (employers without participants lose jobs to those with participants).
32
Gauss and TSP cannot handle large datasets well, the Excel add-in only serves for basic regression analysis and SPSS
has no convenient programming options to use estimated values in further analysis.
In the micro-based approach, the result is typically an improved likelihood of employment. Two
standard forms of the result equation are:
The model of binary outcomes is relatively safe, because relatively few modelling assumptions are
required, compared to for example a duration model. In the standard model of Bartik,19 the
probability Pit to find a job is defined for every individual i in a given period t, and is related to
personal characteristics Xi, the business cycle Xt and a participation variable Iit.
Historical values of Pit are defined as 0 for all periods t in which the individual i did not find a job,
until the moment at which the individual did find a job in which case Pit is defined as 1. The values
a0, a1 and a2 are estimated by choosing the values that minimize the difference between the lefthand side and the right-hand side of the above equation.
For example, consider a minimalist dataset below with data for six participants (indicated with 1)
and six nonparticipants (indicated with 0, in column C in the print-out below). The impact variable is
the regional unemployment rate, which is 6%, 7% or 8% depending on the region (column B in the
print-out below). Finally, the first column indicates whether the person found employment (indicated
with 1) or not (value), in column A of the print-out below. In reality, other factors should be included
and a far larger number of observations is needed, time series effects may need to be taken
account of etcetera; the example only serves to illustrate the principle of estimating a regression
equation.
19
Bartik, T. J., R. W. Eberts and W.-J. Huang (2009), Methodology for adjusting GPRA workforce development program
performance targets for the effects of business cycles. Upjohn Institute Working Paper No. 09-154. Kalamazoo, MI: W.E.
Upjohn Institute for Employment Research.
33
In the above example, half of the participants find employment compared to two-third of the
nonparticipants. This might suggest that the intervention is counterproductive, however the
participants are in regions with a slightly higher unemployment rate (7.0% on average) compared to
non-participants (6.8% on average), and the question is whether the higher unemployment rates in
regions of participants can account for the negative effect of participation in the intervention.
After installing the Data Analysis add-in, in a similar way as the Solver add-in as discussed in
Annex 2, one can go to the tab Data and click on Data Analysis in the ribbon and choose the
option Regression
And select the Y range (the first column including the header), the X range (the second and third
column including the header), indicate the labels (the headers) are included and that the constant
(the value a) needs to be included (is not zero).
34
The result of the regression appears in a new sheet as shown in the print-out below.
The adjusted R square is a measure of the goodness of fit, and is an adjustment of the R square for
small numbers of observations as is the case in the above example. In reality relations are not so
strong and an (adjusted) R square of 15% can be considered as very good.
35
The intercept in the above example is 4.0. With an average unemployment rate of 6.9%, this
implies for non-participants an average result percentage of 4.0 0.49 x 6.9 = 61.9%, which is
somewhat less than the actual 66.7% (four non-participants find work and two do not in the
example). The P-values of 0.3% and 0.7% indicate that the value a and the regional
unemployment rate are different from 0 with high certainty (more than 99%).
However, the estimated effect of participation in the intervention is negligible (between -0.57 and
+0.40 with an average value of -0.09), after controlling for the unemployment rate. Hence the
intervention is not counterproductive as seemed at first sight, but does not improve the success rate
either.
Standard individual characteristics that are generally found to have an impact on the probability to
find a job are for example educational level and age, and standard time-variables are for example
the unemployment rate and the activation/benefit expenditure ratio.
The result equation estimated above is a minimalist model, which does not take account of time
series effects, which are discussed in the next Chapter 5. Also, the appropriate model to estimate
depends on the type of intervention, because different types of interventions influence the likelihood
of employment in different ways. Two examples are given below.
Modelling employment incentives
The intervention variable depends on the type of intervention. For employment incentives, this
could be a tax credit for job-finders. According to the econometric literature, the impact of financial
incentives on the probability to find a job is not always significant. 20 However, if gain in household
income is used instead of the wage, the impact of financial incentives is much more likely to be
significant, as is evident from the empirical literature. The example below illustrates the difference
between wages and changes in net household income:
Where indicates the difference between the new situation (participant finds a job) and the old
situation (participant does not have a job). For example if without the job the husband will have a
monthly unemployment benefit of 1,000 and the wife has a part-time job earning another 1,000
per month which is taxed at 20%, and the husband finds a job earning 2,000 per month causing
the wage of both husband and wife to be taxed at 30%, then the gain in net household income is
not 2,000 but only 300:
Wage
Tax of worker
Lost benefit
Extra tax of spouse
Net gain
+ 2000
600 -/1000 -/100 -/= 300
On an annual basis, this worker stands to gain 3,600 (12 times 300) and a one-time tax credit of
1,200 would increase this gain by one third, which is far more substantial than the gain of 5%
compared to the annual wage of 24,000.
20
36
This discussion is based on Ecorys (2004), Benefit systems and their interaction with active labour market policies,
ec.europa.eu/social/BlobServlet?docId=3195&langId=en. This report describes estimates of micro-models for employment
incentives for six EU15 countries.
Since the net gain is roughly 1/7th of the wage in the above example, the estimate of the value a3
should be roughly 7 times as large if net gain is used as the financial incentive variable instead of
the wage, and the estimate of the impact of the tax credit should also be roughly 7 times as large.
To model employment incentives, it is important to compare future alternatives instead of benefits
and wages from the past. Future benefits can be calculated given the national tax benefit system.
For those unemployed that find a job, the wage of the future job is also known. However, for those
that do not find a job, the wage of potential future jobs is unobserved and needs to be estimated.
These unobserved wages are estimated through a wage equation, relating observed wages to
relevant characteristics, in a similar way as the cost equation described in the previous chapter.
Modelling training
For modelling training, the intervention variable has value 1 for participants and value 0 for nonparticipants. While modelling training, the one complication is to take account of the so-called lockin effect. This lock-in effect implies that during the training, participants have a lower probability of
finding employment. It is only after the training that the chance of finding employment should be
improved. Table 4.1 illustrates that if training lasts six months (including waiting time till the training
starts), a comparison of the results of participants and non-participants would be extremely
negative for the intervention, while positive result of the training could be evident in the longer run.
Table 4.1
Region
Employed on 1 July
Employed on 31
December
Non-participants
0%
50%
60%
Participants
0%
10%
65%
If one estimates the impact of training with a binary probability model (found employment, yes or
no), it suffices to compare the employment of participants and non-participants in the medium term.
If a duration model is estimated, it is important to include both an intervention variable for the
chance of employment during the training, and an intervention variable for the chance of
employment after the training. If training has various durations, even assuming the average
duration for all participants improves the quality of the estimates.
If only one training variable instead of both is used, for example has attended training, the effect of
training tends to be overestimated. It is not intuitive to see why, but the following factors play a role:
The long period after training contributes more to the fit of the model than the short period of
training;
After the period of training, the probability of participants just starting to look for a job is
compared with the already diminished probability of non-participants who started looking for job
earlier. Apparently, a high value of one single training variable to predict employment correctly
after training, contributes more to the overall fit than a low value to predict employment correctly
during training. The solution is, to use two training variables, for during and after training.
37
4.3
20%
40%
2015
2016
2017
2018
2019
Sum
20%
16%
14%
12%
10%
8%
20,000
20,000
16,000
14,000
12,000
10,000
8,000
100,000
2,000
2,000
1,600
1,400
1,200
1,000
800
10,000
The answer is to first add 2% to the success rate for each percentage lower youth unemployment
rate, to predict the success rates for 2015-2020. Secondly, these success rates are multiplied with
the numbers of participants in the respective years to calculate the number in employment (6
months after leaving) in each year. The sum of the participants in employment 6 months after
leaving over all years gives the final result target.
First step: predicting future success rates
The formula to use in this example is:
Success%[t] = success%[2014] 2*(youth unemployment rate[t] youth unemployment rate[2014])
Success%[t] = 40% - 2*(youth unemployment rate[t] 20%).
Table 4.3 shows the results. As before, the shaded cells indicate values that already are provided
or have been calculated. For example, the success rate of 2020 is
64% = 40% - 2*(8% - 20%) = 40% + 2 * 12%.
38
2020
Table 4.3
2015
2016
2017
2018
2019
2020
Sum
20%
20%
16%
14%
12%
10%
8%
40%
40%
48%
52%
56%
60%
64%
20,000
20,000
16,000
14,000
12,000
10,000
8,000
100,000
2,000
2,000
1,600
1,400
1,200
1,000
800
10,000
2015
2016
2017
2018
2019
2020
Sum
20%
20%
16%
14%
12%
10%
8%
40%
40%
48%
52%
56%
60%
64%
20,000
20,000
16,000
14,000
12,000
10,000
8,000
100,000
2,000
2,000
1,600
1,400
1,200
1,000
800
10,000
800
800
768
728
672
600
512
Lastly, the result target for 2014-2020 is the sum of the results of each years, which add up to 4,880
as indicated in Table 4.5: 4,880 = 800 + 800 + + 512.
39
Table 4.5
2015
2016
2017
2018
2019
Sum
20%
20%
16%
14%
12%
10%
8%
40%
40%
48%
52%
56%
60%
64%
20,000
20,000
16,000
14,000
12,000
10,000
8,000
100,000
2,000
2,000
1,600
1,400
1,200
1,000
800
10,000
800
800
768
728
672
600
512
4,880
4.4
40
2020
Stochastic frontier analysis is a method that identifies such outliers and leaves them out of the calculations, but this is
beyond this background paper.
Whether the dashed or the solid line should be used, is a normative issue, based on judgement.
However, the dashed line has the advantage of being based on evidential information and in that
sense helps in setting higher targets.
Figure 4.1
Use of frontier analysis as an aide in the decision whether to set higher targets
In the above example it was assumed that the situation was comparable for all projects. In reality
impact variables such target groups, regions or the phase of the business cycle will have been
different between projects. The result equation discussed in the previous section helps to control for
differences in impact variables. Figure 4.2 illustrates how the result equation can be used to set
targets for new groups (actually, a group with very few observations).
In Figure 4.2, the lower red dashed line connects the three observed results for a new group y.
The results are poorer than for the old group x of Figure 4.1, but the reason might be that the new
group has a lower likelihood of employment with or without the programme. The result equation can
be used to predict the results for the new group y, by replacing the relevant impact variables with
the characteristics of group y. In the example of Figure 4.2 the predicted result levels for group y
are below the observed levels for group x but above the observed level for group y.
Again, it is a normative issue based on judgement whether to set higher result targets for the new
group based on higher predicted values than the observed results and again the frontier analysis
has the advantage of being based on evidential information (in combination with the result
equation).
41
Figure 4.2
4.5
One solution assumes that historical results are available for the relevant target group in the
situation without the programme (the baseline) although not for the situation with the
programme;
The other solution assumes that no historical results are available for the relevant target group
at all, whether with or without the programme.
Limited availability: historical results for baseline and unit costs but not for the programme
In the first case the result for the target group is known for the situation without intervention, for
example 25% of the older unemployed find employment within one year of becoming unemployed.
Then this no policy result can serve as a starting point (first step) for target setting.
In a second step, one estimates what minimum result is needed to justify the cost of the programme
and this would be the target level. The cost of the programme would be justified if the unit gain is
greater than or at least equal to the unit cost. The unit cost is assumed to be known (see Section
3.6 for the case no data on unit costs are available), for example 5,000 per participant. To
calculate the minimum result needed to justify the cost, one also needs to assign a value to a unit
result, for example 50,000 per job-finder.
The requirement that the unit gain is at least the unit cost, can be expressed in a formula:
[Percentage extra job-finders] * Unit value Unit cost
Or equivalently:
Percentage extra job-finders [Unit cost] / [Unit value].
In the above example, the minimum percentage of extra job-finders to justify the cost of the
programme is therefore 5,000 / 50,000 = 10%.
42
Lastly, the minimum percentage of extra job-finders is added to the baseline result to set a
minimum target percentage, in the above example 25% + 10% = 35%. If the output target would be
10,000 older unemployed participants, then the result target would be 3,500 participants in
employment upon leaving in this example
25% of the long-term unemployed ( 1 year) find a job after their first year.
The above rule of the thumb is based on long experience with Dutch unemployment data and
international meta-studies. However the author has no such rule of the thumb for other target
groups, except for his home country. This is because a reasonable target percentage for other
groups depends on national institutional settings including for example:
As an alternative for other target groups besides the long-term unemployed, the unemployment
distribution of the target group can be combined with the above rule of the thumb percentage for
short-term and long-term unemployed. For example if 20% of the older unemployed is short-term
unemployed and 80% is long-term unemployed, then:
The 75% of the target percentage for short-term unemployed is weighted by 20%.
The 25% of the target percentage for long-term unemployed is weighted by 80%.
As a result, the target percentage for the whole group of unemployed older workers is in this
example: target% = 20% * 75% + 80% * 25% = 35%. As in the previous example, if the output
target is 10,000 older unemployed participants, then the result target would be 3,500 participants in
employment upon leaving.
Table 4.6
Long-term
Total
Distribution
20%
80%
100%
Result %
75%
25%
35%
43
5.1
Introduction
This chapter briefly discusses methodologies for adjusting output targets and result targets. When
outputs are adjusted by reallocating a given year budget between different target groups, one can
simply adjust the output targets to the most recent developments, in a similar way as the trend
extrapolation discussed in Section 3.2. Only when budgets are reallocated between years
according to the situation of the labour market, a methodology needs to be developed in advance.
For adjusting result targets, the micro-model of Bartik described in Section 4.2 can be used again,
but then focusing on the variables capturing the impact of external factors on the results.
5.2
Advantages:
-
The required data are usually available, for example from national accounts and PES data;
Disadvantages:
-
The limited number of observations allows inclusion of only a limited number of explanatory
variables (relevant factors);
Time series estimations come with a number of pitfalls (which are discussed below);
Basic software such as the Excel add-in is not sufficient to estimate these models.
Needed:
-
Econometric expertise;
A set of equations for the size of target group (say, the unemployed in group i) in month m can
be formulated as follows:
[
[ ]
[
[ ]
]
]
[
]
[ ]
[ ]
(
)
(
)
Here, the symbol represents a change compared to the previous period, the values a0, a1, a2, b0
and b1 are coefficients to be estimated by running a regression, and N(0,2) indicates Normally
distributed white noise with variance 2.
44
For the number of unemployed, monthly data is typically available. For GDP, typically quarterly data
is available. To predict the development of a target group population, the above set of equations
needs to be estimated, and the development of GDP needs to be predicted.
The second equation of the above set is a time series equation, because the development of total
unemployment is partly related to the development of total unemployment in the previous month,
which in econometrics is called a lagged variable.
The main pitfalls of time series equations are in general:
The unit root problem: the coefficient of the lagged variable is roughly 1;
Autocorrelation: the prediction error of one month is correlated with the prediction error of the
previous month;
End-point bias.
45
Multi-collinearity can be identified through the correlation matrix of the estimates of a1, a2, , but
practically speaking, one can already conclude multi-collinearity if the estimates are extremely
large.
Endpoint bias
The endpoint bias refers to the fact that in a single time series, the first and the last values have a
disproportionate weight in the estimated of time series coefficients. To illustrate the endpoint bias,
Figure 5.1 shows a series with two extreme values, at the 25th in the middle of the time series and
the 50th data point at the end of the time series. If a linear line is estimated based on all 50 data
points, then the estimated 51st data point would be 7.6 instead of 8.1 if both extreme values were
ignored, an error of 6%. If the 50th data point would be ignored but the extreme value in the middle
of the time series is included in the estimation, then the estimated 51st data point would be 8.0, an
error of only 1% compared to 8.1.
To correct for the endpoint bias, one judges by sight and leaves out the first or the last observation
if necessary. In statistical software, the so-called Hodrick-Prescott filter automatically corrects for an
endpoint bias. Correcting for the endpoint bias can be important in short time series. However if the
time series is jointly estimated for a large number of regions, it is unlikely that the endpoint bias is
the same in all regions, and the issue of the endpoint bias can be ignored.
Figure 5.1
46
5.3
[ ]
[ ]
]
[ ]
]
[
[ ]
Unemployment tends to decrease if the economy (GDP) grows and its estimated coefficient of -0.5
reflects this. But unemployment also has its own dynamics, perhaps because it takes time to find a
new job. The estimate of 0.3 for the effect of a change in the number of unemployed in the previous
month reflects this. The unemployment rate tends to decrease with educational level, but also tends
to be more sensitive to the business cycle as reflected by the coefficient of +0.5 in the second
equation. Among the low educated unemployed a relatively large core may exist that remains
unemployed throughout the business cycle, stabilizing the unemployment rate among this group
at a relatively high minimum level. Thus if the share of high educated unemployed increases, the
share of low educated unemployed decreases. The coefficient for medium educated unemployed
may be insignificant and is set at 0 in this example, implying that the share of medium educated
unemployed is constant.
Table 5.1 provides the base data for setting output targets using the above set of equations. The
four quarterly GDP growth rates for 2014 add up to 1.15%, a realistic GDP growth for 2014 as a
whole. The initial unemployment forecast for January 2014 is also provided.
Table 5.1
Base data for setting output targets with macroeconomic time series data
2014
gdp
u
M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
0.25%
0.25%
0.25%
0.50%
0.50%
0.50%
0.30%
0.30%
0.30%
0.10%
0.10%
0.10%
-1.00%
%[high]
10%
%[medium]
50%
%[low]
Unemployed
40%
100,000
High
10,000
Medium
50,000
Low
40,000
To apply the above model, first the change in the number of unemployed needs to be forecasted,
based on the change in the preceding month and the quarterly GDP growth. For example for
February 2014 this gives:
u[February 2014]
u[March 2014]
47
The number of unemployed in February 2014 is therefore the 100,000 of January minus the
predicted 0.43% decline or equivalently 99,575 persons. The number of unemployed in March 2014
is the 99,575 of February minus the predicted 0.25% decline or equivalently 99,324 persons.
Continuing in this way for the other months of 2014 gives the result in Table 5.2 below.
Table 5.2
Base data for setting output targets with predicted changes in unemployment rates
2014
gdp
u
M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
0.25%
0.25%
0.25%
0.50%
0.50%
0.50%
0.30%
0.30%
0.30%
0.10%
0.10%
0.10%
-1.00%
-0.43%
-0.25%
-0.33%
-0.35%
-0.35%
-0.26%
-0.23%
-0.22%
-0.12%
-0.08%
-0.08%
99,575
99,324
99,000
98,656
98,306
98,054
97,832
97,618
97,506
97,423
97,350
M11
M12
%[high]
10%
%[medium]
50%
%[low]
40%
Unemployed
100,000
High
10,000
Medium
50,000
Low
40,000
The forecasts of u in Table 5.2 are in turn used to forecast the shares of high, medium and low
educated unemployed. For February 2014, the predicted change in the share of high educated
unemployed is:
Dshare[high] = +0.5* -0.43% = -0.21%
This results in a share of high educated unemployed of 10% - 0.21% = 9.79% (in Table 5.3 the
rounded figure 9.8% is shown), and applying this percentage to the total number of unemployed in
February 2014 (which is 99,575) gives 9,746 high educated unemployed. Continuing in this way for
the other educational levels and the other months of 2014 gives the table below.
Table 5.3
gdp
M2
M3
M4
M5
M6
M7
M8
M9
M10
0.25%
0.25%
0.25%
0.50%
0.50%
0.50%
0.30%
0.30%
0.30%
0.10%
0.10%
0.10%
-1.00%
-0.43%
-0.25%
-0.33%
-0.35%
-0.35%
-0.26%
-0.23%
-0.22%
-0.12%
-0.08%
-0.08%
%[high]
10%
9.8%
9.7%
9.5%
9.3%
9.1%
9.0%
8.9%
8.8%
8.7%
8.7%
8.7%
%[medium]
50%
50%
50%
50%
50%
50%
50%
50%
50%
50%
50%
50%
%[low]
40%
40.2%
40.3%
40.5%
40.7%
40.9%
41.0%
41.1%
41.2%
41.3%
41.3%
41.3%
100,000
99,575
99,324
99,000
98,656
98,306
98,054
97,832
97,618
97,506
97,423
97,350
Unemployed
High
10,000
9,746
9,596
9,403
9,199
8,992
8,844
8,713
8,587
8,521
8,473
8,430
Medium
50,000
49,788
49,662
49,500
49,328
49,153
49,027
48,916
48,809
48,753
48,712
48,675
Low
40,000
40,042
40,066
40,097
40,129
40,161
40,183
40,203
40,222
40,232
40,239
40,245
The average numbers of unemployed in 2014 can then be used to set output targets for the
subgroups of unemployed, for example assume:
Table 5.4 gives the results. In the second column, the average numbers of unemployed are
calculated by averaging the numbers of January up to December. In the third column, the shares of
high, medium and low unemployed are calculated from the second column. In the fourth and last
48
column, the number of participants is targeted by applying the shares to the total number of
participants for which budget is available.
Table 5.4
Educational level
Average number
Share
Participants
9,042
9.2%
919
Medium
49,194
50.0%
5,000
Low
40,152
40.8%
4,081
Total
98,387
100%
10,000
(=output target)
High
Now suppose that in June, the forecasts for the second half have been adjusted significantly
downward. The new situation is then given as in Table 5.5.
Table 5.5
Base data for adjusting output targets with macroeconomic time series data
2014
M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
0.25%
0.25%
0.25%
0.50%
0.50%
0.50%
-0.70%
-0.70%
-0.70%
-0.90%
-0.90%
-0.90%
-1.00%
-0.43%
-0.25%
-0.33%
-0.35%
-0.35%
%[high]
10%
9.8%
9.7%
9.5%
9.3%
9.1%
%[medium]
50%
50%
50%
50%
50%
50%
%[low]
40%
40.2%
40.3%
40.5%
40.7%
40.9%
gdp
u
Unemployed
100,000
99,575
99,324
99,000
98,656
98,306
High
10,000
9,746
9,596
9,403
9,199
8,992
Medium
50,000
49,788
49,662
49,500
49,328
49,153
Low
40,000
40,042
40,066
40,097
40,129
40,161
Repeating the same calculations as for Table 5.2 and Table 5.3 results in Table 5.6 where due to
the downward adjusted GDP forecasts, the unemployment level and the share of high educated
unemployed people increase.
Table 5.6
Data for adjusting targets with unemployment forecasts and educational breakdown
2014
M1
gdp
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
0.25%
0.25%
0.25%
0.50%
0.50%
0.50%
-0.70%
-0.70%
-0.70%
-0.90%
-0.90%
-0.90%
-1.00%
-0.43%
-0.25%
-0.33%
-0.35%
-0.35%
0.25%
0.42%
0.48%
0.59%
0.63%
0.64%
%[high]
10%
9.8%
9.7%
9.5%
9.3%
9.1%
9.3%
9.5%
9.7%
10.0%
10.3%
10.6%
%[medium]
50%
50%
50%
50%
50%
50%
50.0%
50.0%
50.0%
50.0%
50.0%
50.0%
%[low]
40%
40.2%
40.3%
40.5%
40.7%
40.9%
40.7%
40.5%
40.3%
40.0%
39.7%
39.4%
Unemployed
100,000
99,575
99,324
99,000
98,656
98,306
98,547
98,964
99,436
100,026
100,654
101,297
High
10,000
9,746
9,596
9,403
9,199
8,992
9,133
9,381
9,663
10,017
10,396
10,786
Medium
50,000
49,788
49,662
49,500
49,328
49,153
49,273
49,482
49,718
50,013
50,327
50,648
Low
40,000
40,042
40,066
40,097
40,129
40,161
40,141
40,101
40,055
39,996
39,931
39,863
Suppose that in the first 6 months 5,000 unemployed have participated in the intervention, of which
460 were high educated; 2,500 were medium educated and 2,040 were low educated. Then the
49
analysis for M7-M12 in the new situation can be repeated as was done above for M1-M12 in the old
situation. This gives the result of Table 5.7.
In June 2014, the participants of the first six months of 2014 are given. The remaining 5,000
participants for which budget is left for the second half of 2014, are allocated proportionally to the
new forecasts of the unemployed in the second half of 2014. Lastly, the adjusted target for the
whole of 2014 is the sum of the participants who already have entered the programme in the first
half of 2014, plus the adjusted target for the remainder of 2014.
Table 5.7
M7-M12
M1-M12
Participants
Unemployed
Shares
Participants
Adjusted target
460
9,896
9.9%
496
956
Medium
2,500
49,910
50.0%
2,500
5,000
Low
2,040
40,015
40.1%
2,004
4,044
Total
5,000
99,821
100%
5,000
10,000
High
Comparing the targets of Table 5.4 and Table 5.7 shows that in this case, the adjustment of output
targets would be small. In the course of years instead of months, adjustments may be larger, but
the underlying calculations are the same as illustrated above.
5.4
2015
2016
2017
2018
2019
2020
Unemployment rate
8%
10%
9%
8%
6%
5%
4%
76.7%
74.5%
75.6%
76.7%
78.9%
80.0%
81.1%
Number of unemployed
160,000
200,000
180,000
160,000
120,000
100,000
80,000
1,000,000
Number of participants
32,000
40,000
36,000
32,000
24,000
20,000
16,000
200,000
Targeted job-finders
24,544
29,800
27,216
24,544
18,936
16,000
12,976
154,016
The question is, how to adjust the result targets if the chance to find employment decreases by 1.1
percentage point for every per cent point of unemployment rate? One first needs to set a target
norm, for example at a 5% unemployment rate, 80% of the unemployed participants should find a
22
50
This example is slightly modified from Ecorys (2011), Performance targets for ESF Operational Programmes.
http://www.coprbm.eu/?q=node/494
Sum
job within a six months after leaving an ESF-supported project. If that is the norm, then the adjusted
share of job-finders for higher and lower unemployment rates can be calculated as follows:
[ ]
For example for 2020 the predicted unemployment rate is 4% and the above formula gives a share
of job-finders of 81.1% and in 2018 the predicted unemployment rate is 6% and the above formula
gives a share of job-finders of 78.9%.
Once the share of job-finders is calculated, the result target is easy to calculate:
In 2019, this formula gives that 80% of the 20,000 participants should find a job, or equivalently
16,000 persons. Repeating this for each year and taking the sum over the period 2014-2020 finally
gives the result target of 154,016 participants who should find a job, which is naturally rounded to
154,000. Of course, since the ESF programming period ends in 2020 and the result target is that
the unemployed find a job within a year, the result target actually applies for the period 2014-2021,
since the last unemployed of 2020 that find a job, will likely find that job in 2021.
51
In this annex a list of common indicators is presented. The list of common indicators can be found
in the annex to the draft ESF regulation.23 Further definitions can be found in the 2013 draft ESF
monitoring and evaluation guidance.24
(1) Common output indicators on participants
Participants refer to persons benefiting directly from an ESF investment and who can be identified
and asked for their characteristics, and for whom specific expenditure is earmarked. Other
beneficiaries should not be counted as participants.
long-term unemployed
inactive
below 25 years
above 54 years
disabled
other disadvantaged
The total number of participants is calculated automatically on the basis of the output indicators.
These data on participants entering an ESF supported operation are to be provided in the annual
implementation reports. All data are to be broken down by gender.
(2) Common output indicators for entities
organisations
These data are to be provided in the annual implementation reports. All data are to be broken down
by gender.
23
24
http://ec.europa.eu/regional_policy/sources/docoffic/official/regulation/pdf/2014/proposals/regulation/esf/esf_proposal_en.pdf
European Commission, Programming Period 2014-2020, Monitoring and Evaluation of European Cohesion Policy,
European Social Fund, Draft Guidance document, 9 January 2013,
http://ec.europa.eu/social/main.jsp?catId=701&langId=en
53
These data are to be provided in the annual implementation reports. They are to be collected based
on a representative sample of participants within each priority axis. Internal validity of the sample
should be ensured in such a way that the data can be generalised at the level of priority axis. All
data are to be broken down by gender.
54
In Microsoft Excel, a standard built-in add-in can be of great help in optimizing the result targets
subject to economic (and perhaps political) constraints. This annex illustrates how to install this
add-in in Excel 2010 and how to use it for the first exercise. This add-in is called Solver and solves
linear programming problems. This is very useful for maximizing result targets under budget,
capacity and minimum investment constraints, because budgeting is essentially a linear
programming problem.
The Excel add-in can be found behind Options under the tab File, as shown in the screen-print
below.
In the Excel Options screen, go to the option Add-Ins. Another screen will then pop up as
illustrated below. In this popup-screen, choose Solver Add-in and then click on Go at the
bottom, and then OK.
55
Finally, under the tab Data the Solver add-in is now available.
56
To use the Solver add-in, the unit costs, maximum budget and other constraints need to be
specified, as well as the contribution of each alternative to the target. For Exercise 1, the screen
print below shows how this information can be prepared. Besides the policy alternatives
(counselling, mobility scheme, re-training in this example), the contribution of the no-policy
alternative to the result needs to be specified (50% of the non-participants are in employment in this
example). The cells for the numbers of participants are left blank for the moment, these will be filled
by the Solver add-in in such a way that the result (persons in employment) is maximized.
The budget constraint states that the sum of participants weighted with their unit costs must be less
than 10m euro in the example of exercise 1:
1,000 * (counselling) + 4,000 * (mobility scheme) + 16,000 * (re-training) 10,000,000.
As well as the unit costs for the alternative, the zero unit costs for nonparticipants need to be filled
in.
The capacity constraint for counselling is that one times the number of participants in the
counselling alternative must be less than 5,000, and so on for the other alternatives. Finally, it
needs to be specified that the numbers of participants and non-participants add up to the total,
10,000 in the example of exercise 1:
1*(nonparticipants) + 1*(counselling) + 1*(mobility scheme) + 1*(re-training) = 10,000.
57
After the ground work is done, the result needs to be specified. The number of persons in
employment is:
50%*(nonparticipants) + 55%*(counselling) + 60%*(mobility scheme) + 90%*(re-training).
This formula needs to be filled in in cell B6 in the sheet of the screen print below, where the cells
B3:E3 refer to the expected result per participant, and the cells B4:E4 to the numbers of
participants that will be solved by the add-in.
The formula for the budget constraint needs to be filled in in cell B16 in the sheet of the screen print
below. Here, the cells B9:E9 refer to the unit costs and the cells B4:E4 to the numbers of
participants.
58
The same is done for the capacity restraint of counselling as in the screen print below. Here, cell
B10:E10 refer to the specification that the capacity limit applies to the numbers of participants in
counselling (value 1) and not to the other alternatives (value 0).
59
After filling the similar formulas for the capacity constraints of the mobility scheme and re-training in
cells B18 and B19, the formula for the count restriction is filled in in cell B20 in the screen-print
below.
In cells C16:C20 the right hand side values of cells G9:G13 are copied, but this is purely cosmetic.
After all the formulas have been specified as above, the Solver add-in can be applied to find the
numbers of participants that maximize the result of persons in employment. For this, one needs to
go to the tab Data and then click on the Solver add in at the top right side of the screen, as
illustrated in the screen below.
60
In the pop-up screen of the Solver add-in, the user can specify which cells contain the required
information:
Set objective: cell B6 for the formulas for the result target (persons in employment);
Indicate that the result target needs to be maximized (choose option Max);
61
To add a constraint, one needs to open another popup screen by clicking on the Add button. The
budget and capacity constraints are all subject to maximum limits and can be grouped together as
illustrated in the pop-up below. Reference needs to be made to the cells with the formulas
(B16:B20), not the cells with the numbers that are used in the formulas (B9:E12).
The count restriction is an equality and this constraint needs to be added as well.
62
After specifying the constraints, the Add Constraint window can be exited by clicking on Cancel.
63
Lastly, a Solving Method needs to be specified; in this case the Simplex Linear Programming
method is most appropriate, because the number of persons in employment is a linear function of
the numbers of participants.
Clicking on the Solve button makes the Solver add-in calculate the numbers of participants
(=output target) and the numbers of persons in employment (= result target).
After clicking on Solve, a message should pop up to indicate that a solution is found which
satisfies all the constraints and one can exit this message by clicking on OK.
64
According to the linear programming solver, it is optimal to target 5,000 participants in counselling
(exhausting the capacity constraint for counselling) and 313 participants in re-training. Together
with the 50% of nonparticipants in training who are in employment, this results in 5,375 persons in
employment.
65
The solver identifies the optimal allocation of participants, but still one final check is needed: is
benefit per additional job-finder sufficient to justify the cost per additional participant? Without any
intervention, 50% of all 10,000 persons threatened with job loss would have found a job in this
example, resulting in 5,000. Hence the additional number of job-finders attributable to the mix of
alternatives in this example is (5,375 5,000) = 375 persons.
The cost per additional job-finders is therefore the budget of 10m euro divided by 375, which is
26,667. In this exercise, each additional job-finder is valued at 50,000, which justifies the cost of
26,667.
Finally, the output target is calculated as the sum of participants over all alternatives, namely:
The result target is calculated as the sum of participants weighted with the result% per participant:
66
Watermanweg 44
3067 GG Rotterdam
The Netherlands
W www.ecorys.nl
BELGIUM BULGARIA HUNGARY INDIA THE NETHERLANDS POLAND RUSSIAN FEDERATION SOUTH AFRICA SPAIN TURKEY UNITED KINGDOM