DATA, EVIDENCE AND POLICY

An introduction
Lecture 1
Week 1

1
This Lecture

 Will give you an overview of the module: objectives, organisation and assessment

 Shows how economists use data to build evidence and evaluate public policies

Key Concepts
 Cross-section  Observational data  Extrapolation
 Time series  Sampling error  Internal and external
 Longitudinal  Sample bias validity
 Panel  Correlation  Forecasting
 Cohorts  Causation
 Experimental data  Spurious correlation
 Natural experiments

2
Section 1

DEP: an overview
WHAT WHY AND HOW

3
DATA, EVIDENCE and POLICY – not your usual module

 DEP is different
 Applied and hands-on: start with a question, learn concepts and
methods you need to answer it
 Believes in LEARNING BY DOING
 Helps you build transferable skills and reflect on your strengths

4
WHY?

 Objectives

 To introduce you to the sources and content of economic data and evidence and develop
your appreciation of what methods might be appropriately applied to the analysis of such
data
 To introduce you to the skills required for collection, description and interpretation of
economic data and evidence
 To develop your proficiency in computing techniques appropriate for the collection,
management, description and visualisation of economic data
 To provide hands-on experience, using real-world data, and apply economic reasoning to
policy issues
 To help you develop skills that are transferable to other modules and to the workplace

5
HOW?

 Lectures and independent study Active learning

 Lectures and workshops: join in person and interact
I hear and I forget
 Engage with the material, receive and provide support I see and I remember
I do and I understand

 Projects Learning by doing

 3 Excel projects and 2 Policy briefs
 Group work

 Build transferable skills and know your Strengths Reflective learning

 Group work
 York Strength programme

6
WHAT?

 Lectures – Active Learning

 Introduction
 Topic 1 - Health and Social Policy
 Topic 2 - Inequality and Social mobility
 Topic 3 – Financial stability
 Topic 4 – GDP and Wellbeing

 Applications and small group workshops – Learning By Doing

 Excel Project 1 - Measuring climate change
 Excel Project 2 – Measuring the effect of a sugar tax
 Policy brief 1 – Inequality and social mobility
 Excel Project 3 – Banking systems and the 2008 crisis
 Policy brief 2 – GDP and wellbeing

 Embedding careers in the curriculum – Reflective Learning

 York Strength programme + workshop

7
ASSESSMENT

Essay/Coursework: Reflective piece (10%)

• Individual essay (max 1,500 words) which takes the form of a critical reflection on group work
• You will be assessed against a marking grid, shared in advance: reflection on contribution to group work, before
and after the York strength programme, transferable skills
• You will have the opportunity to take part in the York Strengths programme and attend a development
workshop
• Due at the end of the semester. Not re-assessable

Individual project (90%)

• The project will provide you with the opportunity to use real-world data to investigate a policy issue.
• You will be guided in your analysis with specific tasks to complete and questions to answer. The research question
and key supporting evidence (data, papers, policy briefs) will be shared with you
• You will be assessed on: data skills developed during the module (using Excel): data handling, statistical analysis
and data visualisation; ability to discuss the quality of the data and evidence, critically appraise statistical
methods and discuss the policy implications of their results
• Due during the common assessment period. Re-assessment: revise and resubmit

8
Section 2

DATA
DESIGN, SOURCE AND QUALITY

9
Why do economists need data?

To discriminate among alternative explanations

Is this fact an anomaly?
What are the mechanisms?
Collect data. Use every source of information available to you.
Cross section. Looking at different individuals. Following people over time.
Use multiple sources of data, all of it non experimental.
Use economic theory to help organise your thinking James Heckman – living god of Econometrics
University of Chicago, Nobel prize (2000)

Theories can be useful but we need to know what are the facts
We have to measure these things to make proper comparisons
With better data you can have a more informed discussion
Collect data in a systematic manner
Use taxation data

Thomas Piketty – French superstar

PSE and LSE, Capital in the Twenty-First Century (2013)

10
Dataset Design

 Cross-section (or cross-sectional data): type of data collected by observing many subjects
(individuals, firms, products, countries…) at one time, or period of time.

 Time series: a series of data point indexed in time order. One subject (temperature, financial
index, GDP…) is observed at different and usually regular (annually, quarterly, daily) moments
of time

 Longitudinal data: multidimensional data involving measurements over time

 Repeated cross-sections: cross-sectional studies are repeated in the same population but
not necessarily with the same individuals over different time periods
 Panel data: the same subjects are followed overtime (individuals, firms, products,
countries…)
 Cohort data: a group of people who share a defining characteristics (born or
graduated the same year…) are followed over time at regular intervals

11
Source of data (1/3)

Experimental data: data obtained by running a controlled experiment.

Subjects in the experiment are randomly assigned to a treatment group or a control group.
Traditionally used in medicine (clinical trials)

 RAND study (1974)  estimate the price elasticity of demand for medical care
 Enrolled 5,000 individuals followed for 3 to 5 years
 Individuals were randomly assigned to different insurance schemes

 Esther Duflo: run randomized controlled trials (RCTs) in developing countries

(immunization, protection against malaria, income transfers)

 Experimental economics: small scale experiments, usually in labs, used to test the
validity of economic assumptions and theories (rationality, risk aversion, individual
behaviour, market response etc)

12
Source of data (2/3)

Natural experiments (or quasi experiments): a situation where the economic environment
exogenously changes, perhaps inadvertently, due to a policy or institutional change

Data collected is similar to experimental data with usually a control and a treatment groups. The
assignment however is not under the control of the researcher and not always fully random.

 Vietnam War Draft Lottery (1969) was used to estimate the impact of being a veteran
on lifetime earnings (Angrist, 1990)
 Medicaid lottery in Oregon (2008) was used to estimate the impact of public insurance
on medical consumption and health (Finkelstein et al, 2012)
 Targets on waiting times (2001) targets and sanctions were implemented in NHS
England but not in Scotland (Propper et al, 2008)
 The 2014 tax on sugar-sweetened beverages (Project 2): it was first implemented in
Berkeley on specific products only (Silver et al, 2017)

13
Source of data (2/3)

Observational data (or nonexperimental data): the variable of interest or independent variable, is
not under the control of the researcher. Individuals are observed and certain outcomes are
measured but there is no attempt made to affect their circumstances or outcomes

Surveys: data collected from a sample of respondents

 Cross-section: The Health and Lifestyle Survey 1 (HALS1)
Large representative survey of adults in GB (1984-1985): 9,000 individuals aged 18 and over
 Panel: The Health and Lifestyle Survey 2 (HALS2)
Same population than HALS but 7 years later…of original 9,000 individuals: 9% died, 15% could not be traced, only 59%
were resurveyed (sample attrition)
 Cohort: National child and development study (NCDS)
Cohort of 17,000 individuals born in the week of March the 3 rd 1958. Followed from birth to age 46

Institutional data: GDP, inflation, financial indices…

Administrative data: mortality data, insurance claims, taxation data…
Data mining: “raw data” from transactions, social media… (Big Data)

14
Sampling

Process used in statistical analysis in which a predetermined number of observations are taken
from a larger population.
 Random sample: each member of a population is equally likely to be selected
 Non-random sample: by design, self-selection, attrition

Sample error: difference between a sample statistic used to estimate a population

parameter and the actual but unknown value of the parameter.
 Can occur even when the sample is taken randomly, especially for small sample

Sample bias: a sample is collected in such a way that some members of the intended population are
less likely to be included than others
 Occurs when the sample is not taken randomly

15
Assessing the quality of the data

Always question the quality of data

 How to define the variable of interest: interpretation?
 How to measure it: accuracy? comparability?
 How to collect it: experiment? observational study?
 Is your sample representative of your population of interest: sample error? sample bias?

The Guardian, 31.10.18

16
Application: Measuring climate change

 Interpretation?
 Accuracy?
 Comparability?
 Source?
 Sampling issues?

17
Section 3

EVIDENCE
CORRELATION V CAUSATION, PREDICTIONS AND
FORECASTING

18
What do Economists mean when they talk about “evidence”

Primarily a quantitative approach, based on economic models and large samples of data

Applications:
 Understanding how one variable affects another
 How does education affect health?
 Measuring trends and changes: evolution over time, pre/post analysis after a change in the
economic environment
 How inequality changed in the UK from 1960s to today?
 Evaluating policies: measuring the efficiency and fairness outcomes of a change in policy
 Are sugar taxes effective to prevent obesity?
 Testing theoretical predictions: put theories to the test of real-world data
 Testing the efficiency of financial markets
 Predicting changes
 What will be the impact of an increase in interest rates on inflation?

19
What are we looking for?

A correlation: statistical measure that indicates the extent to which two or more variables fluctuate
together (covariance, correlation coefficient)

Positive Zero Negative

A causal effect: one variable (the independent variable) causes an other variable (the dependent
variable) to change
 We need to estimate causal effects to test hypotheses, make predictions and recommendations

Dependent variable = β Independent variable β: correlation coefficient

Wage = β Education
Health = β Income

20
21
Spurious correlations: how did that happen?

 Spurious correlation: mathematical relationship in which two or more variables are

associated but not causally related

 Numbers go up and down

 Selection (sample bias)
 Omitted variable
 Reverse causality

Click here for more examples

of spurious correlations…

22
When Correlation does not imply Causality

Three reasons why we should not jump to the causality conclusion

 Omitted variable: a variable, left out of a study, that would explain why the two
variables in the study are correlated
 Reverse causality: occurs when we mix up the direction of cause and effect

Dependent variable = β Independent variable

Wage = β Education Skills and abilities?

Health = β Income The other way around?

Pure causal effect cannot be identified because the counterfactual (what has not happened or is not
the case) can never be observed: often concentrate on the average causal/treatment effect (ATE).

23
Application: Correlation or causal effect?

Correlation coefficient: 0.83

 Sample selection?
 Omitted variable?
 Reverse causality?

24
Making predictions

Extrapolation: process of estimating the value beyond the distinct range of the given variable, based on
its relationship with an other variable. Assume that the “trend will continue”
 Internal validity: you can demonstrate a causal relationship within the context of the study
 External validity: you can generalise the findings of a study to other situations

Economic forecasting: process of making predictions about the economy

 Aggregated level: GDP, inflation, exchange rates
 Micro level: firms’ results, individual behaviour

Internal, external validity and accuracy of economic forecasting depend greatly on the source,
quality and design of the data used by economists

25
Application: can we use a linear extrapolation to make predictions on climate change?

 Internal validity?
 External validity?

26
Section 4

POLICY
EVALUATION AND IMPACT

27
Evaluating Public Policy

Public policy: a policy decided by the government

 Public policies are evaluated on the basis of whether their intended outcomes are efficient
and fair, and whether they can be implemented.
 Understanding people’s responses to policies so that we can better understand the likely
outcomes of our policy choices is a major challenge for economists and policymakers.

Examples
 Imposing a tax on sugary beverages to reduce obesity
 Imposing quotas for women in governments and executive boards to reduce gender inequality
 Restricting banks’ leverage ratios to guarantee financial stability
 Reducing corporation taxes to increase economic growth

Problem: how can we evaluate public policies? How can we isolate a causal effect when multiple factors
are likely to affect the outcome of interest?

28
What could go wrong?

Economics as a discipline can be misunderstood

 Concepts of causality, ceteris paribus and “general” reasoning are difficult to grasp
 Sapienza and Zingales (2013) compared the answers of the experts panel of University of Chicago
and 10,000 US households. The more economists seem to agree amongst themselves, the less
they agree with the rest of the population

Uncertainty over results

 Economists are very careful when highlighting causal relationships. Their work usually take years
and a long process of scientific validation before results can be published.
 However, economics is not an exact science: depends on the availability and quality of data, the
ability to isolate the specific impact of one economic policy, the statistical methods (econometrics)
only provide results with large confidence margins

Incorrect forecasts
 Economists are not very good forecasters, especially in trouble times. Too many variables involved,
many are endogenous, past information that informs econometric models is not always relevant

29
Influencing Policy makers

How can economists have a say in the public debate?

Communicating research outcomes effectively

 Academic papers, conferences, press release

Producing policy briefs: a summary of a particular issue, possible policy options and recommendations
 Independent research institutes, Think tanks : e.g. Institute for Fiscal studies in the UK,
Conseil d’Analyse Economique in France, Economic Policy Institute in the US

Working directly with policy makers

 Economists in national or international institutions: Civil service, OECD, IMF, WHO…

30
 Roger Backhouse, Béatrice Cherrier (2017) - The Age of the Applied
Economist
 Link to Béatrice Cherrier’s Blog (shorter version)
 Academic paper published in History of Political Economy (VLE
reading list)
Go Further A very accessible and interesting paper that questions whether Economics
has become an “empirical discipline”. The blog article is based on the
academic paper but a bit shorter.

Agnes Benassy-Quéré, Olivier Blanchard, Jean Tirole (2017) - What

role for Economists in policy-making
 Note du Conseil d’Analyse Economique (VLE reading list)
A 10 page policy brief that questions the influence of economists on public
debates and policy and offers some recommendations to improve the
dialogue between economists and policy makers. Great read, especially if
you are new to economics or still not sure what Economics is about…

31
 TOPIC 1
HEALTH AND SOCIAL POLICY

Next Will a sugar tax reduce the consumption of soft

drinks?
How do we find out if the policy will work? What
data and methods do we need?

32
Data, Evidence and Policy
MATHILDE PERON
[email protected]
A/EC/010

33