October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap.

1 page 1

Chapter 1
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

Introduction to Probability and

Statistics
Probability and Statistics for Economists Downloaded from www.worldscientific.com

Abstract: Probability perhaps has become the best analytic tool to de-
scribe any system involving uncertainties, and statistics provides a math-
ematical foundation to model situations involving uncertainty. As the be-
ginning of this book, this chapter will introduce two fundamental axioms
behind modern econometrics, emphasizes the important role of statistics
in economics and also discusses the limitation of statistical analysis in
economics.
Key words: Chaos, Data generating process, Econometrics, Quantitative
analysis, Probability law, Uncertainty.

1.1 Quantitative Analysis in Economics

The most important feature of modern economics and finance is the wide
use of quantitative analysis. Quantitative analysis consists of mathematical
modeling of economic theory and empirical study of economic data. This
is due to the cumulative effort of many generations of economists to make
economics a science, something like or close to physics, chemistry and bi-
ology, which can make accurate predictions or forecasts. Economic theory,
when formulated via mathematical tools, can achieve its logical consistency
among assumptions, theories, and its implications. Indeed, as Karl Marx
points out, the use of mathematics is an indication of the mature stage of
a science. On the other hand, for any economic theory to be a science, it
must be able to explain important empirical stylized facts and to predict fu-
ture economic evolutions. This requires validating economic models using
the observed economic phenomena, usually in form of data. Mathemati-
cal tools alone cannot achieve this objective. Instead, statistical tools have
proven to be rather useful. The history of the development of economics is

1
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 2

2 Probability and Statistics for Economists

a continuous process of refuting the existing economic theory that cannot

explain new empirical stylized facts and developing new economic theories
that can explain new observed empirical stylized facts. Empirical analytic
tools play a vital role in such a process. In a sense, statistical methods and
techniques are really the heart of the scientific research in economics.
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

As a matter of fact, the main empirical analytic tool in economics is

econometrics. Econometrics is the statistical analysis of economic data in
combination with economic theory. There is a lot of uncertainty in real
Probability and Statistics for Economists Downloaded from www.worldscientific.com

economies and financial markets, and economic agents usually have to make
decisions under uncertainty. Probability is a natural quantitative tool to
describe uncertainty in economics. Historically, probability was motivated
by interest in games of chance. Scholars then began to apply probability
theory to actuarial problems and some aspects of social sciences. Later,
probability and statistics were introduced into physics by L. Boltzmann, J.
Gibbs, and J. Maxwell, and by last century, they had found applications
in all phases of human endeavor that in some way involve an element of
uncertainty or risk. Indeed, probability theory has become the best analytic
tool to describe any system involving uncertainty.
Modern statistics has encompassed the science of basing inferences on
observed data and the entire problem of making decisions in the face of
uncertainty. It would be presumptuous to say that statistics, in its present
state of development, can handle all situations involving uncertainty, but
new techniques are constantly being developed and modern statistics can,
at least, provides a framework for looking at the situations involving un-
certainty in a logical and systematic fashion. It can be said that statistics
provides mathematical models that are needed to study situations involving
uncertainty in the same way as calculus provides mathematical models that
are needed to describe, say the concepts of the Newtonian physics. Indeed,
as Robert Lucas points out, the introduction of stochastic factors into a
dynamic economic system can provide new insight into dynamic economic
laws.

1.2 Fundamental Axioms of Statistical Analysis in

Economics
There are two fundamental axioms behind modern econometrics:
• Axiom A: Any economy can be viewed as a random or stochastic
system governed by some probability law;
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 3

Introduction to Probability and Statistics 3

• Axiom B: Economic phenomenon, often summarized in form of

data, can be viewed as a realization of this stochastic data gener-
ating process.
Economics is about resource allocation in an uncertain environment.
When an economic agent makes a decision, he or she usually does not
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

know precisely the outcome of his or her action, which usually will arise
in an unpredictable manner with time lags. As a consequence, uncertainty
and time are two of the most important features of an economy. Therefore
Probability and Statistics for Economists Downloaded from www.worldscientific.com

it seems reasonable to assume Axiom A. With Axiom A, it is natural to

assume Axiom B under which one can call the economic system a “data
generating process”. It is impossible to prove these two axioms. They are the
philosophic views of econometricians and economists about an economy. We
note that not all economists may agree with these two axioms. For example,
some economists view that an economic system is a chaotic process, which
is deterministic but can generate seemingly random numbers.
To illustrate the different implications between a stochastic view and
a chaotic view on an economy, we consider an example. As a well-known
empirical stylized fact, high frequency stock returns are found to have little
autocorrelation with their own lagged returns. Figure 1.1 plots the obser-
vations on the Standard & Poor 500 daily closing price and daily return
over time respectively. To explain this empirical stylized fact, there are at
least two possible hypotheses or conjectures. The first is to assume that the
stock price follows a geometric random walk, that is,

ln Pt = ln Pt−1 + Xt ,

where its log-return series {Xt = ln(Pt /Pt−1 )} is a statistically independent

sequence over time, which implies zero correlation between stock returns
over time. Figure 1.2 plots the observations on the price level Pt and the
return Xt generated from this geometric random walk model using a ran-
dom number generator on a personal computer. Comparing Figures 1.1 and
1.2, we can observe some similarity between the real data series and the
artificial series generated from the computer.
Alternatively, one may assume that the stock return follows a determin-
istic chaotic logistic map:

Xt = 4Xt−1 (1 − Xt−1 ).

If we generate a large set of observations from this logistic map and calcu-
late the sample autocorrelations between observations over time, we would
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 4

4 Probability and Statistics for Economists

S&P500 daily closing price

1500

1250
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

1000
Probability and Statistics for Economists Downloaded from www.worldscientific.com

750

500

250

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

0.10 daily return

0.05

0.00

-0.05

-0.10

-0.15

-0.20
1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Figure 1.1: Standard & Poor 500 daily closing prices and daily returns
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 5

Introduction to Probability and Statistics 5

price level
700

600
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

500
Probability and Statistics for Economists Downloaded from www.worldscientific.com

400

300

200

100

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000

0.08
return
0.07

0.06

0.05

0.04

0.03

0.02

0.01

0.00

-0.01

-0.02

-0.03

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000

Figure 1.2: Price series Pt and return series Xt

October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 6

6 Probability and Statistics for Economists

also find zero or little correlations. Thus, both the stochastic random walk
hypothesis and the deterministic logistic map hypothesis can explain the
empirical stylized fact of zero or little autocorrelation in high-frequency
stock returns. However, their implications are different: the random walk
hypothesis implies that the future stock return is not predictable using
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

historical stock returns, because a future stock return is independent of

historical stock returns. On the other hand, the time series observations
from a logistic map display zero autocorrelation, but they are not inde-
Probability and Statistics for Economists Downloaded from www.worldscientific.com

pendent over time. In fact, there exists a deterministic nonlinear quadratic

relationship between Xt and Xt−1 from which one can predict Xt perfectly
using Xt−1 . Which view is more realistic to explain stock returns is an issue
for empirical study.
The probability law of a stochastic economic process describes the aver-
age behavior of massive economic phenomena and may be called the “law
of economic motions”. The objective of econometrics is to infer the prob-
ability law of an economic system based on observed economic data, and
then use the inferred probability law to test economic theory and economic
hypotheses, to explain important economic stylized facts, to forecast future
evolutions of the economic system, and to conduct other applications such
as policy analysis. Econometrics provides a bridge linking economic models
and economic reality.
One important implication of Axioms A and B is the need of “stochas-
tic thinking” and “statistical thinking” in economic analysis. For example,
one should expect that economic relationships are stochastic and thus the
outcomes cannot be predicted with certainty. All economic agents must
incorporate this uncertainty when making their economic decisions. More-
over, any observed economic data, as realizations of a stochastic process,
must be subject to sampling variations, thus creating some uncertainties
for inference of the law of economic motions.

1.3 Role of Statistics in Economics

As a science, statistics has been widely applied in many different fields,

including physics, engineering, economics, finance, management, biology,
medical science, public health, and many others. For example, statistical
quality control has been a very successful application of statistical methods
to quality control in manufacture industries in Japan and the United States.
The objective of statistical quality control is to monitor the process of pro-
duction and decides whether the production is in-control or out-of-control
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 7

Introduction to Probability and Statistics 7

by setting some control limits. The idea of control limits is analogous to

that of hypothesis testing in statistics.
Below, we briefly discuss what roles statistics can play in economics and
related fields.
First of all, economic phenomena is usually rather complicated, and
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

there is a vast amount of information in daily economic life. It is very im-

portant to summarize observed economic data in a simple, intuitive and in-
terpretable way so as to convey information to economists, decision makers,
Probability and Statistics for Economists Downloaded from www.worldscientific.com

and the public. Statistics, by its very nature, is an effective tool for doing
so. Economic indicators such as the Consumer Price Index (CPI) and the
unemployment rate are two examples of statistical description of the state
of an macroeconomy which have been indispensable means for governments
and central banks to decide their fiscal and monetary policies. Moreover,
many important empirical stylized facts in economics are presented in form
of a statistical relationship. One example is the well-known Phillips curve
in macroeconomics, which documents that the inflation rate is negatively
correlated with the unemployment rate. Figure 1.3 is a scatter plot of the
U.S. inflation rate, the log-difference of the U.S. CPI series, and the U.S.
unemployment rate, which indicates a negatively correlated relationship.
Another example is volatility clustering in finance, which documents that a
large asset volatility today tends to be followed by another large volatility
tomorrow; a small volatility tends to be followed by another small volatility
tomorrow, and the patterns alternate over time. Figure 1.4 plots the time
series observations of the absolute and squared values of the Standard &
Poor 500 daily return. They indicate the phenomena of volatility clustering.
Concepts in probability and statistics can also provide simple characteriza-
tions of economic ideas and economic theory. Examples include the Lorenz
curve of income inequality and the representation of stochastic dominance
via the probability distribution function. In fact, most financial theory is
represented with the concepts of probability. It would be difficult to imagine
what will be left in modern finance if there is no use of probability concepts
and tools.
Second, a key feature of economic data is the existence of variability in
economics. Much of variability is due to inherent uncertainty in the econ-
omy, which results in risk given the risk-averse nature of economic agents.
A central theme of modern financial risk management is to quantify and
price financial risk via statistical methods and modeling. For such purposes,
statistical models and measures of volatility are indispensable quantitative
tools.
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 8

8 Probability and Statistics for Economists

Third, an ingredient methodology in statistics is sampling. The basic

idea of sampling theory is to use a subset of information called “sample”
to infer knowledge of the entire system or process called “population”, and
then use the knowledge of the system for various applications. This statisti-
cal method is consistent with the economic principle of cost minimization.
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

An example is the aforementioned stochastic quality control, which uses

a similar idea to hypothesis testing to ensure the quality of manufactured
products while minimizing the cost of inspection and monitoring.
Probability and Statistics for Economists Downloaded from www.worldscientific.com

Fourth, perhaps the most important objective of economic analysis is

to discover or validate economic relationships, particularly causal economic
relationships. Statistical inference can play an instrumental role in this re-
gard. For example, statistics can estimate demand elasticities of output with
respect to price for certain product, which may be useful for the marketing
strategy of a company. Combined with well-designed experiments which can
control important economic factors, statistical analysis can identify causal
economic relationships. On the other hand, for observational or historical
economic data for which economists cannot control economic factors, it is
much more challenging if not impossible to identify causal economic rela-
tionships. Nevertheless, aided with economic theory, statistics can be still
very helpful in identifying economic relationships.

Unemployment Rate
11

10

3
CPI
-2 0 2 4 6 8 10 12 14

Figure 1.3: Scatter plots of the U.S. inflation rate and unemployment rate
October 17, 2017 14:27 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 9

Introduction to Probability and Statistics 9

0.22
absolute value of daily return

0.17
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
Probability and Statistics for Economists Downloaded from www.worldscientific.com

0.12

0.07

0.02

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

0.05
squared value of daily return

0.04

0.03

0.02

0.01

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Figure 1.4: Absolute and squared values of the Standard & Poor 500 daily return
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 10

10 Probability and Statistics for Economists

Fifth, economic agents often make a decision under uncertainty, such

as portfolio investment and risk management. Statistics provides a rather
convenient quantitative framework to describe uncertainty and to obtain
optimal decisions under uncertainty.
Finally, in addition to uncertainty, time is another important factor in
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

economics. Oftentimes, economic agents cannot know the outcome due to

their action when making a decision. In general, most outcomes of economic
agents arise after some time periods. Thus, when economic agents make a
Probability and Statistics for Economists Downloaded from www.worldscientific.com

decision in a dynamic setup, they usually have to forecast some important

economic factors. Time series analysis in statistics can provide statistical
methods and models for sensible out-of-sample forecasts.
The above brief summary does not exhaust all major roles that statistics
can play in economics. Nevertheless, it provides us some sense about how
important statistics is in economics and related fields.

1.4 Limitation of Statistical Analysis in Economics

Statistics is an analysis of the “average behavior” of the outcomes of a large
number of repeated experiments. A key assumption in the statistical anal-
ysis of this book is that the repeated experiments are independently and
identically distributed (IID). By independence, it means that the genera-
tion of the outcome of an experiment has nothing to do with the generation
of the outcome of another experiment. Thus, different experiments provide
different pieces of information. By identical distributions, it means that the
mechanisms of different experiments generating outcomes are essentially the
same. Obviously, the IID assumption is a mathematical limiting approxi-
mation of real economic events. It may be difficult to satisfy for economic
processes.
There are some distinctive features of an economic system when viewed
as a stochastic process. These features impose some limitations on the sta-
tistical analysis in economics. First of all, often the observed data is the
joint effect of many (perhaps infinite) factors in an economy, while any eco-
nomic model can only consider some of them. It is therefore very difficult,
if not impossible, to separate the effect of the factors under modeling from
the effect of the omitted factors. This is quite different from Physics, where
a controlled experiment can be done to remove the effect of other factors. In
the recent years, there is an increasing interest in experimental economics,
which can be viewed as studies of controlled economic experiments. This is
similar in spirit to Physics, but it is hard to imagine to extend this method
to the whole economy like China which has 1.3 billion of population!
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 11

Introduction to Probability and Statistics 11

The second feature of economic systems is that an economic process is

irreversible. As a consequence, many important economic variables, such
as Gross Domestic Product (GDP) in each year, only have one realization.
For example, consider the observed time series data for the Chinese GDP
growth rate during 1980-2010. Here, the GDP growth rates in different
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

years are considered as different random variables, and as a result, each

random variable only has one observation. Thus, it is impossible to con-
duct statistical analysis of economic data without making assumptions. In
Probability and Statistics for Economists Downloaded from www.worldscientific.com

practice, it is often assumed that economic variables over different years or

over different cross-sectional units follow the same population distribution
or at least have the same attributes in certain aspects of their probability
distributions. These are usually called “homogeneity” and “stationarity”
assumptions. Thus, the data for variables over different time periods or
different cross-sectional units can be viewed as generated from the same
population or share the same attributes, so that statistical inference of ob-
served data is possible. Obviously, the validity and accuracy of statistical
inference depend on how reasonable these underlying assumptions are in
describing economic systems.
The third feature of economic systems is their time-varying features.
There are many economic events, such as government policy changes,
changes in foreign exchange rate systems, oil shocks, financial crisis, etc,
which may cause economic behaviors to change fundamentally. These
are the so-called economic structural changes or regime shifts. Structural
changes and regime shifts will render the existing economic models invalid
in describing or predicting future economic evolutions.
Finally, most observed economic data may be subject to nontrivial mea-
surement errors, due to the incentive problem of economic agents to report
the true information (e.g., income), aggregation over time or space, the
qualitative or subjective nature of some economic variables (e.g., happiness)
that are difficult to measure, and etc.
Because of these distinctive features of economic systems, we should
be aware of the practical relevance of underlying assumptions and their
limitations when making statistical inference from observed economic data.

1.5 Conclusion

In this chapter, we provide a brief introduction to probability and statis-

tics. As a natural quantitative tool to describe uncertainty in economics,
probability theory has become the best analytic tool to describe any system
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 12

12 Probability and Statistics for Economists

involving uncertainty, and statistics provides a framework for modeling the

situations involving uncertainty.
Statistics has wide applications in economics and related fields, such as
summarizing and interpreting data efficiently, modeling variabilities, mak-
ing inference of a system or process using a subset of sample information,
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

identifying causal economic relationships, and making out-of-sample fore-

casts. However, the non-experimental nature of most economic phenomena,
unobserved heterogeneity among economic agents, unstable economic rela-
Probability and Statistics for Economists Downloaded from www.worldscientific.com

tionships over time, and data quality problems impose certain limitations
on validity and accuracy of statistical analysis in economics.
October 5, 2017 14:56 ws-book9x6 Probability and Statistics for Economists HongWS/Chap. 1 page 13

EXERCISE 1 13

EXERCISE 1
1.1. What are the rationales that probability and statistics are useful in
economics? What are your justifications for these rationales?
1.2. What major roles statistics can play in economics and related fields?
by 81.187.58.148 on 11/25/23. Re-use and distribution is strictly not permitted, except for Open Access articles.

1.3. What are the limitations of statistical analysis in economics?

Probability and Statistics for Economists Downloaded from www.worldscientific.com