Review of Probability and Statistics

ECON 321 Topic 2

Random Variables and Probability Distributions

• Probability theory is the foundation on which econometrics is built.

• An experiment is a process whose outcome is not known in advance.

• Possible outcomes (or realizations) of an experiment are events.

• Set of all possible outcomes is called the sample space.

• Issues relating to probability, experiments and events are represented by a variable (either
continuous or discrete).

• Since outcome of an experiment is not known in advance, this is a random variable.

• Probability reflects the likelihood that an event will occur.

• The probability of event A occurring will be denoted by Pr (A).

Random variables may be discrete or continuous

 A variable is discrete if number of values it can take on is finite (or countable), for
example the number of heads in ten tosses of a coin.

 A variable is continuous if it can take on any value on the real line or in an interval, for
example the amount of energy consumed in an hour.

Example

 An experiment involves rolling a single fair dice.

 Each of the six faces of the dice is equally likely to come up when the dice is tossed

 Sample space is {1, 2, 3, 4, 5, 6}.

 Discrete random variable, A, takes on values 1, 2, 3, 4, 5, 6

1|Page
 Review of Probability and Statistics

1
 Probabilities: Pr ( A=1 )=Pr ( A=2 )=…=Pr ( A=6 )=
6

 Realization of random variable is the value which actually arises (e.g. if the dice is rolled,
a 4 might appear).

Probability Distribution of a Discrete Random Variable

• It is the list of all possible values of the variable and the probability that each value will
occur. These probabilities sum to 1.

Number of computer crashes

0 1 2 3 4

Probability distribution 0.80 0.10 0.06 0.03 0.01

Cumulative Probability distribution 0.80 0.90 0.96 0.99 1

Probability Distribution of a Continuous Random Variable

• Probability density function: the area under the probability density function between any
two points is the probability that the random variable falls between those two points.

2|Page
 Review of Probability and Statistics

Expected Values, Mean and Variance

• Suppose the random variable Y takes on k possible values, where y1 denotes the first value,
y2 denotes the second value and so on, and the probability that Y takes on y1 is p1, the
probability that Y takes on y2 is p2 and so on. The expected value of Y, denoted by E(Y) is

k
E ( Y )= y 1 p1 + y 2 p2 +…+ y k p k =∑ y i p i
i=1

• The expected value is also called the mean of Y and is denoted μY .

3|Page
 Review of Probability and Statistics

The Standard Deviation and Variance

• Standard deviation and variance measure the dispersion or the spread of a probability
distribution.

• The variance of a random variable Y, denoted by var(Y) is

k
σ ❑2 =var ( Y ) =E [ ( Y −μ Y ) ] =∑ ( y i−μY )2 p i
2

i=1

• The standard deviation of Y is σ Y , the square root of the variance.

Other Measures of the Shape of a Distribution

• Skewness: a mathematical way to describe how much a distribution deviates from

symmetry.

E [( Y −μY )3 ]
Skewness=
σ3

4|Page
 Review of Probability and Statistics

• Kurtosis: it measures how thick or “heavy”

are the distribution tails. It is a measure of
how much of the variance Y arises from
extreme values (outliers). The greater the
kurtosis of a distribution, the more likely are
outliers.
E [ ( Y −μY )4 ]
Kurtosis=
σ4
• The kurtosis of a normally distributed
random variable is 3.

Two random variables

Covariance and Correlation

 Covariance: measures the extent to which two random variables move together.

cov ( X , Y )=σ XY =E [ ( X−μ X ) ( Y −μ Y ) ]

k l
cov ( X , Y )=∑ ∑ ( x j−μ X ) ( y i−μY ) Pr ⁡(X =x j ,Y = y i)
i=1 j=1

o If the random variables X and Y move in the same direction, the covariance will be
positive, but if they move in opposite direction, the covariance will be negative.

o If X and Y are independent, the covariance is zero.

5|Page
 Review of Probability and Statistics

 Correlation: an alternative measure of dependence between X and Y that solves the units
problem of the covariance.

cov (X , Y ) σ XY
corr ( X ,Y ) = =
√ var ( X ) var (Y ) σ X σY

o Correlation is unitless.

o If corr ( X ,Y ) =0, X and Y are uncorrelated.

o The correlation is always between -1 and 1.

The Normal Distribution

• A continuous random variable with a normal distribution has the bill shaped probability
density.

• Normal density with mean μ and variance σ 2 is symmetric around its mean and has 95%
of its probability between μ−1.96 σ and μ+1.96 σ .

• The normal distribution is symmetric, so the skewness is zero and the kurtosis is 3.

6|Page
 Review of Probability and Statistics

• The normal distribution with mean μ and variance σ 2 is expressed as N ( μ , σ 2 ).

• The standard normal distribution (Z) is the normal distribution with mean μ=0 and
variance σ 2=1 and is denoted as N ( 0 , 1 ).

• If Y is normally distributed with N ( 0 , 1 ), then Y is standardized by computing

Z=(Y −μ)/σ .

The Chi-square Distribution

• It is the distribution of the sum of squares of n independent standard normal random

variables with n degrees of freedom and is written as χ 2n .

• For example, if n random variables Z1 , Z 2 , … , Z n are all independently and identically

distributed (iid) as the standard normal N ( 0 , 1 ), then

U =Z 1 +Z 2 +…+ Z n=∑ Z i
2 2 2 2

7|Page
 Review of Probability and Statistics

Sampling

• Population: A group that includes all the cases (individuals, objects, or groups) in which
the researcher is interested.

• Sample: A relatively small subset from a population.

• Random Sampling: It is designed to ensure that every member of the population has an
equal chance of being chosen.

Sampling Distribution

• The two most frequently used sample statistics are the sample mean and the sample
variance.
• For n observations Y 1 , … ,Y n

n
1 1
Sample mean: Y = ( Y 1 +Y 2+ …+Y n ) = ∑ Y i
n n i=1

2 1 2 1
Sample variance: s = (Y 1−Y ) + ¿¿
n−1 n−1

Dividing the sample variance by (n−1) is to get unbiased variance, which means the sample
variance tends to approach the true population variance at large sample sizes.

8|Page
 Review of Probability and Statistics

Writing a Research Paper

The summary should be short, and should include the following:

1. A concise statement of the problem.
2. Comments on the information that is available, with one or two key references.
3. A description of the research design that includes:
a. the economic model
b. the econometric estimation and inference methods
c. data sources
d. estimation, hypothesis testing, and prediction procedures, including the
econometric software and version used
4. The potential contribution of the research.

A Format for Writing a Research Report

The following outline for a report is typical.

1. Statement of the problem

2. Review of the literature
3. The economic model
4. The econometric model
5. The data
6. The estimation and inference procedures
7. The empirical results and conclusions
8. Possible extensions and limitations of the study
9. Acknowledgments
10. References

9|Page