shortened

 Notation
 Measures of Location
 Measures of Dispersion
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Outliers
 Population - all items of interest for a particular
decision or investigation
- all married drivers over 25 years old
- all subscribers to Netflix

 Sample - a subset of the population

- a list of individuals who rented a comedy from
Netflix in the past year

 The purpose of sampling is to obtain sufficient

information to draw a valid conclusion about a
population.

Is the Netflix sample above a good sample? Why?

Other ways to select a sample?
 We typically label the elements of a data set using subscripted
variables, x1, x2 , … , and so on, where xi represents the ith
observation. Upper-case letters like X represent often random
variables.

 It is common practice in statistics to use

◦ Greek letters, such as m (mu; mean), s (sigma; std. deviation), and p (pi;
proportion), to represent population measures and
◦ italic letters such as by 𝑥ҧ (called x-bar), s, and p to represent sample statistics.

 N represents the number of items in a population and n represents

the number of observations in a sample.
 Notation
 Measures of Location
 Mean
 Median
 Measures of Dispersion
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Outliers
 Population mean:

 Sample mean:

 Excel function: =AVERAGE(data range)

 Property of the mean:

 Outliers can affect the value of the mean.

 Mean valid for interval/ratio variables and often
questionable for ordinal variables.
Purchase Orders database
 Using formula:

=SUM(B2:B95)/COUNT(B2:B95)

Mean = $2,471,760/94
= $26,295.32

Using Excel AVERAGE Function

=AVERAGE(B2:B95)
 Person Age Person Age
1 17 1 17
2 21 2 21
3 15 3 15
4 18 4 18
5 999 5
6 22 6 22
7 11 7 11
8 25 8 25
Mean 141.00 Mean 18.43

Wikipedia: In statistics, an outlier is an observation point that is distant from

other observations. An outlier may be due to variability in the
measurement or it may indicate experimental error; the latter are
sometimes excluded from the data set.
 The median specifies the middle value when the data are arranged
from least to greatest.
◦ Half the data are below the median, and half the data are above it.
◦ For an odd number of observations, the median is the middle of the
sorted numbers.
◦ For an even number of observations, the median is the mean of the two
middle numbers.
 We could use the Sort option in Excel to rank-order the data and
then determine the median. The Excel function =MEDIAN(data
range) could also be used.

 The median is meaningful for ratio, interval, and ordinal data.

 Not affected by outliers.
 Sort the data from smallest to largest. Since we
have 90 observations, the median is the average
of the 47th and 48th observation.

Median =
($15,562.50 + $15,750.00)/2
= $15,656.25

=MEDIAN(B2:B94)
 Person Age
1 17.00
2 21.00
3 15.00
4 18.00
5 999.00
6 22.00
7 11.00
8 25.00
Mean 141.00
Median 19.50

Median is insensitive to outliers!

The Excel file Computer Repair Times includes 250
repair times for customers.
 What repair time would be
reasonable to quote to a
new customer?
 Median repair time is 2
weeks; mean and mode are
about 15 days.
 Examine the histogram.
 90% are completed within 3 weeks

Distribution is important!
 Notation
 Measures of Location
 Measures of Dispersion
 Range
 Interquartile Range
 Variance
 Standard Deviation
 Empirical Rules
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Outliers
 Dispersion refers to the degree of variation in
the data; that is, the numerical spread (or
compactness) of the data.

 Key measures:
◦ Range
◦ Interquartile range
◦ Variance
◦ Standard deviation
 The range is the simplest and is the difference
between the maximum value and the minimum
value in the data set.

 In Excel, compute as =MAX(data range) -

MIN(data range).

 The range is affected by outliers, and is often

used only for very small data sets.
 Purchase Orders data

 For the cost per order data:

◦ Maximum = $127,500
◦ Minimum = $68.78
 Range = $127,500 - $68.78 = $127,431.22
 The interquartile range (IQR), or the midspread
is the difference between the first and third
quartiles, Q3 – Q1.

 This includes only the middle 50% of the data and,

therefore, is not influenced by extreme values.
 Purchase Orders data
 For the Cost per order data:
 Third Quartile = Q3 = $27,593.75
 First Quartile = Q1 = $6,757.81
 Interquartile Range = $27,593.75 – $6,757.81
=$20,835.94
 The variance is the “average” of the squared
deviations from the mean.
 For a population:

◦ In Excel: =VAR.P(data range)

 For a sample:

◦ In Excel: =VAR.S(data range)

 Note the difference in denominators!

 The standard deviation is the square root of the
variance.
◦ Note that the dimension of the variance is the square of the
dimension of the observations, whereas the dimension of the
standard deviation is the same as the data. This makes the
standard deviation more practical to use in applications.
 For a population:

◦ In Excel: =STDEV.P(data range)

 For a sample:

◦ In Excel: =STDEV.S(data range)

Excel file: Closing Stock
Prices
Intel (INTC):
Mean = $18.81
Standard deviation = $0.50
General Electric (GE):
Mean = $16.19
Standard deviation = $0.35

INTC is a higher risk

investment than GE.
 For many data sets encountered in practice:
 Approximately 68% of the observations fall within one
standard deviation of the mean
 Approximately 95% fall within two standard deviations of
the mean
 Approximately 99.7% fall within three standard deviations
of the mean

 These rules are commonly used to characterize

the natural variation in manufacturing processes
and other business phenomena.
 The empirical Rule comes from the normal distribution .

Most data does not follow a normal distribution!

 For any data set (any distribution), the
proportion of values that lie within +/- k (k > 1)
standard deviations of the mean is at least 1 –
1/k2

 Examples:
◦ For k = 2: at least ¾ or 75% of the data lie within two
standard deviations of the mean
◦ For k = 3: at least 8/9 or 89% of the data lie within
three standard deviations of the mean
 Notation
 Measures of Location
 Measures of Dispersion
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Outliers
 A standardized value, commonly called a z-score,
provides a relative measure of the distance an
observation is from the mean, which is independent of
the units of measurement.
 The z-score for the ith observation in a data set is
calculated as follows:

◦ Excel function: =STANDARDIZE(x, mean, standard_dev).

Standardized data is needed by many predictive

methods since it makes variables comparable.
 Purchase Orders Cost per order data

=(B2 - $B$97)/$B$98, or
=STANDARDIZE(B2,$B$97,$B$98).

0
1
 Notation
 Measures of Location
 Measures of Dispersion
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Outliers
 The proportion, denoted by p, is the fraction of
data that have a certain characteristic.

 Proportions are key descriptive statistics for

categorical data, such as defects or errors in
quality control applications or consumer
preferences in market research.

 Example: Proportion of female students is 60%.

 Proportion of orders placed by Spacetime Technologies
=COUNTIF(A4:A97, “Spacetime Technologies”)/94
= 12/94 = 0.128
 Notation
 Measures of Location
 Measures of Dispersion
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Correlation
 Outliers
 Two variables have a strong statistical relationship
with one another if they appear to “move” together.

 When two variables appear to be related, you

might suspect a cause-and-effect relationship.

 Caution: Correlation does not prove causation!

Statistical relationships may exist even though a
change in one variable is not caused by a change
in the other.
 Covariance is a measure of the linear association between two
variables, X and Y. Like the variance, different formulas are used for
populations and samples.
 Population covariance:

◦ Excel function: =COVARIANCE.P(array1,array2)

 Sample covariance:

◦ Excel function: =COVARIANCE.S(array1,array2)

 The covariance between X and Y is the average of the product of
the deviations of each pair of observations from their respective
means.
 Colleges and
Universities data
 Correlation is a measure of the linear relationship between two
variables, X and Y, which does not depend on the units of
measurement.
 Correlation is measured by the correlation coefficient, also known as
the Pearson product moment correlation coefficient.
 Correlation coefficient for a population:

 Correlation coefficient for a sample:

 The correlation coefficient is scaled between -1 and 1.

 Excel function: =CORREL(array1,array2)
Why is correlation important?
 Colleges and Universities data
 Is a schools graduation rate related to the SAT score of
incoming students?

Is there a causal relationship?

Data >
Data Analysis >
Correlation

 Excel computes the correlation coefficient

between all pairs of variables in the Input Range.
Input Range data must be in contiguous columns.
 Colleges and Universities data

◦ Moderate negative correlation between acceptance rate and

graduation rate, indicating that schools with lower acceptance
rates have higher graduation rates.
◦ Acceptance rate is also negatively correlated with the median
SAT and Top 10% HS, suggesting that schools with lower
acceptance rates have higher student profiles.
◦ The correlations with Expenditures/Student suggest that schools
with higher student profiles spend more money per student.
Value Field Settings include several statistical
measures:
 Average
 Max and Min
 Product
 Standard deviation
 Variance
 Credit Risk Data
 First, create a PivotTable.
 In the PivotTable Field List, move Job to the Row Labels
field and Checking and Savings to the Values field. Then
change the field settings from “Sum of Checking” and
“Sum of Savings” to the averages.
 Notation
 Measures of Location
 Measures of Dispersion
 Standardization
 Proportions for Categorical Variables
 Measures of Association
 Outliers
 There is no standard definition of what constitutes an
outlier!

 Wikipedia: “In statistics, an outlier is an observation point that is

distant from other observations. […] Outliers can occur by
chance in any distribution, but they often indicate either
measurement error or that the population has a heavy-tailed
distribution.”

 If the outlier is due to a measurement error then we often want to

exclude it from the analysis.

 Some typical rules of thumb:

 Normal distribution: z-scores greater than +3 or less than -3
 Boxplot:
 Extreme outliers are more than 3*IQR to the left of Q1 or right of Q3
 Mild outliers are between 1.5*IQR and 3*IQR to the left of Q1 or right of Q3
 Home Market Value data

 None of the z-scores exceed 3. However, while

individual variables might not exhibit outliers,
combinations of them might.
◦ The last observation has a high market value ($120,700) but
a relatively small house size (1,581 square feet) and may be
an outlier.
 Excel file Surgery Infections
◦ Is month 12 simply random variation or some explainable
phenomenon?
 Three-standard deviation empirical rule:

 There is only a 0.3% (for normally distributed data) or a 11% (for any
distribution) chance to see an observation outside +/- 3 std.dev.
 This suggests that month 12 is statistically different from the rest of
the data.