Statistik 2
Regression and Correlation
Capaian Pembelajaran
▪ Mampu menentukan tujuan pengamatan untuk pengolahan data
menggunakan analisis regresi.
▪ Mampu menemukan variabel amatan dalam melakukan analisis
regresi.
2
Correlation Analysis
SCATTER PLOT AND CORRELATION
• Scatter plot (or scatter diagram) is used to show the relationship
between two variables
• Correlation analysis is used to measure strength of the association
(linear relationship) between two variables
→Only concerned with strength of the relationship
→No causal effect is implied
4
SCATTER PLOT EXAMPLES
5
SCATTER PLOT EXAMPLES
6
SCATTER PLOT EXAMPLES
7
CORRELATION COEFFICIENT
✓The population correlation coefficient ρ measures the strength of
the relationship between two variables
✓The sample correlation coefficient r is an estimate of ρ and is used
to measure the strength of the linear relationship in the sample
observations
8
FEATURES OF CORRELATION COEFFICIENT
a. Unit free
b. A correlation coefficient of -1.00 or +1.00
c. The closer to -1.00, the stronger the negative linear relationship
d. The closer to +1.00, the stronger the positive linear relationship
e. The closer to 0, the weaker the linear relationship
9
EXAMPLES OF APPROXIMATE r VALUES
10
CALCULATION OF COEFFICIENT CORRELATION
• Sample correlation coefficient :
𝑺𝒙𝒚
𝒓=
𝑺𝒙𝒙 𝑺𝒚𝒚
With: Where:
r : sample correlation coefficient
𝑆𝑥𝑥 = σ𝑛𝑖=1(𝑥𝑖 − 𝑥)ҧ 2 , n : sample size
𝑆𝑦𝑦 = σ𝑛𝑖=1(𝑦𝑖 − 𝑦)
ത 2, xi : value of observation i in independent variable
yi : value of observation i in dependent variable
𝑆𝑥𝑦 = σ𝑛𝑖=1(𝑥𝑖 − 𝑥)(𝑦
ҧ 𝑖 − 𝑦),
ത 𝑥ҧ : average value of independent variable
𝑦ത : average value of dependent variable
11
EXAMPLE OF COEFFICIENT CORRELATION CALCULATION
We want to evaluate the relationship between the number of
sales calls and the number of products sold.
Calls Sales (𝑥𝑖 − 𝑥)ҧ (𝑥𝑖 − 𝑥)ҧ 2 ത
(𝑦𝑖 − 𝑦) ത 2
(𝑦𝑖 − 𝑦) ҧ 𝑖 − 𝑦)
(𝑥𝑖 −𝑥)(𝑦 ത
20 $ 30.00 -2 4 $ -15.00 $ 225.00 $ 30.00 𝑆𝑥𝑦
40 $ 60.00 18 324 $ 15.00 $ 225.00 $ 270.00 𝑟=
20 $ 40.00 -2 4 $ -5.00 $ 25.00 $ 10.00
𝑆𝑥𝑥 𝑆𝑦𝑦
30 $ 60.00 8 64 $ 15.00 $ 225.00 $ 120.00
10 $ 30.00 -12 144 $ -15.00 $ 225.00 $ 180.00 $ 900
10 $ 40.00 -12 144 $ -5.00 $ 25.00 $ 60.00 = = 0.759
20 $ 40.00 -2 4 $ -5.00 $ 25.00 $ 10.00 760 (1850)
20 $ 50.00 -2 4 $ 5.00 $ 25.00 $ -10.00
20 $ 30.00 -2 4 $ -15.00 $ 225.00 $ 30.00
30 $ 70.00 8 64 $ 25.00 $ 625.00 $ 200.00
22 $ 45.00 𝑺𝒙𝒙 =760 𝑺𝒚𝒚 = $ 1,850.00 𝑺𝒙𝒚 =$ 900.00
12
EXAMPLE OF COEFFICIENT CORRELATION CALCULATION
Using Excel Features: Data – Data Analysis -Correlation
Calls Sales
Calls 1
Sales 0.759014 1 It does not show us any cause-
and-effect relationship between
two variables
r = 0.759 → strong positive linear
relationship between the number of
calls and the number of sales
13
SIGNIFICANCE TEST OF THE CORRELATION COEFFICIENT
On the previous example, we found that r = 0.759
→ It is based on 10 samples observed
→ How can we conclude about the relationship between two variables in the population?
𝐻𝑜 : 𝜌 = 0 𝑛𝑜 𝑐𝑜𝑟𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛
𝐻1 : 𝜌 ≠ 0 (𝑐𝑜𝑟𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛 𝑒𝑥𝑖𝑠𝑡𝑠)
Test statistic:
𝒓 𝒏−𝟐
𝑻=
𝟏 − 𝒓𝟐
With n-2 degrees of freedom
14
SIGNIFICANCE TEST OF THE CORRELATION COEFFICIENT
With α significance level, the critical region will be:
𝑇 < −𝑡𝛼/2,𝑣 or 𝑇 > 𝑡𝛼/2,𝑣
a/2 a/2
Reject H0 Do not reject H0 Reject H0
-tα/2 tα/2
0
15
EXAMPLE OF SIGNIFICANCE TEST OF r
1. State the hypothesis
𝐻𝑜 : 𝜌 = 0 𝑛𝑜 𝑐𝑜𝑟𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛
𝐻1 : 𝜌 ≠ 0 (𝑐𝑜𝑟𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛 𝑒𝑥𝑖𝑠𝑡𝑠)
2. Define critical value and critical region:
With α=0.05 and v=n-2=8 → 𝑡𝛼/2,𝑣 = 𝑡.025,8 = 2.306
Critical region: 𝑇 < −2.306 or 𝑇 > 2.306
3. Compute the T-statistic:
𝑟 𝑛−2 0.756 8
𝑇= = = 3.28
1− 𝑟2 1 − 0.7592
4. Evaluate the hypothesis
T = 3.28 > t = 2.306 → reject H0
5. Conclusion
At 5% significance level,
there is a positive correlation between the number of calls and the number of sales in the population
REGRESSION ANALYSIS
17
INTRODUCTION TO REGRESSION ANALYSIS
Regression analysis is used to:
▪ Predict the value of a dependent variable based on the value of at least one
independent variable
▪ Explain the impact of changes in an independent variable on the dependent variable
Analyse the causality relationship between independent and dependent variables
Independent variable:
• the variable we use to explain the dependent variable
• Predictor variable → use to predict the expected value of dependent variable
Dependent variable:
• the variable we wish to explain
• The variable that is being predicted or estimated
18
SIMPLE LINEAR REGRESSION MODEL
• Only one independent variable (x) → one regressor
• Relationship between x and y is described by a linear
function
• Changes in y are assumed to be caused by changes in x
19
TYPES OF REGRESSION MODELS
20
SIMPLE LINEAR REGRESSION MODEL
A linear relationship form between the response Y and the
regressor x:
𝒀 = 𝜶 + 𝜷𝒙
Where 𝛼 is the intercept, and 𝛽 is the slope
However, the relationship between Y and x is not
deterministic → there must be a random component to the
equation that relates to the variables.
Thus, the model will be:
𝒀 = 𝜶 + 𝜷𝒙 + 𝝐
Where 𝝐 is a random variable that is assumed to be
distributed with E(𝝐)=0 and Var(𝝐)=σ2
21
SIMPLE LINEAR REGRESSION MODEL
Interpretation:
✓ The quantity Y is a random since 𝝐 is random
✓ The value regressor x is not random
22
SIMPLE LINEAR REGRESSION ASSUMPTIONS
✓ Error values (ε) is statistically independent
✓ Error values are normally distributed for any given value of
x and have constant variance
✓ The underlying relationship between the x variable and the
y variable is linear
23
POPULATION AND SAMPLE REGRESSION MODEL
Unknown 𝑦ො = 𝑎 + 𝑏𝑥
relationship
𝑦 = 𝛼 + 𝛽𝑥
24
ESTIMATED REGRESSION MODEL
The sample regression model provides an estimate of the population
regression line
25
LINEAR REGRESSION MODEL
In regression analysis, the objective is to use the data to position a line
that best represent the relationship between the two variables
→How do we find the best fitted line for the data?
the first approach is to use a scatter diagram to visually position the
line
26
SCATTER DIAGRAM
1. Plot of all (Xi,Yi) pairs
2. Suggest how well model will fit
27
SCATTER DIAGRAM
How would you draw a line through the points? How do you determine which line ‘fits best’?
28
SCATTER DIAGRAM
How would you draw a line through the points? How do you determine which line ‘fits best’?
29
SCATTER DIAGRAM
How would you draw a line through the points? How do you determine which line ‘fits best’?
30
SCATTER DIAGRAM
How would you draw a line through the points? How do you determine which line ‘fits best’?
31
LEAST SQUARES PRINCIPLE
• We would like to choose a line that would minimize the error
between the actual data and the line → residual
Error in Fit:
Given a set of regression data [(xi,yi );i:1,2,…,n] and a fitted model
𝑦ො𝑖 = 𝑎 + 𝑏𝑥, the ith residual ei is given by
𝑒𝑖 = 𝑦𝑖 − 𝑦ො𝑖
Using least squares method, we
minimize the sum of squares of
vertical deviations from the points to
the line → LEAST SQUARES VALUE
32
LEAST SQUARES EQUATION
Given a set of regression data [(xi,yi );i:1,2,…,n], the least squares estimate a and
b of the regression coefficients α and β are computed from the formulas:
σ𝒏𝒊=𝟏 𝒙𝒊 − ഥ ഥ) 𝑺𝒙𝒚
𝒙 (𝒚𝒊 − 𝒚
𝒃= 𝒏 𝟐
=
σ𝒊=𝟏(𝒙𝒊 − ഥ
𝒙) 𝑺𝒙𝒙
σ𝒏𝒊=𝟏 𝒚𝒊 − 𝒃 σ𝒏𝒊=𝟏 𝒙𝒊
𝒂= ഥ − 𝒃ഥ
=𝒚 𝒙
𝒏
33
INTERPRETATION OF LEAST SQUARES MODEL
➢ a is the estimated average value of y when the value of x is zero
➢ b is the estimated change in the average value of y as a result of a one-unit
change in x
Note:
The regression equation is not generally used for the points outside the range of
the sample values
34
SIMPLE LINEAR REGRESSION EXAMPLE
Recall the previous example!
We want to evaluate whether the number of sales calls affects the number of
products sold?
Calls (xi) Sales (yi) (𝑥𝑖 − 𝑥)ҧ (𝑥𝑖 − 𝑥)ҧ 2 ത
(𝑦𝑖 − 𝑦) ത 2
(𝑦𝑖 − 𝑦) ҧ 𝑖 − 𝑦)
(𝑥𝑖 −𝑥)(𝑦 ത
20 $ 30.00 -2 4 $ -15.00 $ 225.00 $ 30.00
40 $ 60.00 18 324 $ 15.00 $ 225.00 $ 270.00
-2 4 $ -5.00 $ 25.00 $ 10.00
900
20 $ 40.00 𝑏= = 1.18
30 $ 60.00 8 64 $ 15.00 $ 225.00 $ 120.00 760
10 $ 30.00 -12 144 $ -15.00 $ 225.00 $ 180.00 𝑎 = 45 − 1.18 22 = 18.95
10 $ 40.00 -12 144 $ -5.00 $ 25.00 $ 60.00
20 $ 40.00 -2 4 $ -5.00 $ 25.00 $ 10.00
20 $ 50.00 -2 4 $ 5.00 $ 25.00 $ -10.00
20 $ 30.00 -2 4 $ -15.00 $ 225.00 $ 30.00
30 $ 70.00 8 64 $ 25.00 $ 625.00 $ 200.00
𝑥ҧ =22 𝑦ത = $ 45.00 0 𝑺𝒙𝒙 =760 0 𝑺𝒚𝒚 =$ 1,850.00 𝑺𝒙𝒚 =$ 900.00
35
INTERPRETATION THE MODEL
Linear regression equation:
𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒔𝒂𝒍𝒆𝒔 = 𝟏𝟖. 𝟗𝟓 + 𝟏. 𝟏𝟖(𝐧𝐮𝐦𝐛𝐞𝐫 𝐨𝐟 𝐜𝐚𝐥𝐥𝐬)
a = 18.95 is the estimated average value of Y when the value of X is zero
However, x = 0 is not in the range of the sample values → should not be used to estimate the
number of products sold
→ The number of calls ranged from 10 to 40, so estimates should be limited to that range
The value a = 18.95 is also can be described as the portion of the number of products sold not
explained by the number of calls
→ Probably by any other variable
36
INTERPRETATION THE MODEL
Linear regression equation:
𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒔𝒂𝒍𝒆𝒔 = 𝟏𝟖. 𝟗𝟓 + 𝟏. 𝟏𝟖(𝐧𝐮𝐦𝐛𝐞𝐫 𝐨𝐟 𝐜𝐚𝐥𝐥𝐬)
b = 1.18 is the estimated change in the average value of Y as a result of
one-unit change in X
→ It tells use that the average value of the number of sales increases by
1.18 unit, on average, for each additional number of call
37
Selesai
