Regression – Part 1

Session 7
 Linear regression
• Simple linear regression is a technique that predicts a metric variable from a linear
relation with another metric variable
Metric variable : Interval or Ratio

Example: Predicting Job Performance from IQ

Some company wants to know

can we predict job performance from IQ scores?

The very first step they should take is to measure both (job) performance and IQ on as
many employees as possible. Refer the data file job_performance.sav where 60
employees data are shown below.
Inspect the scatter diagram
It seems that there is a
positive linear relationship
however we need to
examine the correlation
between IQ level and job
performance to check the
significance
Check the correlation
Correlations
Outcome of job Outcome of IQ The Pearson correlation
performance test shows that there is a
test correlation of 0.474 between
IQ and job performance
Pearson Correlation 1 .474**
Outcome of job performance
Sig. (2-tailed) .000 The sig. value is less than
test
N 60 60 0.05 therefore there is a
Pearson Correlation .474** 1 significance relationship
between IQ and job
Outcome of IQ test Sig. (2-tailed) .000
performance
N 60 60
**. Correlation is significant at the 0.01 level (2-tailed).
How can we predict performance from IQ?
• Linear Relation - General Formula
• Any linear relation can be defined as Y’ = A + B * X. Let's see what these numbers mean

• Since X is in our data -in this case, our IQ scores- we can predict performance if we know the
intercept (or constant) and the B coefficient
Regression
 Coefficientsa

Model Unstandardized Coefficients Standardized t Sig.

Coefficients

B Std. Error Beta

(Constant) 49.782 6.976 7.136 .000

1
Outcome of IQ test .266 .065 .474 4.097 .000

a. Dependent Variable: Outcome of job performance test

Job performance = 49.782 + 0.474* IQ

• This output tells us that the best possible prediction for job performance given IQ is
predicted performance = 49.78 + 0.47* IQ

• So the core output of our regression analysis are 2 numbers:

• An intercept (constant) of 49.78 and
•  b coefficient of 0.47
 R-Square - Predictive Accuracy
• R-square is the proportion of variance in the outcome variable that's accounted for by
regression
Model Summary

Model R R Square Adjusted R Std. Error of the

Square Estimate

1 .474a .224 .211 7.130

a. Predictors: (Constant), Outcome of IQ test

• R-square thus indicates the accuracy of our regression model (how well IQ can predict
performance: r-square)
• R-square adjusted is an unbiased estimator of r-square in the population
• Regression computes coefficients that maximize r-square for our data. Applying these to
other data -such as the entire population- probably results in a somewhat lower r-square: r-
square adjusted. This phenomenon is known as shrinkage.
 Statistical Significant
Coefficientsa
Model Unstandardized Standardized t Sig. 95.0% Confidence Interval
Coefficients Coefficients for B
B Std. Error Beta Lower Upper
Bound Bound
(Constant) 49.782 6.976 7.136 .000 35.818 63.747
1 2
1 Outcome of IQ .266 .065 .474 4.097 .000 .136 .395
test
3
a. Dependent Variable: Outcome of job performance test
• This output tells us that the best possible prediction for job performance given IQ
is 1 2
predicted performance = 49.78 + 0.47* IQ
3• The column “Sig.” holds the significance levels for our predictors. As a rule of
thumb, we say that a b coefficient is statistically significant if its p-value is smaller
than 0.05, here IQ is a statistically significant predictor of job performance
Regression Line
Linear Regression- Model
• We'll try to predict job performance from all other variables by means of a multiple
regression analysis
• Therefore, job performance is our criterion (or dependent variable). IQ, motivation
and social support are our predictors (or independent variables). The model is
illustrated below
 Correlation Analysis
Correlations

Outcome of job Outcome of IQ Outcome of job Outcome of

performance test test motivation test social support
test
Pearson Correlation 1 .474** .635** .397**
Outcome of job performance
Sig. (2-tailed) .000 .000 .002
test
N 60 60 60 60
Pearson Correlation .474** 1 .047 -.092
Outcome of IQ test Sig. (2-tailed) .000 .722 .485
N 60 60 60 60
Pearson Correlation .635** .047 1 .363**
Outcome of job motivation
Sig. (2-tailed) .000 .722 .004
test
N 60 60 60 60
Pearson Correlation .397** -.092 .363** 1
Outcome of social support
Sig. (2-tailed) .002 .485 .004
test
N 60 60 60 60
**. Correlation is significant at the 0.01 level (2-tailed).
 Coefficient Table

• The b coefficients tell us how many units job performance increases for a single unit increase
in each predictor. Like so, 1 point increase on the IQ tests corresponds to 0.47 points increase
on the job performance test. Given only the scores on our predictors, we can predict job
performance by computing
• Job performance = 18.1 + (0.47 x intelligence) + (0.52 x motivation) + (0.25 x social support)
• Importantly, note that all b coefficients are positive numbers; higher IQ is associated with
higher job performance and so on. B coefficients having the “wrong direction” often indicate a
problem with the analysis known as multicollinearity.
 Coefficient Table

• The column “Sig.” holds the significance levels for our predictors. As a rule of
thumb, we say that a b coefficient is statistically significant if its p-value is
smaller than 0.05. All of our b coefficients are statistically significant.
Model Summary

• Model predicts job performance, R denotes the correlation between predicted

and observed job performance. In our case, R = 0.81. Since this is a very high
correlation, our model predicts job performance rather precisely.
• R square is simply the square of R. It indicates the proportion of variance in job
performance that can be “explained” by our three predictors.
• Because regression maximizes R square for our sample, it will be somewhat
lower for the entire population, a phenomenon known as shrinkage. The adjusted
R square estimates the population R square for our model and thus gives a more
realistic indication of its predictive power.
Activity
• Use the ODEL data and test the model below

Brand Image

Perceived Quality
EWOM
Purchasing Intention