Correlation and Regression Chapter 10

Linear Regression Analysis:

An estimated line (best fit line) from which we can find dependent variable in the
presence of independent variable.
“Linear regression analysis is a statistical technique for calculating a line that best fits the points in the
scatter diagram. It is the line that minimized the sum of the squared deviations or residuals between
line and actual values. In other words, it is a line that makes the vertical distances from the points to
this line as small as possible. The method used is call least square method. The best fit or the link
(if established) can support forecasting and planning.”
Correlation: Degree of relationship between two variables.
Regression Lines/Estimated Lines/Lines of best fit.
 ̂ ( Y on X)
 ̂ ( X on Y)
 b represents slope/gradient/regression-coefficient/response rate
 a represents fixed value/y-intercept/x-intercept
Formula List:
(Y on X)


∑ ∑ ∑

∑ ∑
∑ ̅ ̅

∑ ̅

 ̅ ̅ ̅ =∑ and ̅ =∑

(X on Y)

∑ ∑ ∑

∑ ∑
∑ ̅ ̅

∑ ̅

 ̅ ̅

M. SHOAIB(M.PHIL) 1
 Correlation and Regression Chapter 10

Co-relation Coefficient r


√
∑ ∑ ∑

√ ∑ ∑ ∑ ∑
∑ ̅ ̅

√ ∑ ̅ ∑ ̅

 √
∑ ̅ ̅


Example: For the data

Sale(X) 50 60 65 70

Advertisement(Y) 2 4 8 10

Find i) Regression Coefficient b

ii) Y-intercept a
iii) Regression line/best fit line/Estimated line
Example:
Sale(X) 10 12 15 20

Advertisement(Y) 2 3 1 4

Find i) Regression Coefficient b

ii) Y-intercept a
iii) Regression line/best fit line/Estimated line

M. SHOAIB(M.PHIL) 2
 Correlation and Regression Chapter 10

Question: If ̅ ,̅ , , , and =10

Find i) ii) Two Regression lines
Sol. , √
Question: Following information is related to income and expenditures of 10 families.
∑ , ∑ , ∑ 1200 , ∑ ∑
Find i) Regression line ii) Coefficient of Co relation
Sol.
∑ ∑ ∑
, ̅ ̅
∑ ∑
Then, ̂ (5.296 + 0.104x)
∑ ∑ ∑
And, , or √
√ ∑ ∑ ∑ ∑

Question: If Var. of x = 36, and Regressions are

5x – y = 22 (x on y)
49x – 30y = 18 (y on x)
Find: i) ̅ ̅ (Solving Simultaneously) as regression lines always passes through mean. (6.36, 9.78)
ii) Co-relation Co-efficient (we will find b from given equations by arranging them in standard
form). (1/5 and 49/30)
iii) , ( ) (17.1)

Question: Regression co-efficient are given, find correlation co-efficient.

i) 0.59 and 0.21
Sol. √

ii) – 0.30 and – 0.70

Sol. √ If slopes are negative r will be negative.

iii) – 0.45 and 0.20

Sol. √
Question: Following are results of height and weight of 200 students.
̅ , ̅ , r = 0.55, = 8kg , =8cm
Find line of estimation. If Bilal weight is 60kg. Estimate his height
Sol.
Height Dependent (as to find)
Weight Independent
Height = a + b weight
̂
(0.55)
̅ ̅ (127)
So, ̂
= 127 + 0.55(60) = 160cm

M. SHOAIB(M.PHIL) 3
 Correlation and Regression Chapter 10

Co-Efficient of Determination: (r2)

 It is square of co-efficient of correlation
 It describes variation in data
 If r = 0.143 then co-efficient of determination r2= (0.143)2 = 0.02 = 2% which means 2% change
in y due to x (independent)
 If r2= 0.6 = 60% it means 60% change in y due to x and remaining 40% change due to other
factors.
Range:
r -1 to +1
2
r 0 to 1

Rank Correlation/Spearman’s Rank correlation Co-efficient:

∑ ,

 d represents difference between rank values

 n represents number of pairs of data
Example:

X Rx Y Ry d= Rx - Ry d2
10 3 6 2 1 1
9 2 13 4 -2 4
7 1 2 1 0 0
15 4 8 3 1 1
Σd2 =6

∑
, where n = 4
= 0.4
Tied-Ranks:
Calculate Rank/Spearman’s rank correlation of the following given data

X 10 8 12 15 8 10
Y 7 4 6 7 9 8

Solution:

∑
{ }

M. SHOAIB(M.PHIL) 4
 Correlation and Regression Chapter 10

 d represents difference between rank values

 n represents number of pairs of data
 m represents multiplicities of the rank. (number of times the value repeated)

X Rx Y Ry d d2
10 3.5 7 3.5 0 0
8 5.5 4 6 - 0.5 0.25
12 2 6 5 -3 9
15 1 7 3.5 -2.5 6.25
8 5.5 9 1 4.5 20.25
10 3.5 8 2 1.5 2.25
Σd2=38

∑
{ }

 m 2 for value 10 as it repeated 2 times, 2 for value 8 as it repeated 2 times,

2 for value 7 as it also repeated 2 times in above data.
 n 6 as number of pairs in above data are 6.
Substituting the values in above formula we will get “r”
Quick Facts:
 Regression lines always pass through means. ̅ ̅
 Sum of deviations of actual values from estimated value is always zero that is ∑ ̂ =0
 b is independent of change of origin.
 b is affected by change of scale.
 If b is positive then regression will be upward
 If b is negative then regression will be downward.
 If b is zero then it is a straight horizontal line with no or zero slope
 In perfect correlation

 r is independent of change of origin and change of scale.
 r has no unit
 Two lines are perpendicular to each other if r = 0

M. SHOAIB(M.PHIL) 5
 Correlation and Regression Chapter 10

Scattered Diagram:

Perfect Positive correlation r=1 Perfect Negative Correlation r=-1

M. SHOAIB(M.PHIL) 6
 Correlation and Regression Chapter 10

Quick Facts:
1. Visual Representation: Scatter plots visually display the relationship between two numerical variables.
They are particularly useful for identifying patterns or trends in data.
2. Data Points: Each point on the plot represents a single data point with a specific value for both the x and
y variables.
3. Correlation: Scatter plots help to determine if there is a correlation or relationship between the two
variables. Positive correlation means that as one variable increases, the other tends to increase as well.
Negative correlation means that as one variable increases, the other tends to decrease.
4. Outliers: Outliers, or data points that deviate significantly from the rest of the data, can often be
identified in a scatter plot.
5. Patterns: Different patterns in scatter plots can indicate different types of relationships. For example, a
linear pattern suggests a linear relationship, while a more random pattern suggests a lack of correlation.
6. Axes: The x-axis represents one variable and the y-axis represents the other. Each axis is labeled to
indicate what it represents.
7. Title and Labels: A good scatter plot includes a title that describes what the plot is about. It also includes
labels for the x and y axes.
8. Range and Scale: The range and scale of the axes are important. They need to be appropriately chosen
to clearly show the relationship between the variables.
9. Strength of Relationship: The tighter the points cluster around a line (in the case of a linear
relationship), the stronger the relationship between the variables.
10. No Causation: It's important to note that while scatter plots show correlation, they do not prove
causation. Just because two variables have a relationship does not mean one causes the other to
change.
Limitations:
a. Only for Numerical Data: Scatter plots are primarily suited for visualizing relationships between two
numerical variables. They are not effective for categorical data.

b. Limited to Two Variables: Scatter plots can only display the relationship between two variables. If there are
more than two relevant variables, additional plots or techniques are needed.

c. Doesn't Prove Causation: As mentioned earlier, a correlation in a scatter plot does not imply causation. Other
factors or variables may be influencing the observed relationship.

d. Sensitive to Outliers: Outliers can significantly impact the interpretation of a scatter plot. They can distort the
appearance of the relationship, especially in smaller datasets.

e. Scaling Issues: Improper scaling of axes can distort the perception of the relationship. It's important to choose
appropriate ranges and scales to accurately represent the data.

f. Non-Linear Relationships: While scatter plots are excellent for identifying linear relationships, they may not
capture more complex non-linear relationships without additional analysis or transformation of the data.

g. Context Dependency: The interpretation of a scatter plot can be highly dependent on the context of the data
and the specific domain of study. What may be meaningful in one context may not be in another.

M. SHOAIB(M.PHIL) 7
 Correlation and Regression Chapter 10

Typical Data:
1. Clustered Around the Line of Best Fit: In a scatter plot showing a strong linear relationship, typical data
points will be closely clustered around the line of best fit.
2. Follow the General Trend: Typical data points tend to follow the general trend or pattern observed in
the scatter plot.
3. Representative of the Majority: They are representative of the majority of the data and contribute to
the overall pattern or trend.
4. Low Residuals: For a regression line, typical data points will have lower residuals (vertical distances from
the data points to the line), indicating that they are well-represented by the model.
5. Contribute to Correlation: They contribute to the correlation coefficient, which measures the strength
and direction of the linear relationship between the variables.
Atypical Data:
1. Outliers: Atypical data points are outliers, meaning they deviate significantly from the general trend or
pattern observed in the scatter plot.
2. Not Representative: They do not represent the majority of the data and may have unique
characteristics or circumstances.
3. High Residuals: For a regression line, atypical data points will have higher residuals, indicating that they
do not conform well to the model.
4. May Indicate Errors or Special Cases: Outliers can sometimes indicate errors in data collection or
represent special cases that require further investigation.
5. Influence on Correlation: Depending on their position and number, outliers can have a significant
influence on the calculated correlation coefficient, potentially inflating or deflating its value.
6. Can Indicate Interesting Phenomena: In some cases, atypical data points may represent interesting or
important phenomena that merit further study or consideration.

Typical Data: Typical data points are those that fall within
the expected or common range of values for a given
dataset. They represent the usual or average behavior of
the data.

Atypical Data: Atypical data points are outliers or

exceptions that significantly deviate from the expected
range of values in a dataset. They represent unusual or
uncommon occurrences.

M. SHOAIB(M.PHIL) 8