Curriculum in Cardiology

Statistics

Linear Regression Analysis Study

Khushbu Kumari, Suniti Yadav
Department of Anthropology, University of Delhi, New Delhi, India
Downloaded from http://journals.lww.com/jpcs by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Abstract
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC1y0abggQZXdtwnfKZBYtws= on 07/17/2024

Linear regression is a statistical procedure for calculating the value of a dependent variable from an independent variable. Linear regression
measures the association between two variables. It is a modeling technique where a dependent variable is predicted based on one or more
independent variables. Linear regression analysis is the most widely used of all statistical techniques. This article explains the basic concepts
and explains how we can do linear regression calculations in SPSS and excel.

Keywords: Continuous variable test, excel and SPSS analysis, linear regression

Introduction of a dependent variable based on the value of at least one

independent variable.
The concept of linear regression was first proposed by
Sir Francis Galton in 1894. Linear regression is a statistical In correlation analysis, the correlation coefficient “r” is a
test applied to a data set to define and quantify the relation dimensionless number whose value ranges from −1 to +1.
between the considered variables. Univariate statistical tests A value toward −1 indicates inverse or negative relationship,
such as Chi‑square, Fisher’s exact test, t‑test, and analysis of whereas towards +1 indicate a positive relation. When there
variance (ANOVA) do not allow taking into account the effect is a normal distribution, the Pearson’s correlation is used,
of other covariates/confounders during analyses (Chang 2004). whereas, in nonnormally distributed data, Spearman’s rank
However, partial correlation and regression are the tests that correlation is used.
allow the researcher to control the effect of confounders The linear regression analysis uses the mathematical
in the understanding of the relation between two variables equation, i.e., y = mx + c, that describes the line of best fit
(Chang 2003). for the relationship between y (dependent variable) and x
In biomedical or clinical research, the researcher often tries (independent variable). The regression coefficient, i.e., r2
to understand or relate two or more independent (predictor) implies the degree of variability of y due to x.[1‑8]
variables to predict an outcome or dependent variable. This
may be understood as how the risk factors or the predictor Significance of Linear Regression
variables or independent variables account for the prediction
The use of linear regression model is important for the
of the chance of a disease occurrence, i.e., dependent
following reasons:
variable. Risk factors (or dependent variables) associate with
a. Descriptive – It helps in analyzing the strength of the
biological (such as age and gender), physical (such as body
association between the outcome (dependent variable)
mass index and blood pressure [BP]), or lifestyle (such as
and predictor variables
smoking and alcohol consumption) variables with the disease.
b. Adjustment – It adjusts for the effect of covariates or the
Both correlation and regression provide this opportunity to
confounders
understand the “risk factors‑disease” relationship (Gaddis and
Gaddis 1990). While correlation provides a quantitative way Address for correspondence: Khushbu Kumari,
of measuring the degree or strength of a relation between two Department of Anthropology, University of Delhi, New Delhi, India.
variables, regression analysis mathematically describes this E‑mail: [email protected]
relationship. Regression analysis allows predicting the value
Access this article online This is an open access journal, and articles are distributed under the terms of the Creative
Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to
Quick Response Code: remix, tweak, and build upon the work non-commercially, as long as appropriate credit
Website: is given and the new creations are licensed under the identical terms.
www.j‑pcs.org
For reprints contact: [email protected]

DOI:
10.4103/jpcs.jpcs_8_18 How to cite this article: Kumari K, Yadav S. Linear regression analysis
study. J Pract Cardiovasc Sci 2018;4:33-6.

© 2018 Journal of the Practice of Cardiovascular Sciences | Published by Wolters Kluwer - Medknow 33
 Kumari and Yadav: Linear regression

c. Predictors – It helps in estimating the important risk factors

Table 1: SPSS table
that affect the dependent variable
Variables entered/removedb d. Extent of prediction – It helps in analyzing the extent of
Model Variables entered Variables removed Method change in the independent variable by one “unit” would
I Age (years)a Enter affect the dependent variable
a
All requested variables entered. bDependent variable: Systolic blood e. Prediction – It helps in quantifying the new cases.
pressure (mmHg)
Downloaded from http://journals.lww.com/jpcs by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Assumptions for Linear Regression

Table 2: SPSS output with R2
The underlying assumptions for linear regression are:
Model summary a. The values of independent variable “x” are set by the
researcher
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC1y0abggQZXdtwnfKZBYtws= on 07/17/2024

Model R R2 Adjusted R2 Std. error of the estimate

I 0.696 a
0.485 0.475 9.10072 b. The independent variable “x” should be measured without
a
Predictors: (constant), age (years) any experimental error
c. For each value of “x,” there is a subpopulation of “y”
Table 3: Analysis of variance with P variables that are normally distributed up and down the
Y‑axis [Figure 1]
ANOVAa d. The variances of the subpopulations of “y” are homogeneous
Model Sum of squares df Mean square F Sig. e. The mean values of the subpopulations of “y” lie on a
I Regression 4128.118 1 4128.118 49.843 0.000b straight line, thus implying the assumption that there exists
Residual 4389.628 53 82.823
Total 8517.745 54 ‑
a
Predictors: (Constant), Age (years). bDependent variable: Systolic blood
pressure (mmHg)

Figure 2: Starting Data Analysis ToolPak. Click the OFFICE button and
choose Excel options.

Figure 1: Scatter plot of systolic blood pressure versus age.

Figure 4: The regression screen. Choose Data > Data Analysis >
Figure 3: The Tool Pak. Choose Add Ins > Choose Analysis ToolPak Regression. Input y Range: A1:A8. Input X Range: B1:C8. Check Labels,
and select Go. Residuals, Output Range as A50.

34 Journal of the Practice of Cardiovascular Sciences ¦ Volume 4 ¦ Issue 1 ¦ January-April 2018

 Kumari and Yadav: Linear regression

Table 4: SPSS equation variables

Coefficientsa
Model Unstandardised coefficients Standardised coefficents t Sig. 95% Confidence interval for B
B Std. error Beta Lower bound Upper bound
I (Constant) 115.706 7.999 0.696 14.465 0.000 99.662 131.749
Downloaded from http://journals.lww.com/jpcs by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Age (years) 1.051 0.149 7.060 0.000 0.752 1.350

a
Dependent variable: Systolic blood pressure (mmHg)

variables through statistical estimation. For example, to know

Table 5: Excel data set
whether the likelihood of having high systolic BP (SBP) is
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC1y0abggQZXdtwnfKZBYtws= on 07/17/2024

A B C influenced by factors such as age and weight, linear regression

Sale of medicine Price (rupees TV advertisements would be used. The variable to be explained, i.e., SBP is called the
(number of units) per tablet) budget (rupees) dependent variable, or alternatively, the response variables that
(dependent (independent (independent explain it age, weight, and sex are called independent variables.
variable) variable) variable)
8888 22 33,333
4444 55 2222
How to Calculate Linear Regression?
5555 33 4444 Linear regression can be tested through the SPSS statistical
7777 22 5555 software (IBM Corp. Released 2011. IBM SPSS Statistics
6666 55 33,333 for Windows, Version 20.0. Armonk, NY: IBM Corp.) in five
7777 33 22,222 steps to analyze data using linear regression. Following is the
5555 44 11,111 procedure followed Tables 1-4:
Testing whether sale of medicine depends on price and advertisements
Click Analyze > Regression > Linear > then select Dependent
and Independent variable > OK (enter).
Table 6: Summary output Example 1 – Data (n = 55) on the age and the SBP were
Regression Values Explanation collected and linear regression model would be tested
statistics to predict BP with age. After checking the normality
Multiple R 0.96332715 Correlation coefficient: 1 means assumptions for both variables, bivariate correlation is tested
perfect correlation and 0 means none
(Pearson’s correlation = 0.696, P < 0.001) and a graphical
R2 0.927999198 Coefficient of determination: How
scatter plot is helpful in that case [Figure 2].
many points fall on the regression line.
Here, 92% points fall within the line Now to check the linear regression, put SBP as the dependent
Adjusted R2 0.891998797 Adjusted R2: Adjusts for multiple and age as the Independent variable.
variables, use with multiple variables
SE 516.3490153 This indicates the dependent and independent variables
Observations 7 included in the test.
Pearson’s correlation between SBP and age is given (r = 0.696).
a linear relation between the dependent and the independent R2 = 0.485 which implies that only 48.5% of the SBP is
variables explained by the age of a person.
f. All the values of “y” are independent from each other,
though dependent on “x.” The ANOVA table shows the “usefulness” of the linear
regression model with P < 0.05.
Coefficient of Determination, R2 This provides the quantification of the relationship between
age and SBP. With every increase of 1 year in age, the SBP
The coefficient of determination is the portion of the total variation
in the dependent variable that can be explained by variation in (on the average) increases by 1.051 (95% confidence interval
the independent variable(s). When R2 is + 1, there exists a perfect 0.752–1.350) units, P < 0.001. The constant here has no
linear relationship between x and y, i.e., 100% of the variation “practical” meaning as it gives the value of the SBP when age = 0.
in y is explained by variation in x. When it is 0< R2 <1, there is a Further, if more than one independent variable is added,
weaker linear relationship between x and y, i.e., some, but not all the linear regression model would adjust for the effect of
of the variation in y is explained by variation in x. other dependent variables when testing the effect of one
variable.
Linear Regression in Biological Data Analysis Example 2 – If we want to see the genetic effect of variables,
In biological or medical data, linear regression is often used to i.e., the effect of increase in per allele dose of any genetic
describe relationships between two variables or among several variant (mutation) on the disease or phenotype, linear

Journal of the Practice of Cardiovascular Sciences ¦ Volume 4 ¦ Issue 1 ¦ January-April 2018 35

 Kumari and Yadav: Linear regression

Table 7: Analysis of variance

df SS MS F Significance F
Regression 2 13,745,386.778 6,872,693.389 25.777 0.005
Residual 4 1,066,465.222 266,616.306
Total 6 14,811,852.000
Coefficients SE t statistic P
Downloaded from http://journals.lww.com/jpcs by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Intercept 8533.99 661.21 12.91 0.000

Price −79.98 15.08 −5.31 0.006
TV adds 0.07 0.02 4.59 0.010
Equation=Sales: 8533.9 − 79.9 (price) + 0.07 (TV adds)
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC1y0abggQZXdtwnfKZBYtws= on 07/17/2024

If we now look at the last three columns, we can create an equation: Equation = Sales: 8533.9 − 79.9 (price) + 0.07 (TV ads). Therefore as the price goes up,
the sales go down (P = 0.006), and the addition of TV ads is less (0.07 multiplied by advertisement money with a P value of 0.01). SE: Standard error, MS:
Mean square, SS: Sum of square

regression is used in a similar way as described above. The the form of an equation. In this article, we have used simple
three genotypes, i.e., normal homozygote AA, heterozygote examples and SPSS and excel to illustrate linear regression
AB and homozygote mutant BB may be coded as 1, 2, and 3, analysis and encourage the readers to analyze their data by
respectively. The test may be preceded, and in a similar way, these techniques.
the unstandardized coefficient (β) would explain the effect on Financial support and sponsorship
the dependent variable with per allele dose increase. Nil.
Example 3 – Using Excel to see the relationship between
Conflicts of interest
sale of medicine with the price of the medicine and TV There are no conflicts of interest.
advertisements.
Table 5 contains data which can be entered into an Excel sheet. References
Follow instructions as shown in figures 2-4. 1. Schneider A, Hommel G, Blettner M. Linear regression analysis:
Part 14 of a series on evaluation of scientific publications. Dtsch Arztebl
As shown in Table 6, Multiple R is the Correlation Coefficient, Int 2010;107:776‑82.
where 1 means a perfect correlation and zero means none. 2. Freedman DA. Statistical Models: Theory and Practice. Cambridge,
R Square is the coefficient of determination which here means USA: Cambridge University Press; 2009.
3. Chan YH. Biostatistics 201: Linear regression analysis. Age (years).
that 92% of the variation can be explained by the variables. Singapore Med J 2004;45:55-61.
Adjusted R square adjusts for multiple variables and should be 4. Chan YH. Biostatistics 103: Qualitative data – Tests of independence.
used here. here. Table 7 shows how to create a linear regression Singapore Med J 2003;44:498‑503.
5. Gaddis ML, Gaddis GM. Introduction to biostatistics: Part 6, correlation
equation from the data. and regression. Ann Emerg Med 1990;19:1462‑8.
6. Mendenhall W, Sincich T. Statistics for Engineering and the Sciences.
Conclusion 7.
3rd ed. New York: Dellen Publishing Co.; 1992.
Panchenko D. 18.443 Statistics for Applications, Section 14, Simple
The techniques for testing the relationship between two Linear Regression. Massachusetts Institute of Technology: MIT
variables are correlation and linear regression. Correlation OpenCourseWare; 2006.
8. Elazar JP. Multiple Regression in Behavioral Research:
quantifies the strength of the linear relationship between a pair Explanation and Prediction. 2nd ed. New York: Holt, Rinehart and
of variables, whereas regression expresses the relationship in Winston; 1982.

36 Journal of the Practice of Cardiovascular Sciences ¦ Volume 4 ¦ Issue 1 ¦ January-April 2018