Finding trends in Data

Regression
•  A statistical method used to estimate the relationship between a
dependent variable (outcome or response) and one or more
independent variables (predictors, covariates, features).
• Can show the strength between two variables
• Can weed out the variables which are not affecting the dependent
variable
• Most common form is linear regression
Types of Regression
Linear/Simple Regression
• Simple regression is the simplest type of regression. There is only one
independent variable and one dependent variable in simple
regression. Simple regression aims to find the regression line that best
fits the data.
• where b1= regression coefficient and bo is the y-intercept
Steps of Regression
• The first step in regression analysis is identifying the independent and
dependent variables. The independent variable is the factor you are
trying to predict or explain, while the dependent variable is the outcome
you are trying to measure.
• The second step is to calculate the linear regression equation. This
equation will show the relationship between the independent and
dependent variables.
• The third step is to perform a regression analysis of your data. That will
determine whether the linear regression equation fits your data well. If it
is not, you may need to adjust the equation accordingly.
Example of Linear/Simple Regression
Regression Residual
• There is a difference in the original value and the predicted value.
• The lower the value of the residual the better the prediction.
Multiple Regression
• Multiple regression is a type of analysis that uses more than one
predictor variable to predict the dependent variable. In multiple
regression, the model simultaneously fits the data using all the
predictor variables. This allows the model to account for the
interdependencies among the predictor variables.
• The equation is as follows:
Stepwise Regression
• Stepwise regression is a type of multiple regression that uses an
iterative algorithm to find the best data model. This regression
algorithm starts by including all of the predictor variables in the
model.
• Then, it removes the predictor variable that has the smallest p-value.
This process is repeated until you can eliminate no more predictors
without increasing the p-value of the model.
• A small p-value means that events at the extremes are extremely
unlikely to occur under the null hypothesis
• A null hypothesis means that there is no relationship b/w 2 variables
Logistic Regression
• Logistic regression is a statistical method for predicting binary classes.
The outcome or target variable is dichotomous.
• Dichotomous means there are only two possible classes. For example,
it can be used for cancer detection problems. It computes the
probability of an event occurrence. e.g win/loss present/absent
yes/no cases
Lasso Regression
• Lasso Regression is used to find the best-fitting line for a data set. It
uses the “least absolute shrinkage and selection operator,” or Lasso,
to find the line.
• Lasso regression is used when there are many variables in the data
set, and the goal is to find the best-fit line while minimizing the
number of variables.
Polynomial Regression
• Polynomial regression models the relationship between an outcome
variable and one or more predictor variables.
• Polynomial regression uses the power of a polynomial function to fit
data instead of just using a linear function like in Linear Regression.
Polynomial regression can be used for both classification and
regression problems.
Regression Applications
• It helps in devising a functional relationship between two variables.
• It is one of the widely used tools in economic, business and scientific
research where statistical interpretations are highly valued as their
analysis is based more on cause-and-effect relationships.
• It helps in predicting the dependent variable value from the
independent variable values.
• The coefficient of correlation and coefficient of determination can be
established with the help of regression coefficients.
Tips and Tricks
• Always inspect your data to make sure it is appropriate for regression
analysis.
• Ensure you understand the type of regression you use and how it works.
• Choose the correct type of regression for your data set.
• Use a linear regression when there is a linear relationship between the
predictor and outcome variables.
• Use a polynomial regression when there is a non-linear relationship
between the predictor and outcome variables.
• Use a logistic regression when the outcome variable is binary (has only
two possible classes).
• Use stepwise regression to find the best model for your data set.