Regression Analysis Is A Statistical Process For Estimating The Relationships Among
Regression Analysis Is A Statistical Process For Estimating The Relationships Among
o It includes many techniques for modeling and analyzing several variables, when the
focus is on the relationship between a dependent variable and one or
more independent variables (or 'predictors').
o The estimation target is a function of the independent variables called the regression
function.
o In regression analysis, it is also of interest to characterize the variation of the
dependent variable around the regression function which can be described by
a probability distribution.
o Regression analysis is widely used for prediction and forecasting, where its use has
substantial overlap with the field of machine learning.
o The performance of regression analysis methods in practice depends on the form of
the data generating process, and how it relates to the regression approach being used
o
Two types
1. Nonlinear regression
2. Linear regression
1. Nonlinear regression
o In statistics, nonlinear regression is a form of regression analysis in which
observational data are modeled by a function which is a nonlinear combination of the
model parameters and depends on one or more independent variables.
o The data are fitted by a method of successive approximations.
o Curve Fitting Procedure:
1. Plot your variables to visualize the relationship
a. What curve does the pattern resemble?
b. What might alternative options be?
2. Decide on the curves you want to compare and run a non-linear regression curve
fitting a. You will have to estimate your parameters from your curve to have starting
values for your curve fitting function.
3. Once you have parameters for your curves compare models with AIC.
4. Plot the model with the lowest AIC on your point data to visualize fit.
2. Linear regression
3. Logistic Regression
Thus, it treats the same set of problems as probit regression using similar techniques,
with the latter using a cumulative normal distribution curve instead.
Equivalently, in the latent variable interpretations of these two methods, the first
assumes a standard logistic distribution of errors and the second a standard normal
distribution of errors.
Logistic regression can be seen as a special case of generalized linear model and thus
analogous to linear regression.
In particular the key differences of these two models can be seen in the following two
features of logistic regression. First, the conditional distribution
is a Bernoulli
distribution rather than a Gaussian distribution, because the dependent variable is
binary. Second, the predicted values are probabilities and are therefore restricted to
(0,1) through the logistic distribution function because logistic regression predicts
the probability of particular outcomes.