The document summarizes key concepts in simple linear regression modeling. It introduces the basic linear regression model as yi = β1 + β2xi + ei, where yi is the dependent variable, xi is the independent variable, and ei is the error term. It describes how to estimate the intercept (β1) and slope (β2) parameters using ordinary least squares. It also discusses evaluating goodness of fit using R-squared and root mean squared error, and performing analysis of variance to partition total sum of squares into explained and residual sums of squares. Finally, it briefly mentions extending the model to handle non-linear relationships.
The document summarizes key concepts in simple linear regression modeling. It introduces the basic linear regression model as yi = β1 + β2xi + ei, where yi is the dependent variable, xi is the independent variable, and ei is the error term. It describes how to estimate the intercept (β1) and slope (β2) parameters using ordinary least squares. It also discusses evaluating goodness of fit using R-squared and root mean squared error, and performing analysis of variance to partition total sum of squares into explained and residual sums of squares. Finally, it briefly mentions extending the model to handle non-linear relationships.
The document summarizes key concepts in simple linear regression modeling. It introduces the basic linear regression model as yi = β1 + β2xi + ei, where yi is the dependent variable, xi is the independent variable, and ei is the error term. It describes how to estimate the intercept (β1) and slope (β2) parameters using ordinary least squares. It also discusses evaluating goodness of fit using R-squared and root mean squared error, and performing analysis of variance to partition total sum of squares into explained and residual sums of squares. Finally, it briefly mentions extending the model to handle non-linear relationships.
The document summarizes key concepts in simple linear regression modeling. It introduces the basic linear regression model as yi = β1 + β2xi + ei, where yi is the dependent variable, xi is the independent variable, and ei is the error term. It describes how to estimate the intercept (β1) and slope (β2) parameters using ordinary least squares. It also discusses evaluating goodness of fit using R-squared and root mean squared error, and performing analysis of variance to partition total sum of squares into explained and residual sums of squares. Finally, it briefly mentions extending the model to handle non-linear relationships.
Linear Regression’s used when dependent variable is numeric. Independent variables can be numeric or categorical. The assumed relationship in a linear regression model has the form: 𝑦𝑖 = β1 + β2𝑥𝑖 + 𝑒𝑖 𝑦𝑖 Is the dependent variable (or Response variable) 𝑥𝑖 Is the independent variable (or Explanatory Variables) 𝑒𝑖 Is the error term 2 σ Is the variance of the error term? β1 Is the intercept parameter or coefficient β2 Is the slope parameter or coefficient
The results of estimation if β1 and β2 using Ordinary Least Squares (OLS) method are: ^ ^ β1 = 𝑦 − β2. 𝑥 𝑁 ∑ (𝑥𝑖−𝑥)(𝑦𝑖−𝑦) 𝑖=1 β2 = 𝑁 2 ∑ (𝑥𝑖−𝑥) 𝑖=1
Goodness of Fit To assess the performance of the predictive regression model, we use: 2 ● 𝑅 : representing the squared correlation between the observed and the predicted values by the model. ⇨ The higher the R-square, the better the model. The closer it is to 1, the better the fit. ● Root Mean Squared Error (𝑅𝑀𝑆𝐸) measures he model prediction error ⇨ The lower the RMSE the better the model.
Total Sum of Squares (TSS) = Explained Sum of Squares (ESS) + Residual Sum of Squares (RSS) 2 𝑅 𝑁 2 𝑁 ^ ^2 ∑ (𝑦𝑖−𝑦) ∑ 𝑒𝑖 2 𝑖=1 𝑖=1 𝑅 = 𝑁 2 =1− 𝑁 2 ∑ (𝑦𝑖−𝑦) ∑ (𝑦𝑖−𝑦) 𝑖=1 𝑖=1
2 𝐸𝑆𝑆 𝑅𝑆𝑆 𝑅 = 𝑇𝑆𝑆 =1− 𝑇𝑆𝑆
2 ⇨ The Closer 𝑅 is to 1 the better the fit. RMSE - Randomly split your data into training set (called𝑁1) and test set (called𝑁2). - Build your Regression model using the training set - Make predictions using the test set and compute the model accuracy metrics.
𝑁2 2 ( ∑ 𝑦𝑖−𝑦𝑖 ^ ) 𝑅𝑀𝑆𝐸 = ⎛ 𝑖=1 ⎞ 𝑁2 ⎝ ⎠ ⇨ The lower the RMSE, the better the model.
