Scribd
Upload
8 views

Lecture 3 Logistic Regression

The document discusses logistic regression models. It explains that logistic regression predicts categorical dependent variables, unlike linear regression which predicts continuous variables. It provides examples of binary and multi-class logistic regression problems. The key aspects of logistic regression covered are: the sigmoid curve shape, estimating logistic regression coefficients using maximum likelihood, and interpreting the results as probabilities between 0 and 1. Sample breast cancer cell data is presented to illustrate logistic regression analysis.

Uploaded by

iamboss086
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
8 views

Lecture 3 Logistic Regression

The document discusses logistic regression models. It explains that logistic regression predicts categorical dependent variables, unlike linear regression which predicts continuous variables. It provides examples of binary and multi-class logistic regression problems. The key aspects of logistic regression covered are: the sigmoid curve shape, estimating logistic regression coefficients using maximum likelihood, and interpreting the results as probabilities between 0 and 1. Sample breast cancer cell data is presented to illustrate logistic regression analysis.

Uploaded by

iamboss086
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Logistic Regression Model

Dr. Kavita Pabreja


Associate Professor
Deptt. of Computer Applications-MSI

1
Linear vs. Logistic Regression
• There are two main
forms of regression
analysis.
• Linear regression has a
continuous set of
results that can easily
be mapped on a graph
as a straight line.
Logistic Regression
• Logistic regressions are non-linear and are portrayed on a
graph with a curved shape called a sigmoid.
• Instead of a continuous set of results, a logistical regression
has two or more categories for data.
• If you want to learn the probability of getting a job based
on how many applications you submit, whether you get a
job is a discrete number of variables.
• Therefore, you would use a logistic regression program for
that data.
Another example
• For instance, a bank might like to develop an estimated regression
equation for predicting whether a person will be approved for a credit
card or not
• The dependent variable can be coded as y =1 if the bank approves the
request for a credit card and y = 0 if the bank rejects the request for a
credit card.
• Using logistic regression we can estimate the probability that the bank
will approve the request for a credit card given a particular set of values
for the chosen independent variables.

4
Types of Logistic Regression
• Binary: A binary output is a variable where there are only two possible
outcomes. These outcomes must be opposite of each other and
mutually exclusive. Whether you have a cat is a binary output.
• Multi-class: A multi-class has three or more categories without any
numerical value, though they usually have a numerical stand-in for
datasets. For example, you could ask if someone if they have a dog, cat
or fish.
• Ordinal: An ordinal output also has three or more categories, though
they're in a ranked output. For example, asking a friend if they like
cats, dislike cats or don't care is an ordinal output.
Estimating the Logistic Regression Equation
• In simple linear and multiple regression the least squares method is used to
compute b0, b1, . . . , bp as estimates of the model parameters (β0 , … , βp)
• The nonlinear form of the logistic regression equation makes the method of
computing estimates more complex
• The estimated logistic regression equation is

Here, y hat provides an estimate of the probability that y = 1, given a particular


set of values for the independent variables.
6
Logistic Regression Equation
• If the two values of the dependent variable y are coded as 0 or 1, the
value of E( y) in equation given below provides the probability that y = 1
given a particular set of values for the independent variables x1, x2, . . . , xp.

7
Logistic Regression Equation
• Because of the interpretation of E( y) as a probability, the logistic
regression equation is often written as follows

8
Logistic regression equation for β0 and β1

9
Logistic regression equation for β0 and β1

• Note that the graph is S-shaped.


• The value of E( y) ranges from 0 to 1, with the value of E( y) gradually
approaching 1 as the value of x becomes larger and the value of E( y)
approaching 0 as the value of x becomes smaller.
• Note also that the values of E( y), representing probability, increase fairly
rapidly as x increases from 2 to 3.
• The fact that the values of E( y) range from 0 to 1 and that the curve is S-
shaped makes equation ideally suited to model the probability the
dependent variable is equal to 1.

10
Example

• Breast Cancer Wisconsin (Diagnostic) Data Set


• Let us consider Features computed from a digitized image of a fine
needle aspiration (FNA) biopsy.
• Features describe characteristics of the cell nuclei present in the image.
• There are 569 records, 30 input features and 1 output columns.

11
Sample dataset

concave fractal_di fractal_di smoothn compactn concave fractal_di


radius_m texture_ perimeter area_me smoothn compactn concavity points_m symmetry mension_ texture_s perimeter smoothn compactn concavity concave symmetry mension_ radius_w texture_ perimeter area_wor ess_wors ess_wors concavity points_w symmetry mension_
id diagnosis ean mean _mean an ess_mean ess_mean _mean ean _mean mean radius_se e _se area_se ess_se ess_se _se points_se _se se orst worst _worst st t t _worst orst _worst worst
842302 M 17.99 10.38 122.8 1001 0.1184 0.2776 0.3001 0.1471 0.2419 0.07871 1.095 0.9053 8.589 153.4 0.006399 0.04904 0.05373 0.01587 0.03003 0.006193 25.38 17.33 184.6 2019 0.1622 0.6656 0.7119 0.2654 0.4601 0.1189
842517 M 20.57 17.77 132.9 1326 0.08474 0.07864 0.0869 0.07017 0.1812 0.05667 0.5435 0.7339 3.398 74.08 0.005225 0.01308 0.0186 0.0134 0.01389 0.003532 24.99 23.41 158.8 1956 0.1238 0.1866 0.2416 0.186 0.275 0.08902
84300903 M 19.69 21.25 130 1203 0.1096 0.1599 0.1974 0.1279 0.2069 0.05999 0.7456 0.7869 4.585 94.03 0.00615 0.04006 0.03832 0.02058 0.0225 0.004571 23.57 25.53 152.5 1709 0.1444 0.4245 0.4504 0.243 0.3613 0.08758
84348301 M 11.42 20.38 77.58 386.1 0.1425 0.2839 0.2414 0.1052 0.2597 0.09744 0.4956 1.156 3.445 27.23 0.00911 0.07458 0.05661 0.01867 0.05963 0.009208 14.91 26.5 98.87 567.7 0.2098 0.8663 0.6869 0.2575 0.6638 0.173
84358402 M 20.29 14.34 135.1 1297 0.1003 0.1328 0.198 0.1043 0.1809 0.05883 0.7572 0.7813 5.438 94.44 0.01149 0.02461 0.05688 0.01885 0.01756 0.005115 22.54 16.67 152.2 1575 0.1374 0.205 0.4 0.1625 0.2364 0.07678
843786 M 12.45 15.7 82.57 477.1 0.1278 0.17 0.1578 0.08089 0.2087 0.07613 0.3345 0.8902 2.217 27.19 0.00751 0.03345 0.03672 0.01137 0.02165 0.005082 15.47 23.75 103.4 741.6 0.1791 0.5249 0.5355 0.1741 0.3985 0.1244
844359 M 18.25 19.98 119.6 1040 0.09463 0.109 0.1127 0.074 0.1794 0.05742 0.4467 0.7732 3.18 53.91 0.004314 0.01382 0.02254 0.01039 0.01369 0.002179 22.88 27.66 153.2 1606 0.1442 0.2576 0.3784 0.1932 0.3063 0.08368
84458202 M 13.71 20.83 90.2 577.9 0.1189 0.1645 0.09366 0.05985 0.2196 0.07451 0.5835 1.377 3.856 50.96 0.008805 0.03029 0.02488 0.01448 0.01486 0.005412 17.06 28.14 110.6 897 0.1654 0.3682 0.2678 0.1556 0.3196 0.1151

A tumor can be malignant (cancerous) or benign (not cancerous).


Let’s see python code

ROC and Logistic regression.ipynb

Path is D:\Subjects of BCA or BBA\sub machine learning\ML codes


Thank You

14

You might also like