Scribd
Upload
16 views

Linear Regression With Multiple Variables

Linear regression can be performed with multiple variables (features). Gradient descent can be used to minimize the cost function and calculate the parameters for the regression. Feature scaling is important to make sure features are on a similar scale. The learning rate used in gradient descent should be chosen carefully - if it is too small, convergence will be slow, and if it is too large, gradient descent may not converge properly.

Uploaded by

abdala sabry
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
16 views

Linear Regression With Multiple Variables

Linear regression can be performed with multiple variables (features). Gradient descent can be used to minimize the cost function and calculate the parameters for the regression. Feature scaling is important to make sure features are on a similar scale. The learning rate used in gradient descent should be chosen carefully - if it is too small, convergence will be slow, and if it is too large, gradient descent may not converge properly.

Uploaded by

abdala sabry
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Linear Regression with

multiple variables

Multiple features

Machine Learning
Multiple features (variables).

Size (feet2) Price ($1000)

2104 460
1416 232
1534 315
852 178
… …

Andrew Ng
Multiple features (variables).
Size (feet2) Number of Number of Age of home Price ($1000)
bedrooms floors (years)

2104 5 1 45 460
1416 3 2 40 232
1534 3 2 30 315
852 2 1 36 178
… … … … …
Notation:
= number of features
= input (features) of training example.
= value of feature in training example.

Andrew Ng
Hypothesis:
Previously:

Andrew Ng
For convenience of notation, define .

Multivariate linear regression.


Andrew Ng
Linear Regression with
multiple variables

Gradient descent for


multiple variables
Machine Learning
Hypothesis:
Parameters:
Cost function:

Gradient descent:
Repeat

(simultaneously update for every )


Andrew Ng
New algorithm :
Gradient Descent
Repeat
Previously (n=1):
Repeat
(simultaneously update for
)

(simultaneously update )

Andrew Ng
Linear Regression with
multiple variables
Gradient descent in
practice I: Feature Scaling

Machine Learning
Feature Scaling
Idea: Make sure features are on a similar scale.
E.g. = size (0-2000 feet2) size (feet2)
= number of bedrooms (1-5)
number of bedrooms

Andrew Ng
Feature Scaling
Get every feature into approximately a range.

Andrew Ng
Mean normalization
Replace with to make features have approximately zero mean
(Do not apply to ).
E.g.

Andrew Ng
Linear Regression with
multiple variables

Gradient descent in
practice II: Learning rate
Machine Learning
Gradient descent

- “Debugging”: How to make sure gradient


descent is working correctly.
- How to choose learning rate .

Andrew Ng
Making sure gradient descent is working correctly.

Example automatic
convergence test:

Declare convergence if
decreases by less than
in one iteration.
0 100 200 300 400
No. of iterations
Andrew Ng
Making sure gradient descent is working correctly.
Gradient descent not working.
Use smaller .

No. of iterations

No. of iterations No. of iterations

- For sufficiently small , should decrease on every iteration.


- But if is too small, gradient descent can be slow to converge.
Andrew Ng
Summary:
- If is too small: slow convergence.
- If is too large: may not decrease on
every iteration; may not converge.

To choose , try

Andrew Ng

You might also like