Scribd
Upload
13 views4 pages

Hyperparameter Tuning

The document discusses hyperparameter tuning in machine learning, emphasizing the importance of setting external parameters for optimal model performance while internal parameters are managed by the model. It outlines methods for hyperparameter search, including Grid Search and Random Search, and highlights the significance of cross-validation techniques to ensure robust model training and evaluation. Additionally, it provides resources for further reading on hyperparameter tuning and cross-validation methods.

Uploaded by

Shreya Parekh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
13 views4 pages

Hyperparameter Tuning

The document discusses hyperparameter tuning in machine learning, emphasizing the importance of setting external parameters for optimal model performance while internal parameters are managed by the model. It outlines methods for hyperparameter search, including Grid Search and Random Search, and highlights the significance of cross-validation techniques to ensure robust model training and evaluation. Additionally, it provides resources for further reading on hyperparameter tuning and cross-validation methods.

Uploaded by

Shreya Parekh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

11-08-2024

Machine Learning
Hyperparameter Tuning

Hyperparameter Tuning
• Models have internal and external parameters.
• External parameters are called hyperparameters that need to be set
for model to perform optimally
• Internal parameters are taken care by the model itself.
• Hyperparameters generally control the over-fitting and under-fitting
of the model. Optimal hyperparameters often differ for different
datasets.
• For each proposed hyperparameter setting, the model is evaluated
• The hyperparameters that give the best model are selected

1
11-08-2024

Hyperparameter Tuning

• Simple linear regression does not have any hyperparameters


• Other linear regressions such as Lasso, Ridge, and Elastic Net have
hyperparameters
• All other machine learnings algorithms except simple linear
regression have hyperparameters
• For example: Learning rate , a hyperparameter, needs to be set in
such a way that algorithm reaches optimal solution.

Hyperparameters Search
• Grid Search: It picks a grid of hyperparameter values and evaluates each of
them, one at a time. Min and max of each hyperparameter must be defined,
basing on the educated guess.
• Method: GridSearchCV
• Random Search: It chooses the hyperparameter sample combinations
randomly from the grid space, in contrast to Grid Search that tries with every
possible combination supplied to the algorithm. It is very effective in practice
as it reduces computational time but there is not guarantee that we will find
the best result like Grid Search.
• Method: RandomizedSearchCV
• URL: https://www.geeksforgeeks.org/hyperparameter-tuning/
• Complete List: https://neptune.ai/blog/hyperparameter-tuning-in-python-complete-guide

2
11-08-2024

Cross Validation (CV)


• We train algorithm on training data and check performance on unseen
data which is testing data
• Best model: learnt well and not getting to much noise
• To build robust model, cross validation needs to be performed
• CV is a technique in which model is trained on the subset of the dataset
and thereafter is evaluated on the complimentary subset of the dataset
• K-fold Cross Validation, K-fold Time-Series Cross Validation
• Full List: https://scikit-learn.org/stable/modules/cross_validation.html

Cross Validation (CV)


• 5-fold Cross Validation

3
11-08-2024

Cross Validation (CV)


• 4-fold Time-Series Cross Validation

You might also like