Scribd
Upload
53 views8 pages

Hyperparameters and Parameters

The document discusses the difference between parameters and hyperparameters in machine learning models. Parameters are internal to a model and are estimated from data, while hyperparameters are external and set before training to determine aspects like the network structure and training process. The document also covers techniques for tuning hyperparameters like random search, grid search, Bayesian optimization, and gradient-based methods.

Uploaded by

SANJIDA AKTER
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
53 views8 pages

Hyperparameters and Parameters

The document discusses the difference between parameters and hyperparameters in machine learning models. Parameters are internal to a model and are estimated from data, while hyperparameters are external and set before training to determine aspects like the network structure and training process. The document also covers techniques for tuning hyperparameters like random search, grid search, Bayesian optimization, and gradient-based methods.

Uploaded by

SANJIDA AKTER
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 8

Hyperparameters &

Parameters
Soft Computing
Parameters
❖ Internal to the model and whose value can be estimated from data
❖ Required by the model when making predictions
❖ Learned from data
❖ Often not set manually by the practitioner
❖ Often saved as part of the learned model
❖ Define the skill of the model on your problem
❖ Examples:
➢ Weight matrix of a model
➢ Coefficients in a linear regression
➢ Depth of a Decision tree
Parameters
Parameters
3×4+4×2+1×4+1×2

= 12 + 8 + 4 + 2

= 26
Hyperparameters
External to the model and whose value cannot be estimated from data
❖ Set before training
❖ Determines the network structure
➢ used in processes to help estimate model parameters
❖ Determine how the network is trained
❖ Tuned for a given predictive modeling problem
❖ Examples
➢ Related to Network Structure
■ Dropout, # of Hidden Layer, Weight Initialization, Active Function, etc
➢ Related to Training
■ Epoch, Batch Size, Optimizer, Learning Rate
Tuning Techniques
❖ Random Search
● Randomly chooses combinations of hyperparameters from searchable
space (all possible combinations).

❖ Grid Search
● Tests all possible combinations of hyperparameters of given Machine
Learning algorithm
Tuning Techniques
❖ Bayesian optimizer
● Based upon Bayes Rule and considers previously known knowledge to
help narrow down the search space of good hyperparameter
combinations.

❖ Gradient-based
● A methodology to optimise several hyperparameters, based on the
computation of the gradient of a machine learning model selection
criterion with respect to the hyperparameters.
Useful Links

https://machinelearningmastery.com/difference-between-a-parameter-and-a-hyper

parameter/

https://towardsdatascience.com/what-are-hyperparameters-and-how-to-tune-the-h

Yperparameters-in-a-deep-neural-network-d0604917584a

https://towardsdatascience.com/number-of-parameters-in-a-feed-forward-neural-
network-4e4e33a53655

You might also like