tfm.optimization.OptimizerFactory
Stay organized with collections
Save and categorize content based on your preferences.
Optimizer factory class.
tfm.optimization.OptimizerFactory(
config: tfm.optimization.OptimizationConfig
)
This class builds learning rate and optimizer based on an optimization config.
To use this class, you need to do the following:
(1) Define optimization config, this includes optimizer, and learning rate
schedule.
(2) Initialize the class using the optimization config.
(3) Build learning rate.
(4) Build optimizer.
This is a typical example for using this class:
params = {
'optimizer': {
'type': 'sgd',
'sgd': {'momentum': 0.9}
},
'learning_rate': {
'type': 'stepwise',
'stepwise': {'boundaries': [10000, 20000],
'values': [0.1, 0.01, 0.001]}
},
'warmup': {
'type': 'linear',
'linear': {'warmup_steps': 500, 'warmup_learning_rate': 0.01}
}
}
opt_config = OptimizationConfig(params)
opt_factory = OptimizerFactory(opt_config)
lr = opt_factory.build_learning_rate()
optimizer = opt_factory.build_optimizer(lr)
Args |
config
|
OptimizationConfig instance contain optimization config.
|
Methods
build_learning_rate
View source
build_learning_rate()
Build learning rate.
Builds learning rate from config. Learning rate schedule is built according
to the learning rate config. If learning rate type is consant,
lr_config.learning_rate is returned.
Returns |
tf.keras.optimizers.schedules.LearningRateSchedule instance. If
learning rate type is consant, lr_config.learning_rate is returned.
|
build_optimizer
View source
build_optimizer(
lr: Union[tf.keras.optimizers.schedules.LearningRateSchedule, float],
gradient_aggregator: Optional[Callable[[List[Tuple[tf.Tensor, tf.Tensor]]], List[Tuple[tf.Tensor,
tf.Tensor]]]] = None,
gradient_transformers: Optional[List[Callable[[List[Tuple[tf.Tensor, tf.Tensor]]], List[Tuple[tf.
Tensor, tf.Tensor]]]]] = None,
postprocessor: Optional[Callable[[tf.keras.optimizers.Optimizer], tf.keras.optimizers.
Optimizer]] = None,
use_legacy_optimizer: bool = True
)
Build optimizer.
Builds optimizer from config. It takes learning rate as input, and builds
the optimizer according to the optimizer config. Typically, the learning
rate built using self.build_lr() is passed as an argument to this method.
Args |
lr
|
A floating point value, or a
tf.keras.optimizers.schedules.LearningRateSchedule instance.
|
gradient_aggregator
|
Optional function to overwrite gradient aggregation.
|
gradient_transformers
|
Optional list of functions to use to transform
gradients before applying updates to Variables. The functions are
applied after gradient_aggregator. The functions should accept and
return a list of (gradient, variable) tuples. clipvalue, clipnorm,
global_clipnorm should not be set when gradient_transformers is passed.
|
postprocessor
|
An optional function for postprocessing the optimizer. It
takes an optimizer and returns an optimizer.
|
use_legacy_optimizer
|
A boolean that indicates if using legacy optimizers.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates. Some content is licensed under the numpy license.
Last updated 2024-02-02 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-02-02 UTC."],[],[],null,["# tfm.optimization.OptimizerFactory\n\n\u003cbr /\u003e\n\n|------------------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/models/blob/v2.15.0/official/modeling/optimization/optimizer_factory.py#L102-L266) |\n\nOptimizer factory class.\n\n#### View aliases\n\n\n**Main aliases**\n\n[`tfm.optimization.optimizer_factory.OptimizerFactory`](https://www.tensorflow.org/api_docs/python/tfm/optimization/OptimizerFactory)\n\n\u003cbr /\u003e\n\n tfm.optimization.OptimizerFactory(\n config: ../../tfm/optimization/OptimizationConfig\n )\n\nThis class builds learning rate and optimizer based on an optimization config.\nTo use this class, you need to do the following:\n(1) Define optimization config, this includes optimizer, and learning rate\nschedule.\n(2) Initialize the class using the optimization config.\n(3) Build learning rate.\n(4) Build optimizer.\n\nThis is a typical example for using this class: \n\n params = {\n 'optimizer': {\n 'type': 'sgd',\n 'sgd': {'momentum': 0.9}\n },\n 'learning_rate': {\n 'type': 'stepwise',\n 'stepwise': {'boundaries': [10000, 20000],\n 'values': [0.1, 0.01, 0.001]}\n },\n 'warmup': {\n 'type': 'linear',\n 'linear': {'warmup_steps': 500, 'warmup_learning_rate': 0.01}\n }\n }\n opt_config = OptimizationConfig(params)\n opt_factory = OptimizerFactory(opt_config)\n lr = opt_factory.build_learning_rate()\n optimizer = opt_factory.build_optimizer(lr)\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|----------|----------------------------------------------------------|\n| `config` | OptimizationConfig instance contain optimization config. |\n\n\u003cbr /\u003e\n\nMethods\n-------\n\n### `build_learning_rate`\n\n[View source](https://github.com/tensorflow/models/blob/v2.15.0/official/modeling/optimization/optimizer_factory.py#L163-L182) \n\n build_learning_rate()\n\nBuild learning rate.\n\nBuilds learning rate from config. Learning rate schedule is built according\nto the learning rate config. If learning rate type is consant,\nlr_config.learning_rate is returned.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| tf.keras.optimizers.schedules.LearningRateSchedule instance. If learning rate type is consant, lr_config.learning_rate is returned. ||\n\n\u003cbr /\u003e\n\n### `build_optimizer`\n\n[View source](https://github.com/tensorflow/models/blob/v2.15.0/official/modeling/optimization/optimizer_factory.py#L184-L266) \n\n build_optimizer(\n lr: Union[tf.keras.optimizers.schedules.LearningRateSchedule, float],\n gradient_aggregator: Optional[Callable[[List[Tuple[tf.Tensor, tf.Tensor]]], List[Tuple[tf.Tensor,\n tf.Tensor]]]] = None,\n gradient_transformers: Optional[List[Callable[[List[Tuple[tf.Tensor, tf.Tensor]]], List[Tuple[tf.\n Tensor, tf.Tensor]]]]] = None,\n postprocessor: Optional[Callable[[tf.keras.optimizers.Optimizer], tf.keras.optimizers.\n Optimizer]] = None,\n use_legacy_optimizer: bool = True\n )\n\nBuild optimizer.\n\nBuilds optimizer from config. It takes learning rate as input, and builds\nthe optimizer according to the optimizer config. Typically, the learning\nrate built using self.build_lr() is passed as an argument to this method.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `lr` | A floating point value, or a tf.keras.optimizers.schedules.LearningRateSchedule instance. |\n| `gradient_aggregator` | Optional function to overwrite gradient aggregation. |\n| `gradient_transformers` | Optional list of functions to use to transform gradients before applying updates to Variables. The functions are applied after gradient_aggregator. The functions should accept and return a list of (gradient, variable) tuples. clipvalue, clipnorm, global_clipnorm should not be set when gradient_transformers is passed. |\n| `postprocessor` | An optional function for postprocessing the optimizer. It takes an optimizer and returns an optimizer. |\n| `use_legacy_optimizer` | A boolean that indicates if using legacy optimizers. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| [`tf.keras.optimizers.legacy.Optimizer`](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/legacy/Optimizer) or [`tf.keras.optimizers.experimental.Optimizer`](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Optimizer) instance. ||\n\n\u003cbr /\u003e"]]