LazyLinear

class torch.nn.LazyLinear(out_features, bias=True, device=None, dtype=None)

A torch.nn.Linear module where in_features is inferred.

In this module, the weight and bias are of torch.nn.UninitializedParameter class. They will be initialized after the first call to forward is done and the module will become a regular torch.nn.Linear module. The in_features argument of the Linear is inferred from the input.shape[-1].

Check the torch.nn.modules.lazy.LazyModuleMixin for further documentation on lazy modules and their limitations.

Parameters

• out_features (int) – size of each output sample

• bias (UninitializedParameter) – If set to False, the layer will not learn an additive bias. Default: True

Variables

• weight (torch.nn.parameter.UninitializedParameter) – the learnable weights of the module of shape $(\text{out\_features}, \text{in\_features})$. The values are initialized from $\mathcal{U}(-\sqrt{k}, \sqrt{k})$, where $k = \frac{1}{\text{in\_features}}$

• bias (torch.nn.parameter.UninitializedParameter) – the learnable bias of the module of shape $(\text{out\_features})$. If bias is True, the values are initialized from $\mathcal{U}(-\sqrt{k}, \sqrt{k})$ where $k = \frac{1}{\text{in\_features}}$

cls_to_become

alias of Linear

initialize_parameters(input)

Infers in_features based on input and initializes parameters.

reset_parameters()

Resets parameters based on their initialization used in __init__.