Optimizer

class Optimizer(params, defaults)

Base class for all optimizers.

Parameters:

• params (Union[Iterable[Parameter], dict]) – specifies what Tensors should be optimized.

• defaults (dict) – a dict of default parameters of Optimizer, like learning rate or momentum.

add_param_group(param_group)

Add a param group to param_groups of the Optimizer.

This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to the Optimizer as training progresses.

Parameters:

param_group (dict) – specifies what tensors should be optimized along with group.

clear_grad()

Set the grad attribute to None for all parameters.

load_state_dict(state)

Loads the optimizer state.

Parameters:

state (dict) – optimizer state. Should be an object returned from a call to state_dict.

state_dict(keep_var=False)

Export the optimizer state.

Return type:

Dict

Returns:

optimizer state. Can be loaded by load_state_dict.

step()

Performs a single optimization step.

zero_grad()

Deprecated since version 1.0: use clear_grad instead