Adam

class Adam(params, lr, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.0)[源代码]

实现 “Adam: A Method for Stochastic Optimization” 中提出的Adam算法。

参数
  • params (Union[Iterable[Parameter], dict]) – 可迭代对象,可以是一组待优化的参数,或定义几组参数的dict类型。

  • lr (float) – learning rate.

  • betas (Tuple[float, float]) – coefficients used for computing running averages of gradient and its square. Default: (0.9, 0.999).

  • eps (float) – term added to the denominator to improve numerical stability. Default: 1e-8.

  • weight_decay (float) – weight decay (L2 penalty). Default: 0.

返回

An instance of the Adam optimizer.