megengine.functional.nn.layer_norm#

layer_norm(inp, normalized_shape, affine, weight=None, bias=None, eps=1e-5)[source]#

Applies layer normalization to the input. Support tensor of any shape as input. Reference: https://arxiv.org/pdf/1803.08494.pdf.

Parameters:

  • inp (Tensor) – input tensor.

  • normalized_shape (tuple) – the shape that you want to be normalizated

  • affine (bool) – whether to use weight and bias

  • weight (Optional[Tensor]) – must not be None when the affine is true

  • bias (Optional[Tensor]) – must not be None when the affine is true

  • eps (float) – a value added to the denominator for numerical stability. Default: 1e-5