megengine.module.init.msra_normal_#

msra_normal_(tensor, a=0, mode='fan_in', nonlinearity='leaky_relu')[source]#

Fills tensor wilth random values sampled from \(\mathcal{N}(0, \text{std}^2)\) where

\[\text{std} = \sqrt{\frac{2}{(1 + a^2) \times \text{fan_in}}}\]

Detailed information can be retrieved from Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification

Parameters:

  • tensor (Tensor) – tensor to be initialized

  • a (float) – optional parameter for calculating gain for leaky_relu. See calculate_gain for details.

  • mode (str) – fan_in” or “fan_out”, used to calculate \(gain\), the scaling factor for \(gain\). See calculate_fan_in_and_fan_out for details.

  • nonlinearity (str) – name of the non-linear function used to calculate \(gain\). See calculate_gain for details.

Return type:

None