LeakyReLU

class LeakyReLU(negative_slope=0.01, **kwargs)

对每个元素应用函数：

$$\text{LeakyReLU}(x)=max(0,x)+negative\mathrm{\_}slope\times min(0,x)$$

或者

$$\begin{array}{r}\text{LeakyReLU}(x)=\{\begin{array}{ll}x,& \text{ if }x\ge 0\\ negative\mathrm{\_}slope\times x,& \text{ otherwise }\end{array}\end{array}$$

实际案例

>>> import numpy as np
>>> data = mge.tensor(np.array([-8, -12, 6, 10]).astype(np.float32))
>>> leakyrelu = M.LeakyReLU(0.01)
>>> output = leakyrelu(data)
>>> output.numpy()
array([-0.08, -0.12,  6.  , 10.  ], dtype=float32)

上一页

ReLU

下一页

PReLU