LeakyReLU

class LeakyReLU(negative_slope=0.01, **kwargs)

对每个元素应用函数：

$$\text{LeakyReLU}(x)=max(0,x)+negative\mathrm{\_}slope\times min(0,x)$$

或者

$$\begin{array}{r}\text{LeakyReLU}(x)=\{\begin{array}{ll}x,& \text{ if }x\ge 0\\ negative\mathrm{\_}slope\times x,& \text{ otherwise }\end{array}\end{array}$$

实际案例

import numpy as np
import megengine as mge
import megengine.module as M
data = mge.tensor(np.array([-8, -12, 6, 10]).astype(np.float32))

leakyrelu = M.LeakyReLU(0.01)
output = leakyrelu(data)
print(output.numpy())

输出：

[-0.08 -0.12  6.   10.  ]

上一页

ReLU

下一页

PReLU