megengine.module.GELU¶

class GELU(name=None)[源代码]¶

Applies the element-wise function:

\[\text{GELU}(x) = x\Phi(x)\]

where \(\Phi(x)\) is the Cumulative Distribution Function for Gaussian Distribution.

实际案例

import numpy as np
import megengine as mge
import megengine.module as M

data = mge.tensor(np.array([-2,-1,0,1,2,]).astype(np.float32))
gelu = M.GELU()
output = gelu(data)
with np.printoptions(precision=4):
    print(output.numpy())

Outputs:

[-0.0455 -0.1587  0.      0.8413  1.9545]

Methods

`apply`(fn)	Applies function `fn` to all the modules within this module, including itself.
`buffers`([recursive])	Returns an iterable for the buffers of the module.
`children`(**kwargs)	Returns an iterable for all the submodules that are direct attributes of this module.
`disable_quantize`([value])	Sets `module`'s `quantize_disabled` attribute and return `module`.
`eval`()	Sets training mode of all the modules within this module (including itself) to `False`.
`forward`(inputs)
`load_state_dict`(state_dict[, strict])	Loads a given dictionary created by `state_dict` into this module.
`modules`(**kwargs)	Returns an iterable for all the modules within this module, including itself.
`named_buffers`([prefix, recursive])	Returns an iterable for key buffer pairs of the module, where `key` is the dotted path from this module to the buffer.
`named_children`(**kwargs)	Returns an iterable of key-submodule pairs for all the submodules that are direct attributes of this module, where 'key' is the attribute name of submodules.
`named_modules`([prefix])	Returns an iterable of key-module pairs for all the modules within this module, including itself, where 'key' is the dotted path from this module to the submodules.
`named_parameters`([prefix, recursive])	Returns an iterable for key `Parameter` pairs of the module, where `key` is the dotted path from this module to the `Parameter`.
`named_tensors`([prefix, recursive])	Returns an iterable for key tensor pairs of the module, where `key` is the dotted path from this module to the tensor.
`parameters`([recursive])	Returns an iterable for the `Parameter` of the module.
`register_forward_hook`(hook)	Registers a hook to handle forward results.
`register_forward_pre_hook`(hook)	Registers a hook to handle forward inputs.
`replace_param`(params, start_pos[, seen])	Replaces module's parameters with `params`, used by `ParamPack` to
`state_dict`([rst, prefix, keep_var])
`tensors`([recursive])	Returns an iterable for the `Tensor` of the module.
`train`([mode, recursive])	Sets training mode of all the modules within this module (including itself) to `mode`.
`zero_grad`()	Sets all parameters' grads to zero

megengine.module.SiLU.zero_grad

megengine.module.GELU.apply