BNNSActivationFunctionGELUApproximation2
An activation function that provides a fast evaluation of the Gaussian error linear units (GELU) approximation on its input.
Declaration
var BNNSActivationFunctionGELUApproximation2: BNNSActivationFunction { get }Discussion
This constant defines an activation function that returns values using the following operation:
x * (ReLU 6(x + 3.0) * 1.0 / 6.0)The following illustrates the output that the activation function generates from inputs in the range -10...10, an alpha of 0.1, and a beta of 1.0. The thinner, dashed line shows, for comparison, the result of BNNSActivationFunctionGELUApproximation using the same alpha and beta values:
[Image]
See Also
Raw Values
init(_:)init(rawValue:)rawValueBNNSActivationFunctionAbsBNNSActivationFunctionCELUBNNSActivationFunctionClampedLeakyRectifiedLinearBNNSActivationFunctionELUBNNSActivationFunctionErfBNNSActivationFunctionGELUBNNSActivationFunctionGELUApproximationBNNSActivationFunctionGELUApproximationSigmoidBNNSActivationFunctionGumbelBNNSActivationFunctionGumbelMaxBNNSActivationFunctionHardShrinkBNNSActivationFunctionHardSigmoid