rmsNorm(scale:epsilon:)
Adds an RMS spatial normalization operation to the current graph.
Declaration
func rmsNorm(scale: some BNNSGraph.Builder.OperationParameter<T>, epsilon: Float = .ulpOfOne.squareRoot()) -> BNNSGraph.Builder.Tensor<T>Parameters
- scale:
The scale.
- epsilon:
The epsilon value that the function uses to avoid division by zero.
Discussion
This function performs y = x / sqrt(sum_square(x) / size(x) + epsilon) * scale.