Contents

clipByGlobalNorm(threshold:inputs:outputs:globalNorm:)

Clips the input tensors to a global Euclidean norm and writes the result to the output tensors.

Declaration

static func clipByGlobalNorm(threshold: Float, inputs: [BNNSNDArrayDescriptor], outputs: [BNNSNDArrayDescriptor], globalNorm: Float = 0) throws

Parameters

  • threshold:

    The maximum Euclidean norm to clip the gradient to.

  • inputs:

    An array of input descriptors.

  • outputs:

    An array of output descriptors.

  • globalNorm:

    The global norm to use. Set to 0 to specify that the function computes the norm from the input descriptors.

Discussion

Use this function to clip the values in an array of input tensors to a maximum global Euclidean norm. If you know the global norm of the input tensors, pass this value as the globalNorm. Otherwise, pass 0 to specify that the function calculates the norm.

The Euclidean norm is the square root of the sum of squares of the two tensors. The following code clips the Euclidean norm of two input tensors to half of the global Euclidean norm:

static func clipToGlobalNorm() {
    let inputOne = BNNSNDArrayDescriptor.allocate(
        initializingFrom: [1, 2, 3, 4] as [Float],
        shape: .vector(4))
    let inputTwo = BNNSNDArrayDescriptor.allocate(
        initializingFrom: [5, 6, 7, 8] as [Float],
        shape: .vector(4))
    
    let outputOne = BNNSNDArrayDescriptor.allocateUninitialized(
        scalarType: Float.self,
        shape: .vector(4))
    let outputTwo = BNNSNDArrayDescriptor.allocateUninitialized(
        scalarType: Float.self,
        shape: .vector(4))
    
    try? BNNS.clipByGlobalNorm(threshold: 0.5 * 14.2828568570857,
                               inputs: [inputOne, inputTwo],
                               outputs: [outputOne, outputTwo])
    
    // Prints: `[0.5, 1.0, 1.5, 2.0]`
    print(outputOne.makeArray(of: Float.self)!)
    
    // Prints: `[2.5, 3.0, 3.5, 4.0]`
    print(outputTwo.makeArray(of: Float.self)!)

    inputOne.deallocate()
    inputTwo.deallocate()
    outputOne.deallocate()
    outputTwo.deallocate()
}

See Also

Utility functions