quantize(_:scaleTensor:zeroPoint:dataType:axis:name:)

Declaration

func quantize(_ tensor: MPSGraphTensor, scaleTensor: MPSGraphTensor, zeroPoint: Double, dataType: MPSDataType, axis: Int, name: String?) -> MPSGraphTensor

Parameters

• tensor:

  Input tensor to be quantized

• scaleTensor:

  Scale 1D Tensor parameter with size == tensor.shape[axis]

• zeroPoint:

  Bias scalar parameter (converted to dataType of resultTensor)

• dataType:

  Data type of the result tensor. Supports MPSDataTypeInt8, MPSDataTypeUInt8, MPSDataTypeFloat8E4M3, and MPSDataTypeFloat8E5M2. Float8 output requires symmetric quantization (zeroPoint = 0).

• axis:

  Axis on which the scale 1D value is being broadcasted

• name:

  The name for the operation.

Return Value

A valid MPSGraphTensor array of datatype dataType

Discussion

Convert the float tensor to a quantized tensor by applying a scale + bias transform: result = round(tensor / scaleTensor) + zeroPoint