Contents

InferenceFunction

A function that performs inference on input values and produces output values.

Declaration

struct InferenceFunction

Mentioned in

Overview

An InferenceFunction owns the resources needed for inference, including model weights and intermediate buffers. You load a function from an AIModel and call run(inputs:states:outputViews:) to perform inference.

This type is Sendable, so you can run it concurrently from multiple tasks. The function automatically allocates additional intermediate buffers as needed to support concurrency.

Topics

Running inference

Inspecting a function

Supporting types

Asynchronous inference

See Also

Inference