Contents

ToolCallEvaluator

An evaluator that verifies agentic tool calls against an expected trajectory.

Declaration

struct ToolCallEvaluator<Input> where Input : ModelSampleProtocol, Input.Expectation == TrajectoryExpectation

Mentioned in

Overview

This evaluator produces both a strict and partial result from a single evaluation pass. It also supports ordered sequences, unordered expectations, disallowed tool checks, and group steps.

Use the toolsAllPass and toolsPercentagePass conveniences for the standard metrics:

let evaluator = ToolCallEvaluator<ModelSample<String>>(
    allPass: .toolsAllPass, percentagePass: .toolsPercentagePass
)

Topics

Initializers

Instance Properties

Instance Methods

See Also

Tool-call evaluation