ToolCallEvaluator
An evaluator that verifies agentic tool calls against an expected trajectory.
Declaration
struct ToolCallEvaluator<Input> where Input : ModelSampleProtocol, Input.Expectation == TrajectoryExpectationMentioned in
Overview
Produces both a strict and partial result from a single evaluation pass.
Supports ordered sequences, unordered expectations, disallowed tool checks, and group steps.
let toolsAllPass = Metric("Tools All Pass")
let toolsPercentagePass = Metric("Tools Percentage Pass")
let evaluator = ToolCallEvaluator<ModelSample<String>>(
allPass: toolsAllPass, percentagePass: toolsPercentagePass
)