Contents

TrajectoryExpectation

The expected pattern of tool calls for an evaluation.

Declaration

struct TrajectoryExpectation

Mentioned in

Overview

TrajectoryExpectation(ordered: [
    ToolExpectation("authenticate"),
    ToolExpectation("processResults"),
])

TrajectoryExpectation specifies expected tool-calling behavior across three axes:

  • Ordered: Tool calls that must occur in a specific sequence. Use ToolExpectation for single sequential steps, or anyOrder(_:) when multiple tools must all be called at a given position but their relative order doesn’t matter.

  • Unordered: Tool calls that must occur at some point, regardless of when.

  • Disallowed: Tool calls that must NOT occur.

TrajectoryExpectation(ordered: [
    ToolExpectation("authenticate"),
    ToolExpectation("processResults"),
])
TrajectoryExpectation(ordered: [
    ToolExpectation("authenticate"),
    .anyOrder([
        ToolExpectation("fetchData"),
        ToolExpectation("fetchMetadata"),
    ]),
    ToolExpectation("processResults"),
], allowsAdditionalToolCalls: false)
TrajectoryExpectation(
    ordered: [
        ToolExpectation("findActivities"),
        ToolExpectation("estimateTravelTime"),
    ],
    unordered: [ToolExpectation("getWeather")],
    disallowed: [ToolExpectation("deleteData")]
)
TrajectoryExpectation(expected: "getWeather", arguments: [
    .exact(argumentName: "location", value: "Paris, France")
])

Topics

Creating a single-tool expectation

Creating multi-tool expectations

Combining expectations

See Also

Tool-call evaluation