Contents

init(allPass:percentagePass:argumentMatchModel:)

Creates a new tool-call expectations evaluator with a custom language model for semantic matching of natural language argument matchers.

Declaration

init(allPass: Metric, percentagePass: Metric, argumentMatchModel: any LanguageModel)

Parameters

  • allPass:

    The metric for the strict pass or fail result.

  • percentagePass:

    The metric for the partial score result.

  • argumentMatchModel:

    The language model to use for semantic matching.

Discussion

let allPassMetric = Metric("Tools All Pass")
let percentagePassMetric = Metric("Tools Percentage Pass")

let evaluator = ToolCallEvaluator<ModelSample<String>>(
    allPass: allPassMetric,
    percentagePass: percentagePassMetric,
    argumentMatchModel: SystemLanguageModel.default
)