Contents

run(info:)

Runs the evaluation against the dataset and computes metric results.

Declaration

nonisolated(nonsending) func run(info: [String : String] = [:]) async throws -> EvaluationResult

Parameters

  • info:

    User-defined key-value pairs attached to the result, such as model name or dataset version.

Return Value

An EvaluationResult that contains the summary statistics and sample-level results.

Discussion

This method executes the evaluation by iterating through your dataset, producing subjects, applying evaluators, and producing summary statistics. The evaluation logs and skips inference errors rather than halting the run.

The resulting EvaluationResult contains typed data in its DataFrames:

  • The Input column contains the full Sample from the dataset.

  • The Response column contains the full subject that the system under test produces.

  • Metric columns contain Metric values directly, preserving pass/fail/score semantics and rationale.

See Also

Running an evaluation