Contents

evaluates(_:info:recordTranscripts:)

Creates a trait that runs a single evaluation and makes its result available through the current evaluation context.

Declaration

static func evaluates(_ evaluation: any Evaluation, info: [String : String] = [:], recordTranscripts: Bool = false) -> Self

Parameters

evaluation:
The evaluation to run.
info:
User-defined key-value pairs attached to the result, such as model name or dataset version.
recordTranscripts:
When true, each row’s transcript snapshot is embedded in the .xcevalresult attachment under the Transcript column. Recoverable on load via EvaluationResult/transcriptSnapshots. Defaults to false so the attachment stays small for runs that don’t need transcripts.

Discussion

let accuracyMetric = Metric("Accuracy")

@Test(.evaluates(myEvaluation, info: ["model": "gpt-4"]))
func testAccuracy() async throws {
    let result = EvaluationContext.current.result
    #expect(result.aggregateValue(.mean(of: accuracyMetric)) >= 0.8)
}