EvaluatorProtocol
A type that evaluates subjects and produces metrics.
Declaration
protocol EvaluatorProtocol<Input, Subject> : SendableOverview
Conform to EvaluatorProtocol to create custom evaluators that measure the system’s output against expected criteria. Each evaluator returns an array of Metric values — one per DataFrame column produced.
The protocol is parameterized by Input (the sample type). Subject is an associated type constrained to EvaluationSubject, ensuring the subject’s value type matches the sample’s expected value type.
Conforming types must be Sendable.
struct MyEvaluator<Input: SampleProtocol>: EvaluatorProtocol
where Input.ExpectedValue: Sendable & Codable {
let metric = Metric("Quality")
func metrics(
subject: ModelSubject<Input.ExpectedValue>,
input: Input
) async throws -> [Metric] {
return [metric.scoring(1.0)]
}
}