ModelJudgeEvaluator
An evaluator that uses a language model as a judge to score responses.
Declaration
struct ModelJudgeEvaluator<Input> where Input : ModelSampleProtocolMentioned in
Overview
ModelJudgeEvaluator sends the query, response, and optional reference data to a judge model, which returns scores for one or more dimensions. The response is automatically serialized as JSON, because OutputType is Codable, or is customizable via ModelJudgePrompt.
Topics
Creating a single-dimension evaluator
Creating a multi-dimension evaluator
Creating a pairwise evaluator
pairwise(_:scale:judge:scoringMode:evaluationTarget:)pairwise(judge:dimensions:scoringMode:evaluationTarget:)