Contents

init(_:scale:judge:scoringMode:prompt:)

Creates a single-metric evaluator with a custom model prompt.

Declaration

init(_ name: String, scale: ScoringScale, judge: any LanguageModel, scoringMode: ScoringMode = .discrete, prompt: ModelJudgePrompt<Input>)

Parameters

  • name:

    The metric name, used as the DataFrame column name.

  • scale:

    The scoring scale for this metric.

  • judge:

    The language model to use as judge.

  • scoringMode:

    Whether scores are discrete (default) or allow any floating-point value.

  • prompt:

    Configuration for the model prompt, including instructions, response presentation, and reference.

Discussion

let prompt = ModelJudgePrompt<ModelSample<String>>(
    instructions: "Score the relevance of the response."
)
let evaluator = ModelJudgeEvaluator<ModelSample<String>>(
    "Relevance",
    scale: .passFail(passDescription: "Relevant", failDescription: "Not relevant"),
    judge: SystemLanguageModel.default,
    prompt: prompt
)

See Also

Creating a single-dimension evaluator