---
title: Evaluation
framework: evaluations
role: symbol
role_heading: Protocol
path: evaluations/evaluation
---

# Evaluation

A type that defines an evaluation.

## Declaration

```swift
protocol Evaluation : Sendable
```

## Mentioned in

Designing effective evaluations

## Overview

Overview Implement this protocol to create custom evaluations. The evaluation runs your system under test against a dataset and applies evaluators to measure performance. struct MyEvaluation: Evaluation {     let metric = Metric("Match")

let dataset = ArrayLoader(samples: [         ModelSample(prompt: "One plus one is...", expected: "Two.")     ])

func subject(from sample: ModelSample<String>) async throws -> ModelSubject<String> {         ModelSubject(value: "Two.")     }

var evaluators: Evaluators {         Evaluator { sample, subject in             let metric = Metric("Match")             guard let expected = sample.expected else { return metric.ignore() }             return subject.value == expected ? metric.passing() : metric.failing()         }     }

func aggregateMetrics(using aggregator: inout MetricsAggregator) {         aggregator.computeMean(of: metric)     } }

## Topics

### Providing data

- [Sample](evaluations/evaluation/sample.md)
- [SampleLoader](evaluations/evaluation/sampleloader.md)
- [dataset](evaluations/evaluation/dataset.md)

### Testing an intelligent feature

- [Subject](evaluations/evaluation/subject.md)
- [subject(from:)](evaluations/evaluation/subject(from:).md)
- [EvaluationSubject](evaluations/evaluationsubject.md)
- [ModelSubject](evaluations/modelsubject.md)
- [name](evaluations/evaluation/name.md)

### Scoring results

- [evaluators](evaluations/evaluation/evaluators-swift.property.md)
- [Evaluation.Evaluators](evaluations/evaluation/evaluators-swift.typealias.md)
- [EvaluatorProtocol](evaluations/evaluatorprotocol.md)
- [EvaluatorsBuilder](evaluations/evaluatorsbuilder.md)
- [aggregateMetrics(using:)](evaluations/evaluation/aggregatemetrics(using:).md)

### Running an evaluation

- [EvaluationTrait](evaluations/evaluationtrait.md)
- [EvaluationContext](evaluations/evaluationcontext.md)
- [EvaluationResult](evaluations/evaluationresult.md)
- [run(info:)](evaluations/evaluation/run(info:).md)

### Inspecting detailed results

- [inputColumn](evaluations/evaluation/inputcolumn.md)
- [responseColumn](evaluations/evaluation/responsecolumn.md)
- [expectedColumn](evaluations/evaluation/expectedcolumn.md)

### Errors

- [EvaluationError](evaluations/evaluationerror.md)
- [EvaluationResultsError](evaluations/evaluationresultserror.md)

## Relationships

### Inherits From

- [Sendable](swift/sendable.md)
- [SendableMetatype](swift/sendablemetatype.md)

## See Also

### Essentials

- [Evaluating language model responses](evaluations/evaluating-language-model-responses.md)
- [Designing effective evaluations](evaluations/designing-effective-evaluations.md)
- [Book Tracker: Using Evaluations to evaluate an intelligent feature](evaluations/book-tracker-using-evaluations-to-evaluate-an-intelligent-feature.md)