random(top:seed:)

A sampling mode that considers a fixed number of high-probability tokens.

Declaration

static func random(top k: Int, seed: UInt64? = nil) -> GenerationOptions.SamplingMode

Parameters

k:
The number of tokens to consider.
seed:
An optional random seed used to make output more deterministic.

Discussion

Also known as top-k.

During the token-selection process, the vocabulary is sorted by probability a token is selected from among the top K candidates. Smaller values of K will ensure only the most probable tokens are candidates for selection, resulting in more deterministic and confident answers. Larger values of K will allow less probably tokens to be selected, raising non-determinism and creativity.