random(probabilityThreshold:seed:)

A mode that considers a variable number of high-probability tokens based on the specified threshold.

Declaration

static func random(probabilityThreshold: Double, seed: UInt64? = nil) -> GenerationOptions.SamplingMode

Parameters

probabilityThreshold:
A number between 0.0 and 1.0 that increases sampling pool size.
seed:
An optional random seed used to make output more deterministic.

Discussion

Also known as top-p or nucleus sampling.

With nucleus sampling, tokens are sorted by probability and added to a pool of candidates until the cumulative probability of the pool exceeds the specified threshold, and then a token is sampled from the pool.

Because the number of tokens isn’t predetermined, the selection pool size will be larger when the distribution is flat and smaller when it is spikey. This variability can lead to a wider variety of options to choose from, and potentially more creative responses.