Per-call sampling knobs consumed by the TokenSampler. More...

Public Attributes
float	temperature = 1.0f
int	top_k = 0
float	top_p = 1.0f

Detailed Description

Per-call sampling knobs consumed by the TokenSampler.

Deliberately free of loop control (max_new_tokens) and lifetime state (seed): loop bounds belong to GenerateParams, and reproducibility is a stream property seeded once via LanguageModel::seedSampler, not a per-call value.

The documentation for this struct was generated from the following file:

Mila/Src/Dnn/Components/Transformers/SamplingParams.ixx

Public Attributes

Detailed Description