Contract for SamplingOp: in-place token sampling from a logits tensor. More...

Concept definition

    template<typename TOp, typename TLogits, typename TToken, typename TParams>
    concept SamplingOpConcept = requires( const TOp& op,
                                          const TLogits& logits, TToken& token_out,
                                          const TParams& params, float r )
    {
        op.forward( logits, token_out, params, r );
    }

Detailed Description

Contract for SamplingOp: in-place token sampling from a logits tensor.

Per-call sampling knobs travel in a SamplingParams struct (temperature/top_k/ top_p/seed) – a struct, not loose scalars, so adding a filter does not churn the signature. The host-drawn uniform r is passed as a scalar, keeping the op pure and deterministic. token_out is a device INT32 tensor written in-place; the caller provides the buffer (the model's decode_token_device_).

Template Parameters

TOp	Candidate op type.
TLogits	Logits tensor type (model compute precision).
TToken	Output tensor type (INT32 device tensor).
TParams	Per-call sampling parameter struct (SamplingParams).