Buffers BufSize tokens before forwarding a contiguous span to Sink. More...

Public Member Functions
	BufferedTokenStreamer (Sink &sink)
void	finish ()
void	flush ()
void	operator() (int32_t tok)

Detailed Description

template<class Sink, size_t BufSize = 8>
requires TokenSink<Sink>
class Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >

Buffers BufSize tokens before forwarding a contiguous span to Sink.

Reduces per-token overhead when Sink involves cross-thread posting or I/O. Satisfies TokenStreamer via operator()(int32_t). Call finish() after generation completes to flush any partial buffer and invoke Sink::on_finish() if present.

Sink is held by reference; the caller must ensure Sink outlives this object.

Template Parameters

BufSize	Tokens to buffer before each flush. Defaults to 8.
Sink	A TokenSink callable.

The documentation for this class was generated from the following file:

Mila/Src/Dnn/Core/TokenStreamer.ixx

Public Member Functions

Detailed Description