Everything this block carries between chunks – all of it, in one value. More...

Public Attributes
ConvType::StateSnapshot	query_key_window
DeltaRuleType::StateSnapshot	recurrent
ConvType::StateSnapshot	value_window

Detailed Description

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>
struct Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::StateSnapshot

Everything this block carries between chunks – all of it, in one value.

THREE PIECES, NOT ONE, and that is the point of aggregating them here. The mixer's recurrent state is the large one, but the two convolution windows are equally load-bearing: restoring the mixer alone would leave each convolution's kernel width - 1 retained rows describing tokens the mixer no longer believes it has seen. The block's continuation would then be wrong by three positions per layer, in a way that produces plausible output rather than an error.

The documentation for this struct was generated from the following file:

Mila/Src/Dnn/Components/Transformers/Qwen/Qwen.DeltaNetBlock.ixx

Public Attributes

Detailed Description