One Gated DeltaNet decoder block. More...

Inheritance diagram for Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >:

Classes
struct	BlockContexts
struct	StateSnapshot
	Everything this block carries between chunks – all of it, in one value. More...

Public Types
using	CompositeComponentBase = CompositeComponent<TDeviceType, TPrecision>
using	ConvActivationType = Activation<TDeviceType, TPrecision, ActivationType::Silu>
using	ConvType = CausalConv1d<TDeviceType, TPrecision>
using	DeltaRuleType = GatedDeltaRule<TDeviceType, TPrecision>
using	FeedForwardDownType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::FeedForwardDown>
using	FeedForwardGateUpType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::FeedForwardGateUp>
using	GateProjectionType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::DeltaNetValueGateOutput>
using	GatingProjectionType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::DeltaNetGating>
using	MR = typename DeviceTypeTraits<TDeviceType>::memory_resource
using	OutputGateType = AttentionOutputGate<TDeviceType, TPrecision, ActivationType::Silu>
using	OutputProjectionType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::DeltaNetValueGateOutput>
using	PrecisionPlan = QwenPrecisionPlanFor<TWeightPlan>
	The resolved plan. Named so a caller can ask a block what it decided.
using	QueryKeyProjectionType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::DeltaNetQueryKey>
using	ResidualType = Residual<TDeviceType, TPrecision>
using	RmsNormType = RmsNorm<TDeviceType, TPrecision>
using	SwigluType = Swiglu<TDeviceType, TPrecision, ActivationType::Silu>
using	TensorType = Tensor<TPrecision, MR>
using	ValueProjectionType = Linear<TDeviceType, TPrecision, typename PrecisionPlan::DeltaNetValueGateOutput>
Public Types inherited from Mila::Dnn::CompositeComponent< TDeviceType, TPrecision >
using	ComponentBase = Component<TDeviceType, TPrecision>
using	ComponentPtr = std::shared_ptr<Component<TDeviceType, TPrecision>>
Public Types inherited from Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >
using	MR = typename DeviceTypeTraits<TDeviceType>::memory_resource
using	TensorType = Tensor<TPrecision, MR>

Public Member Functions
	QwenDeltaNetBlock (const std::string &name, const QwenConfig &config, std::optional< DeviceId > device_id=std::nullopt)
TensorType &	decode (const TensorType &input, dim_t position) override
	Single-token decode at an absolute position (T == 1).
dim_t	gatingWidth () const noexcept
MemoryStats	getMemoryStats () const override
	Return the current memory allocation breakdown for this component.
std::vector< ObservableStage >	getObservableStages () const override
	Stages this component publishes, and the passes each is published on.
MemoryStats	getRequiredMemory (const BuildContext &context) const override
	Report what build( context ) would allocate, without allocating it.
const ComponentType	getType () const override
	Get the component type identifier.
void	installSharedWorkspace (const QwenDeltaNetBlockWorkspace< TDeviceType, TPrecision > &workspace)
	Install the transformer-owned shared activation workspace (pooling).
StateSnapshot	makeStateSnapshot () const
	Snapshots sized for the built geometry. Allocates; hold them and reuse them.
TensorType &	prefill (const TensorType &input, dim_t position_offset) override
	Chunked prefill: process [B, T_chunk, model_dim] at an absolute offset.
dim_t	queryKeyWidth () const noexcept
void	resetKvCache () override
	Start a new generation session: drop the recurrent state and the conv window.
void	restoreState (const StateSnapshot &source)
	Replace everything the block carries. All three pieces, or the block is torn.
bool	rewindKvCache (dim_t) override
	Always false: a recurrent state cannot be rewound.
void	setState (const GqaState &) override
	No-op: this block has no grouped-query attention and no shared GQA workspace.
void	snapshotState (StateSnapshot &destination) const
	Copy out everything the block carries.
bool	supportsKvCache () const noexcept override
	False: the mixer's memory is a recurrent state, not a key/value cache.
std::string	toString () const override
	Generate a human-readable description.
dim_t	valueWidth () const noexcept
Public Member Functions inherited from Mila::Dnn::CompositeComponent< TDeviceType, TPrecision >
	CompositeComponent (CompositeComponent &&) noexcept=default
	CompositeComponent (const CompositeComponent &)=delete
	CompositeComponent (const std::string &name)
	Construct composite component with name.
CompositeComponent &	addComponent (ComponentPtr component)
	Add a pre-constructed child component (chainable).
size_t	childCount () const noexcept
	Get the number of direct children.
void	clearComponents ()
	Clear all child components.
std::vector< std::string >	componentPaths () const
	Every component path in this subtree, in traversal order.
ComponentPtr	findComponent (const std::string &path) const
	Resolve a dot-separated component path within this composite.
ComponentPtr	getComponent (const std::string &name) const
	Retrieve a direct child component by name.
const std::vector< ComponentPtr > &	getComponents () const
	Get all child components in insertion order.
DeviceId	getDeviceId () const override
	Get the compute device for this composite.
std::vector< ITensor * >	getGradients () const override
	Get all parameter gradients from all children.
std::vector< ITensor * >	getParameters () const override
	Get all parameters from all children.
bool	hasChildren () const noexcept
	Check if this composite has any children.
bool	hasComponent (const std::string &name) const
	Check if a named child component exists.
size_t	observe (std::string_view pattern, Compute::ComputePassMask passes, Compute::ActivationObserver sink)
	Attach sink to every component in this subtree whose path matches.
CompositeComponent &	operator= (CompositeComponent &&) noexcept=default
CompositeComponent &	operator= (const CompositeComponent &)=delete
dim_t	parameterCount () const override
	Count parameters across all children.
bool	removeComponent (const std::string &name)
	Remove a child component by name.
void	saveFlatTensors (Serialization::SafeTensorsWriter &writer, const std::string &prefix, Serialization::TensorSavePass pass) const override
	Recurse into children, extending the flat dotted prefix.
void	stopObserving ()
	Detach every observer in this subtree and clear the shared sink.
void	synchronize () override
	Synchronize all child components.
ComponentPtr	tryFindComponent (const std::string &path) const
	Try to resolve a dot-separated component path within this composite.
Public Member Functions inherited from Mila::Dnn::Component< TDeviceType, TPrecision >
	Component (const std::string &name)
	Construct component with required name identifier.
virtual void	build (const BuildContext &context) final
	Build the component with the provided BuildContext (canonical overload).
IExecutionContext *	getExecutionContext () const
	Get the shared execution context.
const std::string	getName () const
	Get the component's name identifier.
Compute::ComputePassMask	getObservedPasses () const noexcept
virtual std::vector< const ITensor * >	getOutputs () const
	Return non-owning pointers to the tensors this component produces.
virtual std::vector< std::string >	getParameterNames () const
	List all available parameter names for this component.
RuntimeMode	getRuntimeMode () const noexcept
TrainingMode	getTrainingMode () const noexcept
	The current runtime behavioral mode of this Component.
virtual bool	isBuilt () const final
	Returns true if build() has completed successfully.
bool	isInferenceMode () const noexcept
bool	isTrainingMode () const noexcept
virtual void	loadParameter (const std::string &, const Serialization::ITensorBlob &)
	Load a parameter from serialized tensor data.
void	setObservedPasses (Compute::ComputePassMask passes) noexcept
	Select which passes this component publishes on.
void	setTrainingMode (TrainingMode mode)
	Set the runtime behavioral mode for this Component.
virtual void	zeroGradients ()
	Clear all model-owned gradients for this component.

Protected Member Functions
void	onBuilding (const BuildContext &context) override
	Hook invoked by build() to allocate component buffers.
void	onTrainingModeChanging (TrainingMode training_mode) override
	Hook invoked when training mode is about to change.
BlockContexts	resolveBlockContexts (const BuildContext &context) const
Protected Member Functions inherited from Mila::Dnn::CompositeComponent< TDeviceType, TPrecision >
template<typename TComponent>
std::shared_ptr< TComponent >	getComponentAs (const std::string &name) const
	Retrieve a typed child component by name.
void	load_ (ModelArchive &archive, SerializationMode mode) override
	Restore children from their nested scopes, mirroring save_().
void	onExecutionContextSet () override
	Hook invoked after ExecutionContext is set.
virtual void	optimize ()
	Virtual hook for graph optimization after construction.
void	requireSerializableParameters () const override
	No-op override: a composite names no parameters of its own.
void	save_ (ModelArchive &archive, SerializationMode mode) const override
Protected Member Functions inherited from Mila::Dnn::Component< TDeviceType, TPrecision >
bool	hasExecutionContext () const noexcept
	Check if execution context has been set.
template<TensorDataType TParameterPrecision, typename TMemoryResource>
void	loadParameterFromBlob (const std::string &param_name, const Serialization::ITensorBlob &blob, Tensor< TParameterPrecision, TMemoryResource > &target, const shape_t &expected_shape)
	Load a tensor blob into a parameter tensor with validation.
void	publish (Compute::ComputePass pass, std::string_view stage, const ITensor &value) const
	Publish one activation to the observer, if this pass is being observed.
template<TensorDataType TParameterPrecision, typename TMemoryResource>
void	saveParameterToArchive (ModelArchive &archive, const std::string &parameter_name, const Tensor< TParameterPrecision, TMemoryResource > &parameter) const
	Write one parameter tensor into the archive under "tensors/<name>".
template<TensorDataType TParameterPrecision, typename TMemoryResource>
void	saveParameterToWriter (Serialization::SafeTensorsWriter &writer, const std::string &flat_name, const Tensor< TParameterPrecision, TMemoryResource > &parameter, Serialization::TensorSavePass pass) const
	Drive one parameter through one pass of a flat safetensors save.
void	setExecutionContext (IExecutionContext *context)
	Set the execution context for this component.

Additional Inherited Members
Static Public Member Functions inherited from Mila::Dnn::Component< TDeviceType, TPrecision >
static constexpr DeviceType	getDeviceType ()
	Compile-time device type for this component instance.
static constexpr TensorDataType	getPrecision () noexcept
	Compile-time tensor precision for this component instance.
Protected Types inherited from Mila::Dnn::Component< TDeviceType, TPrecision >
using	HostStagingMemoryResource = typename DeviceTypeTraits<TDeviceType>::host_staging_memory_resource
	Host memory a device-resident parameter stages through.
Protected Attributes inherited from Mila::Dnn::Component< TDeviceType, TPrecision >
BuildContext	build_context_ { shape_t{ 1 }, RuntimeMode::Training }
	The BuildContext stored at build time.

Detailed Description

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>
requires PrecisionSupportedOnDevice<TPrecision, TDeviceType> && FeedForwardPrecisionRoles<QwenPrecisionPlanFor<TWeightPlan>> && DeltaNetPrecisionRoles<QwenPrecisionPlanFor<TWeightPlan>>
class Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >

One Gated DeltaNet decoder block.

Template Parameters

TWeightPlan A precision plan, or a bare policy meaning "uniform". Lifted through QwenPrecisionPlanFor rather than the generic PrecisionPlanFor, because the generic uniform plan deliberately does not name this family's DeltaNet roles.

Member Function Documentation

◆ decode()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

TensorType & Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::decode	(	const TensorType &	input,
		dim_t	position )

inlineoverridevirtual

Single-token decode at an absolute position (T == 1).

Returns: Reference to the block-owned output [B, 1, model_dim].

Implements Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >.

◆ getMemoryStats()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

MemoryStats Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::getMemoryStats ( ) const

inlineoverridevirtual

Return the current memory allocation breakdown for this component.

Reflects allocations at the moment of the call. The returned stats naturally track the component lifecycle:

After construction – nothing; construction allocates none After build( Inference ) – parameters + T=1 state buffers After build( Training ) – parameters + T=full state buffers After setTrainingMode( Train ) – parameters + state + gradients

For CompositeComponent and Network, the returned stats are the recursive aggregate of all child components.

May be called at any time – no lifecycle preconditions.

Returns: MemoryStats reflecting current allocations.

Implements Mila::Dnn::Component< TDeviceType, TPrecision >.

◆ getObservableStages()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

std::vector< ObservableStage > Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::getObservableStages ( ) const

inlineoverridevirtual

Stages this component publishes, and the passes each is published on.

Lets a caller tell an uninstrumented component from a clean one before running anything. The pass qualifier is load-bearing rather than decorative: the stage set genuinely differs by pass, since a prefill can produce intermediates its decode never materializes.

Returns: Vector of stages; empty for components that publish nothing.

Reimplemented from Mila::Dnn::Component< TDeviceType, TPrecision >.

◆ getRequiredMemory()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

MemoryStats Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::getRequiredMemory ( const BuildContext & context ) const

inlineoverridevirtual

Report what build( context ) would allocate, without allocating it.

The predictive counterpart to getMemoryStats(): what this component would need against what it currently has. Called on a constructed but unbuilt component, which is possible because construction commits no device memory – see Specifications/MemoryFootprint.md.

Implementations mirror their own onBuilding(), which is why the two take the same BuildContext: an implementation cannot answer for a different shape, runtime mode, or prefill size than the one build() would receive.

Composites mirror whatever their getMemoryStats() twin does – children, plus directly-owned buffers, minus any pooling, output-sharing or weight- tying correction. A plain sum over children overcounts wherever a buffer is installed rather than self-allocated.

Parameters

context The context that would be passed to build().

Returns: MemoryStats the component would allocate.

Exceptions

std::logic_error if this component has not implemented the contract.

Reimplemented from Mila::Dnn::Component< TDeviceType, TPrecision >.

◆ getType()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

const ComponentType Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::getType ( ) const

inlineoverridevirtual

Get the component type identifier.

Used for serialization and runtime type identification.

Returns: Component type enum value.

Implements Mila::Dnn::Component< TDeviceType, TPrecision >.

◆ installSharedWorkspace()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

void Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::installSharedWorkspace ( const QwenDeltaNetBlockWorkspace< TDeviceType, TPrecision > & workspace )

inline

Install the transformer-owned shared activation workspace (pooling).

Must be called before build(); onBuilding then routes each slot into the matching child via installSharedOutput and keeps the split scratch for the block's own q/k views. Self-allocation remains the default for standalone blocks (tests, and the layer-streamed parity harness).

◆ onBuilding()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

void Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::onBuilding ( const BuildContext & context )

inlineoverrideprotectedvirtual

Hook invoked by build() to allocate component buffers.

Receives the stored BuildContext. Implementations must use config.allocationSeqLen() when sizing output buffers – this is the single call that makes Inference and Training allocate the correct buffer sizes automatically without per-component logic.

// Example -- Linear component:
shape_t out_shape =
{
    config.batchSize(),
    config.allocationSeqLen(),   // 1 for Inference, T for Training
    config_.getOutputFeatures()
};
output_ = std::make_unique<TensorType>( device, out_shape,
    this->getName() + ".output" );

The default implementation forwards to the legacy onBuilding( const shape_t& ) overload for backwards compatibility. New components should override this overload directly.

Note: Do not call build() or onBuilding() from within this hook.; Implementations should either succeed fully or leave no partial state, as a failed build() may be retried.

Takes the build-time configuration; use its allocationSeqLen() to obtain the correct output buffer sequence dimension.

Reimplemented from Mila::Dnn::Component< TDeviceType, TPrecision >.

◆ onTrainingModeChanging()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

void Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::onTrainingModeChanging ( TrainingMode training_mode )

inlineoverrideprotectedvirtual

Hook invoked when training mode is about to change.

Propagates the new mode to all child components. The hook runs with the Component's training mutex held; it MUST NOT call setTrainingMode().

Parameters

training_mode New training mode (Normal or Eval)

Reimplemented from Mila::Dnn::CompositeComponent< TDeviceType, TPrecision >.

◆ prefill()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

TensorType & Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::prefill	(	const TensorType &	input,
		dim_t	position_offset )

inlineoverridevirtual

Chunked prefill: process [B, T_chunk, model_dim] at an absolute offset.

Returns: Reference to the block-owned output [B, T_chunk, model_dim].

Implements Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >.

◆ resetKvCache()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

void Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::resetKvCache ( )

inlineoverridevirtual

Start a new generation session: drop the recurrent state and the conv window.

Implements Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >.

◆ rewindKvCache()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

bool Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::rewindKvCache ( dim_t )

inlineoverridevirtual

Always false: a recurrent state cannot be rewound.

A KV cache stores each position separately, so dropping a suffix is exact. This block's state is a LOSSY SUMMARY of every position it has seen – the information needed to undo the last N steps is not in it. Returning true would silently corrupt a prefix-reuse session rather than fail it, so prompt-prefix reuse is unavailable for any model containing these layers.

Implements Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >.

◆ setState()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

void Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::setState ( const GqaState & )

inlineoverridevirtual

No-op: this block has no grouped-query attention and no shared GQA workspace.

The interface names a concrete GqaState because it was written for a family whose blocks all attend (Qwen3.8.md section 7 records the mismatch). Accepting and ignoring it is honest here – there is no attention transient to wire – and the alternative, generalizing the interface, is a change across three families.

Implements Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >.

◆ snapshotState()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

void Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::snapshotState ( StateSnapshot & destination ) const

inline

Copy out everything the block carries.

The mechanism prompt caching needs on this layer kind, and the reason rewindKvCache above can only ever return false. See PromptCaching.md section 2 for what an attention layer does instead, and why it costs nothing there.

◆ supportsKvCache()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

bool Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::supportsKvCache ( ) const

inlineoverridevirtualnoexcept

False: the mixer's memory is a recurrent state, not a key/value cache.

Not a limitation being reported – a different mechanism. The state is already fixed-size and always live; there is no cache path to support or refuse.

Implements Mila::Dnn::ITransformerBlock< TDeviceType, TPrecision >.

◆ toString()

template<DeviceType TDeviceType, TensorDataType TPrecision, typename TWeightPlan = QwenReferencePrecisionPlan>

std::string Mila::Dnn::QwenDeltaNetBlock< TDeviceType, TPrecision, TWeightPlan >::toString ( ) const

inlineoverridevirtual

Generate a human-readable description.

Returns: String representation showing children

Reimplemented from Mila::Dnn::CompositeComponent< TDeviceType, TPrecision >.

The documentation for this class was generated from the following file:

Mila/Src/Dnn/Components/Transformers/Qwen/Qwen.DeltaNetBlock.ixx

Classes

Public Types

Public Member Functions

Protected Member Functions

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ decode()

◆ getMemoryStats()

◆ getObservableStages()

◆ getRequiredMemory()

◆ getType()

◆ installSharedWorkspace()

◆ onBuilding()

◆ onTrainingModeChanging()

◆ prefill()

◆ resetKvCache()

◆ rewindKvCache()

◆ setState()

◆ snapshotState()

◆ supportsKvCache()

◆ toString()