Mila
Deep Neural Network Library
Loading...
Searching...
No Matches
Mila::Dnn::QwenDeltaNetBlockWorkspace< TDeviceType, TPrecision > Struct Template Referenceexport
module Dnn.Components.QwenDeltaNetBlock

Transformer-owned shared activation workspace for QwenDeltaNetBlock (pooling). More...

Public Types

using MR = typename DeviceTypeTraits<TDeviceType>::memory_resource
using TensorType = Tensor<TPrecision, MR>

Public Member Functions

std::size_t deviceStorageBytes () const
 Total device bytes held, for memory accounting.

Public Attributes

std::shared_ptr< TensorType > a
std::shared_ptr< TensorType > act_qk
std::shared_ptr< TensorType > act_v
std::shared_ptr< TensorType > b
std::shared_ptr< TensorType > conv_qk
std::shared_ptr< TensorType > conv_v
std::shared_ptr< TensorType > core
std::shared_ptr< TensorType > core_normed
std::shared_ptr< TensorType > ffn_act
std::shared_ptr< TensorType > ffn_down
std::shared_ptr< TensorType > ffn_in
std::shared_ptr< TensorType > gate_up
std::shared_ptr< TensorType > gated
std::shared_ptr< TensorType > k
std::shared_ptr< TensorType > mixed
std::shared_ptr< TensorType > normed
std::shared_ptr< TensorType > q
std::shared_ptr< TensorType > qk
std::shared_ptr< TensorType > res1
std::shared_ptr< TensorType > stream
std::shared_ptr< TensorType > v
std::shared_ptr< TensorType > z

Detailed Description

template<DeviceType TDeviceType, TensorDataType TPrecision>
requires PrecisionSupportedOnDevice<TPrecision, TDeviceType>
struct Mila::Dnn::QwenDeltaNetBlockWorkspace< TDeviceType, TPrecision >

Transformer-owned shared activation workspace for QwenDeltaNetBlock (pooling).

The SECOND workspace struct QwenAttentionBlockWorkspace's comment anticipates. Not a wider version of it: a DeltaNet layer's slots are shaped by the mixer's own geometry – fused [query|key], the value stream, two per-head gating scalars – and share no width with the attention block's beyond the residual stream itself.

One slot set serves every DeltaNet layer, because inference is strictly sequential and exactly one block is live at a time. Slots are sized [B, chunk, width] and components view prefixes of them. Aliasing is the same argument the attention workspace makes and it has three live points here: normed is read by all five input projections, z survives from its projection to the output gate, and res1 is read again at res_2 – so each of those is its own slot, and stream (the block's own input, and its output) is only overwritten at block end.

Without this, every DeltaNet layer self-allocated its outputs: 138.2 MiB per layer on the 27B at chunk 512, ~6.5 GiB over 48 layers, which is what capped the model at 512 context and made WDDM page its weights (Qwen3.8.md section 8, Phase 5).

The documentation for this struct was generated from the following file: