Mila::Dnn::GemmaBlock< TDeviceType, TPrecision, kGlobal, TWeightQuant, TKvPolicy >::BlockBuildContexts Struct Reference

The per-child build contexts and split-scratch geometry this block implies. More...

Public Member Functions
dim_t	splitKvElements () const noexcept
dim_t	splitQElements () const noexcept

Public Attributes
dim_t	batch { 0 }
dim_t	chunk { 0 }
BuildContext	gate_up
dim_t	head_dim { 0 }
BuildContext	hidden
BuildContext	kknorm
dim_t	num_heads { 0 }
dim_t	num_kv_heads { 0 }
BuildContext	qknorm
BuildContext	qkv
BuildContext	qproj
BuildContext	stream

Detailed Description

template<DeviceType TDeviceType, TensorDataType TPrecision, bool kGlobal, WeightQuantPolicy TWeightQuant = NoWeightQuant, KvCachePolicy TKvPolicy = NoKvCompression>
struct Mila::Dnn::GemmaBlock< TDeviceType, TPrecision, kGlobal, TWeightQuant, TKvPolicy >::BlockBuildContexts

The per-child build contexts and split-scratch geometry this block implies.

A block does NOT cascade one context to its children: the QK-norms see per-head rows, the gate/up projection is double width, and the attention layer is built at the FULL context length so it sizes the KV cache while everything else is built at the prefill chunk. Derived once here so onBuilding() and getRequiredMemory() cannot disagree about any of it – see Specifications/MemoryFootprint.md section 4.4.

---

The documentation for this struct was generated from the following file:

• Mila/Src/Dnn/Components/Transformers/Gemma/Gemma.Block.ixx