The Section 5 oracle: FP4 across the body, everything else as the plan has it. More...

Public Types
using	DeltaNetGating = NoWeightQuant
using	DeltaNetQueryKey = PerGroupFp4<128>
using	DeltaNetValueGateOutput = PerGroupFp4<128>
using	EmbeddingTable = NoWeightQuant
using	FeedForwardDown = PerGroupFp4<128>
using	FeedForwardGateUp = PerGroupFp4<128>
using	LanguageModelHead = PerGroupFp4<128>
using	OutputProjection = PerGroupFp4<128>
using	QkvProjection = PerGroupFp4<128>

Detailed Description

The Section 5 oracle: FP4 across the body, everything else as the plan has it.

The near-lossless arm of the Phase 5 quality gate. 4.125 bits everywhere the plan quantizes, 12.31 GiB of weights – past the 12 GiB target card and onto the 16 GiB one, which is what that card is for ("The 16 GiB oracle").

NOT QwenUniformPrecisionPlan<PerGroupFp4<128>>, and the difference is the point. A true uniform lift would also quantize the two roles the plan deliberately leaves alone, and both refuse: TokenEmbedding static-asserts that a table policy is per-channel and that a host-resident table is stored at compute precision. Those refusals are correct – the table is a gather over PCIe, never a multiply – and the measurement wants them anyway. Two arms that differ in their embedding AND their body do not attribute a perplexity gap to either.

DeltaNetGating stays BF16 for the same reason it does in the plan: error in the forget gate compounds exponentially over the sequence, so quantizing 0.1% of the parameters would put the oracle's own quality in question.

The documentation for this struct was generated from the following file:

Mila/Src/Dnn/Components/Transformers/Qwen/Qwen.PrecisionPlan.ixx

Public Types

Detailed Description