Mila::Dnn::DeltaNetPrecisionRoles Concept Reference

Roles a Gated DeltaNet block instantiates, on top of the feed-forward pair. More...

Concept definition

        template<typename TPlan>
        concept DeltaNetPrecisionRoles =
        requires {
            typename TPlan::DeltaNetQueryKey;
            typename TPlan::DeltaNetValueGateOutput;
            typename TPlan::DeltaNetGating;
        }
        && WeightQuantPolicy<typename TPlan::DeltaNetQueryKey>
        && WeightQuantPolicy<typename TPlan::DeltaNetValueGateOutput>
        && WeightQuantPolicy<typename TPlan::DeltaNetGating>

Detailed Description

Roles a Gated DeltaNet block instantiates, on top of the feed-forward pair.

The DeltaNet block is Phase 3 and does not exist yet; this concept is here because the plan below already names its roles, and a role no concept ever checks is a comment. When the block lands it requires this alongside FeedForwardPrecisionRoles, exactly as QwenAttentionBlock requires AttentionPrecisionRoles.

The query/key pair separates from the value/gate/output group because section 5 ranks them a half-step apart: the recurrence reads q and k to address the state matrix, while v, the output gate and o carry the magnitude that enters it.