GatedDeltaRule.Config.ixx File Reference

Configuration for the gated delta rule (linear-attention mixer). More...

#include <stdexcept>
#include <string>
#include <sstream>
#include <utility>
import Serialization.Metadata;
import Dnn.ComponentConfig;
import Dnn.TensorTypes;

Classes
class	Mila::Dnn::GatedDeltaRuleConfig

Namespaces
namespace	Mila
	Mila main API namespace.

Detailed Description

Configuration for the gated delta rule (linear-attention mixer).

All four geometry values are structurally required. Qwen 3.8 27B uses 16 key heads, 48 value heads, and 128 for both head dimensions.