Mila::Dnn::RopeConfig Class Reference

Inheritance diagram for Mila::Dnn::RopeConfig:

Public Member Functions
	RopeConfig (dim_t channels, dim_t n_heads, dim_t n_kv_heads, dim_t max_seq_len)
	Construct with all structurally required parameters.
void	fromMetadata (const SerializationMetadata &meta) override
	Populate configuration from provided metadata.
float	getBase () const noexcept
dim_t	getEmbeddingDim () const noexcept
dim_t	getHeadDim () const noexcept
	Per-head dimension, derived as channels / n_heads.
dim_t	getMaxSequenceLength () const noexcept
	Returns the training maximum sequence length.
dim_t	getNumHeads () const noexcept
dim_t	getNumKVHeads () const noexcept
dim_t	getRotaryDim () const noexcept
RotaryLayout	getRotaryLayout () const noexcept
SerializationMetadata	toMetadata () const override
	Convert configuration into a SerializationMetadata object.
std::string	toString () const override
	Produce a short, human-readable summary of the configuration.
void	validate () const override
	Validate configuration.
template<typename Self>
decltype(auto)	withBase (this Self &&self, float base)
	Set frequency base for rotary angle computation.
template<typename Self>
decltype(auto)	withRotaryDim (this Self &&self, dim_t rotary_dim)
	Set rotary sub-dimension per head (number of channels to rotate).
template<typename Self>
decltype(auto)	withRotaryLayout (this Self &&self, RotaryLayout rotary_layout)
	Select which dimension pairs a partial rotary rotates.
Public Member Functions inherited from Mila::Dnn::ComponentConfig
virtual	~ComponentConfig ()=default
	Virtual destructor for polymorphic base.

Constructor & Destructor Documentation

RopeConfig()

Mila::Dnn::RopeConfig::RopeConfig ( dim_t channels, dim_t n_heads, dim_t n_kv_heads, dim_t max_seq_len )

inline

Construct with all structurally required parameters.

Parameters

channels	Total Q embedding width (n_heads * head_dim).
n_heads	Number of query heads.
n_kv_heads	Number of key/value heads (GQA: <= n_heads).
max_seq_len	Trained maximum sequence length. A build longer than this is refused; the cos/sin tables are sized to the build's own length.

Member Function Documentation

fromMetadata()

void Mila::Dnn::RopeConfig::fromMetadata ( const SerializationMetadata & meta )

inlineoverridevirtual

Populate configuration from provided metadata.

Implementations should read available keys and leave missing keys at their current/default values to preserve forward/backward compatibility.

Parameters

meta	Metadata to read configuration values from.

Implements Mila::Dnn::ComponentConfig.

getHeadDim()

dim_t Mila::Dnn::RopeConfig::getHeadDim ( ) const

inlinenoexcept

Per-head dimension, derived as channels / n_heads.

Valid only after validate() has confirmed consistency.

getMaxSequenceLength()

dim_t Mila::Dnn::RopeConfig::getMaxSequenceLength ( ) const

inlinenoexcept

Returns the training maximum sequence length.

Returns

The maximum sequence length.

toMetadata()

SerializationMetadata Mila::Dnn::RopeConfig::toMetadata ( ) const

inlineoverridevirtual

Convert configuration into a SerializationMetadata object.

Implementations should include any fields required to fully reconstruct the configuration via fromMetadata.

Returns

SerializationMetadata Metadata representation of the config.

Implements Mila::Dnn::ComponentConfig.

toString()

std::string Mila::Dnn::RopeConfig::toString ( ) const

inlineoverridevirtual

Produce a short, human-readable summary of the configuration.

Implementations should return a compact, single-line description suitable for logging and debugging.

Returns

std::string Human-readable summary of the configuration.

Implements Mila::Dnn::ComponentConfig.

validate()

void Mila::Dnn::RopeConfig::validate ( ) const

inlineoverridevirtual

Validate configuration.

Enforces: required fields are positive, channels is divisible by n_heads, head_dim is even (RoPE requires paired dimensions), n_kv_heads <= n_heads, and rotary_dim (if set) does not exceed head_dim.

Exceptions

std::invalid_argument

on any violated constraint.

Implements Mila::Dnn::ComponentConfig.

withBase()

template<typename Self>

decltype(auto) Mila::Dnn::RopeConfig::withBase ( this Self && self, float base )

inline

Set frequency base for rotary angle computation.

Standard RoPE default is 10000.0f. Llama 3 uses 500000.0f. Default: 10000.0f.

withRotaryDim()

template<typename Self>

decltype(auto) Mila::Dnn::RopeConfig::withRotaryDim ( this Self && self, dim_t rotary_dim )

inline

Set rotary sub-dimension per head (number of channels to rotate).

Default: 0 – the full head_dim is rotated.

withRotaryLayout()

template<typename Self>

decltype(auto) Mila::Dnn::RopeConfig::withRotaryLayout ( this Self && self, RotaryLayout rotary_layout )

inline

Select which dimension pairs a partial rotary rotates.

Default: WholeHead, which is what Gemma's reference does and what every family in the tree used before Qwen. Only Qwen needs RotaryPrefix.

---

The documentation for this class was generated from the following file:

• Mila/Src/Dnn/Components/Encodings/Rope/Rope.Config.ixx