Mila::Dnn::Serialization::WeightsMetadata Struct Reference

The model configuration a weights file carries. More...

Public Attributes
std::string	activation
std::string	architecture
bool	attention_output_gate = false
std::string	attention_type
uint32_t	embedding_dim
uint32_t	expert_hidden_dim = 0
float	final_logit_softcapping
uint32_t	full_attention_interval = 0
uint32_t	global_head_dim
uint32_t	global_rotary_dim
uint32_t	head_dim
uint32_t	hidden_dim
bool	key_equals_value
uint32_t	linear_conv_kernel_dim = 0
uint32_t	linear_head_dim = 0
uint32_t	linear_num_key_heads = 0
uint32_t	linear_num_value_heads = 0
uint32_t	max_seq_length
std::string	model_name
float	norm_epsilon
std::string	norm_type
uint32_t	num_experts = 0
uint32_t	num_global_kv_heads
uint32_t	num_heads
uint32_t	num_kv_heads
uint32_t	num_layers
float	partial_rotary_factor = 0.0f
std::string	positional_encoding
float	rope_theta
float	rope_theta_global
float	rope_theta_local
uint32_t	sliding_window_pattern
bool	tie_word_embeddings = false
uint32_t	top_k_experts = 0
bool	use_bias
uint32_t	vocab_size
uint32_t	window

Detailed Description

The model configuration a weights file carries.

---

The documentation for this struct was generated from the following file:

• Mila/Src/Dnn/Serialization/WeightsReader.ixx