Mila::Dnn::Serialization::PretrainedMetadata Struct Reference

Metadata for pretrained model. More...

Public Attributes
std::string	activation
std::string	architecture
std::string	attention_type
uint32_t	embedding_dim
float	final_logit_softcapping
uint32_t	global_head_dim
uint32_t	global_rotary_dim
uint32_t	head_dim
uint32_t	hidden_dim
bool	key_equals_value
uint32_t	max_seq_length
std::string	model_name
float	norm_epsilon
std::string	norm_type
uint32_t	num_global_kv_heads
uint32_t	num_heads
uint32_t	num_kv_heads
uint32_t	num_layers
std::string	positional_encoding
float	rope_theta
float	rope_theta_global
float	rope_theta_local
uint32_t	sliding_window_pattern
bool	tie_word_embeddings = false
bool	use_bias
uint32_t	vocab_size
uint32_t	window

Detailed Description

Metadata for pretrained model.

---

The documentation for this struct was generated from the following file:

• Mila/Src/Dnn/Serialization/PretrainedReader.ixx