Mila
Deep Neural Network Library
Loading...
Searching...
No Matches
Mila::Dnn::Quant::Weight::LanguageModelHeadRole Concept Referenceexport

The output (language-model head) table. More...

Concept definition

template<typename TPlan>
requires { typename TPlan::LanguageModelHead; }
The output (language-model head) table.
Definition PrecisionPlan.ixx:102

Detailed Description

The output (language-model head) table.

Separate from EmbeddingTable because the two tables only share a policy when they share an allocation. A tied family (Gemma) names one policy twice and loses nothing; an untied one (Qwen 3.8, tie_word_embeddings: false) reads the head every decode step and gathers one row from the table, so the two are priced against different costs and land at different widths.