|
Mila
Deep Neural Network Library
|
The output (language-model head) table. More...
The output (language-model head) table.
Separate from EmbeddingTable because the two tables only share a policy when they share an allocation. A tied family (Gemma) names one policy twice and loses nothing; an untied one (Qwen 3.8, tie_word_embeddings: false) reads the head every decode step and gathers one row from the table, so the two are priced against different costs and land at different widths.