Dnn.Components.LlamaTransformer Module Reference

Exported Modules
module	Serialization.PretrainedReader
module	Dnn.Components.TokenEmbedding
module	Dnn.ActivationType
module	Dnn.Components.RmsNorm
module	Compute.DeviceTypeTraits
module	Serialization.Tensor
module	Serialization.ModelArchive
module	Dnn.Components.Linear
module	Dnn.Quantization.Weight.Policies
module	Dnn.TensorDataTypeTraits
module	Compute.GqaState
module	Dnn.TensorDataType
module	Dnn.ComponentType
module	Compute.ExecutionContextFactory
module	Dnn.Components.Rope
module	Compute.DeviceId
module	Dnn.ModelType
module	Compute.Device
module	Dnn.Tensor
module	Compute.ExecutionContext
module	Dnn.LanguageNetwork
module	Compute.DeviceType
module	Dnn.ITensor
module	Dnn.TensorTypes
module	Compute.CpuMemoryResource
module	Dnn.Component
module	Dnn.Quantization.KvCache.Policy

Classes
class	Mila::Dnn::LlamaBlock< TDeviceType, TPrecision, TWeightQuant, TKvPolicy >
class	Mila::Dnn::LlamaConfig
	Network-level configuration for LLaMA-style transformer networks. More...
class	Mila::Dnn::LlamaTransformer< TDeviceType, TPrecision, TWeightQuantization, TKvCachePolicy >
	LLaMA-style transformer (decoder-only) for autoregressive token prediction. More...

Functions
int64_t	Mila::Dnn::computePrefillChunkSize (int64_t batch, int64_t num_heads, int64_t head_dim, int64_t context_length, int64_t precision_bytes)
LlamaConfig	Mila::Dnn::Llama2_13B ()
	Llama 2 13B.
LlamaConfig	Mila::Dnn::Llama2_70B ()
	Llama 2 70B.
LlamaConfig	Mila::Dnn::Llama2_7B ()
	Llama 2 7B.
LlamaConfig	Mila::Dnn::Llama3_1_405B ()
	Llama 3.1 405B.
LlamaConfig	Mila::Dnn::Llama3_1_70B ()
	Llama 3.1 70B.
LlamaConfig	Mila::Dnn::Llama3_1_8B ()
	Llama 3.1 8B.
LlamaConfig	Mila::Dnn::Llama3_2_1B ()
	Usage Examples:
LlamaConfig	Mila::Dnn::Llama3_2_3B ()
	Llama 3.2 3B.
LlamaConfig	Mila::Dnn::Llama3_70B ()
	Llama 3 70B (Original release).
LlamaConfig	Mila::Dnn::Llama3_8B ()
	Llama 3 8B (Original release).

Variables
constexpr int64_t	Mila::Dnn::kPrefillScratchByteCap = int64_t{ 1536 } * 1024 * 1024

Files
file	Mila/Src/Dnn/Components/Transformers/LlaMa/Llama.ixx
	LLaMA-style decoder-only transformer network.
file	Mila/Src/Dnn/Components/Transformers/LlaMa/Llama.Block.ixx
	LLaMA transformer block – module partition of LlamaTransformer.
file	Mila/Src/Dnn/Components/Transformers/LlaMa/Llama.Config.ixx
	LLaMA network-level configuration.
file	Mila/Src/Dnn/Components/Transformers/LlaMa/Llama.Presets.ixx