Dnn.Components.MixtureOfExperts Module Reference

Exported Modules
module	Compute.DeviceAllocation
module	Compute.Observation
module	Dnn.ActivationType
module	Dnn.TensorTypes
module	Dnn.Component
module	Dnn.TensorDataType
module	Dnn.TensorDataTypeTraits
module	Serialization.ModelArchive
module	Dnn.Quantization.Weight.Policies
module	Dnn.ComponentType
module	Compute.ExecutionContextFactory
module	Serialization.Tensor
module	Dnn.TensorOps
module	Serialization.Metadata
module	Compute.OperationTraits
module	Dnn.Tensor
module	Compute.DeviceId
module	Serialization.SafeTensors
module	Dnn.Components.MixtureOfExpertsConfig
module	Compute.IExecutionContext
module	Compute.DeviceTypeTraits
module	Dnn.Components.Activation
module	Serialization.Mode
module	Compute.ExecutionContext
module	Compute.DeviceType
module	Dnn.ITensor

Classes
class	Mila::Dnn::MixtureOfExperts< TDeviceType, TPrecision, TGate, TWeightQuantization >
	Stacked experts: gate_up_proj [E, 2I, H] and down_proj [E, H, I], checkpoint-named. More...

Files
file	Mila/Src/Dnn/Components/MixtureOfExperts/MixtureOfExperts.ixx
	Mixture-of-experts bank: the stacked experts and the weighted combine, routed from outside.