OperationType.ixx File Reference

Defines the operation types supported by the compute framework. More...

#include <string>
#include <stdexcept>

Namespaces
namespace	Mila
	Mila main API namespace.

Enumerations
enum class	Mila::Dnn::Compute::OperationType {   CrossEntropyOp , TokenEmbeddingOp , LpeOp , RopeOp ,   FusedOp , LinearOp , GeluOp , ElementwiseActivationOp ,   SwigluOp , GegluOp , LayerNormOp , RmsNormOp ,   MultiHeadAttentionOp , GroupedQueryAttentionOp , ResidualOp , SoftmaxOp ,   DropoutOp , SamplingOp , SoftmaxCrossEntropyOp }
	Enumeration of all supported neural network operation types. More...

Functions
std::string_view	Mila::Dnn::Compute::operationTypeToString (OperationType op)

Variables
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::CrossEntropy = "CrossEntropyOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Dropout = "DropoutOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::ElementwiseActivation = "ElementwiseActivationOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Fused = "FusedOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Geglu = "GegluOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Gelu = "GeluOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::GroupedQueryAttention = "GroupedQueryAttentionOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::LayerNorm = "LayerNormOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Linear = "LinearOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Lpe = "LpeOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::MultiHeadAttention = "MultiHeadAttentionOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Residual = "ResidualOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::RmsNorm = "RmsNormOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Rope = "RopeOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Sampling = "SamplingOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Softmax = "SoftmaxOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::SoftmaxCrossEntropy = "SoftmaxCrossEntropyOp"
	WIP – targeted for Llama training.
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::Swiglu = "SwigluOp"
constexpr std::string_view	Mila::Dnn::Compute::OperationNames::TokenEmbedding = "TokenEmbeddingOp"

Detailed Description

Defines the operation types supported by the compute framework.

ARCHITECTURAL NOTE (TODO): OperationType is an internal dispatch key used by the compute layer. It is not part of the public Mila API – ComponentType (Dnn.ComponentType) is the user-facing component identity. OperationType should be moved to Dnn::Core and removed from the public Mila.ixx re-exports so it is inaccessible to library consumers. Operations are an implementation detail of Components; users should never need to reference OperationType directly.

Enumeration Type Documentation

OperationType

enum class Mila::Dnn::Compute::OperationType

exportstrong

Enumeration of all supported neural network operation types.

This enumeration defines the different types of operations that can be executed by the compute framework. Each operation type corresponds to a specific neural network function or layer.

Enumerator
CrossEntropyOp	Cross entropy loss operation (host-based; used by GPT reference implementation).
TokenEmbeddingOp	Token embedding operation.
LpeOp	Learned Positional Embedding operation for transformer architecture.
RopeOp	Rotary Position Embedding operation for transformer architecture.
FusedOp	Fused operation combining multiple operations for performance optimization.
LinearOp	Linear (fully connected/dense) layer operation.
GeluOp	Gaussian Error Linear Unit activation function.
ElementwiseActivationOp	Functor-templated elementwise activation (GELU/SiLU/ReLU/Tanh/Sigmoid/LeakyReLU/Mish).
SwigluOp	SwiGLU (SiLU-gated) GLU FFN activation.
GegluOp	GeGLU (GELU-gated) GLU FFN activation – Gemma.
LayerNormOp	Layer normalization operation.
RmsNormOp	RMS normalization operation.
MultiHeadAttentionOp	Multi-head attention operation (MHA) for transformers.
GroupedQueryAttentionOp	Grouped Query Attention (GQA).
ResidualOp	Residual connection operation.
SoftmaxOp	Softmax activation function.
DropoutOp	Dropout regularization operation.
SamplingOp	Device-side token sampling from logits.
SoftmaxCrossEntropyOp	WIP: Fused softmax + cross-entropy loss – targeted for Llama training.