TokenEmbedding.ixx File Reference

Device-templated TokenEmbedding component. More...

#include <memory>
#include <vector>
#include <string>
#include <sstream>
#include <iostream>
#include <stdexcept>
#include <optional>
#include <cmath>
#include <algorithm>
#include <numeric>
#include <format>
import Serialization.Tensor;
import Serialization.Mode;
import Serialization.Metadata;
import Serialization.ModelArchive;
import Logging.Logger;
import Compute.MemoryResource;
import Compute.ExecutionContextFactory;
import Compute.ExecutionContext;
import Compute.DeviceTypeTraits;
import Dnn.Components.TokenEmbeddingConfig;
import Dnn.Tensor;
import Dnn.Quantization.Weight.Policies;
import Compute.DeviceId;
import Dnn.Component;
import Dnn.ITensor;
import Compute.DeviceType;
import Dnn.TensorOps;
import Dnn.ComponentType;
import Dnn.TensorDataType;
import Dnn.TensorHelpers;
import Compute.OperationTraits;
import Dnn.TensorDataTypeTraits;
import Serialization.SafeTensors;
import Compute.CpuMemoryResource;
import Dnn.TensorTypes;
import Compute.Device;

Classes
class	Mila::Dnn::TokenEmbedding< TDeviceType, TIndex, TPrecision, TTableQuantization >
	Pure token embedding component (device-templated). More...

Namespaces
namespace	Mila
	Mila main API namespace.

Detailed Description

Device-templated TokenEmbedding component.

Pure vocabulary lookup: maps token indices [B, T] to dense vectors [B, T, C]. Owns the wte parameter and its gradient. No positional encoding – that is handled downstream by a dedicated encoding component (RoPE, ALiBi, or Learned).

Derived from Lpe with all wpe / IPositionalDecode / decode() concerns removed.