IDecoderLayer.ixx File Reference

Virtual inference interface for a heterogeneous decoder layer list. More...

#include <memory>
import Compute.GqaState;
import Compute.DeviceTypeTraits;
import Compute.DeviceType;
import Dnn.TensorDataType;
import Dnn.TensorTypes;
import Compute.Device;
import Dnn.Tensor;

Classes
class	Mila::Dnn::IDecoderLayer< TDeviceType, TPrecision >
	Polymorphic inference interface for one decoder layer. More...

Namespaces
namespace	Mila
	Mila main API namespace.

Detailed Description

Virtual inference interface for a heterogeneous decoder layer list.

Gemma interleaves two structurally different block types (local sliding and global full attention), so its transformer cannot hold a homogeneous vector<Block> the way GptTransformer / LlamaTransformer do. Both GemmaBlock instantiations implement this interface; the transformer drives the layer list polymorphically (one virtual call per layer per token step – negligible against the per-layer GEMMs). See Specifications/Gemma.md section 8.

Inference-only: Gemma is an inference target, so the interface exposes the prefill/decode KV-cache path and the shared-GQA-workspace wiring, not training forward/backward (those remain on the concrete CompositeComponent if ever needed).