QwenModel.ixx File Reference

Qwen 3.8 inference model. More...

#include <memory>
#include <vector>
#include <span>
#include <unordered_set>
#include <string>
#include <sstream>
#include <cstdint>
#include <stdexcept>
#include <filesystem>
#include <format>
#include <optional>
#include <algorithm>
#include <functional>
#include <stop_token>
#include <cstring>
#include <type_traits>
import Logging.Logger;
import Serialization.SafeTensors;
import Compute.ExecutionContextFactory;
import Compute.CpuMemoryResource;
import Compute.DeviceTypeTraits.Cpu;
import Compute.DeviceTypeTraits;
import Compute.DeviceType;
import Compute.DeviceId;
import Dnn.Components.QwenPrecisionPlan;
import Dnn.Components.QwenConfig;
import Compute.Device;
import Dnn.LanguageModel;
import Dnn.Components.QwenTransformer;
import Dnn.LanguageModelConfig;
import Dnn.ITensor;
import Dnn.GenerateStatus;
import Dnn.LanguageModelNetwork;
import Serialization.WeightsReader;
import Dnn.Quantization.Weight.Policies;
import Dnn.Models.QwenModelConfig;
import Dnn.Quantization.KvCache.Policy;
import Dnn.GenerateParams;
import Dnn.TensorDataType;
import Dnn.TensorDataTypeTraits;
import Serialization.Mode;
import Dnn.Tensor;
import Dnn.RuntimeMode;
import Dnn.TensorTypes;
import Dnn.Component;

Classes
class	Mila::Dnn::QwenModel< TDeviceType, TPrecision >
	Qwen 3.8 compatible inference model. More...

Namespaces
namespace	Mila
	Mila main API namespace.

Detailed Description

Qwen 3.8 inference model.

Inference-only wrapper around a loaded QwenTransformer network. Follows GemmaModel, which is the family this one is modelled on – both drive a heterogeneous layer stack – with two deliberate divergences, each forced by the Gated DeltaNet mixer:

• NO PROMPT-PREFIX REUSE. A recurrent state is a lossy summary of every position it has seen, so it cannot be rewound; QwenDeltaNetBlock::rewindKvCache always refuses and the transformer ANDs that into a stack-wide refusal. Gemma's reuse block is therefore absent here rather than present and permanently failing – machinery that can never fire reads as a capability. See Qwen3.8.md section 7.
• THE MIXER STATE IS SELF-CLEANING AT PREFILL, not reset by a call. prefill at position 0 zeroes the recurrent state and starts the convolution window cold (GatedDeltaRule::prefill, CausalConv1d::prefill), so a fresh generation cannot inherit the previous one's state. That is also precisely why prefillFrom at a non-zero offset must never be reached on this family: it would carry state forward for a prefix the caches no longer hold.

REFERENCE PRECISION ONLY, for now. Section 5's allocation is a per-role PLAN over codebook formats, and the artifact that carries it does not exist yet, so this entry point accepts the BF16 reference plan and refuses the uniform quantization modes rather than building something that is not the plan. That refusal is the Phase 4 / Phase 5 boundary, and it is named in the error.