GptModelConfig.ixx File Reference

Deployment configuration for Gpt2 language models. More...

#include <string>
import Dnn.TensorTypes;
import Dnn.LanguageModelConfig;

Classes
struct	Mila::Dnn::GptModelConfig
	Deployment configuration for Gpt language models. More...

Namespaces
namespace	Mila
	Mila main API namespace.

Detailed Description

Deployment configuration for Gpt2 language models.

GptModelConfig is the concrete configuration type passed to GptModel::fromPretrained(). It inherits all universal language model deployment concerns from LanguageModelConfig<GemmaModelConfig>:

• context_length – maximum sequence length
• weight_quantization – Linear weight storage strategy
• kv_cache_compression – GroupedQueryAttention cache strategy