AttentionType.ixx File Reference

Defines attention mechanism types used by transformer components. More...

#include <string>
#include <stdexcept>

Namespaces
namespace	Mila
	Mila main API namespace.

Enumerations
enum class	Mila::Dnn::AttentionType { MultiHead , GroupedQuery , MultiQuery }
	Enumeration of supported attention mechanism types. More...

Functions
std::string	Mila::Dnn::attentionTypeToString (AttentionType t)
	Convert AttentionType to string.
AttentionType	Mila::Dnn::stringToAttentionType (const std::string &v)
	Parse string to AttentionType.

Detailed Description

Defines attention mechanism types used by transformer components.

Enumeration Type Documentation

AttentionType

enum class Mila::Dnn::AttentionType

exportstrong

Enumeration of supported attention mechanism types.

Enumerator
MultiHead	Multi-Head Attention (MHA): independent Q, K, V per head.
GroupedQuery	Grouped Query Attention (GQA): Q heads grouped over shared K/V heads.
MultiQuery	Multi-Query Attention (MQA): all Q heads share a single K/V head.

Function Documentation

stringToAttentionType()

AttentionType Mila::Dnn::stringToAttentionType ( const std::string & v )

inlineexport

Parse string to AttentionType.

Exceptions

std::invalid_argument

on unknown value.