Mila::Dnn::SequenceLogLikelihood Struct Reference

Teacher-forced log-likelihood of a token sequence under a model. More...

Public Attributes
dim_t	scored_positions { 0 }
	Positions that contributed.
double	total_log_probability { 0.0 }
	Summed natural log of the probability the model assigned to each actual next token.

Detailed Description

Teacher-forced log-likelihood of a token sequence under a model.

The two fields are reported separately rather than pre-averaged because the caller usually sums several sequences before dividing: perplexity over a corpus is exp( -total_log_probability / total_scored_positions ), and averaging per sequence first would weight a short sequence like a long one.

Accumulated in double. Per-position log-probabilities are small negative numbers and a corpus contributes tens of thousands of them, so the running total is where precision is actually at risk – not in any single term.

Member Data Documentation

scored_positions

dim_t Mila::Dnn::SequenceLogLikelihood::scored_positions { 0 }

Positions that contributed.

One less than the sequence length: the first token has no preceding context, so nothing predicts it.

---

The documentation for this struct was generated from the following file:

• Mila/Src/Dnn/Core/LanguageModelNetwork.ixx