Mila::Dnn::MemoryStats Struct Reference

Memory allocation breakdown for a single component. More...

Public Member Functions
MemoryStats &	operator+= (const MemoryStats &rhs) noexcept
	Accumulate another MemoryStats into this one.
std::string	toString () const
	Produce a human-readable summary of this stats instance.
std::size_t	totalBytes () const noexcept
	Total memory allocated across all categories and locations.
std::size_t	totalDeviceBytes () const noexcept
	Total device memory allocated by this component.
std::size_t	totalHostBytes () const noexcept
	Total host memory allocated by this component.

Public Attributes
std::size_t	device_gradient_bytes { 0 }
	Input and parameter gradient buffers.
std::size_t	device_parameter_bytes { 0 }
	Learnable parameter buffers (weights, biases).
std::size_t	device_state_bytes { 0 }
	Forward and decode output buffers, KV cache.
std::size_t	host_gradient_bytes { 0 }
	Gradient buffers on host (if any).
std::size_t	host_parameter_bytes { 0 }
	Learnable parameter buffers pinned on host (if any).
std::size_t	host_state_bytes { 0 }
	Forward and decode output buffers on host (if any).

Detailed Description

Memory allocation breakdown for a single component.

Reflects the current allocation state at the moment of the call. Categories map directly onto the component build lifecycle:

After construction nothing – construction allocates no device memory After build() parameters + state After setTrainingMode() parameters + state + gradients

Parameters are allocated in onBuilding(), not in the constructor. That is what lets Component::getRequiredMemory() report a footprint from a constructed but unbuilt graph. See Specifications/MemoryFootprint.md section 3.1.

All sizes are in bytes. Device and host allocations are tracked separately as they represent distinct, independently constrained resources.

Member Function Documentation

operator+=()

MemoryStats & Mila::Dnn::MemoryStats::operator+= ( const MemoryStats & rhs )

inlinenoexcept

Accumulate another MemoryStats into this one.

Used by CompositeComponent and Network to aggregate child stats.

toString()

std::string Mila::Dnn::MemoryStats::toString ( ) const

inlinenodiscard

Produce a human-readable summary of this stats instance.

Returns

Formatted multi-line string.

Member Data Documentation

device_gradient_bytes

std::size_t Mila::Dnn::MemoryStats::device_gradient_bytes { 0 }

Input and parameter gradient buffers.

Allocated lazily on first setTraining(true). Retained thereafter.

device_parameter_bytes

std::size_t Mila::Dnn::MemoryStats::device_parameter_bytes { 0 }

Learnable parameter buffers (weights, biases).

Allocated at construction. Static for the component lifetime.

device_state_bytes

std::size_t Mila::Dnn::MemoryStats::device_state_bytes { 0 }

Forward and decode output buffers, KV cache.

Allocated at build(). Static after build().

---

The documentation for this struct was generated from the following file:

• Mila/Src/Dnn/Core/Component.MemoryStats.ixx