Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory > Class Template Reference

Device-agnostic buffer for storing tensor data with abstract type system. More...

Public Types
using	DataType = TensorDataType
	Abstract data type enumeration.
using	DataTypeTraits = TensorDataTypeTraits<TDataType>
	Compile-time data type characteristics.

Public Member Functions
	TensorBuffer (const TensorBuffer &)=delete
	Copy operations explicitly deleted for performance safety.
	TensorBuffer (int device_id, size_t logical_size)
	Constructs buffer with owned memory.
	TensorBuffer (TensorBuffer &&other) noexcept
	Move constructor for efficient ownership transfer.
	~TensorBuffer ()
	Destructor with automatic memory cleanup via RAII.
size_t	alignedSize () const noexcept
	Returns the aligned memory allocation size in bytes.
const void *	data () const noexcept
	Returns const raw pointer to buffer data.
void *	data () noexcept
	Returns raw pointer to buffer data.
bool	empty () const noexcept
	Checks if the buffer is empty.
Compute::MemoryResource *	getMemoryResource () const noexcept
	Returns pointer to the memory resource managing this buffer's storage.
bool	isAligned () const noexcept
	Checks if buffer memory is properly aligned.
TensorBuffer &	operator= (const TensorBuffer &)=delete
TensorBuffer &	operator= (TensorBuffer &&other) noexcept
	Move assignment operator for efficient ownership transfer.
void	resize (size_t new_logical_size)
	Resizes buffer WITHOUT preserving existing data.
size_t	size () const noexcept
	Returns the number of logical elements in the buffer.
size_t	storageBytes () const noexcept
	Returns the storage size in bytes.

Static Public Attributes
static constexpr size_t	alignment = Detail::get_alignment<TDataType, TMemoryResource>()
	Optimal memory alignment.
static constexpr TensorDataType	data_type = TDataType
	Compile-time data type constant.
static constexpr size_t	element_size = DataTypeTraits::size_in_bytes
	Storage size per element.
static constexpr bool	is_device_only = DataTypeTraits::is_device_only
	Device-only type restriction.
static constexpr bool	is_float_type = DataTypeTraits::is_float_type
	Floating-point type classification.
static constexpr bool	is_integer_type = DataTypeTraits::is_integer_type
	Integer type classification.

Detailed Description

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>
requires isValidTensor<TDataType, TMemoryResource>
class Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >

Device-agnostic buffer for storing tensor data with abstract type system.

Advanced memory management container providing efficient storage for tensor data across heterogeneous compute environments using abstract TensorDataType enumeration. Supports device-specific alignment optimization and automatic compatibility validation.

Core architectural principles:

• Abstract data types prevent device-specific compilation issues
• Automatic memory alignment optimization for target hardware
• Support for precision formats including FP32, FP16, BF16, FP8, and integer types
• Device-agnostic memory operations with compile-time dispatch optimization
• Optional allocation tracking for memory profiling and debugging
• Exception-safe design with strong safety guarantees

The buffer supports both owned memory management and external memory wrapping, enabling integration with existing memory pools and external libraries while maintaining optimal performance characteristics.

Template Parameters

TDataType	Abstract tensor data type from TensorDataType enumeration
TMemoryResource	Memory resource type determining allocation strategy and device targeting
TrackMemory	When true, enables detailed memory allocation tracking and profiling

Note

Thread Safety: Buffer operations are not thread-safe; external synchronization required

Exception Safety: Strong guarantee for most operations; basic guarantee for constructors

Memory Layout: Automatic optimization for device-specific alignment

Memory Transfers: Transfer operations belong at the Tensor level where device contexts are meaningful

See also

TensorDataType for supported abstract data type enumeration

TensorDataTypeTraits for compile-time data type characteristics

MemoryResource for device memory abstraction layer

Example usage:

// CPU buffer with FP32 data type
TensorBuffer<TensorDataType::FP32, Compute::CpuMemoryResource> cpuBuffer(device_context, 100);
 
// GPU buffer with FP16 data type
TensorBuffer<TensorDataType::FP16, Compute::CudaDeviceMemoryResource> gpuBuffer(device_context, 100);
 
// Buffer with memory tracking enabled
TensorBuffer<TensorDataType::BF16, Compute::CudaDeviceMemoryResource, true> trackedBuffer(device_context, 100);

Constructor & Destructor Documentation

TensorBuffer() [1/3]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::TensorBuffer ( int device_id, size_t logical_size )

inlineexplicit

Constructs buffer with owned memory.

Allocates optimally aligned memory using the specified memory resource and initializes all memory to zero for deterministic behavior. Memory alignment is optimized based on data type and target hardware.

Parameters

device_id	Device identifier for memory resource initialization
logical_size	Number of logical elements to store in the buffer

Exceptions

std::invalid_argument	If device_id is invalid
std::overflow_error	If size causes overflow in storage calculations
std::bad_alloc	If memory allocation fails
std::runtime_error	If memory resource operations fail

Note

Memory is aligned according to hardware optimization requirements

Zero-sized buffers are handled efficiently without allocation

~TensorBuffer()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::~TensorBuffer ( )

inline

Destructor with automatic memory cleanup via RAII.

Automatically deallocates owned memory through the memory resource. External memory is not deallocated, maintaining safe resource management. Provides optional deallocation tracking for memory profiling.

TensorBuffer() [2/3]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::TensorBuffer ( const TensorBuffer< TDataType, TMemoryResource, TrackMemory > & )

delete

Copy operations explicitly deleted for performance safety.

Prevents accidental expensive copy operations involving large memory transfers across different memory spaces and devices.

TensorBuffer() [3/3]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::TensorBuffer ( TensorBuffer< TDataType, TMemoryResource, TrackMemory > && other )

inlinenoexcept

Move constructor for efficient ownership transfer.

Transfers all resources from source buffer without memory copying, leaving source in valid but empty state.

Member Function Documentation

alignedSize()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::alignedSize ( ) const

inlinenoexcept

Returns the aligned memory allocation size in bytes.

This represents the actual memory allocated, which includes alignment padding for optimal hardware performance.

Returns

Total allocated memory size including alignment padding

data() [1/2]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

const void * Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::data ( ) const

inlinenoexcept

Returns const raw pointer to buffer data.

Provides read-only access to the underlying memory buffer.

Returns

Const raw pointer to buffer memory

Warning

No type safety or bounds checking

data() [2/2]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

void * Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::data ( )

inlinenoexcept

Returns raw pointer to buffer data.

Provides direct access to the underlying memory buffer for performance-critical operations and device kernel interfacing.

Returns

Raw pointer to buffer memory

Warning

No type safety or bounds checking

Note

Use with appropriate casting based on data type requirements

empty()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::empty ( ) const

inlinenoexcept

Checks if the buffer is empty.

Returns

true if buffer contains no logical elements, false otherwise

getMemoryResource()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Compute::MemoryResource * Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::getMemoryResource ( ) const

inlinenoexcept

Returns pointer to the memory resource managing this buffer's storage.

Provides access to the memory resource for efficient dispatch optimization, zero-copy operations when memory resources are compatible, and type-safe downcasting to specific memory resource types.

Returns

Pointer to memory resource (nullptr if using external memory)

Note

Returns nullptr for buffers constructed with external memory

For owned memory, returns pointer to the underlying memory resource

Can be safely cast to TMemoryResource* based on template parameter

Enables efficient memory resource compatibility checks

Example:

TensorBuffer<TensorDataType::FP32, CudaMemoryResource> buffer(ctx, 100);
auto* mr = buffer.getMemoryResource();
 
// Type-safe downcast check
if (auto* cuda_mr = dynamic_cast<CudaMemoryResource*>(mr)) {
    // Use CUDA-specific operations
}

isAligned()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::isAligned ( ) const

inlinenoexcept

Checks if buffer memory is properly aligned.

Verifies that the buffer's data pointer meets the alignment requirements for optimal performance on the target hardware.

Returns

true if properly aligned, false otherwise

operator=()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

TensorBuffer & Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::operator= ( TensorBuffer< TDataType, TMemoryResource, TrackMemory > && other )

inlinenoexcept

Move assignment operator for efficient ownership transfer.

Safely transfers ownership while properly cleaning up existing resources.

resize()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

void Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::resize ( size_t new_logical_size )

inline

Resizes buffer WITHOUT preserving existing data.

Allocates new optimally aligned memory. Existing data is DISCARDED. Use for buffer reuse scenarios where data preservation is not required.

Parameters

new_logical_size

New number of logical elements

Exceptions

std::runtime_error	If buffer uses external memory
std::overflow_error	If new size causes storage overflow
std::bad_alloc	If memory allocation fails

Note

Does NOT preserve existing data (caller must reinitialize)

New memory is uninitialized for performance

size()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::size ( ) const

inlinenoexcept

Returns the number of logical elements in the buffer.

Returns

Number of logical elements stored in the buffer

storageBytes()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::storageBytes ( ) const

inlinenoexcept

Returns the storage size in bytes.

Provides the actual memory storage used.

Returns

Storage size in bytes

---

The documentation for this class was generated from the following file:

• Mila/Src/Dnn/Tensors/TensorBuffer.ixx