TensorOps.Transfer.ixx File Reference

Tensor transfer utilities – copy/dispatch helpers for tensor data movement. More...

#include <concepts>
#include <string>
#include <memory>
#include <span>
#include <type_traits>
#include <stdexcept>
import Serialization.Tensor;
import Compute.CpuMemoryResource;
import Compute.IExecutionContext;
import Compute.ExecutionContext;
import Compute.DeviceType;
import Compute.DeviceId;
import Compute.Device;
import Dnn.TensorDataTypeMap;
import Dnn.TensorOps.Base;
import Dnn.TensorDataTypeTraits;
import Dnn.TensorDataType;
import Dnn.Tensor;

Namespaces
namespace	Mila
	Mila main API namespace.

Functions
template<TensorDataType TSrcDataType, typename TSrcMemoryResource, TensorDataType TDstDataType, typename TDstMemoryResource> requires isValidTensor<TSrcDataType, TSrcMemoryResource> && isValidTensor<TDstDataType, TDstMemoryResource>
void	Mila::Dnn::copy (const Tensor< TSrcDataType, TSrcMemoryResource > &src, Tensor< TDstDataType, TDstMemoryResource > &dst, IExecutionContext *exec_context=nullptr)
	Copies tensor data from source to destination tensor with optional ExecutionContext.
template<TensorDataType TDstDataType, typename TDstMemoryResource> requires isValidTensor<TDstDataType, TDstMemoryResource>
void	Mila::Dnn::copyFromBlob (const Serialization::ITensorBlob &blob, Tensor< TDstDataType, TDstMemoryResource > &dst, IExecutionContext *exec_context=nullptr)
template<TensorDataType TSrcDataType, TensorDataType TDstDataType, typename TDstMemoryResource> requires isValidTensor<TDstDataType, TDstMemoryResource>
void	Mila::Dnn::copyFromBlobWithConversion (const Serialization::ITensorBlob &blob, Tensor< TDstDataType, TDstMemoryResource > &dst, IExecutionContext *exec_context=nullptr)
	Copy a serialized blob into a destination tensor, converting element types.
template<TensorDataType TDstDataType, TensorDataType TSrcDataType, typename TSrcMemoryResource> requires isValidTensor<TSrcDataType, TSrcMemoryResource> && isValidTensor<TDstDataType, CpuMemoryResource>
Tensor< TDstDataType, CpuMemoryResource >	Mila::Dnn::toHost (const Tensor< TSrcDataType, TSrcMemoryResource > &src, IExecutionContext *exec_context=nullptr)
	Create a host (CPU) tensor from src and copy data into it.

Detailed Description

Tensor transfer utilities – copy/dispatch helpers for tensor data movement.

Provides an exported generic copy() template that validates shapes, handles host/device accessibility, and dispatches to device-specific TensorOps implementations. Supports an optional execution context for stream control.

Function Documentation

copy()

template<TensorDataType TSrcDataType, typename TSrcMemoryResource, TensorDataType TDstDataType, typename TDstMemoryResource>
requires isValidTensor<TSrcDataType, TSrcMemoryResource> && isValidTensor<TDstDataType, TDstMemoryResource>

void Mila::Dnn::copy ( const Tensor< TSrcDataType, TSrcMemoryResource > & src, Tensor< TDstDataType, TDstMemoryResource > & dst, IExecutionContext * exec_context = nullptr )

export

Copies tensor data from source to destination tensor with optional ExecutionContext.

Transfers data from source tensor to pre-allocated destination tensor. Both tensors must have compatible shapes (same dimensions). Supports type conversion and cross-device transfers with explicit stream control.

Device compatibility rules:

• Host-accessible to host-accessible: Always allowed
• Host-accessible to device-only: Uses destination device
• Device-only to host-accessible: Uses source device
• Device-only to device-only: Must be same device type (e.g., both CUDA)

ExecutionContext handling:

• Optional ExecutionContext parameter for stream control (borrowed, not owned)
• When provided, operations use the context's stream (caller controls sync)
• When null, operations use default stream and synchronize before returning
• Raw pointer semantics ensure zero overhead

Template Parameters

TSrcDataType	Source tensor data type
TSrcMemoryResource	Source memory resource type
TDstDataType	Destination tensor data type
TDstMemoryResource	Destination memory resource type

Parameters

src	Source tensor to copy from
dst	Destination tensor to copy to (must be pre-allocated)
exec_context	Optional execution context for stream control (borrowed, not owned)

Exceptions

std::runtime_error

If device-only tensors are on incompatible device types

Note

exec_context must outlive this function call

When exec_context provided, caller controls synchronization

When exec_context is null, uses default stream and synchronizes before returning

For CPU-only operations, exec_context parameter is ignored

Example:

// With explicit context (async)
auto ctx = std::make_unique<CudaExecutionContext>(0);
copy(src_tensor, dst_tensor, ctx.get());
ctx->synchronize();
 
// Without context (sync)
copy(src_tensor, dst_tensor);  // Returns after completion

copyFromBlobWithConversion()

template<TensorDataType TSrcDataType, TensorDataType TDstDataType, typename TDstMemoryResource>
requires isValidTensor<TDstDataType, TDstMemoryResource>

void Mila::Dnn::copyFromBlobWithConversion ( const Serialization::ITensorBlob & blob, Tensor< TDstDataType, TDstMemoryResource > & dst, IExecutionContext * exec_context = nullptr )

export

Copy a serialized blob into a destination tensor, converting element types.

Intended for quantize-on-load paths where the checkpoint dtype (TSrcDataType) differs from the weight storage dtype (TDstDataType). Shape is validated against the destination tensor. Dispatches to the device-specific backend.

Template Parameters

TSrcDataType	Blob element dtype (e.g. BF16).
TDstDataType	Destination tensor dtype (e.g. FP8_E4M3).
TDstMemoryResource	Destination memory resource.

Parameters

blob	Source tensor blob.
dst	Pre-allocated destination tensor.
exec_context	Optional execution context for stream control (borrowed).

Exceptions

std::invalid_argument

if blob shape != dst shape.

toHost()

template<TensorDataType TDstDataType, TensorDataType TSrcDataType, typename TSrcMemoryResource>
requires isValidTensor<TSrcDataType, TSrcMemoryResource> && isValidTensor<TDstDataType, CpuMemoryResource>

Tensor< TDstDataType, CpuMemoryResource > Mila::Dnn::toHost ( const Tensor< TSrcDataType, TSrcMemoryResource > & src, IExecutionContext * exec_context = nullptr )

export

Create a host (CPU) tensor from src and copy data into it.

By default the destination data type matches the source data type. The destination tensor preserves the source shape. An optional execution context may be supplied for device-side stream control when the source is device-resident.

Template Parameters

TSrcDataType	Source tensor data type
TSrcMemoryResource	Source memory resource type
TDstDataType	Destination tensor data type (defaults to source type)

Parameters

src	Source tensor to copy from
exec_context	Optional execution context for stream control (borrowed)

Returns

Tensor on CPU with copied data