Type-erased execution context interface. More...

Inheritance diagram for Mila::Dnn::Compute::IExecutionContext:

Public Member Functions
const ActivationObserver &	getActivationObserver () const noexcept
virtual DeviceId	getDeviceId () const noexcept=0
	Get the device identifier.
bool	hasActivationObserver () const noexcept
virtual void	releaseLoadStaging () noexcept
	Free the device buffer a load staged full-precision weights through.
virtual void	reserveScratch (std::size_t bytes)
	Reserve the forward scratch buffer at its final size.
void	setActivationObserver (ActivationObserver observer)
	Install the observer components publish their activations to.
virtual void	synchronize ()=0
	Synchronize all pending operations.

Detailed Description

Type-erased execution context interface.

Provides a minimal virtual interface for execution contexts. Specializations (CPU, CUDA, Metal, ROCm) inherit from this to enable polymorphic usage when the device type is not known at compile time.

For performance-critical code where the device type is known statically, use the templated ExecutionContext<TDeviceType> directly or cast_context<>() to avoid runtime overhead.

Member Function Documentation

◆ getDeviceId()

virtual DeviceId Mila::Dnn::Compute::IExecutionContext::getDeviceId ( ) const

nodiscardpure virtualnoexcept

Get the device identifier.

Returns: DeviceId Device identifier (type + index).

Implemented in Mila::Dnn::Compute::CpuExecutionContext, Mila::Dnn::Compute::CudaExecutionContext, Mila::Dnn::Compute::ExecutionContext< DeviceType::Metal >, and Mila::Dnn::Compute::ExecutionContext< DeviceType::Vulkan >.

◆ releaseLoadStaging()

virtual void Mila::Dnn::Compute::IExecutionContext::releaseLoadStaging ( )

inlinevirtualnoexcept

Free the device buffer a load staged full-precision weights through.

A load that fits full-precision weights on the device stages each tensor through one buffer, which is needed only until the last tensor has landed. The model load calls this when it returns, so the buffer is not held for the model's lifetime. Safe to call when nothing was staged; a no-op for contexts that stage nothing.

Reimplemented in Mila::Dnn::Compute::CudaExecutionContext.

◆ reserveScratch()

virtual void Mila::Dnn::Compute::IExecutionContext::reserveScratch ( std::size_t bytes )

inlinevirtual

Reserve the forward scratch buffer at its final size.

A network calls this at the end of its build with the largest scratch request any of its operations makes, which its footprint reports. A larger request afterwards is a prediction defect, and throws instead of growing. A context that never reserves grows on demand, which is what a component built on its own gets. A no-op for contexts that allocate no scratch.

Reimplemented in Mila::Dnn::Compute::CudaExecutionContext.

◆ setActivationObserver()

void Mila::Dnn::Compute::IExecutionContext::setActivationObserver ( ActivationObserver observer )

inline

Install the observer components publish their activations to.

The context is the transport because it is already shared parent to child across a whole model tree, so an observer installed here reaches every component without further plumbing. Pass an empty function to detach.

ONE CONTEXT, ONE MODEL TREE. Every transformer creates its own context from a DeviceId and no constructor accepts an existing one, so an observer installed here sees exactly one model. A future overload that accepted a context – a shared CUDA stream across two models is a reasonable thing to want – would make this a cross-model observation leak, so the contract must hold or this must change with it.

Not virtual and stored here rather than per backend: the behaviour is identical for every device.

◆ synchronize()

virtual void Mila::Dnn::Compute::IExecutionContext::synchronize ( )

pure virtual

Synchronize all pending operations.

Blocks until all operations submitted to this context complete. For CPU contexts, this is typically a no-op.