Mila::Dnn::CosineLRScheduler Class Reference

Cosine annealing scheduler. More...

Inheritance diagram for Mila::Dnn::CosineLRScheduler:

Public Member Functions
	CosineLRScheduler (double initial_lr, double final_lr, std::size_t total_steps)
double	getLearningRate (std::size_t step) const override
	Get the learning rate for the provided (zero-based) step.
std::string	toString () const override
	Human-readable description of the scheduler.

Detailed Description

Cosine annealing scheduler.

Uses cosine schedule from initial_lr to final_lr over total_steps. For step >= total_steps the scheduler returns final_lr.

Formula (t in [0,1]): lr = final + 0.5*(initial - final)*(1 + cos(pi * t))

Preconditions:

• total_steps > 0
• initial_lr >= 0, final_lr >= 0

Member Function Documentation

getLearningRate()

double Mila::Dnn::CosineLRScheduler::getLearningRate ( std::size_t step ) const

inlineoverridevirtual

Get the learning rate for the provided (zero-based) step.

Parameters

step	Zero-based step index.

Returns

Scalar learning rate to use at this step.

Implements Mila::Dnn::LearningRateScheduler.

toString()

std::string Mila::Dnn::CosineLRScheduler::toString ( ) const

inlineoverridevirtual

Human-readable description of the scheduler.

Implements Mila::Dnn::LearningRateScheduler.

---

The documentation for this class was generated from the following file:

• Mila/Src/Dnn/Core/LearningRateScheduler.ixx