ActivationType.ixx File Reference

Definition of activation function types used throughout the Mila library. More...

#include <string>
#include <stdexcept>

Namespaces
namespace	Mila
	Mila main API namespace.

Enumerations
enum class	Mila::Dnn::ActivationType {   None , Relu , Gelu , Silu ,   Swiglu , Tanh , Sigmoid , LeakyRelu ,   Mish }
	Enumeration of supported activation function types. More...

Functions
std::string	Mila::Dnn::activationTypeToString (ActivationType type)
	Converts an ActivationType enum value to its string representation.
ActivationType	Mila::Dnn::stringToActivationType (const std::string &name)
	Converts a string to its corresponding ActivationType enum value.

Detailed Description

Definition of activation function types used throughout the Mila library.

Enumeration Type Documentation

ActivationType

enum class Mila::Dnn::ActivationType

exportstrong

Enumeration of supported activation function types.

This enum class defines the different activation functions that can be used throughout the Mila library, particularly in neural network layers.

Enumerator
None	No activation (identity function).
Relu	Rectified Linear Unit: max(0, x).
Gelu	Gaussian Error Linear Unit: x * phi(x) where phi() is the standard Gaussian CDF.
Silu	Sigmoid Linear Unit (Swish): x * sigmoid(x).
Swiglu	SwiGLU: gated activation x1 * GELU(x2).
Tanh	Hyperbolic Tangent: tanh(x).
Sigmoid	Sigmoid function: 1 / (1 + exp(-x)).
LeakyRelu	Leaky ReLU: max(alpha * x, x) where alpha is typically 0.01.
Mish	Mish: x * tanh(softplus(x)).

Function Documentation

activationTypeToString()

std::string Mila::Dnn::activationTypeToString ( ActivationType type )

inlineexport

Converts an ActivationType enum value to its string representation.

Parameters

type	The ActivationType to convert

Returns

std::string The string representation of the activation type

stringToActivationType()

ActivationType Mila::Dnn::stringToActivationType ( const std::string & name )

inlineexport

Converts a string to its corresponding ActivationType enum value.

Parameters

name	The string representation of an activation function

Returns

ActivationType The corresponding enum value

Exceptions

std::invalid_argument

if the string doesn't match any known activation function