Router.Config.ixx File Reference

Configuration for mixture-of-experts routing: how many experts, and how many run per token. More...

#include <stdexcept>
#include <string>
#include <sstream>
#include <format>
#include <utility>
#include <cstdint>
import Serialization.Metadata;
import Dnn.ComponentConfig;
import Dnn.TensorTypes;

Classes
class	Mila::Dnn::RouterConfig

Namespaces
namespace	Mila
	Mila main API namespace.

Detailed Description

Configuration for mixture-of-experts routing: how many experts, and how many run per token.

Gemma 4 26B-A4B routes each 2816-wide token to 8 of 128 experts. See Specifications/Gemma4MoE.md Phase 5.