Mila
Deep Neural Network Library
Loading...
Searching...
No Matches
MixtureOfExperts Directory Reference

Files

 
MixtureOfExperts.Config.ixx
 Configuration for a mixture-of-experts bank: expert geometry, count, and experts per token.
 
MixtureOfExperts.ixx
 Mixture-of-experts bank: the stacked experts and the weighted combine, routed from outside.
 
Router.Config.ixx
 Configuration for mixture-of-experts routing: how many experts, and how many run per token.
 
Router.ixx
 Mixture-of-experts router: which experts each token runs, and with what combine weights.