|
Mila
Deep Neural Network Library
|
Files | |
| MixtureOfExperts.Config.ixx | |
| Configuration for a mixture-of-experts bank: expert geometry, count, and experts per token. | |
| MixtureOfExperts.ixx | |
| Mixture-of-experts bank: the stacked experts and the weighted combine, routed from outside. | |
| Router.Config.ixx | |
| Configuration for mixture-of-experts routing: how many experts, and how many run per token. | |
| Router.ixx | |
| Mixture-of-experts router: which experts each token runs, and with what combine weights. | |