Mila::Dnn::Quant::Weight::FeedForwardPrecisionRoles Concept Reference

Fused gate+up projection and the down projection of a gated feed-forward network. More...

Concept definition

        template<typename TPlan>
        concept FeedForwardPrecisionRoles =
        requires { typename TPlan::FeedForwardGateUp; typename TPlan::FeedForwardDown; }
        && WeightQuantPolicy<typename TPlan::FeedForwardGateUp>
        && WeightQuantPolicy<typename TPlan::FeedForwardDown>

Detailed Description

Fused gate+up projection and the down projection of a gated feed-forward network.