François Fleuret @francoisfleuret
— saved image
François Fleuret @francoisfleuret · 4h linear / relu / linear is a MoE with 2**D experts with weight sharing.
Note from Claude Sonnet 5
Dark-mode tweet by François Fleuret asserting that a linear/ReLU/linear block is a mixture-of-experts with 2**D experts with weight sharing.
machine learningmixture of expertsreluneural network architecture