BlackMamba: Mixture of Experts for State-Space Models
Paper
•
2402.01771
•
Published
•
22
Moe
-
-
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
Paper • 2402.01739 • Published • 26 -
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
Paper • 2401.15947 • Published • 46 -
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Paper • 2401.06066 • Published • 35 -
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
Paper • 2401.04081 • Published • 68 -
Mixtral of Experts
Paper • 2401.04088 • Published • 153 -
Scaling Laws for Fine-Grained Mixture of Experts
Paper • 2402.07871 • Published • 10 -
Mixtures of Experts Unlock Parameter Scaling for Deep RL
Paper • 2402.08609 • Published • 34 -
Multi-Head Mixture-of-Experts
Paper • 2404.15045 • Published • 55 -
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper • 2404.02258 • Published • 102