Mixture of Experts (MoE)
A model architecture where different sub-networks specialize in different inputs — gets you GPT-4 quality at smaller compute.
Definition
MoE models have many "expert" sub-networks but only activate a few per input. Mixtral, GPT-4, Llama 4 use MoE. From a worker view, MoE is invisible — you still see prompts and responses.
Examples
- Mixtral 8x22B
- Llama 4 Behemoth
- GPT-4 (per leaks)
Related terms
Looking for actual gigs that use Mixture of Experts (MoE)? Browse all live listings or filter by $100+/hr.