Expert parallelism

MoE-specific technique: instead of splitting every layer, the experts (conditional sub-networks) are distributed across GPUs. A token only invokes the active experts, so VRAM is used more efficiently than with tensor parallelism.