Tagged with

When Less Is Actually More: Cerebras' REAP Exposes Expert Merging as Flawed MoE Strategy

REAP pruning outperforms merging in MoE models, enabling near-lossless compression of 480B giants to local hardware