Expert Parallelism

Mixture-of-Experts routing and communication benchmarks

Overview

Benchmarks for evaluating all-to-all communication patterns used in Mixture-of-Experts (MoE) model architectures.