âš¡

Megatron / NeMo

6 test cases

NVIDIA's Megatron-LM and NeMo frameworks for large-scale LLM pre-training with tensor parallelism, pipeline parallelism, expert parallelism, and sequence parallelism.

âš¡

Megatron-LM

NVIDIA's framework for training multi-billion parameter transformer models

Megatron-LMTensor ParallelPipeline ParallelExpert Parallel
âš¡

NVIDIA NeMo

End-to-end framework for building, training, and deploying AI models

NeMoPre-trainingFine-tuningPEFTMulti-modal
âš¡

NeMo RL

Reinforcement learning from human feedback with NeMo

NeMoRLHFPPOReward Models
🧪

BioNeMo

NVIDIA's framework for biomolecular AI model training

BioNeMoProteinDrug DiscoveryESM
âš¡

Megatron-Bridge

NVIDIA Megatron-Bridge + UCCL-EP for MoE training with expert-parallel all-to-all over EFA

Megatron-BridgeMoEExpert ParallelUCCLEFA
âš¡

NeMo 1.0 (Legacy)

Legacy NeMo 1.0 training examples — superseded by NeMo 2.x

NeMoLegacyDeprecated