⚡

Megatron / NeMo

4 test cases

NVIDIA's Megatron-LM and NeMo frameworks for large-scale LLM pre-training with tensor parallelism, pipeline parallelism, expert parallelism, and sequence parallelism.

⚡

Megatron-LM

NVIDIA's framework for training multi-billion parameter transformer models

Megatron-LMTensor ParallelPipeline ParallelExpert Parallel

⚡

NVIDIA NeMo

End-to-end framework for building, training, and deploying AI models

NeMoPre-trainingFine-tuningPEFTMulti-modal

⚡

NeMo RL

Reinforcement learning from human feedback with NeMo

NeMoRLHFPPOReward Models

🧪

BioNeMo

NVIDIA's framework for biomolecular AI model training

BioNeMoProteinDrug DiscoveryESM