âš¡

Megatron / NeMo

6 test cases

NVIDIA's Megatron-LM and NeMo frameworks for large-scale LLM pre-training with tensor parallelism, pipeline parallelism, expert parallelism, and sequence parallelism.

âš¡

Megatron-LM

NVIDIA's framework for training multi-billion parameter transformer models

Megatron-LMTensor ParallelPipeline ParallelExpert Parallel
🧪

BioNeMo

NVIDIA's framework for biomolecular AI model training

BioNeMoProteinDrug DiscoveryESM
âš¡

NVIDIA NeMo

NVIDIA NeMo distributed training on AWS

NeMoPre-trainingFine-tuning
âš¡

NeMo RL

NeMo RL distributed training on AWS

NeMoRLHFPPO
âš¡

Megatron Bridge

Megatron Bridge distributed training on AWS

MegatronMoE
🎯

miles

GRPO reinforcement learning post-training with miles on Amazon SageMaker HyperPod EKS

RLGRPOPost-TrainingSGLangRay