DeepSpeed All the tricks to scale to gigantic models By: Mark Saroufim

MUG '24 Day 26 DeepSpeed and Trillion parameter LLMs By: MVAPICH

DeepSpeed Ulysses System Optimizations for Enabling Training of Long Sequence Transformer Models By: Arxiv Papers

Large Model Training and Inference with DeepSpeed Samyam Rajbhandari LLMs in Prod Conference By: MLOps.community

ZeRO & Fastest BERT Increasing the scale and speed of deep learning training in DeepSpeed By: Microsoft Research

Colossal AI A Unified Deep Learning System For Large Scale Parallel Training By: HPC-AI Tech

Microsoft DeepSpeed introduction at KAUST By: KAUST Supercomputing Laboratory

Lecture 124 Scaling up Mixed precision Data parallelism FSDP By: DLVU