DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
[2025/12] DeepSpeed Core API updates: PyTorch-style backward and low-precision master states [2025/10] SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips [2025/10] Study of ZenFlow and ZeRO offload performance with DeepSpeed CPU core binding [2025/08] ZenFlow: Stall-Free Offloading Engine for LLM Training [2025/06] Arctic Long Sequence Training (ALST) with DeepSpeed: Scalable And Efficient Training For Multi-Million Token Sequences DeepSpeed has been used to train many different large-scale models. Below is a list of several examples that we are aware of (if you’d like to include your model please submit a PR): DeepSpeed has been integrated with several different popular open-source DL frameworks such as: DeepSpeed is an integral part of Microsoft’s AI at Scale initiative to enable next-generation AI capabilities at scale. DeepSpeed welcomes your contributions! Please see our contributing guide for more details on formatting, testing, etc. This project welcomes contributions and suggestions. Most contributions require you to agree to a Developer Certificate of Origin (DCO)[https://wiki.linuxfoundation.org/dco] stating that they agree to the terms published at https://developercertificate.org for that particular contribution. DCOs are per-commit, so each commit needs to be signed off. These can be signed in the commit by adding the -s flag. DCO enforcement can also be signed off in the PR itself by clicking on the DCO enforcement check. Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko, Masahiro Tanaka, Stas Bekman, Olatunji Ruwase, Minjia Zhang. (2024) Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training arXiv:2406.18820
Beam
Run sandboxes, inference, and training with ultrafast boot times, instant autoscaling, and a developer experience that just works.
Run sandboxes, inference, and training with ultrafast boot times, instant autoscaling, and a developer experience that just works.
DeepSpeed
Beam
DeepSpeed
Beam
Only in DeepSpeed (1)
DeepSpeed
No screenshots
Beam
DeepSpeed
Beam
