Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
bigscience-workshop/Megatron-DeepSpeed
View on GitHub ↗Ongoing research training transformer language models at scale, including: BERT & GPT-2
Megatron-DeepSpeed grew -0.1% in the last 30 days, ranks 6th of 103 in Pretraining & Training, last commit 946 days ago.
Star trajectory
113 days tracked2026-07-021.4k stars today2026-10-23
Momentum
-0.1% in 30d, below the Pretraining & Training median of +0.0%.
Maintenance
Last commit 946 days ago, staler than the median of 161d.
How it compares in Pretraining & Training
Megatron-DeepSpeed
1.4k
stars
-0.1% 30d
commit 946 days ago
Pretraining & Training median
316
stars
+0.0% 30d
commit ~161d ago
Leader · maxtext
2.4k
stars
+0.7% 30d
commit today
Ranked 6th of 103 by stars.
kubeflow/katib1.7k-259intelligent-machine-learning/dlrover1.7k-241
bigscience-workshop/Megatron-DeepSpeed1.4k
ant-research/4DAnyone1.4k+7uclaml/SPIN1.3k+193- Language
- Python
- License
- Other
- Contributors
- 46
- Forks
- 225
- Watchers
- 20
- Open issues
- 76
- Releases
- 0
- Created
- 2021-07-02
- Stars this week
- -1
You've seen where Megatron-DeepSpeed stands today.
Get the moves before everyone else: the 5 Pretraining & Training repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.