LLMRepos·Pretraining & Training LLM projects

A simple, performant, and scalable Jax LLM!

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

[ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct

Automated Machine Learning on Kubernetes

DLRover: An Automatic Distributed Deep Learning System

Ongoing research training transformer language models at scale, including: BERT & GPT-2

The official implementation of Self-Play Fine-Tuning (SPIN)

(ICLR 2025) TabM: Advancing Tabular Deep Learning With Parameter-Efficient Ensembling

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

AxisRL is an agentic RL post-training framework built on SGLang rollout, Megatron training, and real-world agent workflows.

Fast & Simple Resource-Constrained Learning of Deep Network Structure

Byted PyTorch Distributed for Hyperscale Training of LLMs and RLs

🚀 Ultra Recipe for Training Long-Horizon Search Agents - matching frontier AI's search capability with a 20B model + stateful harness

Pure Rust implementation of a minimal Generative Pretrained Transformer

This repo contains a PyTorch implementation of a pretrained BERT model for multi-label text classification.

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Unified Reinforcement Learning Framework

The most atomic way to train and inference a GPT in pure, dependency-free C

An open-source solution for full parameter fine-tuning of DeepSeek-V3/R1 671B, including complete code and scripts from training to inference, as well as some practical experiences and conclusions. (DeepSeek-V3/R1 满血版 671B 全参数微调的开源解决方案，包含从训练到推理的完整代码和脚本，以及实践中积累一些经验和结论。)

Scaling Deep Research via Reinforcement Learning in Real-world Environments.

Building DeepSeek R1 from Scratch

The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

The code for the bark-voicecloning model. Training and inference.

Official repository for DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

[ICML 2026 & EMNLP 2026] Multimodal deep-research MLLM and benchmark. The first long-horizon multimodal deep-research MLLM, extending the number of reasoning turns to dozens and the number of search-engine interactions to hundreds.

[SIGGRAPH Asia 2026] 4DAnyone: Create Anyone in 4D from a Casual Monocular Video

Official implementation of OneDiffusion paper (CVPR 2025)

LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis

Generative Adversarial Text to Image Synthesis / Please Star -->

Official implementation for "Blended Diffusion for Text-driven Editing of Natural Images" [CVPR 2022]

An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale

从0到1构建一个MiniLLM (pretrain+sft+dpo实践中)

unofficial vits2-TTS implementation in pytorch

Named Entity Recognition using multilayered bidirectional LSTM

Official implementation for "Break-A-Scene: Extracting Multiple Concepts from a Single Image" [SIGGRAPH Asia 2023]

Implementation of paper Data Engineering for Scaling Language Models to 128K Context

Projects from basic algorithms to MARL. Implements MADDPG,MATD3,MA/HAPPO in Predator-Prey pursuit games with PettingZoo MPE environments.

Agentic RAG R1 Framework via Reinforcement Learning

Classify Kaggle Consumer Finance Complaints into 11 classes. Build the model with CNN (Convolutional Neural Network) and Word Embeddings on Tensorflow.