moe-l2
MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
yalun753/moe-l2
View on GitHub ↗MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
moe-l2 grew +0.7% in the last 30 days, ranks 232nd of 343 in Inference Optimization, last commit 51 days ago.
Star trajectory
82 days tracked+0.7% in 30d, below the Inference Optimization median of +1.6%.
Last commit 51 days ago, staler than the median of 16d.
How it compares in Inference Optimization
Ranked 232nd of 343 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere moe-l2 sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB…
+193 (+40%) in 30d · Committed 4d ago · Swift · MIT
Shares Inference Optimization, Local Runtime / Inference Engine
The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models…
+17 (+28%) in 30d · Committed today · Swift · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…
+3.4k (+37%) in 30d · Committed 5d ago · C · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization
Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash…
+1.6k (+108%) in 30d · Committed today · Cuda · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…
+962 (+243%) in 30d · Committed 3d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
- wici-ai/inferssd1.3k
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Committed today · C++ · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp…
+425 (+65%) in 30d · Committed today · C++ · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine
Run Kimi K3 locally on CPU with ~55GB measured runtime RAM. A single-file Linux inference server powered by…
Committed 25d ago · HTML · Other license
Shares Local Runtime / Inference Engine, Inference Optimization
RDNA-native LLM inference engine in Rust.
+95 (+13%) in 30d · Committed today · Rust · Other license
Shares Local Runtime / Inference Engine, Inference Optimization
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine
- Language
- Cuda
- License
- Apache License 2.0
- Contributors
- 2
- Forks
- 6
- Watchers
- 1
- Open issues
- 1
- Releases
- 15
- Created
- 2026-07-26
- Stars this week
- +0
You've seen where moe-l2 stands today.
Get the moves before everyone else: the 5 Inference Optimization repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.