llama-modchip
Zero-GPU CPU inference for 30B-284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.
synchopate/llama-modchip
View on GitHub ↗Zero-GPU CPU inference for 30B-284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.
llama-modchip ranks 118th of 539 in Local Runtime / Inference Engine, actively maintained, last commit today.
Star trajectory
20 days tracked- in 30d, below the Local Runtime / Inference Engine median of +1.9%.
Last commit today, fresher than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 118th of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere llama-modchip sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…
+962 (+243%) in 30d · Committed 3d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every…
+18 (+16%) in 30d · Committed 16d ago · Python · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models…
+341 (+32%) in 30d · Committed 3d ago · Shell · Other license
Shares Local Runtime / Inference Engine, Inference Optimization
RDNA-native LLM inference engine in Rust.
+95 (+13%) in 30d · Committed today · Rust · Other license
Shares Local Runtime / Inference Engine, Inference Optimization
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs…
+280 (+61%) in 30d · Committed yesterday · C · GPL-3.0
Shares Local Runtime / Inference Engine, Inference Optimization
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console…
Committed today · C# · BSD-3-Clause
Shares Local Runtime / Inference Engine, Inference Optimization
Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM)…
+264 (+90%) in 30d · Committed today · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
Rust + CUDA inference engine for NVIDIA RTX PRO 6000 Blackwell and RTX 5090. Serves safetensors and GGUF over…
+37 (+13%) in 30d · Committed today · OpenEdge ABL · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.
+2 (+0.7%) in 30d · Committed 2d ago · Go · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa…
+9 (+4.1%) in 30d · Committed 27d ago · Python · Other license
Shares Local Runtime / Inference Engine, Inference Optimization
- Language
- Python
- License
- Apache License 2.0
- Contributors
- 5
- Forks
- 137
- Watchers
- 12
- Open issues
- 27
- Releases
- 14
- Created
- 2026-10-29
- Stars this week
- +152
You've seen where llama-modchip stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.