inferssd
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
wici-ai/inferssd
View on GitHub ↗Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
inferssd ranks 123rd of 539 in Local Runtime / Inference Engine, actively maintained, last commit today.
Star trajectory
20 days tracked- in 30d, below the Local Runtime / Inference Engine median of +1.9%.
Last commit today, fresher than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 123rd of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere inferssd sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…
+962 (+243%) in 30d · Committed 3d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment
Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090…
Committed today · C++ · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models…
+17 (+28%) in 30d · Committed today · Swift · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment
Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash…
+1.6k (+108%) in 30d · Committed today · Cuda · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Production Serving & Deployment, Inference Optimization, Local Runtime / Inference Engine
Run many AI agents at once on your Mac, up to 3x faster.
+273 (+91%) in 30d · Committed today · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM)…
+264 (+90%) in 30d · Committed today · C++ · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features…
+34 (+7.5%) in 30d · Committed today · C++ · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment
lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.
+43 (+14%) in 30d · Committed 31d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
The inference engine the open-source world built for itself.
+6 (+2.5%) in 30d · Committed 56d ago · Python
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
- Language
- C++
- License
- Apache License 2.0
- Contributors
- 12
- Forks
- 115
- Watchers
- 10
- Open issues
- 16
- Releases
- 8
- Created
- 2026-10-22
- Stars this week
- +130
You've seen where inferssd stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.