RustingStrata

Pure-Rust LLM inference engine for GGUF (Qwen, DeepSeek, GLM, Hy4, Flint). Runs models bigger than your VRAM on CUDA and Vulkan, with MTP drafts and an OpenAI/Anthropic server. 12 GB RTX 3060 vs llama.cpp: Qwen3.6-35B-A3B UD-Q5_K_M 80.5 vs 64.4 tok/s, Flint UD-Q5_K_XL 64.6 vs 48.6. With MTP: 108 tok/s (UD-Q3_K_M), up to 160 (UD-Q2_K_XL).

RustingStudio/RustingStrata

View on GitHub ↗

Pure-Rust LLM inference engine for GGUF (Qwen, DeepSeek, GLM, Hy4, Flint). Runs models bigger than your VRAM on CUDA and Vulkan, with MTP drafts and an OpenAI/Anthropic server. 12 GB RTX 3060 vs llama.cpp: Qwen3.6-35B-A3B UD-Q5_K_M 80.5 vs 64.4 tok/s, Flint UD-Q5_K_XL 64.6 vs 48.6. With MTP: 108 tok/s (UD-Q3_K_M), up to 160 (UD-Q2_K_XL).

RustingStrata ranks 280th of 539 in Local Runtime / Inference Engine, actively maintained, last commit today.

Star trajectory

14 days tracked
2026-11-08345 stars today2026-11-22
Momentum

- in 30d, below the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit today, fresher than the median of 6d.

How it compares in Local Runtime / Inference Engine

RustingStrata
345
stars
- 30d
commit today
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 280th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Production Serving & Deployment, Inference Optimization, Local Runtime / Inference Engine

  2. Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash…

    +1.6k (+108%) in 30d · Committed today · Cuda · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  3. Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…

    +962 (+243%) in 30d · Committed 3d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment

  4. Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM)…

    +264 (+90%) in 30d · Committed today · C++ · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  5. Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090…

    Committed today · C++ · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  6. Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in…

    Committed today · Rust · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  7. The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models…

    +17 (+28%) in 30d · Committed today · Swift · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment

  8. An inference engine for NVIDIA RTX card card family, built for a coding agent and its subagents, and a…

    Committed today · Rust · Apache-2.0

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  9. A Rust-native LLM serving engine. One binary, no Python runtime.

    +1 (+4.8%) in 30d · Committed today · Rust · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  10. Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

    Committed today · C++ · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment

Language
Rust
License
Other
Contributors
12
Forks
40
Watchers
2
Open issues
13
Releases
7
Created
2026-10-10
Stars this week
+107

You've seen where RustingStrata stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain RustingStrata?

Claim this page to add your own call-to-action.

Claim this page