sparkinfer

Fastest MoE/LLM inference runtime for consumer and edge Blackwell GPUs. SN74 on Gittensor.

gittensor-ai-lab/sparkinfer

View on GitHub ↗

Fastest MoE/LLM inference runtime for consumer and edge Blackwell GPUs. SN74 on Gittensor.

sparkinfer grew +5.7% in the last 30 days, ranks 300th of 343 in Inference Optimization, last commit 41 days ago.

Star trajectory

71 days tracked
2026-09-1193 stars today2026-11-22
Momentum

+5.7% in 30d, above the Inference Optimization median of +1.6%.

Maintenance

Last commit 41 days ago, staler than the median of 16d.

How it compares in Inference Optimization

sparkinfer
93
stars
+5.7% 30d
commit 41 days ago
Inference Optimization median
242
stars
+1.6% 30d
commit ~16d ago
Leader · supertonic
13.8k
stars
-0.1% 30d
commit 74 days ago

Ranked 300th of 343 by stars.

Ranked by shared tags, then keywords, then stars
  1. Run Mixtral-8x7B models in Colab or consumer desktops

    -3 (-0.1%) in 30d · Committed 957d ago · Python · MIT

    Shares Inference Optimization, Model Optimization, Web / Edge / On-Device Runtime

  2. OrbitKV is a High Performance KV Cache System for LLM Inference.

    -52 (-15%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Inference Optimization, Model Optimization, Web / Edge / On-Device Runtime

  3. An inference chip design that runs a real LLM (Qwen2.5-0.5B) on FPGA — one transformer decoder layer in RTL…

    -123 (-22%) in 30d · Committed 95d ago · Python · Apache-2.0

    Shares Web / Edge / On-Device Runtime, Inference Optimization, Model Optimization

  4. On-device LLM Inference Powered by X-Bit Quantization

    +0 (+0.0%) in 30d · Committed 9d ago · Python · Apache-2.0

    Shares Web / Edge / On-Device Runtime, Inference Optimization, Model Optimization

  5. Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.

    +5 (+3.6%) in 30d · Committed 162d ago · Python · MIT

    Shares Inference Optimization, Web / Edge / On-Device Runtime, Model Optimization

  6. 2bit quants, kernels and runtime for consumer devices.

    Committed 16d ago · C++ · Other license

    Shares Web / Edge / On-Device Runtime, Inference Optimization, Model Optimization

  7. Run a 1-billion parameter LLM on a $10 board with 256MB RAM

    +6 (+0.3%) in 30d · Committed 272d ago · C · MIT

    Shares Web / Edge / On-Device Runtime, Inference Optimization, Model Optimization

  8. Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache…

    +2 (+0.6%) in 30d · Committed 549d ago · Python · Other license

    Shares Inference Optimization, Model Optimization, Web / Edge / On-Device Runtime

  9. RDNA-native LLM inference engine in Rust.

    +95 (+13%) in 30d · Committed today · Rust · Other license

    Shares Inference Optimization, Model Optimization

  10. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Inference Optimization, Model Optimization

Language
C++
License
MIT License
Contributors
42
Forks
82
Watchers
0
Open issues
4
Releases
60
Created
2026-06-22
Stars this week
+0

You've seen where sparkinfer stands today.

Get the moves before everyone else: the 5 Inference Optimization repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain sparkinfer?

Claim this page to add your own call-to-action.

Claim this page