engine

2bit quants, kernels and runtime for consumer devices.

cactus-compute/engine

View on GitHub ↗

2bit quants, kernels and runtime for consumer devices.

engine ranks 46th of 539 in Local Runtime / Inference Engine, last commit 16 days ago.

Star trajectory

14 days tracked
2026-11-086.2k stars today2026-11-22
Momentum

- in 30d, below the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit 16 days ago, staler than the median of 6d.

How it compares in Local Runtime / Inference Engine

engine
6.2k
stars
- 30d
commit 16 days ago
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 46th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

    +83 (+5.1%) in 30d · Committed 2d ago · Rust · EUPL-1.2

    Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Inference Optimization, Language Bindings

  2. True on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM inference…

    +20 (+9.7%) in 30d · Committed yesterday · Kotlin · Other license

    Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Language Bindings, Inference Optimization

  3. A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console…

    Committed today · C# · BSD-3-Clause

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime, Language Bindings

  4. A model-agnostic GPU runtime for shipping models as verified programs.

    +84 (+63%) in 30d · Committed 2d ago · Rust · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization, Language Bindings

  5. OrbitKV is a High Performance KV Cache System for LLM Inference.

    -52 (-15%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Inference Optimization, Model Optimization, Web / Edge / On-Device Runtime, Local Runtime / Inference Engine

  6. Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…

    +10 (+1.8%) in 30d · Committed yesterday · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime

  7. Run generative AI models in sophgo BM1684X/BM1688

    +3 (+0.9%) in 30d · Committed 4d ago · C++ · Other license

    Shares Local Runtime / Inference Engine, Model Optimization, Inference Optimization

  8. Swift package to run local LLMs on iOS, macOS, Linux

    +0 (+0.0%) in 30d · Committed 87d ago · Swift · MIT

    Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Language Bindings

  9. A practical lab for exploring Apple's Core AI framework, model assets, specialization, and on-device…

    +0 (+0.0%) in 30d · Committed 141d ago · Swift · MIT

    Shares Web / Edge / On-Device Runtime, Local Runtime / Inference Engine, Model Optimization

  10. A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…

    +3.4k (+37%) in 30d · Committed 5d ago · C · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

Language
C++
License
Other
Contributors
52
Forks
515
Watchers
49
Open issues
46
Releases
23
Created
2025-04-23
Stars this week
+5

You've seen where engine stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain engine?

Claim this page to add your own call-to-action.

Claim this page