engine
2bit quants, kernels and runtime for consumer devices.
cactus-compute/engine
View on GitHub ↗2bit quants, kernels and runtime for consumer devices.
engine ranks 46th of 539 in Local Runtime / Inference Engine, last commit 16 days ago.
Star trajectory
14 days tracked- in 30d, below the Local Runtime / Inference Engine median of +1.9%.
Last commit 16 days ago, staler than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 46th of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere engine sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
+83 (+5.1%) in 30d · Committed 2d ago · Rust · EUPL-1.2
Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Inference Optimization, Language Bindings
True on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM inference…
+20 (+9.7%) in 30d · Committed yesterday · Kotlin · Other license
Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Language Bindings, Inference Optimization
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console…
Committed today · C# · BSD-3-Clause
Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime, Language Bindings
A model-agnostic GPU runtime for shipping models as verified programs.
+84 (+63%) in 30d · Committed 2d ago · Rust · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization, Language Bindings
OrbitKV is a High Performance KV Cache System for LLM Inference.
-52 (-15%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Inference Optimization, Model Optimization, Web / Edge / On-Device Runtime, Local Runtime / Inference Engine
Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…
+10 (+1.8%) in 30d · Committed yesterday · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime
Run generative AI models in sophgo BM1684X/BM1688
+3 (+0.9%) in 30d · Committed 4d ago · C++ · Other license
Shares Local Runtime / Inference Engine, Model Optimization, Inference Optimization
Swift package to run local LLMs on iOS, macOS, Linux
+0 (+0.0%) in 30d · Committed 87d ago · Swift · MIT
Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Language Bindings
A practical lab for exploring Apple's Core AI framework, model assets, specialization, and on-device…
+0 (+0.0%) in 30d · Committed 141d ago · Swift · MIT
Shares Web / Edge / On-Device Runtime, Local Runtime / Inference Engine, Model Optimization
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…
+3.4k (+37%) in 30d · Committed 5d ago · C · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
- Language
- C++
- License
- Other
- Contributors
- 52
- Forks
- 515
- Watchers
- 49
- Open issues
- 46
- Releases
- 23
- Created
- 2025-04-23
- Stars this week
- +5
You've seen where engine stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.