kindi
Kindi 0.6.1: fast local LLM inference on low-cost GPUs and CPUs. DeepSeek-V4.1-Flash (552B) at 18.7 tok/s on one RTX 3060 12 GB with a dual-Xeon DDR4 server. Ternary Bonsai 2B-27B, AH-Nanbeige4.3-33B, Nemotron 3.5 Lightning 30B-A3B. NVIDIA from the Quadro K420 to the RTX 50 series, AMD Radeon graphics, Windows and Linux. OpenAI-compatible server.
ahtx/kindi
View on GitHub ↗Kindi 0.6.1: fast local LLM inference on low-cost GPUs and CPUs. DeepSeek-V4.1-Flash (552B) at 18.7 tok/s on one RTX 3060 12 GB with a dual-Xeon DDR4 server. Ternary Bonsai 2B-27B, AH-Nanbeige4.3-33B, Nemotron 3.5 Lightning 30B-A3B. NVIDIA from the Quadro K420 to the RTX 50 series, AMD Radeon graphics, Windows and Linux. OpenAI-compatible server.
kindi ranks 366th of 539 in Local Runtime / Inference Engine, actively maintained, last commit 5 days ago.
Star trajectory
21 days tracked- in 30d, below the Local Runtime / Inference Engine median of +1.9%.
Last commit 5 days ago, fresher than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 366th of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere kindi sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Production Serving & Deployment, Inference Optimization, Distributed Inference, Local Runtime / Inference Engine
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…
+962 (+243%) in 30d · Committed 3d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment
Native Windows vLLM 0.33.0 (Python 3.13/CUDA 13.0), plus a Python 3.14/CUDA 13.2 build; older versions on the…
+5 (+7.9%) in 30d · Committed 2d ago · Python · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM)…
+264 (+90%) in 30d · Committed today · C++ · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo…
+36 (+15%) in 30d · Committed 2d ago · Python · MIT
Shares Production Serving & Deployment, Local Runtime / Inference Engine, Inference Optimization
One-click Qwen3.6-27B inference on Windows. 158 tok/s on RTX 5090, 72 tok/s on RTX 3090. Native, no WSL, no…
-1 (-0.4%) in 30d · Committed 191d ago · Python
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and…
+31 (+19%) in 30d · Committed 35d ago · Python
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ…
+39 (+45%) in 30d · Committed today · C++ · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar…
Committed yesterday · Python · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment
The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up…
Committed 17d ago · MIT
Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization
- Language
- Python
- License
- GNU Affero General Public License v3.0
- Contributors
- 1
- Forks
- 42
- Watchers
- 6
- Open issues
- 16
- Releases
- 7
- Created
- 2026-10-20
- Stars this week
- +32
You've seen where kindi stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.