bonsai-ada-surgery

Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.

professorpalmer/bonsai-ada-surgery

View on GitHub ↗

Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.

bonsai-ada-surgery ranks 474th of 539 in Local Runtime / Inference Engine, actively maintained, last commit yesterday.

Star trajectory

14 days tracked
2026-11-08102 stars today2026-11-22
Momentum

- in 30d, below the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit yesterday, fresher than the median of 6d.

How it compares in Local Runtime / Inference Engine

bonsai-ada-surgery
102
stars
- 30d
commit yesterday
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 474th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…

    +962 (+243%) in 30d · Committed 3d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment

  2. Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM)…

    +264 (+90%) in 30d · Committed today · C++ · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  3. Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo…

    +36 (+15%) in 30d · Committed 2d ago · Python · MIT

    Shares Production Serving & Deployment, Local Runtime / Inference Engine, Inference Optimization

  4. run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and…

    +31 (+19%) in 30d · Committed 35d ago · Python

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  5. The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up…

    Committed 17d ago · MIT

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  6. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Production Serving & Deployment, Inference Optimization, Local Runtime / Inference Engine

  7. Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…

    +10 (+1.8%) in 30d · Committed yesterday · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

  8. a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features…

    +34 (+7.5%) in 30d · Committed today · C++ · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment

  9. llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

    +2 (+0.7%) in 30d · Committed 2d ago · Go · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Production Serving & Deployment

  10. One-click Qwen3.6-27B inference on Windows. 158 tok/s on RTX 5090, 72 tok/s on RTX 3090. Native, no WSL, no…

    -1 (-0.4%) in 30d · Committed 191d ago · Python

    Shares Local Runtime / Inference Engine, Production Serving & Deployment, Inference Optimization

Language
Python
License
Other
Contributors
2
Forks
6
Watchers
0
Open issues
0
Releases
44
Created
2026-09-19
Stars this week
+2

You've seen where bonsai-ada-surgery stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain bonsai-ada-surgery?

Claim this page to add your own call-to-action.

Claim this page