OScaR-KV-Quant
🏆 OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond — redefining the accuracy-efficiency Pareto front for X-LLMs KV quantization.
ZunhaiSu/OScaR-KV-Quant
View on GitHub ↗🏆 OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond — redefining the accuracy-efficiency Pareto front for X-LLMs KV quantization.
OScaR-KV-Quant grew +1.4% in the last 30 days, ranks 83rd of 108 in Model Optimization, last commit 185 days ago.
Star trajectory
82 days tracked+1.4% in 30d, above the Model Optimization median of +0.6%.
Last commit 185 days ago, staler than the median of 43.5d.
How it compares in Model Optimization
Ranked 83rd of 108 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere OScaR-KV-Quant sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger…
+9 (+10%) in 30d · Committed 16d ago · C++ · Apache-2.0
Shares Inference Optimization, Model Optimization
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…
+3.4k (+37%) in 30d · Committed 5d ago · C · Apache-2.0
Shares Inference Optimization, Model Optimization
- syv-ai/HyperQwen2.8k
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127…
+592 (+27%) in 30d · Committed 12d ago · Python · Apache-2.0
Shares Inference Optimization, Model Optimization
Run Mixtral-8x7B models in Colab or consumer desktops
-3 (-0.1%) in 30d · Committed 957d ago · Python · MIT
Shares Inference Optimization, Model Optimization
- RWKV/rwkv.cpp1.6k
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
+1 (+0.1%) in 30d · Committed 609d ago · C++ · MIT
Shares Inference Optimization, Model Optimization
Open deep learning compiler stack for Kendryte AI accelerators ✨
+2 (+0.2%) in 30d · Committed 127d ago · C# · Apache-2.0
Shares Inference Optimization, Model Optimization
Suno AI's Bark model in C/C++ for fast text-to-speech generation
+0 (+0.0%) in 30d · Committed 735d ago · C++ · MIT
Shares Inference Optimization, Model Optimization
RDNA-native LLM inference engine in Rust.
+95 (+13%) in 30d · Committed today · Rust · Other license
Shares Inference Optimization, Model Optimization
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Inference Optimization, Model Optimization
Frontier MoE models on RTX PRO 6000 / RTX 5090 (sm_120): DeepSeek-V4.1-Flash on 4x RTX PRO 6000, as a GPU+CPU…
+36 (+6.6%) in 30d · Committed 3d ago · Sass · Apache-2.0
Shares Inference Optimization, Model Optimization
- Language
- C++
- License
- MIT License
- Contributors
- 3
- Forks
- 15
- Watchers
- 5
- Open issues
- 1
- Releases
- 0
- Created
- 2026-05-19
- Stars this week
- +0
You've seen where OScaR-KV-Quant stands today.
Get the moves before everyone else: the 5 Model Optimization repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.