OScaR-KV-Quant

🏆 OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond — redefining the accuracy-efficiency Pareto front for X-LLMs KV quantization.

ZunhaiSu/OScaR-KV-Quant

View on GitHub ↗

🏆 OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond — redefining the accuracy-efficiency Pareto front for X-LLMs KV quantization.

OScaR-KV-Quant grew +1.4% in the last 30 days, ranks 83rd of 108 in Model Optimization, last commit 185 days ago.

Star trajectory

82 days tracked
2026-08-31144 stars today2026-11-22
Momentum

+1.4% in 30d, above the Model Optimization median of +0.6%.

Maintenance

Last commit 185 days ago, staler than the median of 43.5d.

How it compares in Model Optimization

OScaR-KV-Quant
144
stars
+1.4% 30d
commit 185 days ago
Model Optimization median
271
stars
+0.6% 30d
commit ~43.5d ago
Leader · kimi-k3-in-c
12.6k
stars
+37.0% 30d
commit 5 days ago

Ranked 83rd of 108 by stars.

Ranked by shared tags, then keywords, then stars

Where OScaR-KV-Quant sits

Hover or focus a node for details. Click to open it.

  • This repo
  • Related repo
  • Tag or category
  1. Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger…

    +9 (+10%) in 30d · Committed 16d ago · C++ · Apache-2.0

    Shares Inference Optimization, Model Optimization

  2. A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…

    +3.4k (+37%) in 30d · Committed 5d ago · C · Apache-2.0

    Shares Inference Optimization, Model Optimization

  3. Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127…

    +592 (+27%) in 30d · Committed 12d ago · Python · Apache-2.0

    Shares Inference Optimization, Model Optimization

  4. Run Mixtral-8x7B models in Colab or consumer desktops

    -3 (-0.1%) in 30d · Committed 957d ago · Python · MIT

    Shares Inference Optimization, Model Optimization

  5. INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

    +1 (+0.1%) in 30d · Committed 609d ago · C++ · MIT

    Shares Inference Optimization, Model Optimization

  6. Open deep learning compiler stack for Kendryte AI accelerators ✨

    +2 (+0.2%) in 30d · Committed 127d ago · C# · Apache-2.0

    Shares Inference Optimization, Model Optimization

  7. Suno AI's Bark model in C/C++ for fast text-to-speech generation

    +0 (+0.0%) in 30d · Committed 735d ago · C++ · MIT

    Shares Inference Optimization, Model Optimization

  8. RDNA-native LLM inference engine in Rust.

    +95 (+13%) in 30d · Committed today · Rust · Other license

    Shares Inference Optimization, Model Optimization

  9. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Inference Optimization, Model Optimization

  10. Frontier MoE models on RTX PRO 6000 / RTX 5090 (sm_120): DeepSeek-V4.1-Flash on 4x RTX PRO 6000, as a GPU+CPU…

    +36 (+6.6%) in 30d · Committed 3d ago · Sass · Apache-2.0

    Shares Inference Optimization, Model Optimization

Language
C++
License
MIT License
Contributors
3
Forks
15
Watchers
5
Open issues
1
Releases
0
Created
2026-05-19
Stars this week
+0

You've seen where OScaR-KV-Quant stands today.

Get the moves before everyone else: the 5 Model Optimization repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain OScaR-KV-Quant?

Claim this page to add your own call-to-action.

Claim this page