KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
dipampaul17/KVSplit
View on GitHub ↗Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
KVSplit grew -0.3% in the last 30 days, ranks 122nd of 303 in Inference Optimization, last commit 521 days ago.
Star trajectory
115 days tracked-0.3% in 30d, below the Inference Optimization median of +1.5%.
Last commit 521 days ago, staler than the median of 15d.
How it compares in Inference Optimization
Ranked 122nd of 303 by stars.
- Language
- Python
- License
- Other
- Contributors
- 3
- Forks
- 13
- Watchers
- 5
- Open issues
- 0
- Releases
- 0
- Created
- 2025-05-16
- Stars this week
- +0
You've seen where KVSplit stands today.
Get the moves before everyone else: the 5 Inference Optimization repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.