TyloQuant

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

Tylogi/TyloQuant

View on GitHub ↗

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

TyloQuant grew +10.5% in the last 30 days, ranks 296th of 343 in Inference Optimization, last commit 16 days ago.

Star trajectory

82 days tracked
2026-08-3195 stars today2026-11-22
Momentum

+10.5% in 30d, above the Inference Optimization median of +1.6%.

Maintenance

Last commit 16 days ago, staler than the median of 16d.

How it compares in Inference Optimization

TyloQuant
95
stars
+10.5% 30d
commit 16 days ago
Inference Optimization median
242
stars
+1.6% 30d
commit ~16d ago
Leader · supertonic
13.8k
stars
-0.1% 30d
commit 74 days ago

Ranked 296th of 343 by stars.

Ranked by shared tags, then keywords, then stars
  1. A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…

    +3.4k (+37%) in 30d · Committed 5d ago · C · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  2. RDNA-native LLM inference engine in Rust.

    +95 (+13%) in 30d · Committed today · Rust · Other license

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  3. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization

  4. Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM…

    +8 (+1.5%) in 30d · Committed 10d ago · C++ · Other license

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  5. The inference engine the open-source world built for itself.

    +6 (+2.5%) in 30d · Committed 56d ago · Python

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  6. Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K…

    +22 (+18%) in 30d · Committed 37d ago · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  7. Memra is a Rust + CUDA LLM inference engine with hand-written kernels, NVFP4 and mixed-precision execution…

    +4 (+100%) in 30d · Committed today · Rust · Other license

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  8. Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127…

    +592 (+27%) in 30d · Committed 12d ago · Python · Apache-2.0

    Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization

  9. INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

    +1 (+0.1%) in 30d · Committed 609d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

  10. Suno AI's Bark model in C/C++ for fast text-to-speech generation

    +0 (+0.0%) in 30d · Committed 735d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization

Language
C++
License
Apache License 2.0
Contributors
14
Forks
15
Watchers
0
Open issues
4
Releases
7
Created
2026-07-24
Stars this week
+0

You've seen where TyloQuant stands today.

Get the moves before everyone else: the 5 Inference Optimization repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain TyloQuant?

Claim this page to add your own call-to-action.

Claim this page