HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
syv-ai/HyperQwen
View on GitHub ↗Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
HyperQwen grew +27.0% in the last 30 days, ranks 16th of 343 in Inference Optimization, actively maintained, last commit 12 days ago.
Star trajectory
93 days tracked+27.0% in 30d, above the Inference Optimization median of +1.6%.
Last commit 12 days ago, fresher than the median of 16d.
How it compares in Inference Optimization
Ranked 16th of 343 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere HyperQwen sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86…
+0 (+0.0%) in 30d · Committed 52d ago · Python · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization
RDNA-native LLM inference engine in Rust.
+95 (+13%) in 30d · Committed today · Rust · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo…
+36 (+15%) in 30d · Committed 2d ago · Python · MIT
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one…
+31 (+18%) in 30d · Committed today · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K…
+22 (+18%) in 30d · Committed 37d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
LLM Inference on consumer devices
+0 (+0.0%) in 30d · Committed 614d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Memra is a Rust + CUDA LLM inference engine with hand-written kernels, NVFP4 and mixed-precision execution…
+4 (+100%) in 30d · Committed today · Rust · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Suno AI's Bark model in C/C++ for fast text-to-speech generation
+0 (+0.0%) in 30d · Committed 735d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM…
+8 (+1.5%) in 30d · Committed 10d ago · C++ · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
- Language
- Python
- License
- Apache License 2.0
- Contributors
- 39
- Forks
- 350
- Watchers
- 19
- Open issues
- 5
- Releases
- 0
- Created
- 2026-08-15
- Stars this week
- +69
You've seen where HyperQwen stands today.
Get the moves before everyone else: the 5 Inference Optimization repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.