sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
jpezzulli/sglang-rtxpro6000
View on GitHub ↗Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
sglang-rtxpro6000 grew +17.7% in the last 30 days, ranks 356th of 539 in Local Runtime / Inference Engine, actively maintained, last commit today.
Star trajectory
57 days tracked+17.7% in 30d, above the Local Runtime / Inference Engine median of +1.9%.
Last commit today, fresher than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 356th of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere sglang-rtxpro6000 sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
- syv-ai/HyperQwen2.8k
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127…
+592 (+27%) in 30d · Committed 12d ago · Python · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization
RDNA-native LLM inference engine in Rust.
+95 (+13%) in 30d · Committed today · Rust · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo…
+36 (+15%) in 30d · Committed 2d ago · Python · MIT
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K…
+22 (+18%) in 30d · Committed 37d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86…
+0 (+0.0%) in 30d · Committed 52d ago · Python · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization
LLM Inference on consumer devices
+0 (+0.0%) in 30d · Committed 614d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Memra is a Rust + CUDA LLM inference engine with hand-written kernels, NVFP4 and mixed-precision execution…
+4 (+100%) in 30d · Committed today · Rust · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Suno AI's Bark model in C/C++ for fast text-to-speech generation
+0 (+0.0%) in 30d · Committed 735d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Inference Optimization, Local Runtime / Inference Engine, Model Optimization
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM…
+8 (+1.5%) in 30d · Committed 10d ago · C++ · Other license
Shares Local Runtime / Inference Engine, Inference Optimization, Model Optimization
- Language
- Python
- License
- Apache License 2.0
- Contributors
- 1,840
- Forks
- 28
- Watchers
- 6
- Open issues
- 10
- Releases
- 23
- Created
- 2026-08-25
- Stars this week
- +6
You've seen where sglang-rtxpro6000 stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.