vLLM-2080Ti-Definitive

The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )

weicj/vLLM-2080Ti-Definitive

View on GitHub ↗

The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )

vLLM-2080Ti-Definitive grew +5.6% in the last 30 days, ranks 126th of 539 in Local Runtime / Inference Engine, last commit 51 days ago.

Star trajectory

82 days tracked
2026-08-311.2k stars today2026-11-22
Momentum

+5.6% in 30d, above the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit 51 days ago, staler than the median of 6d.

How it compares in Local Runtime / Inference Engine

vLLM-2080Ti-Definitive
1.2k
stars
+5.6% 30d
commit 51 days ago
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 126th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…

    +962 (+243%) in 30d · Committed 3d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  2. Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one…

    +31 (+18%) in 30d · Committed today · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  3. Kindi 0.6.1: fast local LLM inference on low-cost GPUs and CPUs. DeepSeek-V4.1-Flash (552B) at 18.7 tok/s on…

    Committed 5d ago · Python · AGPL-3.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  4. WeeLLM runs large diffusion models with as little as 4 GB of VRAM, without any quantization. It dynamically…

    +42 (+36%) in 30d · Committed 13d ago · Python

    Shares Local Runtime / Inference Engine, Inference Optimization

  5. Native Windows vLLM 0.33.0 (Python 3.13/CUDA 13.0), plus a Python 3.14/CUDA 13.2 build; older versions on the…

    +5 (+7.9%) in 30d · Committed 2d ago · Python · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  6. Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

    +442 (+6.3%) in 30d · Committed 7d ago · Swift · Apache-2.0

    Shares Inference Optimization, Local Runtime / Inference Engine

  7. TokenSpeed is a speed-of-light LLM inference engine.

    +176 (+6.4%) in 30d · Committed today · Python · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  8. Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127…

    +592 (+27%) in 30d · Committed 12d ago · Python · Apache-2.0

    Shares Inference Optimization, Local Runtime / Inference Engine

  9. Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.

    +97 (+4.9%) in 30d · Committed today · C++ · Other license

    Shares Local Runtime / Inference Engine, Inference Optimization

  10. A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is…

    +23 (+1.4%) in 30d · Committed today · C++ · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization

Language
Python
License
Apache License 2.0
Contributors
3,441
Forks
175
Watchers
14
Open issues
30
Releases
25
Created
2026-06-02
Stars this week
+18

You've seen where vLLM-2080Ti-Definitive stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain vLLM-2080Ti-Definitive?

Claim this page to add your own call-to-action.

Claim this page