micro-vllm

A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.

nopnoping/micro-vllm

View on GitHub ↗

A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.

micro-vllm grew +3.4% in the last 30 days, ranks 515th of 539 in Local Runtime / Inference Engine, last commit 24 days ago.

Star trajectory

82 days tracked
2026-08-3161 stars today2026-11-22
Momentum

+3.4% in 30d, above the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit 24 days ago, staler than the median of 6d.

How it compares in Local Runtime / Inference Engine

micro-vllm
61
stars
+3.4% 30d
commit 24 days ago
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 515th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. TokenSpeed is a speed-of-light LLM inference engine.

    +176 (+6.4%) in 30d · Committed today · Python · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  2. NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

    +83 (+5.1%) in 30d · Committed 2d ago · Rust · EUPL-1.2

    Shares Local Runtime / Inference Engine, Inference Optimization

  3. Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…

    +962 (+243%) in 30d · Committed 3d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  4. The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with…

    +66 (+5.6%) in 30d · Committed 51d ago · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  5. Library for faster pinned CPU <-> GPU transfer in Pytorch

    -1 (-0.1%) in 30d · Committed 2,465d ago · Python · MIT

    Shares Inference Optimization, Local Runtime / Inference Engine

  6. LLM Benchmark for Throughput via Ollama (Local LLMs)

    +2 (+0.5%) in 30d · Committed 25d ago · Python · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  7. A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

    +2 (+0.6%) in 30d · Committed 530d ago · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  8. Inference for hybrid LLMs: Gemma, RWKV, and all kinds of hybrids.

    +0 (+0.0%) in 30d · Committed 73d ago · Python

    Shares Local Runtime / Inference Engine, Inference Optimization

  9. Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one…

    +31 (+18%) in 30d · Committed today · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  10. Kindi 0.6.1: fast local LLM inference on low-cost GPUs and CPUs. DeepSeek-V4.1-Flash (552B) at 18.7 tok/s on…

    Committed 5d ago · Python · AGPL-3.0

    Shares Local Runtime / Inference Engine, Inference Optimization

Language
Python
License
-
Contributors
1
Forks
5
Watchers
0
Open issues
0
Releases
0
Created
2026-07-26
Stars this week
+0

You've seen where micro-vllm stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain micro-vllm?

Claim this page to add your own call-to-action.

Claim this page