micro-vllm
A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.
nopnoping/micro-vllm
View on GitHub ↗A small, educational vLLM implementation for exploring LLM inference, scheduling, KV-cache management, and GPU execution.
micro-vllm grew +3.4% in the last 30 days, ranks 515th of 539 in Local Runtime / Inference Engine, last commit 24 days ago.
Star trajectory
82 days tracked+3.4% in 30d, above the Local Runtime / Inference Engine median of +1.9%.
Last commit 24 days ago, staler than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 515th of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere micro-vllm sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
TokenSpeed is a speed-of-light LLM inference engine.
+176 (+6.4%) in 30d · Committed today · Python · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
+83 (+5.1%) in 30d · Committed 2d ago · Rust · EUPL-1.2
Shares Local Runtime / Inference Engine, Inference Optimization
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…
+962 (+243%) in 30d · Committed 3d ago · C++ · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with…
+66 (+5.6%) in 30d · Committed 51d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization
Library for faster pinned CPU <-> GPU transfer in Pytorch
-1 (-0.1%) in 30d · Committed 2,465d ago · Python · MIT
Shares Inference Optimization, Local Runtime / Inference Engine
LLM Benchmark for Throughput via Ollama (Local LLMs)
+2 (+0.5%) in 30d · Committed 25d ago · Python · MIT
Shares Local Runtime / Inference Engine, Inference Optimization
A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…
+2 (+0.6%) in 30d · Committed 530d ago · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization
- xiaol/wkvm359
Inference for hybrid LLMs: Gemma, RWKV, and all kinds of hybrids.
+0 (+0.0%) in 30d · Committed 73d ago · Python
Shares Local Runtime / Inference Engine, Inference Optimization
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one…
+31 (+18%) in 30d · Committed today · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Inference Optimization
- ahtx/kindi198
Kindi 0.6.1: fast local LLM inference on low-cost GPUs and CPUs. DeepSeek-V4.1-Flash (552B) at 18.7 tok/s on…
Committed 5d ago · Python · AGPL-3.0
Shares Local Runtime / Inference Engine, Inference Optimization
- Language
- Python
- License
- -
- Contributors
- 1
- Forks
- 5
- Watchers
- 0
- Open issues
- 0
- Releases
- 0
- Created
- 2026-07-26
- Stars this week
- +0
You've seen where micro-vllm stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.