llm-inference-capacity-handbook
大模型推理案头手册(开源版):给定 GPU 算力,一个模型能扛多少 QPS?三层模型 × 三面墙 × 排队论 × 开环压测
shtjww/llm-inference-capacity-handbook
View on GitHub ↗大模型推理案头手册(开源版):给定 GPU 算力,一个模型能扛多少 QPS?三层模型 × 三面墙 × 排队论 × 开环压测
llm-inference-capacity-handbook grew +0.5% in the last 30 days, ranks 205th of 343 in Inference Optimization, last commit 63 days ago.
Star trajectory
75 days tracked+0.5% in 30d, below the Inference Optimization median of +1.6%.
Last commit 63 days ago, staler than the median of 16d.
How it compares in Inference Optimization
Ranked 205th of 343 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere llm-inference-capacity-handbook sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
- xLLM-AI/xllm1.6k
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is…
+23 (+1.4%) in 30d · Committed today · C++ · Apache-2.0
Shares Inference Optimization, Production Serving & Deployment
DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various…
+0 (+0.0%) in 30d · Committed 472d ago · C · Apache-2.0
Shares Production Serving & Deployment, Inference Optimization
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool…
+38 (+27%) in 30d · Committed today · Python · MIT
Shares Production Serving & Deployment, Inference Optimization
Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model…
+12 (+10%) in 30d · Committed today · Python · Apache-2.0
Shares Inference Optimization, Production Serving & Deployment
Lucebox inference engine for heterogeneous hardware & consumer GPUs
+76 (+2.6%) in 30d · Committed today · C++ · Apache-2.0
Shares Inference Optimization, Production Serving & Deployment
Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
+342 (+13%) in 30d · Committed 3d ago · Python · Apache-2.0
Shares Inference Optimization, Production Serving & Deployment
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…
+962 (+243%) in 30d · Committed 3d ago · C++ · MIT
Shares Inference Optimization, Production Serving & Deployment
- wici-ai/inferssd1.3k
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Committed today · C++ · Apache-2.0
Shares Inference Optimization, Production Serving & Deployment
- eagle-hq/EAGLE-41.2k
Official Inference Code of EAGLE-4
+334 (+38%) in 30d · Committed 8d ago · Python · Other license
Shares Inference Optimization, Production Serving & Deployment
TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC
+21 (+1.8%) in 30d · Committed 55d ago · C++ · MIT
Shares Inference Optimization, Production Serving & Deployment
- Language
- -
- License
- Other
- Contributors
- 1
- Forks
- 0
- Watchers
- 0
- Open issues
- 0
- Releases
- 0
- Created
- 2026-09-02
- Stars this week
- +0
You've seen where llm-inference-capacity-handbook stands today.
Get the moves before everyone else: the 5 Inference Optimization repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.
Maintain llm-inference-capacity-handbook?
Claim this page to add your own call-to-action.
Claim this page