llm-inference-at-scale
A Practitioner handbook for production llm serving.
harshuljain13/llm-inference-at-scale
View on GitHub ↗A Practitioner handbook for production llm serving.
llm-inference-at-scale grew +6.5% in the last 30 days, ranks 195th of 397 in Production Serving & Deployment, actively maintained, last commit today.
Star trajectory
82 days tracked+6.5% in 30d, above the Production Serving & Deployment median of +1.3%.
Last commit today, fresher than the median of 3d.
How it compares in Production Serving & Deployment
Ranked 195th of 397 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere llm-inference-at-scale sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
A Mac-local specialist-model factory for data, post-training, evaluation, packaging, and evidence-backed…
+0 (+0.0%) in 30d · Committed yesterday · Swift · MIT
Shares Production Serving & Deployment, Getting Started / Hands-On
Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with…
+22 (+0.1%) in 30d · Committed 186d ago · Jupyter Notebook · MIT
Shares Getting Started / Hands-On
Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each…
+838 (+11%) in 30d · Committed today · Rust · Apache-2.0
Shares Production Serving & Deployment
Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference &…
Committed yesterday · Jupyter Notebook
Shares Getting Started / Hands-On
Lucebox inference engine for heterogeneous hardware & consumer GPUs
+76 (+2.6%) in 30d · Committed today · C++ · Apache-2.0
Shares Production Serving & Deployment
Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
+342 (+13%) in 30d · Committed 3d ago · Python · Apache-2.0
Shares Production Serving & Deployment
cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台,算法全链路流程,多租户,算力租赁平台,token中转,拖拉拽任务流pipeline编排,多机多卡分布式…
+50 (+1.9%) in 30d · Committed 40d ago · Python · Other license
Shares Production Serving & Deployment
- ddalcu/mlx-serve2.6k
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend…
+504 (+25%) in 30d · Committed today · Zig · Other license
Shares Production Serving & Deployment
Manages Unified Access to Generative AI Services built on Envoy Gateway
+157 (+6.9%) in 30d · Committed 2d ago · Go · Apache-2.0
Shares Production Serving & Deployment
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch…
+750 (+48%) in 30d · Committed 4d ago · Python
Shares Getting Started / Hands-On
- Language
- Jupyter Notebook
- License
- Other
- Contributors
- 3
- Forks
- 56
- Watchers
- 3
- Open issues
- 6
- Releases
- 0
- Created
- 2026-05-18
- Stars this week
- +1
You've seen where llm-inference-at-scale stands today.
Get the moves before everyone else: the 5 Production Serving & Deployment repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.