shard
Pipeline-parallel LLM inference across GPUs on separate machines.
leyten/shard
View on GitHub ↗Pipeline-parallel LLM inference across GPUs on separate machines.
shard grew +1.5% in the last 30 days, ranks 40th of 99 in Distributed Inference, last commit 24 days ago.
Star trajectory
82 days tracked+1.5% in 30d, above the Distributed Inference median of +1.2%.
Last commit 24 days ago, staler than the median of 5d.
How it compares in Distributed Inference
Ranked 40th of 99 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere shard sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
+170 (+8.9%) in 30d · Committed today · Python · Apache-2.0
Shares Distributed Inference
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL…
+39 (+2.5%) in 30d · Committed yesterday · C++ · Apache-2.0
Shares Distributed Inference
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
+25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0
Shares Distributed Inference
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM…
+71 (+11%) in 30d · Committed today · Rust · Apache-2.0
Shares Distributed Inference
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management…
+10 (+1.9%) in 30d · Committed today · Go · Apache-2.0
Shares Distributed Inference
A high-performance inference system for large language models, designed for production environments.
-1 (-0.2%) in 30d · Committed 338d ago · C++ · Apache-2.0
Shares Distributed Inference
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs…
+3 (+0.6%) in 30d · Committed 320d ago · Python · Apache-2.0
Shares Distributed Inference
🧱 Experimental library for building inference engines
Committed today · Python · Apache-2.0
Shares Distributed Inference
InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the…
-1 (-0.2%) in 30d · Committed 458d ago · Python · Apache-2.0
Shares Distributed Inference
Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream…
+6 (+1.5%) in 30d · Committed 87d ago · Shell · MIT
Shares Distributed Inference
- Language
- Python
- License
- Apache License 2.0
- Contributors
- 2
- Forks
- 66
- Watchers
- 9
- Open issues
- 2
- Releases
- 2
- Created
- 2026-06-15
- Stars this week
- +0
You've seen where shard stands today.
Get the moves before everyone else: the 5 Distributed Inference repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.