shard

Pipeline-parallel LLM inference across GPUs on separate machines.

leyten/shard

View on GitHub ↗

Pipeline-parallel LLM inference across GPUs on separate machines.

shard grew +1.5% in the last 30 days, ranks 40th of 99 in Distributed Inference, last commit 24 days ago.

Star trajectory

82 days tracked
2026-08-31472 stars today2026-11-22
Momentum

+1.5% in 30d, above the Distributed Inference median of +1.2%.

Maintenance

Last commit 24 days ago, staler than the median of 5d.

How it compares in Distributed Inference

shard
472
stars
+1.5% 30d
commit 24 days ago
Distributed Inference median
299
stars
+1.2% 30d
commit ~5d ago
Leader · exo
48.5k
stars
+1.1% 30d
commit 15 days ago

Ranked 40th of 99 by stars.

Ranked by shared tags, then keywords, then stars

Where shard sits

Hover or focus a node for details. Click to open it.

  • This repo
  • Related repo
  • Tag or category
  1. Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

    +170 (+8.9%) in 30d · Committed today · Python · Apache-2.0

    Shares Distributed Inference

  2. UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL…

    +39 (+2.5%) in 30d · Committed yesterday · C++ · Apache-2.0

    Shares Distributed Inference

  3. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Distributed Inference

  4. Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM…

    +71 (+11%) in 30d · Committed today · Rust · Apache-2.0

    Shares Distributed Inference

  5. Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management…

    +10 (+1.9%) in 30d · Committed today · Go · Apache-2.0

    Shares Distributed Inference

  6. A high-performance inference system for large language models, designed for production environments.

    -1 (-0.2%) in 30d · Committed 338d ago · C++ · Apache-2.0

    Shares Distributed Inference

  7. JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs…

    +3 (+0.6%) in 30d · Committed 320d ago · Python · Apache-2.0

    Shares Distributed Inference

  8. 🧱 Experimental library for building inference engines

    Committed today · Python · Apache-2.0

    Shares Distributed Inference

  9. InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the…

    -1 (-0.2%) in 30d · Committed 458d ago · Python · Apache-2.0

    Shares Distributed Inference

  10. Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream…

    +6 (+1.5%) in 30d · Committed 87d ago · Shell · MIT

    Shares Distributed Inference

Language
Python
License
Apache License 2.0
Contributors
2
Forks
66
Watchers
9
Open issues
2
Releases
2
Created
2026-06-15
Stars this week
+0

You've seen where shard stands today.

Get the moves before everyone else: the 5 Distributed Inference repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain shard?

Claim this page to add your own call-to-action.

Claim this page