FlexLLMGen

Running large language models on a single GPU for throughput-oriented scenarios.

FMInference/FlexLLMGen

View on GitHub ↗

Running large language models on a single GPU for throughput-oriented scenarios.

FlexLLMGen grew -0.1% in the last 30 days, ranks 9th of 99 in Distributed Inference, last commit 755 days ago.

Star trajectory

149 days tracked
2026-06-259.3k stars today2026-11-22
Momentum

-0.1% in 30d, below the Distributed Inference median of +1.2%.

Maintenance

Last commit 755 days ago, staler than the median of 5d.

How it compares in Distributed Inference

FlexLLMGen
9.3k
stars
-0.1% 30d
commit 755 days ago
Distributed Inference median
299
stars
+1.2% 30d
commit ~5d ago
Leader · exo
48.5k
stars
+1.1% 30d
commit 15 days ago

Ranked 9th of 99 by stars.

Ranked by shared tags, then keywords, then stars

Where FlexLLMGen sits

Hover or focus a node for details. Click to open it.

  • This repo
  • Related repo
  • Tag or category
  1. Run frontier AI locally.

    +524 (+1.1%) in 30d · Committed 15d ago · Python · Apache-2.0

    Shares Distributed Inference

  2. Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for…

    +245 (+0.6%) in 30d · Committed today · Python · Apache-2.0

    Shares Distributed Inference

  3. Making large AI models cheaper, faster and more accessible

    -7 (-0.0%) in 30d · Committed 3d ago · Python · Apache-2.0

    Shares Distributed Inference

  4. The Cloud Native AI Platform

    +52 (+0.3%) in 30d · Committed 52d ago · Apache-2.0

    Shares Distributed Inference

  5. Easy-to-use and powerful LLM and SLM library with awesome model zoo.

    +23 (+0.2%) in 30d · Committed 183d ago · Python · Apache-2.0

    Shares Distributed Inference

  6. The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so…

    +135 (+1.3%) in 30d · Committed today · Python · Apache-2.0

    Shares Distributed Inference

  7. 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

    +69 (+0.6%) in 30d · Committed 805d ago · Python · MIT

    Shares Distributed Inference

  8. Build, Manage and Deploy AI/ML Systems

    +44 (+0.4%) in 30d · Committed 2d ago · Python · Apache-2.0

    Shares Distributed Inference

  9. A Datacenter Scale Distributed Inference Serving Framework

    +175 (+2.1%) in 30d · Committed today · Rust · Other license

    Shares Distributed Inference

  10. FlashInfer: Kernel Library for LLM Serving

    +234 (+3.5%) in 30d · Committed today · Cuda · Apache-2.0

    Shares Distributed Inference

Language
Python
License
Apache License 2.0
Contributors
19
Forks
589
Watchers
7
Open issues
52
Releases
0
Created
2023-02-15
Stars this week
-2

You've seen where FlexLLMGen stands today.

Get the moves before everyone else: the 5 Distributed Inference repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain FlexLLMGen?

Claim this page to add your own call-to-action.

Claim this page