FlexLLMGen
Running large language models on a single GPU for throughput-oriented scenarios.
FMInference/FlexLLMGen
View on GitHub ↗Running large language models on a single GPU for throughput-oriented scenarios.
FlexLLMGen grew -0.1% in the last 30 days, ranks 9th of 99 in Distributed Inference, last commit 755 days ago.
Star trajectory
149 days tracked-0.1% in 30d, below the Distributed Inference median of +1.2%.
Last commit 755 days ago, staler than the median of 5d.
How it compares in Distributed Inference
Ranked 9th of 99 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere FlexLLMGen sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
- exo-explore/exo48.5k
Run frontier AI locally.
+524 (+1.1%) in 30d · Committed 15d ago · Python · Apache-2.0
Shares Distributed Inference
- ray-project/ray44.4k
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for…
+245 (+0.6%) in 30d · Committed today · Python · Apache-2.0
Shares Distributed Inference
- hpcaitech/ColossalAI41.4k
Making large AI models cheaper, faster and more accessible
-7 (-0.0%) in 30d · Committed 3d ago · Python · Apache-2.0
Shares Distributed Inference
The Cloud Native AI Platform
+52 (+0.3%) in 30d · Committed 52d ago · Apache-2.0
Shares Distributed Inference
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
+23 (+0.2%) in 30d · Committed 183d ago · Python · Apache-2.0
Shares Distributed Inference
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so…
+135 (+1.3%) in 30d · Committed today · Python · Apache-2.0
Shares Distributed Inference
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
+69 (+0.6%) in 30d · Committed 805d ago · Python · MIT
Shares Distributed Inference
- Netflix/metaflow10.4k
Build, Manage and Deploy AI/ML Systems
+44 (+0.4%) in 30d · Committed 2d ago · Python · Apache-2.0
Shares Distributed Inference
- ai-dynamo/dynamo8.5k
A Datacenter Scale Distributed Inference Serving Framework
+175 (+2.1%) in 30d · Committed today · Rust · Other license
Shares Distributed Inference
FlashInfer: Kernel Library for LLM Serving
+234 (+3.5%) in 30d · Committed today · Cuda · Apache-2.0
Shares Distributed Inference
- Language
- Python
- License
- Apache License 2.0
- Contributors
- 19
- Forks
- 589
- Watchers
- 7
- Open issues
- 52
- Releases
- 0
- Created
- 2023-02-15
- Stars this week
- -2
You've seen where FlexLLMGen stands today.
Get the moves before everyone else: the 5 Distributed Inference repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.