lite_llama
RapidLLM is an LLM inference framework with continuous batching, tensor and data parallelism, quantized weights, and with pluggable Triton/CUDA kernels.
harleyszhang/lite_llama
View on GitHub ↗RapidLLM is an LLM inference framework with continuous batching, tensor and data parallelism, quantized weights, and with pluggable Triton/CUDA kernels.
lite_llama grew +0.0% in the last 30 days, ranks 340th of 494 in Local Runtime / Inference Engine, last commit 15 days ago.
Star trajectory
115 days tracked+0.0% in 30d, below the Local Runtime / Inference Engine median of +1.7%.
Last commit 15 days ago, staler than the median of 5d.
How it compares in Local Runtime / Inference Engine
Ranked 340th of 494 by stars.
- Language
- Python
- License
- -
- Contributors
- 4
- Forks
- 33
- Watchers
- 5
- Open issues
- 0
- Releases
- 0
- Created
- 2024-10-06
- Stars this week
- +0
You've seen where lite_llama stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.