olla
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
thushan/olla
View on GitHub ↗High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
olla grew +4.2% in the last 30 days, ranks 90th of 268 in LLM Gateway & Routing, actively maintained, last commit 8 days ago.
Star trajectory
115 days tracked+4.2% in 30d, above the LLM Gateway & Routing median of +2.1%.
Last commit 8 days ago, staler than the median of 5d.
How it compares in LLM Gateway & Routing
Ranked 90th of 268 by stars.
- Language
- Go
- License
- Apache License 2.0
- Contributors
- 6
- Forks
- 58
- Watchers
- 4
- Open issues
- 23
- Releases
- 24
- Created
- 2025-05-23
- Stars this week
- -1
You've seen where olla stands today.
Get the moves before everyone else: the 5 LLM Gateway & Routing repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.