GoRightLLM

A torch-free, lightweight LLM routing middleware. Lossless compression via fp32 ONNX embeddings + pure-numpy scorers (100% routing parity with the original), four routing modes (base / pro / ultra / agent), fully local inference with no data leaving your premises. Drop-in OpenAI-compatible proxy.

Enlightment-to-truth/GoRightLLM

View on GitHub ↗

A torch-free, lightweight LLM routing middleware. Lossless compression via fp32 ONNX embeddings + pure-numpy scorers (100% routing parity with the original), four routing modes (base / pro / ultra / agent), fully local inference with no data leaving your premises. Drop-in OpenAI-compatible proxy.

GoRightLLM ranks 189th of 339 in LLM Gateway & Routing, last commit 121 days ago.

Star trajectory

21 days tracked
2026-11-01142 stars today2026-11-22
Momentum

- in 30d, below the LLM Gateway & Routing median of +1.5%.

Maintenance

Last commit 121 days ago, staler than the median of 5d.

How it compares in LLM Gateway & Routing

GoRightLLM
142
stars
- 30d
commit 121 days ago
LLM Gateway & Routing median
162
stars
+1.5% 30d
commit ~5d ago
Leader · OmniRoute
89.5k
stars
+11.9% 30d
commit today

Ranked 189th of 339 by stars.

Ranked by shared tags, then keywords, then stars
  1. Local proxy that learns your app's typed LLM decisions and answers them with a Laya head. Jev and OpenAI…

    +3 (+4.1%) in 30d · Committed 41d ago · Python · Apache-2.0

    Shares Inference Optimization, LLM Gateway & Routing, Production Serving & Deployment

  2. Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely…

    -22 (-41%) in 30d · Committed today · Go · Apache-2.0

    Shares Production Serving & Deployment, Inference Optimization, LLM Gateway & Routing

  3. Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo…

    +36 (+15%) in 30d · Committed 2d ago · Python · MIT

    Shares Production Serving & Deployment, LLM Gateway & Routing, Inference Optimization

  4. The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up…

    Committed 17d ago · MIT

    Shares Production Serving & Deployment, Inference Optimization, LLM Gateway & Routing

  5. Local decision engine with typed, calibrated outputs (Noul/Choice/Score): cheap→strong cascade, ECE/Brier…

    Committed yesterday · Python · Apache-2.0

    Shares LLM Gateway & Routing, Inference Optimization, Production Serving & Deployment

  6. Run many AI agents at once on your Mac, up to 3x faster.

    +273 (+91%) in 30d · Committed today · Python · Apache-2.0

    Shares Production Serving & Deployment, Inference Optimization, LLM Gateway & Routing

  7. This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs…

    +1 (+0.5%) in 30d · Committed 706d ago · TypeScript · MIT

    Shares Production Serving & Deployment, Inference Optimization, LLM Gateway & Routing

  8. Reproducible LLM routing and deployment optimization.

    +0 (+0.0%) in 30d · Committed 88d ago · Python · Apache-2.0

    Shares LLM Gateway & Routing, Inference Optimization, Production Serving & Deployment

  9. LLMs and Machine Learning done easily

    +0 (+0.0%) in 30d · Committed 283d ago · Python · MIT

    Shares Production Serving & Deployment, LLM Gateway & Routing

  10. Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI…

    +9 (+9.0%) in 30d · Committed 38d ago · Go · Apache-2.0

    Shares LLM Gateway & Routing, Production Serving & Deployment

Language
Python
License
MIT License
Contributors
1
Forks
0
Watchers
1
Open issues
0
Releases
0
Created
2026-07-23
Stars this week
+0

You've seen where GoRightLLM stands today.

Get the moves before everyone else: the 5 LLM Gateway & Routing repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain GoRightLLM?

Claim this page to add your own call-to-action.

Claim this page