grps_trtllm
Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.
NetEase-Media/grps_trtllm
View on GitHub ↗Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.
grps_trtllm grew +0.0% in the last 30 days, ranks 231st of 320 in Production Serving & Deployment, last commit 321 days ago.
Star trajectory
79 days tracked+0.0% in 30d, below the Production Serving & Deployment median of +1.2%.
Last commit 321 days ago, staler than the median of 3d.
How it compares in Production Serving & Deployment
Ranked 231st of 320 by stars.
- Language
- Python
- License
- Apache License 2.0
- Contributors
- 3
- Forks
- 12
- Watchers
- 5
- Open issues
- 7
- Releases
- 0
- Created
- 2024-08-21
- Stars this week
- +0
You've seen where grps_trtllm stands today.
Get the moves before everyone else: the 5 Production Serving & Deployment repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.