TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
NVIDIA/TensorRT-LLM
View on GitHub ↗TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
TensorRT-LLM grew +1.5% in the last 30 days, ranks 12th of 318 in Production Serving & Deployment, actively maintained, last commit today.
Star trajectory
120 days tracked+1.5% in 30d, above the Production Serving & Deployment median of +1.1%.
Last commit today, fresher than the median of 3d.
How it compares in Production Serving & Deployment
Ranked 12th of 318 by stars.
- Language
- Python
- License
- Other
- Contributors
- 502
- Forks
- 2.9k
- Watchers
- 122
- Open issues
- 679
- Releases
- 94
- Created
- 2023-08-16
- Stars this week
- +96
You've seen where TensorRT-LLM stands today.
Get the moves before everyone else: the 5 Production Serving & Deployment repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.