exllamav2
A fast inference library for running LLMs locally on modern consumer-class GPUs
turboderp-org/exllamav2
View on GitHub ↗A fast inference library for running LLMs locally on modern consumer-class GPUs
exllamav2 grew +0.3% in the last 30 days, ranks 54th of 539 in Local Runtime / Inference Engine, last commit 262 days ago.
Star trajectory
149 days tracked+0.3% in 30d, below the Local Runtime / Inference Engine median of +1.9%.
Last commit 262 days ago, staler than the median of 6d.
How it compares in Local Runtime / Inference Engine
Ranked 54th of 539 by stars.
Related repos
Ranked by shared tags, then keywords, then starsWhere exllamav2 sits
Hover or focus a node for details. Click to open it.
- This repo
- Related repo
- Tag or category
Official inference library for Mistral models
-4 (-0.0%) in 30d · Committed 158d ago · Jupyter Notebook · Apache-2.0
Shares Local Runtime / Inference Engine, Language Bindings
2bit quants, kernels and runtime for consumer devices.
Committed 16d ago · C++ · Other license
Shares Local Runtime / Inference Engine, Language Bindings
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model…
+16 (+0.7%) in 30d · Committed today · TypeScript · MIT
Shares Local Runtime / Inference Engine, Language Bindings
Community maintained hardware plugin for vLLM on Apple Silicon
+109 (+5.7%) in 30d · Committed today · Python · Apache-2.0
Shares Local Runtime / Inference Engine, Language Bindings
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
+83 (+5.1%) in 30d · Committed 2d ago · Rust · EUPL-1.2
Shares Local Runtime / Inference Engine, Language Bindings
LM Studio Python SDK
+8 (+0.9%) in 30d · Committed 391d ago · Python · MIT
Shares Language Bindings, Local Runtime / Inference Engine
Go With Your Own Intelligence! Use Go for hardware accelerated local inference with llama.cpp, whisper.cpp…
+22 (+2.6%) in 30d · Committed today · Go · Apache-2.0
Shares Local Runtime / Inference Engine, Language Bindings
Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on…
+2 (+0.2%) in 30d · Committed 1,207d ago · Rust · Apache-2.0
Shares Local Runtime / Inference Engine, Language Bindings
Simple, modular, and observable Go framework for backend applications.
-1 (-0.1%) in 30d · Committed 31d ago · Go · MIT
Shares Local Runtime / Inference Engine, Language Bindings
A modular Swift SDK for audio processing with MLX on Apple Silicon
+20 (+2.5%) in 30d · Committed yesterday · Swift · MIT
Shares Language Bindings, Local Runtime / Inference Engine
- Language
- Python
- License
- MIT License
- Contributors
- 57
- Forks
- 351
- Watchers
- 34
- Open issues
- 141
- Releases
- 43
- Created
- 2023-08-30
- Stars this week
- +4
You've seen where exllamav2 stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.