exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

turboderp-org/exllamav2

View on GitHub ↗

A fast inference library for running LLMs locally on modern consumer-class GPUs

exllamav2 grew +0.3% in the last 30 days, ranks 54th of 539 in Local Runtime / Inference Engine, last commit 262 days ago.

Star trajectory

149 days tracked
2026-06-254.7k stars today2026-11-22
Momentum

+0.3% in 30d, below the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit 262 days ago, staler than the median of 6d.

How it compares in Local Runtime / Inference Engine

exllamav2
4.7k
stars
+0.3% 30d
commit 262 days ago
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 54th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. Official inference library for Mistral models

    -4 (-0.0%) in 30d · Committed 158d ago · Jupyter Notebook · Apache-2.0

    Shares Local Runtime / Inference Engine, Language Bindings

  2. 2bit quants, kernels and runtime for consumer devices.

    Committed 16d ago · C++ · Other license

    Shares Local Runtime / Inference Engine, Language Bindings

  3. Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model…

    +16 (+0.7%) in 30d · Committed today · TypeScript · MIT

    Shares Local Runtime / Inference Engine, Language Bindings

  4. Community maintained hardware plugin for vLLM on Apple Silicon

    +109 (+5.7%) in 30d · Committed today · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Language Bindings

  5. NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

    +83 (+5.1%) in 30d · Committed 2d ago · Rust · EUPL-1.2

    Shares Local Runtime / Inference Engine, Language Bindings

  6. LM Studio Python SDK

    +8 (+0.9%) in 30d · Committed 391d ago · Python · MIT

    Shares Language Bindings, Local Runtime / Inference Engine

  7. Go With Your Own Intelligence! Use Go for hardware accelerated local inference with llama.cpp, whisper.cpp…

    +22 (+2.6%) in 30d · Committed today · Go · Apache-2.0

    Shares Local Runtime / Inference Engine, Language Bindings

  8. Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on…

    +2 (+0.2%) in 30d · Committed 1,207d ago · Rust · Apache-2.0

    Shares Local Runtime / Inference Engine, Language Bindings

  9. Simple, modular, and observable Go framework for backend applications.

    -1 (-0.1%) in 30d · Committed 31d ago · Go · MIT

    Shares Local Runtime / Inference Engine, Language Bindings

  10. A modular Swift SDK for audio processing with MLX on Apple Silicon

    +20 (+2.5%) in 30d · Committed yesterday · Swift · MIT

    Shares Language Bindings, Local Runtime / Inference Engine

Language
Python
License
MIT License
Contributors
57
Forks
351
Watchers
34
Open issues
141
Releases
43
Created
2023-08-30
Stars this week
+4

You've seen where exllamav2 stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain exllamav2?

Claim this page to add your own call-to-action.

Claim this page