WeeLLM
WeeLLM runs large diffusion models with as little as 4 GB of VRAM, without any quantization. It dynamically determines how many layers can fit within the available VRAM and streams the text encoder and transformer layers to the GPU layer by layer, enabling inference on hardware with limited VRAM. It supports both safetensors and GGUF models.
Jit-Roy/WeeLLM
View on GitHub ↗WeeLLM runs large diffusion models with as little as 4 GB of VRAM, without any quantization. It dynamically determines how many layers can fit within the available VRAM and streams the text encoder and transformer layers to the GPU layer by layer, enabling inference on hardware with limited VRAM. It supports both safetensors and GGUF models.
WeeLLM ranks 410th of 494 in Local Runtime / Inference Engine, actively maintained, last commit today.
Star trajectory
14 days tracked- in 30d, below the Local Runtime / Inference Engine median of +1.7%.
Last commit today, fresher than the median of 5d.
How it compares in Local Runtime / Inference Engine
Ranked 410th of 494 by stars.
- Language
- Python
- License
- -
- Contributors
- 1
- Forks
- 13
- Watchers
- 2
- Open issues
- 0
- Releases
- 0
- Created
- 2026-08-02
- Stars this week
- +22
You've seen where WeeLLM stands today.
Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.