toshllm

Run large language models locally on Intel Macs with AMD GPUs - native macOS app with Metal acceleration

engeldlgado/toshllm

View on GitHub ↗

Run large language models locally on Intel Macs with AMD GPUs - native macOS app with Metal acceleration

toshllm grew +21.9% in the last 30 days, ranks 313th of 539 in Local Runtime / Inference Engine, actively maintained, last commit 3 days ago.

Star trajectory

82 days tracked
2026-08-31267 stars today2026-11-22
Momentum

+21.9% in 30d, above the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit 3 days ago, fresher than the median of 6d.

How it compares in Local Runtime / Inference Engine

toshllm
267
stars
+21.9% 30d
commit 3 days ago
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 313th of 539 by stars.

Ranked by shared tags, then keywords, then stars
  1. Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

    +442 (+6.3%) in 30d · Committed 7d ago · Swift · Apache-2.0

    Shares Web / Edge / On-Device Runtime, Inference Optimization, Local Runtime / Inference Engine

  2. Open-source local AI workspace — advancing on-device inference.

    -6 (-1.5%) in 30d · Committed 34d ago · TypeScript · Apache-2.0

    Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Inference Optimization

  3. The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models…

    +17 (+28%) in 30d · Committed today · Swift · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime

  4. Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.

    +15 (+1.8%) in 30d · Committed 49d ago · Rust · Other license

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime

  5. DeepSeek-V4-Flash-0731 284B inference in ~25 GB of RAM / Qwen3.8-Next-Flash inference in ~15 GB of RAM on any…

    +3 (+2.0%) in 30d · Committed 25d ago · Python · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime

  6. A high-performance inference engine for AI models

    +837 (+33%) in 30d · Committed today · Rust · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime

  7. The fastest way to run Inkling 2 Mini, Qwen 3.8 and Ternary Bonsai 2 27B on a Mac: Inkling 2 Mini at up to…

    +107 (+4.1%) in 30d · Committed today · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization, Web / Edge / On-Device Runtime

  8. Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.

    +97 (+4.9%) in 30d · Committed today · C++ · Other license

    Shares Web / Edge / On-Device Runtime, Local Runtime / Inference Engine, Inference Optimization

  9. Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB…

    +193 (+40%) in 30d · Committed 4d ago · Swift · MIT

    Shares Web / Edge / On-Device Runtime, Inference Optimization, Local Runtime / Inference Engine

  10. Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices…

    +5 (+0.8%) in 30d · Committed 4d ago · Swift · Apache-2.0

    Shares Local Runtime / Inference Engine, Web / Edge / On-Device Runtime, Inference Optimization

Language
Swift
License
GNU General Public License v3.0
Contributors
9
Forks
31
Watchers
0
Open issues
25
Releases
147
Created
2026-06-14
Stars this week
+9

You've seen where toshllm stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain toshllm?

Claim this page to add your own call-to-action.

Claim this page