halogen-flash-server

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

peonist-ai/halogen-flash-server

View on GitHub ↗

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

halogen-flash-server grew +31.7% in the last 30 days, ranks 116th of 539 in Local Runtime / Inference Engine, actively maintained, last commit 3 days ago.

Star trajectory

71 days tracked
2026-09-111.4k stars today2026-11-22
Momentum

+31.7% in 30d, above the Local Runtime / Inference Engine median of +1.9%.

Maintenance

Last commit 3 days ago, fresher than the median of 6d.

How it compares in Local Runtime / Inference Engine

halogen-flash-server
1.4k
stars
+31.7% 30d
commit 3 days ago
Local Runtime / Inference Engine median
360
stars
+1.9% 30d
commit ~6d ago
Leader · ollama
187.8k
stars
+1.9% 30d
commit today

Ranked 116th of 539 by stars.

Ranked by shared tags, then keywords, then stars

Where halogen-flash-server sits

Hover or focus a node for details. Click to open it.

  • This repo
  • Related repo
  • Tag or category
  1. Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4…

    +962 (+243%) in 30d · Committed 3d ago · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  2. Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM)…

    +264 (+90%) in 30d · Committed today · C++ · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  3. llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

    +2 (+0.7%) in 30d · Committed 2d ago · Go · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  4. Zero-GPU CPU inference for 30B-284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup…

    Committed today · Python · Apache-2.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  5. RDNA-native LLM inference engine in Rust.

    +95 (+13%) in 30d · Committed today · Rust · Other license

    Shares Local Runtime / Inference Engine, Inference Optimization

  6. Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    +25 (+3.5%) in 30d · Committed yesterday · Rust · Apache-2.0

    Shares Inference Optimization, Local Runtime / Inference Engine

  7. 37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs…

    +280 (+61%) in 30d · Committed yesterday · C · GPL-3.0

    Shares Local Runtime / Inference Engine, Inference Optimization

  8. A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console…

    Committed today · C# · BSD-3-Clause

    Shares Local Runtime / Inference Engine, Inference Optimization

  9. Rust + CUDA inference engine for NVIDIA RTX PRO 6000 Blackwell and RTX 5090. Serves safetensors and GGUF over…

    +37 (+13%) in 30d · Committed today · OpenEdge ABL · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

  10. Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo…

    +36 (+15%) in 30d · Committed 2d ago · Python · MIT

    Shares Local Runtime / Inference Engine, Inference Optimization

Language
Shell
License
Other
Contributors
1
Forks
103
Watchers
41
Open issues
39
Releases
0
Created
2026-08-26
Stars this week
+67

You've seen where halogen-flash-server stands today.

Get the moves before everyone else: the 5 Local Runtime / Inference Engine repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain halogen-flash-server?

Claim this page to add your own call-to-action.

Claim this page