tool-eval-bench
Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, and structured output. Supports vLLM, SGLang, and llama.cpp.
SeraphimSerapis/tool-eval-bench
View on GitHub ↗Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, and structured output. Supports vLLM, SGLang, and llama.cpp.
tool-eval-bench grew +13.4% in the last 30 days, ranks 261st of 746 in Evaluation & Guardrails, actively maintained, last commit today.
Star trajectory
101 days tracked+13.4% in 30d, above the Evaluation & Guardrails median of +1.2%.
Last commit today, fresher than the median of 14.5d.
How it compares in Evaluation & Guardrails
Ranked 261st of 746 by stars.
- Language
- Python
- License
- MIT License
- Contributors
- 22
- Forks
- 53
- Watchers
- 2
- Open issues
- 0
- Releases
- 36
- Created
- 2026-04-17
- Stars this week
- +5
You've seen where tool-eval-bench stands today.
Get the moves before everyone else: the 5 Evaluation & Guardrails repos climbing fastest, every Tuesday.
Free weekly digest. Double opt-in, unsubscribe anytime.