audit-the-judge

Auditing LLM-as-judge systems with paired synthetic controls, bootstrap CIs, and BH-FDR bias detection.

1norqaq/audit-the-judge

View on GitHub ↗

Auditing LLM-as-judge systems with paired synthetic controls, bootstrap CIs, and BH-FDR bias detection.

audit-the-judge grew -3.7% in the last 30 days, ranks 830th of 872 in Evaluation & Guardrails, last commit 86 days ago.

Star trajectory

82 days tracked
2026-08-3152 stars today2026-11-22
Momentum

-3.7% in 30d, below the Evaluation & Guardrails median of +1.4%.

Maintenance

Last commit 86 days ago, staler than the median of 15d.

How it compares in Evaluation & Guardrails

audit-the-judge
52
stars
-3.7% 30d
commit 86 days ago
Evaluation & Guardrails median
235
stars
+1.4% 30d
commit ~15d ago
Leader · impeccable
114.7k
stars
+0.0% 30d
commit today

Ranked 830th of 872 by stars.

Ranked by shared tags, then keywords, then stars
  1. Benchmarking LLMs via Uncertainty Quantification

    +0 (+0.0%) in 30d · Committed 1,027d ago · Python · MIT

    Shares Interpretability & Alignment, Evaluation & Guardrails

  2. RuLES: a benchmark for evaluating rule-following in language models

    +1 (+0.4%) in 30d · Committed 636d ago · Python · Apache-2.0

    Shares Interpretability & Alignment, Evaluation & Guardrails

  3. A Mac-local specialist-model factory for data, post-training, evaluation, packaging, and evidence-backed…

    +0 (+0.0%) in 30d · Committed yesterday · Swift · MIT

    Shares Interpretability & Alignment, Evaluation & Guardrails

  4. [NeurIPS 2025] BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language…

    +8 (+2.4%) in 30d · Committed 75d ago · Python · MIT

    Shares Evaluation & Guardrails, Interpretability & Alignment

  5. Research infrastructure for understanding, testing, and modifying SAE features in LLMs.

    Committed today · Python · Apache-2.0

    Shares Interpretability & Alignment, Evaluation & Guardrails

  6. Deterministic local LLM engine and debugger — run GGUF models on your own GPU, get bit-identical output, and…

    Committed today · Python · MIT

    Shares Interpretability & Alignment, Evaluation & Guardrails

  7. PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of…

    +3 (+0.6%) in 30d · Committed 208d ago · Python · MIT

    Shares Interpretability & Alignment, Evaluation & Guardrails

  8. [USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large…

    +7 (+2.3%) in 30d · Committed 298d ago · Python · MIT

    Shares Interpretability & Alignment, Evaluation & Guardrails

  9. Find, rank and steer MLP neurons of GGUF models on llama.cpp; export control vectors; 3D web UI and MCP server

    Committed yesterday · Python · MIT

    Shares Interpretability & Alignment, Evaluation & Guardrails

  10. [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness…

    +3 (+0.5%) in 30d · Committed 48d ago · Python · MIT

    Shares Evaluation & Guardrails

Language
Python
License
MIT License
Contributors
1
Forks
1
Watchers
0
Open issues
0
Releases
0
Created
2026-06-23
Stars this week
+0

You've seen where audit-the-judge stands today.

Get the moves before everyone else: the 5 Evaluation & Guardrails repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain audit-the-judge?

Claim this page to add your own call-to-action.

Claim this page