LLMRepos·Interpretability & Alignment LLM projects

Updated daily

Browse 53 open-source interpretability & alignment projects in Model Development. Compare GitHub stars, recent growth, languages, licenses, and repository activity.

Interpretability & Alignment

Browse 53 open-source interpretability & alignment projects in Model Development. Compare GitHub stars, recent growth, languages, licenses, and repository activity.

53
Repositories
452
Model Development

Top Interpretability & Alignment repositories

Ranked by current GitHub stars from the latest LLMRepos snapshot.

Showing 40 of 53

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

PythonOther+5 stars in 7dupdated today

A JAX research toolkit for building, editing, and visualizing neural networks.

PythonApache License 2.0+0 stars in 7dupdated 428d ago

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

PythonApache License 2.0-1 stars in 7dupdated 273d ago

Research of DeepSeek Engram Architecture based on Qwen-3 and Stable Diffusion series.

Python+95 stars in 7dupdated 95d ago

Observe how memory shapes an answer. Pensieve is an interactive system for visualizing, interpreting, and managing how Large Language Models (LLMs) “remember” a user. It bridges the gap between model-level mechanisms and user-level understanding, making AI memory observable, explainable, and partially controllable.

TypeScriptMIT License+171 stars in 7dupdated 27d ago

[ACL2023] We introduce LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the diverse strengths of multiple open-source LLMs. LLM-Blender cut the weaknesses through ranking and integrate the strengths through fusing generation to enhance the capability of LLMs.

PythonApache License 2.0+0 stars in 7dupdated 671d ago

Official Implementation for "Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models" (SIGGRAPH 2023)

Jupyter NotebookMIT License+0 stars in 7dupdated 941d ago

[ICML 2026 & EMNLP 2026] Multimodal deep-research MLLM and benchmark. The first long-horizon multimodal deep-research MLLM, extending the number of reasoning turns to dozens and the number of search-engine interactions to hundreds.

PythonMIT License+2 stars in 7dupdated 16d ago

Official implementation for "Blended Diffusion for Text-driven Editing of Natural Images" [CVPR 2022]

Jupyter NotebookMIT License+0 stars in 7dupdated 811d ago

Underexposed Photo Enhancement Using Deep Illumination Estimation

Python+0 stars in 7dupdated 1,249d ago

A flexible, adaptive classification system for dynamic text classification

PythonApache License 2.0+0 stars in 7dupdated 321d ago

Named Entity Recognition using multilayered bidirectional LSTM

Python+0 stars in 7dupdated 2,725d ago

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

PythonMIT License+2 stars in 7dupdated 119d ago

A Python toolbox for conformal prediction research on deep learning models, using PyTorch.

PythonGNU Lesser General Public License v3.0+1 stars in 7dupdated 19d ago

Graph convolutional neural network for multirelational link prediction

Jupyter NotebookMIT License+1 stars in 7dupdated 1,372d ago

EvaLearn is a pioneering benchmark designed to evaluate large language models (LLMs) on their learning capability and efficiency in challenging tasks.

PythonApache License 2.0+0 stars in 7dupdated 104d ago
422
stars

A PyTorch implementation of ACM SIGKDD 2019 paper "Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks"

PythonMIT License+0 stars in 7dupdated 760d ago

Advanced Quantitative Factor Research: ML-powered stock return prediction with 72% performance improvement. Features comprehensive alpha factor library, systematic feature selection, and deep learning models (LSTM+ResNet achieving IC=0.06476).

Jupyter Notebook+0 stars in 7dupdated 367d ago

Backdoors Framework for Deep Learning and Federated Learning. A light-weight tool to conduct your research on backdoors.

PythonMIT License+0 stars in 7dupdated 1,296d ago

We jailbreak GPT-3.5 Turbo’s safety guardrails by fine-tuning it on only 10 adversarially designed examples, at a cost of less than $0.20 via OpenAI’s APIs.

PythonMIT License+1 stars in 7dupdated 913d ago

Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs

Jupyter NotebookApache License 2.0+1 stars in 7dupdated 808d ago

End-to-end training of sparse deep neural networks with little-to-no performance loss.

PythonApache License 2.0+0 stars in 7dupdated 1,306d ago

The purpose of this repository is to make prototypes as case study in the context of proof of concept(PoC) and research and development(R&D) that I have written in my website. The main research topics are Auto-Encoders in relation to the representation learning, the statistical machine learning for energy-based models, adversarial generation networks(GANs), Deep Reinforcement Learning such as Deep Q-Networks, semi-supervised learning, and neural network language model for natural language processing.

PythonGNU General Public License v2.0+0 stars in 7dupdated 972d ago

[NeurIPS 2025] BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models

PythonMIT License+1 stars in 7dupdated 164d ago

Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs (ACL 2024)

Jupyter NotebookApache License 2.0+0 stars in 7dupdated 608d ago

TrojanZoo provides a universal pytorch platform to conduct security researches (especially backdoor attacks/defenses) of image classification in deep learning.

PythonMIT License+0 stars in 7dupdated 364d ago

[USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

PythonMIT License+6 stars in 7dupdated 209d ago

Code and data for the research paper "Towards Open Set Deep Networks" A Bendale, T Boult, CVPR 2016

PythonOther-1 stars in 7dupdated 3,039d ago

This is the repo of the research paper, "Evaluating Shallow and Deep Neural Networks for Network Intrusion Detection Systems in Cyber Security".

Python+0 stars in 7dupdated 976d ago

RuLES: a benchmark for evaluating rule-following in language models

PythonApache License 2.0+0 stars in 7dupdated 546d ago

I replicated Ng's RYS method and found that duplicating 3 specific layers in Qwen2.5-32B boosts reasoning by 17% and duplicating layers 12-14 in Devstral-24B improves logical deduction from 0.22→0.76 on BBH — no training, no weight changes, just routing hidden states through the same circuit twice. Tools included. Two AMD GPUs, one evening.

Python+0 stars in 7dupdated 158d ago

Data Synthesis for Deep Research Based on Semi-Structured Data

PythonApache License 2.0+0 stars in 7dupdated 41d ago

Stanford NLP Python library for benchmarking the utility of LLM interpretability methods

PythonApache License 2.0+1 stars in 7dupdated 165d ago

Deep neural network kernel for Gaussian process

PythonApache License 2.0+0 stars in 7dupdated 2,176d ago