LLMRepos·GPU Kernels LLM projects

Efficient Triton Kernels for LLM Training

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

Benchmarking Deep Learning operations on different hardware

Pure Rust implementation of a minimal Generative Pretrained Transformer

The most atomic way to train and inference a GPT in pure, dependency-free C

Cleora AI is a general-purpose open-source model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data. Created by Synerise.com team.

Rust + CUDA inference engine for NVIDIA RTX PRO 6000 Blackwell and RTX 5090. Serves safetensors and GGUF over an OpenAI-compatible API, with per-device tuned defaults and speculative decode gated byte-identical to plain decode. Hosted instance: inference.tiyuvta.ai

An official lightweight library for the RaBitQ algorithm and its applications in vector search.

Quantized LLM training in pure CUDA/C++.

Sparse Inferencing for transformer based LLMs

Tiny ASIC implementation for "The Era of 1-bit LLMs All Large Language Models are in 1.58 Bits" matrix multiplication unit

Simulation framework for nonsmooth dynamical systems

OrbitKV: a Rust attention-state compiler and lifetime-safe KV block manager.

OrbitKV: a Rust attention-state compiler and lifetime-safe KV block manager.

OrbitKV: a Rust attention-state compiler and lifetime-safe KV block manager.

CUDA编程练习项目-Hands-on CUDA kernels and performance optimization, covering GEMM, FlashAttention, Tensor Cores, CUTLASS, quantization, KV cache, NCCL, and profiling.

[HPCA'21] SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning

An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch-1 LLM decode, paper: https://arxiv.org/abs/2606.09682