Fast, Modern, and Low Precision PyTorch Optimizers

Muon is an optimizer for hidden layers in neural networks

A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning

A project to improve skills of large language models

Tilus is a tile-level kernel programming language with explicit control over shared memory and registers.

Simple RL training for reasoning

The Python programming language

TensorDict is a pytorch dedicated tensor container.

A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.

An implementation of AlphaZero and MCTS with neural networks for Tetris

Official electron build of draw.io

FlashRNN - Fast RNN Kernels with I/O Awareness

maximal update parametrization (µP)

A PowerPoint add-in to insert LaTeX equations into PowerPoint presentations on Windows and Mac

🚀 Efficient implementations of state-of-the-art linear attention models

This is a successor of prboom+ with extra tooling for demo recording and playback, with a focus on speedrunning and quality of life.

Constrained optimization toolkit for PyTorch

Accelerated First Order Parallel Associative Scan

Beyond Straight-Through

Schedule-Free Optimization in PyTorch

Implementations of various linear RNN layers using pytorch and triton

Structured state space sequence models

Python library for arbitrary-precision floating-point arithmetic

Haste: a fast, simple, and open RNN library

Rational Activation Functions - Replacing Padé Activation Units

Implementation of Bootstrap DQN and Randomized Prior Functions on ALE