Phil Wang Claim this profile

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Implementation of Alphafold 3 from Google Deepmind in Pytorch

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Vector (and Scalar) Quantization, in Pytorch

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

Simplest working implementation of Stylegan2, state of the art generative adversarial network, in Pytorch. Enabling everyone to experience disentanglement

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

Reformer, the efficient Transformer, in Pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

Unofficial implementation of Titans, SOTA memory for transformers, in Pytorch

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI

Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

Implementation of Natural Speech 2, Zero-shot Speech and Singing Synthesizer, in Pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

Implementation of TabTransformer, attention network for tabular data, in Pytorch

An All-MLP solution for Vision, from Google AI

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

Transformer based on a variant of attention that is linear complexity in respect to sequence length

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification

A concise but complete implementation of CLIP with various experimental improvements from recent papers

Implementation of Voicebox, new SOTA Text-to-speech network from MetaAI, in Pytorch

Implementation of Bottleneck Transformer in Pytorch

Implementation of MagViT2 Tokenizer in Pytorch

A simple way to keep track of an Exponential Moving Average (EMA) version of your Pytorch model

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

Implementation of the Point Transformer layer, in Pytorch

Implementation of π₀, the robotic foundation model architecture proposed by Physical Intelligence

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch

Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch

Implementation of E2-TTS, "Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS", in Pytorch

Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function

An implementation of local windowed attention for language modeling

Implementation of Slot Attention from GoogleAI

Implementation of rectified flow and some of its followup research / improvements in Pytorch

Implementation of RT1 (Robotic Transformer) in Pytorch

Implementation of the convolutional module from the Conformer paper, for use in Transformers

Implementation of Autoregressive Diffusion in Pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind

Implementation of Axial attention - attending to multi-dimensional data efficiently

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Implementation of Segformer, Attention + MLP neural network for segmentation, in Pytorch

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Implementation of Bit Diffusion, Hinton's group's attempt at discrete denoising diffusion, in Pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repository is geared towards integration with eventual Alphafold2 replication.

Implementation of the proposed minGRU in Pytorch

Explorations into training LLMs to use clinical calculators from patient history, using open sourced models. Will start with Wells' Criteria

Implementation of Linformer for Pytorch

Explorations into some recent techniques surrounding speculative decoding

Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Fully featured implementation of Routing Transformer

Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI

Implementation of a U-net complete with efficient attention as well as the latest research findings

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding

Implementation of Lumiere, SOTA text-to-video generation from Google Deepmind, in Pytorch

Implementation of a Transformer, but completely in Triton

Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch

Implementation of Pixel-level Contrastive Learning, proposed in the paper "Propagate Yourself", in Pytorch

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

Implementation of MetNet-3, SOTA neural weather model out of Google Deepmind, in Pytorch

Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2

A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch

Implementation of Flash Attention in Jax

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

Implementation of Block Recurrent Transformer - Pytorch

Implementation of Recurrent Interface Network (RIN), for highly efficient generation of images and video without cascading networks, in Pytorch

Implementation of Danijar's latest iteration for his Dreamer line of work

A simple cross attention that updates both the source and target in one step

Unofficial implementation of Tiny Recursive Model (TRM), improvement to HRM from Sapient AI, by Alexia Jolicoeur-Martineau

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)

Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public

Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch

Implementation of TiTok, proposed by Bytedance in "An Image is Worth 32 Tokens for Reconstruction and Generation"

Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

Implementation of ProteinBERT in Pytorch

Implementation of Chroma, generative models of protein using DDPM and GNNs, in Pytorch

Implementation of the new SOTA for model based RL, from the paper "Improving Transformer World Models for Data-Efficient RL", in Pytorch

Implementation of ETSformer, state of the art time-series Transformer, in Pytorch

A simple to use pytorch wrapper for contrastive self-supervised learning on any neural network

Implementation of Nyström Self-attention, from the paper Nyströmformer

Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch

A denoising diffusion probabilistic model (DDPM) tailored for conditional generation of protein distograms

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind

Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics

Pytorch implementation of MIMO, Controllable Character Video Synthesis with Spatial Decomposed Modeling, from Alibaba Intelligence Group

A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch

Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification

Implementation of RQ Transformer, proposed in the paper "Autoregressive Image Generation using Residual Quantization"

Implementation of the Large Behavioral Model architecture for dexterous manipulation from Toyota Research Institute

An implementation of PPO in Pytorch

Implementation of Mimic-Video, Video-Action Models for SOTA Generalizable Robot Control Beyond VLAs

LocoFormer - Generalist Locomotion via Long-Context Adaptation

Explorations into some of the approaches advocated by Yann LeCun, and just a more wholistic architecture (JEPA) in general

Pytorch implementation of Evolutionary Policy Optimization, from Wang et al. of the Robotics Institute at Carnegie Mellon University

Implementation of HS-TasNet, "Real-time Low-latency Music Source Separation using Hybrid Spectrogram-TasNet"

Implementation of a multimodal diffusion transformer in Pytorch

Implementation of the MetaController proposed in "Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning" from the Paradigms of Intelligence team at Google

Implementation of LVSM, SOTA Large View Synthesis with Minimal 3d Inductive Bias, from Adobe Research

Implementation of the proposed DeepCrossAttention by Heddes et al at Google research, in Pytorch

Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"

A Transformer made of Rotation-equivariant Attention using Vector Neurons

Explorations into the recently proposed Taylor Series Linear Attention

Implementation of AlphaGenome, Deepmind's updated genomic attention model

A repository to house some personal attempts to beat some state-of-the-art for medical datasets

Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch

Implementation of GateLoop Transformer in Pytorch and Jax

Thispersondoesnotexist went down, so this time, while building it back up, I am going to open source all of it.

Implementation of the transformer proposed in "Building Blocks for a Complex-Valued Transformer Architecture"

Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorch

Implementation of the dynamic chunking mechanism in H-net by Hwang et al. of Carnegie Mellon

Exploration into the proposed architecture from Sapient Intelligence of Singapore 🇸🇬

The Gaussian Histogram Loss (HL-Gauss) proposed by Imani et al. with a few convenient wrappers for regression, in Pytorch

Implementation of D4RT, Efficiently Reconstructing Dynamic Scenes, from Deepmind

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Implementation of a transformer for reinforcement learning using `x-transformers`

Explorations into whether a transformer with RL can direct a genetic algorithm to converge faster

Efficient implementation (and explorations) into polar coordinate positional embedding (PoPE) - from Gopalakrishnan et al. under Schmidhuber

An implementation of (Induced) Set Attention Block, from the Set Transformers paper

Implementation of Soft Actor Critic and some of its improvements in Pytorch

Unofficial implementation of GotenNet, new SOTA 3d equivariant transformer, in Pytorch

Contrastive Reinforcement Learning

Implementation of RL-100, Performant Robotic Manipulation with Real-World Reinforcement Learning

Implementation of the world model architecture for self driving out of Wayve

Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"

Toy genetic algorithm in Pytorch

Implementation of Denoising Diffusion for protein design, but using the new Equiformer (successor to SE3 Transformers) with some additional improvements

Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)

Unofficial implementation of Hippoformer, Integrating Hippocampus-inspired Spatial Memory with Transformers

Implementation of Poly-attention, a higher-order self-attention proposed by Chakrabarti et al. of Columbia

Some utility functions to help myself (and perhaps others) go faster with ML/AI work

Implementation of various evolutionary algorithms, starting with evolutionary strategies

Just a repository that will house some MLPs and their variants, so to avoid having to reimplement them again and again for different projects (especially RL)

Implementation of a Light Recurrent Unit in Pytorch

Implementation of 2-simplicial attention proposed by Clift et al. (2019) and the recent attempt to make practical in Fast and Simplex, Roy et al. (2025)

Implementation of Humanoid Standing Up, from the paper "Learning Humanoid Standing-up Control across Diverse Postures" out of Shanghai, in Pytorch

Implementation of Kalmanformer, modeling the Kalman gain with a transformer

Explorations into NEAT and some of its derivative research

Exploration into Discrete Distribution Network, by Lei Yang out of Beijing

Implementation of Fast Weight Attention

Explorations into the proposed Streaming Deep Reinforcement Learning, from University of Alberta

Explorations into the proposed SDFT, Self-Distillation Enables Continual Learning, from Shenfeld et al. of MIT

Implementation of the model architecture for SRT-H

Implementation of RigidFormer, Learning Rigid Dynamics using Transformers

Causal Attention with Lookahead Keys

Implementation of WorldParticle, Unified World Simulation of Lagrangian Particle Dynamics via Transformer

Implementation of ViLLA-X, Enhancing Latent Action Modeling in Vision-Language-Action Models

Implementation and explorations into DiscoRL, Discovering state-of-the-art reinforcement learning algorithms, David Silver's last work at Deepmind

Usable implementation of Mogrifier, a circuit for enhancing LSTMs and potentially other networks, from Deepmind

Embedding and readout for simple multi-categorical and gaussian continuous

Implementation of the fast weight product key memory from Sakana AI

Implementation of Multiscreen proposed by Ken Nakanishi for "Screening is Enough"

Implementation of Attention Schema-based Attention Control (ASAC), A Cognitive-Inspired Approach for Attention Management in Transformers

Implementation of the Hierarchical Latent Action Model, proposed by Hanjung Kim et al. of Yonsei University

Implementation and explorations into PopuLoRA, Co-Evolving LLM Populations for Reasoning Self-Play

Implementation of Lift4d, Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild, from Litman et al. of Carnegie Mellon

Wrappers around environments that will take care of providing representations from self supervised learning automagically

Implementation and explorations into MPO / DMPO

A simple numpy memmap replay buffer for RL and personal use-cases

Exploration and eventually practical implementation for the Multistep Quasimetric Estimation proposed by Zheng et al. of Berkeley

Contrastive Language-Image Pretraining

Implementation of RISE

DeepSeek 4 Flash local inference engine for Metal and CUDA