[go: up one dir, main page]

Skip to content

A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and more, for researchers and developers in the Kolmogorov-Arnold Network field.

Notifications You must be signed in to change notification settings

kunpengxu/awesome-kan

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 

Repository files navigation

Awesome KAN(Kolmogorov-Arnold Network)

Awesome Contributions Commits GitHub stars

A curated list of awesome libraries, projects, tutorials, papers, and other resources related to Kolmogorov-Arnold Network (KAN). This repository aims to be a comprehensive and organized collection that will help researchers and developers in the world of KAN!

image

Table of Contents

Papers

Theorem

Library

  • pykan : Offical implementation for Kolmogorov Arnold Networks | Github stars
  • efficient-kan : An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN). | Github stars
  • FastKAN : Very Fast Calculation of Kolmogorov-Arnold Networks (KAN) | Github stars
  • FasterKAN : FasterKAN = FastKAN + RSWAF bases functions and benchmarking with other KANs. Fastest KAN variation as of 5/13/2024, 2 times slower than MLP in backward speed. | Github stars
  • FourierKAN : Pytorch Layer for FourierKAN. It is a layer intended to be a substitution for Linear + non-linear activation | Github stars
  • Vision-KAN : PyTorch Implementation of Vision Transformers with KAN layers, built on top ViT. 95% accuracy on CIFAR100 (top-5), 80% on ImageNet1000 (training in progress) | Github stars
  • ChebyKAN : Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines. | Github stars
  • GraphKAN : Implementation of Graph Neural Network version of Kolmogorov Arnold Networks (GraphKAN) | Github stars
  • FCN-KAN : Kolmogorov–Arnold Networks with modified activation (using fully connected network to represent the activation) | Github stars
  • X-KANeRF : KAN based NeRF with various basis functions like B-Splines, Fourier, Radial Basis Functions, Polynomials, etc | Github stars
  • Large Kolmogorov-Arnold Networks : Variations of Kolmogorov-Arnold Networks (including CUDA-supported KAN convolutions) | Github stars
  • xKAN : Kolmogorov-Arnold Networks with various basis functions like B-Splines, Fourier, Chebyshev, Wavelets etc | Github stars
  • JacobiKAN : Kolmogorov-Arnold Networks (KAN) using Jacobi polynomials instead of B-splines. | Github stars
  • GraphKAN : Implementation of Graph Neural Network version of Kolmogorov Arnold Networks (GraphKAN) | Github stars
  • OrthogPolyKAN : Kolmogorov-Arnold Networks (KAN) using orthogonal polynomials instead of B-splines. | Github stars
  • kansformers : Kansformers: Transformers using KANs | Github stars
  • Deep-KAN: Better implementation of Kolmogorov Arnold Network | Github stars
  • RBF-KAN: RBF-KAN is a PyTorch module that implements a Radial Basis Function Kolmogorov-Arnold Network | Github stars
  • KolmogorovArnold.jl : Very fast Julia implementation of KANs with RBF and RSWAF basis. Extra speedup is gained by writing custom gradients to share work between forward and backward pass. | Github stars
  • Wav-KAN: Wav-KAN: Wavelet Kolmogorov-Arnold Networks | Github stars
  • KANX : Fast Implementation (Approximation) of Kolmogorov-Arnold Network in JAX | Github stars
  • FlashKAN: Grid size-independent computation of Kolmogorov Arnold networks | Github stars
  • BSRBF_KAN: Combine B-Spline (BS) and Radial Basic Function (RBF) in Kolmogorov-Arnold Networks (KANs) | Github stars
  • TaylorKAN: Kolmogorov-Arnold Networks (KAN) using Taylor series instead of Fourier | Github stars
  • fKAN: fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions | Github stars
  • Initial Investigation of Kolmogorov-Arnold Networks (KANs) as Feature Extractors for IMU Based Human Activity Recognition
  • rKAN: rKAN: Rational Kolmogorov-Arnold Networks | Github stars
  • TKAN: Temporal Kolmogorov-Arnold Networks Keras3 layer implementations multibackend (Jax, Tensorflow, Torch) | Github stars
  • TKAT: Temporal Kolmogorov-Arnold Transformer Tensorflow 2.x model implementation | Github stars
  • SigKAN: Path Signature-Weighted Kolmogorov-Arnold Networks tensorflow 2.x layer implementations, based on iisignature | Github stars
  • KAN-SGAN: Semi-supervised learning with Generative Adversarial Networks (GANs) using Kolmogorov-Arnold Network Layers (KANLs) | Github stars

Library-based

ConvKANs

  • Convolutional-KANs : This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to non linear activations in each pixel. | Github stars
  • Torch Conv KAN : This repository implements Convolutional Kolmogorov-Arnold Layers with various basis functions. The repository includes implementations of 1D, 2D, and 3D convolutions with different kernels, ResNet-like, Unet-like, and DenseNet-like models, training code based on accelerate/PyTorch, and scripts for experiments with CIFAR-10/100, Tiny ImageNet and ImageNet1k. Pretrained weights on ImageNet1k are also available | Github stars
  • convkan : Implementation of convolutional layer version of KAN (drop-in replacement of Conv2d) | Github stars
  • KA-Conv : Kolmogorov-Arnold Convolutional Networks with Various Basis Functions (Optimization for Efficiency and GPU memory usage) | Github stars
  • KAN-Conv2D : Drop-in Convolutional KAN built on multiple implementations (Original pykan / efficient-kan / FastKAN) to support the original paper hyperparameters. | Github stars
  • CNN-KAN : A modified CNN architecture using Kolmogorov-Arnold Networks | Github stars
  • ConvKAN3D : 3D Convolutional Layer built on top of the efficient-kan implementation (importable Python package from PyPi), drop-in replacement of Conv3d.

Benchmark

  • KAN-benchmarking : Benchmark for efficiency in memory and time of different KAN implementations. | Github stars
  • seydi1370/Basis_Functions : This packaege investigates the performance of 18 different polynomial basis functions, grouped into several categories based on their mathematical properties and areas of application. The study evaluates the effectiveness of these polynomial-based KANs on the MNIST dataset for handwritten digit classification. | Github stars

Non-Python

Alternative

  • high-order-layers-torch : High order piecewise polynomial neural networks using Chebyshev polynomials at Gauss Lobatto nodes (lagrange polynomials). Includes convolutional layers as well HP refinement for non convolutional layers, linear initialization and various applications in the linked repos with varrying levels of success. Euler equations of fluid dynamics, nlp, implicit representation and more | Github stars
  • Training based on Kaczmarz, not Broyden method : The training process is independent of the basis functions, the provided link shows alternative to Broyden method originally suggested in MIT paper. It outperforms MLP siginificantly, benchmarks provided.

Project

  • KAN-GPT : The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling | Github stars
  • KAN-GPT-2 : Training small GPT-2 style models using Kolmogorov-Arnold networks.(despite the KAN model having 25% fewer parameters!). | Github stars
  • KANeRF : Kolmogorov-Arnold Network (KAN) based NeRF | Github stars
  • Vision-KAN : KAN for Vision Transformer | Github stars
  • Simple-KAN-4-Time-Series : A simple feature-based time series classifier using Kolmogorov–Arnold Networks | Github stars
  • KANU_Net : U-Net architecture with Kolmogorov-Arnold Convolutions (KA convolutions) | Github stars
  • kanrl : Kolmogorov-Arnold Network for Reinforcement Leaning, initial experiments | Github stars
  • kan-diffusion : Applying KANs to Denoising Diffusion Models with two-layer KAN able to restore images almost as good as 4-layer MLP (and 30% less parameters). | Github stars
  • KAN4Rec : Implementation of Kolmogorov-Arnold Network (KAN) for Recommendations | Github stars
  • CF-KAN : Kolmogorov-Arnold Network (KAN) implementation for collaborative filtering (CF) | Github stars
  • X-KANeRF : X-KANeRF: KAN-based NeRF with Various Basis Functions to explain the the NeRF formula | Github stars
  • KAN4Graph : Implementation of Kolmogorov-Arnold Network (KAN) for Graph Neural Networks (GNNs) and Tasks on Graphs | Github stars
  • ImplicitKAN : Kolmogorov-Arnold Network (KAN) as an implicit function for images and other modalities | Github stars
  • ThangKAN : Kolmogorov-Arnold Network (KAN) for text classification over GLUE tasks | Github stars
  • JianpanHuang/KAN : This repository contains a demo of regression task (curve fitting) using an efficient Kolmogorov-Arnold Network. | Github stars
  • Fraud Detection in Supply Chains Using Kolmogorov Arnold NetworksGithub stars
  • CL-KAN-ViT : Kolmogorov-Arnold Network (KAN) based vision transformer for class-based continual learning to mitigate catastrophic forgetting | Github stars
  • KAN-Autoencoder : KAE KAN-based AutoEncoder (AE, VAE, VQ-VAE, RVQ, etc.) | Github stars
  • OpenKAN
  • KAN-DQN : An experiment where KAN replaces MLP in Deep Q-Network to play Flappy Bird as a Reinforcement Learning agent. | GitHub Repo stars

Discussion

Tutorial

YouTube

Contributing

We welcome your contributions! Please follow these steps to contribute:

  1. Fork the repo.
  2. Create a new branch (e.g., feature/new-kan-resource).
  3. Commit your changes to the new branch.
  4. Create a Pull Request, and provide a brief description of the changes/additions.

Please make sure that the resources you add are relevant to the field of Kolmogorov-Arnold Network. Before contributing, take a look at the existing resources to avoid duplicates.

License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Star History

Star History Chart

About

A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and more, for researchers and developers in the Kolmogorov-Arnold Network field.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published