8000 GitHub - rajatvd/NTK: Code for experiments in my blog post on the Neural Tangent Kernel: https://eigentales.com/NTK · GitHub
[go: up one dir, main page]

Skip to content
  • rajatvdPublic

    About

    Code for experiments in my blog post on the Neural Tangent Kernel: https://eigentales.com/NTK

    Resources

    Stars

    174 stars

    Watchers

    3 watching

    Forks

    Repository files navigation

    Neural Tangent Kernel

    This repo contains code for my blog post on Understanding the Neural Tangent Kernel.

    Notebooks

    • ntk.ipynb has code for generating the gifs and some other visualizations (like the kernel function) not included in the post.
    • one_layer_asymptotics.ipynb empirically verifies the asymptotics for the gradient and Hessian derived in the blog post. Play with it in colab here: colab link.

    Scripts

    • utils.py - has some helper functions to create models, run gradient descent, and calculate NTKs.
    • train.py - a sacred script which trains a simple fully connected network on 1-D data points.
    • grid_run.py - run a grid of sacred scripts with different configs. For example, I used this to create the runs with different alphas in the ellipse gif.

    About

    Code for experiments in my blog post on the Neural Tangent Kernel: https://eigentales.com/NTK

    Resources

    Stars

    174 stars

    Watchers

    3 watching

    Forks

    Releases

    Packages

    Used by

    Contributors

    Languages

    0