8000 GitHub - Ryvikk/LSUV: LSUV - Layer-Wise Sequential Unit-Variance · GitHub
[go: up one dir, main page]

Skip to content
RyvikkPublic

About

LSUV - Layer-Wise Sequential Unit-Variance

Resources

Stars

0 stars

Watchers

1 watching

Forks

Repository files navigation

LSUV and Modified ReLUs

(base concepts and methods from the Fastai22-23P2 activation and initialization nbs)

We start with a baseline training of a CNN on Fashion MNIST, then progressively improve the accuracy and quality of convergence with normalization methods.
Then we reimplement the LSUV - Layer-Wise Sequential Unit-Variance algorithm and compare it to the previous normalization steps.
We also introduce modified ReLU units with leakiness and learnable parameters.

About

LSUV - Layer-Wise Sequential Unit-Variance

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages

0