A lightweight deep learning framework built from first principles in modern C++.
- Tensor abstraction (contiguous N-dimensional storage).
- Matrix-based dense (fully-connected) layers: weight matrix + bias vector.
- Refactor per-neuron object model to vectorized layers.
- Feedforward networks with arbitrary topology.
- Bias neurons for learning offsets.
- Convolutional layers.
- Max/average pooling (downscaling).
- Transpose convolutions (upscaling).
- Batch normalization.
- Layer normalization.
- Sequential model API.
- Backpropagation with automatic gradient computation.
- Autograd engine (dynamic computational graph).
- Gradient descent with momentum optimizer.
- Configurable learning rate and momentum.
- Mini-batch training.
- Exponential moving average.
- L2 regularization (weight decay).
- Dropout.
- Gradient clipping.
- Data loaders with shuffling.
- Seedable RNG for reproducible runs.
- Model serialization (save/load weights).
- Model checkpointing (save/resume training).
- Weight initialization strategies: Xavier/Glorot, He/Kaiming.
- Advanced optimizers: Adam, AdamW, RMSprop.
- Learning rate schedulers: step decay, reduce on plateau, cosine annealing.
- Data augmentation (random flips, crops, rotations).
- Mixed precision training (FP16/FP32).
- Linear (identity).
- Hyperbolic tangent.
- ReLU (Rectified Linear Unit).
- Sigmoid.
- Softmax.
- ELU (Exponential Linear Unit).
- GELU (Gaussian Error Linear Unit).
- Swish/SiLU.
- Mean Squared Error (MSE).
- Mean Absolute Error (MAE).
- Binary Cross-Entropy (BCE).
- Categorical Cross-Entropy.
- Dice Loss.
- Combo Loss.
- IoU/Jaccard Loss.
- Focal Loss.
- Tversky Loss.
- Huber Loss.
- Memory pool allocators.
- Multi-threaded batch processing.
- SIMD intrinsics (AVX2/AVX-512)
- CUDA support (GPU acceleration).
- cuDNN integration (optimized conv/pooling).
- Built-in metrics: accuracy, precision, recall, F1.
- Model summary (print architecture).
- Training history logging.
- Visualization tools.
- Confusion matrix.
- Youden's J index.
- Gradient checking (finite-difference verification).
- Overfit a single batch to near-zero loss (Karpathy, 2019).
- David E. Rumelhart et al., Learning representations by back-propagating errors (1986).
- George Cybenko, Approximation by superpositions of a sigmoidal function (1989).
- Yann LeCun et al., Efficient BackProp (1998).
- David Miller, Neural Net in C++ Tutorial.
- Michael Nielsen, Neural Networks and Deep Learning (2015).
- John D. Kelleher, Deep Learning (2019).
- Andrej Karpathy, A Recipe for Training Neural Networks (2019).
This repository is released under the MIT license. See LICENSE for more information.