ECCV22 paper "Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality" and the T-PAMI extension "Orthogonal SVD Covariance Conditioning and Latent Disentanglement"

We propose nearest-orthogonal gradient (nog) and optimal learning rate (olr) to enforce strict/relaxed orthogonality into the training of differentiable SVD layer, which can simultaneously improve the conditioning and generalization. The combination with orthogonal convolution could further boost the performance.

In the expanded journal version, we further show that the proposed orthogonality techniques can be also used for unsupervised latent disentanglement of generative models such as EigenGAN and vanilla/simple GAN. For EigenGAN, we validate our orthogonality techniques on AnimeFace and FFHQ. For vanilla/simple GAN, we conduct experiments on relatively simpler CelebA and LSUN Church.

Run decorrelated BN experiments with proposed techniques to improve covariance conditioning:

CUDA_VISIBLE_DEVICES=0 python main_cifar100.py --norm='zcanormbatch' --batch_size=128 --nog --olr --ow

Run orthogonal EigenGAN on FFHQ as:

CUDA_VISIBLE_DEVICES=0 python train_ffhq.py DATA_ROOT  --size 256 --batch 64 --reg_type nog --name ffhq_nog

Check run_ffhq.sh and run_animeface for the detailed training and test commands.

All scripts inlcuding training and test can be found in the folder of training scripts. The examplery usage of training orthogonal vannila GAN on CelebA is:

CUDA_VISIBLE_DEVICES=0 python train.py --dataset_mode celeba --model gan128 --nz 30 --reg_type nog --dataroot CELEBA_ROOT --name celeba_nog  

For the VP score evaluation, after generating images using gen_pairs.py please use the official VP metric to evaluate the disentanglement score.

Check the environment file latent.yml for the full list of required packages.

Please consider citing our paper if you think the code is helpful to your research.

@inproceedings{song2022improving,
  title={Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality},
  author={Song, Yue and Sebe, Nicu and Wang, Wei},
  booktitle={ECCV},
  year={2022}
}

@article{song2022orthogonal,
  title={Orthogonal SVD Covariance Conditioning and Latent Disentanglement},
  author={Song, Yue and Sebe, Nicu and Wang, Wei},
  journal={IEEE TPAMI},
  year={2022},
  publisher={IEEE}
}

If you have any questions or suggestions, please feel free to contact me

yue.song@unitn.it