This code also requires the Synchronized-BatchNorm-PyTorch rep.
cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../
To reproduce the results reported in the paper, you would need an NVIDIA DGX1 machine with 8 V100 GPUs.
Varible name should be a combination one or more parts below:
normalized- normalized. If not indicated, then unnormalizedpilorimage-PIL.Imageobject. If not indicated, thentorch.Tensorsubnet- variable is for subnet, and should not be fed into the main networkresized- resized from original image size (default:256x256) to the size of the correspondence matrix (default:64x64) withbilinearwarped- warped output created according to the attentiongt- should be used as ground truth when calculating loss
-
VGG perceptual loss
- input: normalized RGB
-
discriminator
- input: LAB
-
smoothness loss
- input: LAB
-
FID
- input: RGB (not normalized)
-
subnet_transformer:
- input:
image[wxh] PIL Image in RGB - output:
(ref[256x256], ref_warp[64x64]),(target[256x256], target_gt[64x64]),(index_image[256x256], index_image_warp[64x64])all of them arePIL.Imagein RGB- we use indexes stored in
index_image_warp[64x64]to index intoref_warp[64x64]and extractABchannels. ForLchannel,Lfromtarget_gt[64x64]is used.
- we use indexes stored in
- input:
-
subnet
- input:
target_L&reference_LAB(256x256) normalized
- input:
-
discriminator
- input:
warped_LAB,target_LAB(256x256) normalized
- input:
-
FID
- input:
warped_RGB&target_RGB(not normalized)
- input:
-
smoothness
- input:
warped_LAB(64x64)
- input:
-
softmax
- input: raw
attentionfrom subnet (64x64x64^2) & index tesnor (64x64)
- input: raw
-
VGG
- input:
target_RGB&generated_RGB(64x64) (이미 네트워크의 인풋으로 normalized 를 넣어주는데 이때 genearted LAB 는 또 normalize 를 해줘야하나?)
- input: