8000 GitHub - alzayats/UDnet: Adaptive Uncertainty Distribution in Deep Learning for Unsupervised Underwater Image Enhancement · GitHub
[go: up one dir, main page]

Skip to content
alzayatsPublic

About

Adaptive Uncertainty Distribution in Deep Learning for Unsupervised Underwater Image Enhancement

Topics

Resources

Stars

71 stars

Watchers

2 watching

Forks

Repository files navigation

UDnet

Underwater image enhancement trained without ground truth. UDnet removes the blue–green cast and haze from underwater photos and video frames, at the original resolution. Official code for Adaptive deep learning framework for robust unsupervised underwater image enhancement (Expert Systems with Applications, 2025).

Paper Model Gallery License PyTorch

Split-screen animation over eight underwater scenes: the original photo on the left of a moving divider, UDnet V2 + AI Max on the right

Left of the divider: original · Right: UDnet V2 + AI Max (experimental, ai-max branch). Images from UIEB (C60) and LSUI. Full-quality MP4.
Open the gallery to compare the original, V1, V2 and AI Max with before/after sliders on 24 scenes.

Models

Model Status Weights Speed (256×256) Notes
UDnet V1 Released UDnet.pth · 5.6 MB 457 img/s GPU
11 img/s CPU
Paper model. Fully unsupervised, 1.4 M parameters, runs on a CPU.
UDnet V2 Released UDnet_v2.pth · 26 MB 209 img/s GPU The UDnet design modernised and trained on paired images for stronger correction. 12.8 M parameters. See the gallery.
UDnet V2 + AI Preview V2 + Real-ESRGAN 45 img/s GPU V2 plus an AI detail pass: sharper, 2× upscaled. Can invent fine texture, so it's for viewing, not measurement.

Speed: one 256×256 image on an RTX 4090 (CPU: 24 threads), model time only.

The code exactly as published with the paper is tagged v1.0-paper.

Quick start

git clone https://github.com/alzayats/UDnet.git
cd UDnet
pip install -r requirements.txt

# V1, the paper model
python Inference.py --input path/to/images --output results/
# V2
python Inference.py --input path/to/images --output results/ --model v2
# V2 + AI detail, 2× upscaled
python Inference.py --input path/to/images --output results/ --model v2 --ai x2
# a video
python Inference.py --input dive.mp4 --output results/ --model v2

The weights are downloaded on first run. A GPU is used if available, otherwise the CPU. Videos (.mp4, .mov, .avi, .mkv, .m4v) are enhanced frame by frame and written as .mp4 at the original frame rate; the audio track is not copied. On an RTX 4090, 1080p video runs at about 9 frames/s with V1 and 7 frames/s with V2.

Option         Default
--input required an image or video file, or a folder of them
--output results/ output folder; file names are kept
--model v1 v1 (paper model) or v2
--ai none x2 or x4: Real-ESRGAN detail pass that sharpens and upscales; for viewing, not measurement
--weights auto a checkpoint, e.g. from Train.py or Train_v2.py; by default the released weights
--device auto cpu, cuda, cuda:1, ...
--format png image output: png, jpg, or same as the input
Use it from Python
import numpy as np, torch
from PIL import Image
from model_utils.UDnet import load_udnet

model = load_udnet('weights/UDnet.pth', device='cuda')
img = np.asarray(Image.open('photo.jpg').convert('RGB'), dtype=np.float32) / 255
x = torch.from_numpy(img).permute(2, 0, 1)[None].cuda()  # (1, 3, H, W), 0-1
out = model.enhance(x)                                    # same shape, 0-1
out = (out[0].permute(1, 2, 0).cpu().numpy() * 255).round().astype(np.uint8)
Image.fromarray(out).save('out.png')

For V2, load it with from model_utils.UDnet_v2 import load_udnet_v2 and model = load_udnet_v2('weights/UDnet_v2.pth', device='cuda'); model.enhance works the same way.

V2 + AI Max (experimental, separate branch)

The most detailed results in the gallery add a diffusion-based detail-restoration stage (DiffBIR) after V2. Because it needs its own environment, about 15 GB of disk, a GPU with about 10 GB of memory and roughly 25 s per image, it is kept on the ai-max branch rather than in the main installation. It can synthesise detail that was not captured, so it is intended for viewing, not for measurement. Requirements, limitations and licences are described in AI_MAX.md.

Evaluate

Score results with PSNR, SSIM, UIQM and UCIQE

evaluate.py scores a folder of results. With --ref it computes PSNR and SSIM against reference images (matched by file name, any extension); UIQM and UCIQE are always reported.

python Inference.py --input EUVP/test_samples/Inp --output results/EUVP
python evaluate.py --pred results/EUVP --ref EUVP/test_samples/GTr \
    --csv results/EUVP.csv

Add --list splits/...txt to score only the images in a split.

UDnet v1 is fully unsupervised: it never sees a reference image during training, so it is not trying to reproduce the reference style that PSNR and SSIM reward. UIQM and UCIQE need no reference.

The NAFNet model released as the V2 preview before UDnet V2 is kept as UDnet_v2_preview.pth and loads with the same load_udnet_v2. If you use the older v2.0-preview tag of this repository, download that file and pass it with --weights weights/UDnet_v2_preview.pth, because that code expects the preview model.

Train

Train or fine-tune V1 on your own images

UDnet V1 needs only a folder of underwater images. Training targets are generated from each image on the fly.

# from scratch
python Train.py --train_dir path/to/images
# fine-tune the released model
python Train.py --train_dir path/to/images --pretrained weights/UDnet.pth \
    --nEpochs 50
# continue a run
python Train.py --train_dir path/to/images --resume_train True
# use a split
python Train.py --train_dir UIEB/raw-890 --train_list splits/UIEB/train800.txt

Checkpoints go to weights/: epoch_last.pth every 2 epochs and epoch_<N>.pth every --snapshots epochs. See python Train.py -h for all options.

Train or fine-tune V2 on paired images

V2 is trained on input/reference pairs from UIEB (raw-890, reference-890) and LSUI (input, GT), using the lists in splits/V2. It needs a CUDA GPU and a few extra packages.

pip install kornia lpips tensorboard
# from scratch
python Train_v2.py --uieb path/to/UIEB --lsui path/to/LSUI --name my_v2
# continue a run
python Train_v2.py --uieb path/to/UIEB --lsui path/to/LSUI --name my_v2 --resume
# fine-tune the released model
python Train_v2.py --uieb path/to/UIEB --lsui path/to/LSUI --name ft \
    --init weights/UDnet_v2.pth --iters 20000

The loss combines Charbonnier, SSIM, LPIPS and a frequency-domain term; training uses AdamW with warm-up and cosine decay, bfloat16 mixed precision and an EMA of the weights, plus V1's KL term between the posterior and prior latent codes. The released UDnet_v2.pth is the EMA model at 75k of the default 100k iterations (the full run takes about 6 hours on an RTX 4090). Checkpoints go to weights/v2_runs/<name>/; use one with Inference.py --model v2 --weights weights/v2_runs/<name>/best.pth.

Test splits

Which images are in each test split

Each file in splits/ lists image names, one per line.

File Images Taken from
paper_test/UIEB.txt 50 UIEB as released by PUIE-Net: test/image, reference test/label/O
paper_test/UFO120.txt 50 UFO-120 TEST/lrd, reference TEST/hr
paper_test/EUVP.txt 50 EUVP test_samples/Inp, reference 83B1 test_samples/GTr
paper_test/RUIE.txt 50 RUIE UCCS
paper_test/DETRAC.txt 50 Fish-DETRAC sequence V1_Leleiwi_26June19_17
paper_test/SUIM.txt 50 SUIM
paper_test/DeepFish.txt 50 DeepFish
paper_test/FishID.txt 50 FishID
UIEB/train800.txt, UIEB/test90.txt 800 / 90 UIEB raw-890, fixed random split (seed 0)
V2/UIEB_train.txt 738 V2 training images: UIEB/train800.txt without near-duplicates of any test image
V2/LSUI_train.txt, V2/LSUI_test400.txt 3160 / 400 LSUI; test images chosen by near-duplicate group, training images near-duplicate-free

How it works

UDnet V1: the unsupervised paper model

Supervised enhancement models need clean reference images, which almost never exist underwater. UDnet instead builds its own reference for each training image with classical contrast stretching, refined by a learned multi-colour-space stretch module (SGMCSS). A conditional variational autoencoder learns from these noisy references, and a probabilistic adaptive instance normalisation block (PAdaIN) models how uncertain they are. At test time only the image encoder and decoder run. Details are in the paper.

UDnet V2: the modernised, supervised model

V2 trades V1's independence from reference images for stronger correction, but keeps V1's design: a conditional variational autoencoder whose latent code modulates the decoder features. During training a posterior encoder sees the input together with its reference, refined by V1's SGMCSS module, and a KL term ties it to a prior encoder that sees only the input; at test time only the prior runs. The building blocks are modernised: NAFNet blocks (Chen et al., ECCV 2022) over four scales, and the latent code modulates every decoder stage instead of only the last. A small global colour branch first predicts a per-image 3×3 colour matrix and offset; because underwater casts are largely global, it removes most of the cast before the network restores local contrast and detail. The model has 12.8 M parameters, of which 10.9 M run at inference. V2 is trained on about 3,900 input/reference pairs from UIEB and LSUI (see Train), so it learns the look of those references; V1 remains the better choice when no reference style should be imposed, or when only a CPU is available.

The optional --ai pass runs Real-ESRGAN after V2 to sharpen and upscale. It is generative and can add texture that was not captured.

Citation

@article{SALEH2025126314,
  title   = {Adaptive deep learning framework for robust unsupervised underwater image enhancement},
  author  = {Alzayat Saleh and Marcus Sheaves and Dean Jerry and Mostafa {Rahimi Azghadi}},
  journal = {Expert Systems with Applications},
  volume  = {268},
  pages   = {126314},
  year    = {2025},
  doi     = {10.1016/j.eswa.2024.126314}
}

License and acknowledgements

MIT license; intended for academic research. UDnet builds on PUIE-Net, USLN and PyTorch-Underwater-Image-Enhancement. The UIQM code is adapted from FUnIE-GAN. UDnet V2 uses NAFNet blocks, and the --ai pass uses the official Real-ESRGAN weights (BSD-3-Clause), downloaded from their releases.

About

Adaptive Uncertainty Distribution in Deep Learning for Unsupervised Underwater Image Enhancement

Topics

Resources

Stars

71 stars

Watchers

2 watching

Forks

Releases

Packages

Used by

Contributors

Languages

0