up all repo

albertpumarola · Feb 15, 2021 · faa9329 · faa9329
1 parent 4a09283
commit faa9329
Show file tree

Hide file tree

Showing 33 changed files with 4,518 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -0,0 +1,76 @@
+<img src='https://www.albertpumarola.com/images/2021/D-NeRF/dnerf.gif' align="right" width=90>
+
+# D-NeRF
+### [[Project]](https://www.albertpumarola.com/research/D-NeRF/index.html)[ [Paper]](https://arxiv.org/abs/2011.13961) 
+
+[D-NeRF](https://www.albertpumarola.com/research/D-NeRF/index.html) is a method for synthesizing novel views, at an arbitrary point in time, of dynamic scenes with complex non-rigid geometries. We optimize an underlying deformable volumetric function from a sparse set of input monocular views without the need of ground-truth geometry nor multi-view images.
+
+This project is an extension of [NeRF](http://www.matthewtancik.com/nerf) enabling it to model dynmaic scenes. The code heavily relays on [NeRF-pytorch](https://github.com/yenchenlin/nerf-pytorch). 
+
+![D-NeRF](https://www.albertpumarola.com/images/2021/D-NeRF/model.png)
+
+## Installation
+```
+git clone https://github.com/albertpumarola/D-NeRF.git
+cd D-NeRF
+conda create -n dnerf python=3.6
+conda activate dnerf
+pip install -r requirements.txt
+cd torchsearchsorted
+pip install .
+cd ..
+```
+
+## Download Pre-trained Weights
+ You can download the pre-trained models from [drive](https://drive.google.com/file/d/1VN-_DkRLL1khDVScQJEaohpbA2gC2I2K/view?usp=sharing) or [dropbox](https://www.dropbox.com/s/25sveotbx2x7wap/logs.zip?dl=0). Place the downloaded directory in `./logs` in order to test it later. See the following directory structure for an example:
+```
+├── logs 
+│   ├── mutant
+│   ├── standup 
+│   ├── ...
+```
+
+## Download Dataset
+ You can download the datasets from [drive](https://drive.google.com/file/d/19Na95wk0uikquivC7uKWVqllmTx-mBHt/view?usp=sharing) or [dropbox](https://www.dropbox.com/s/0bf6fl0ye2vz3vr/data.zip?dl=0). Place the downloaded directory in `./data` in order to train. See the following directory structure for an example:
+```
+├── data 
+│   ├── mutant
+│   ├── standup 
+│   ├── ...
+```
+
+## Demo
+We provide simple jupyter notebooks to explore the model. To use them first download the pre-trained weights and dataset.
+
+| Description      | Jupyter Notebook |
+| ----------- | ----------- |
+| Synthesize novel views at an arbitrary point in time. | render.ipynb|
+| Reconstruct mesh at an arbitrary point in time. | reconstruct.ipynb|
+| Quantitatively evaluate trained model. | metrics.ipynb|
+
+## Test
+First download pre-trained weights and dataset. Then, 
+```
+python run_dnerf.py --config configs/mutant.txt --render_only --render_test
+```
+To quantitatively evaluate model run `metrics.ipynb` notebook
+
+## Train
+First download the dataset. Then,
+```
+conda activate dnerf
+export PYTHONPATH='path/to/D-NeRF'
+export CUDA_VISIBLE_DEVICES=0
+python run_dnerf.py --config configs/mutant.txt
+```
+
+## Citation
+If you use this code or ideas from the paper for your research, please cite our paper:
+```
+@article{pumarola2020d,
+  title={D-NeRF: Neural Radiance Fields for Dynamic Scenes},
+  author={Pumarola, Albert and Corona, Enric and Pons-Moll, Gerard and Moreno-Noguer, Francesc},
+  journal={arXiv preprint arXiv:2011.13961},
+  year={2020}
+}
+```
diff --git a/configs/bouncingballs.txt b/configs/bouncingballs.txt
@@ -0,0 +1,26 @@
+expname = bouncingballs
+basedir = ./logs
+datadir = ./data/bouncingballs
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 10000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
diff --git a/configs/hellwarrior.txt b/configs/hellwarrior.txt
@@ -0,0 +1,27 @@
+expname = hellwarrior
+basedir = ./logs
+datadir = ./data/hellwarrior
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 100000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
+
diff --git a/configs/hook.txt b/configs/hook.txt
@@ -0,0 +1,27 @@
+expname = hook
+basedir = ./logs
+datadir = ./data/hook
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 100000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
+
diff --git a/configs/jumpingjacks.txt b/configs/jumpingjacks.txt
@@ -0,0 +1,27 @@
+expname = jumpingjacks
+basedir = ./logs
+datadir = ./data/jumpingjacks
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 100000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
+
diff --git a/configs/lego.txt b/configs/lego.txt
@@ -0,0 +1,26 @@
+expname = lego
+basedir = ./logs
+datadir = ./data/lego
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 10000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
diff --git a/configs/mutant.txt b/configs/mutant.txt
@@ -0,0 +1,27 @@
+expname = mutant
+basedir = ./logs
+datadir = ./data/mutant
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 100000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
+
diff --git a/configs/standup.txt b/configs/standup.txt
@@ -0,0 +1,27 @@
+expname = standup
+basedir = ./logs
+datadir = ./data/standup
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 100000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
+
diff --git a/configs/trex.txt b/configs/trex.txt
@@ -0,0 +1,27 @@
+expname = trex
+basedir = ./logs
+datadir = ./data/trex
+dataset_type = blender
+
+nerf_type = direct_temporal
+no_batching = True
+not_zero_canonical = False
+use_viewdirs = True
+
+use_viewdirs = True
+white_bkgd = True
+lrate_decay = 500
+
+N_iter = 800000
+N_samples = 64
+N_importance = 128
+N_rand = 500
+testskip = 1
+
+precrop_iters = 500
+precrop_iters_time = 10000
+precrop_frac = 0.5
+
+half_res = True
+do_half_precision = False
+