Joint-Embedding Predictive Architecture on Point Clouds
- Python 3.10.4
- CUDA 11.6 or above
- cuDNN 8.4.0 or above
- GCC >= 6 and <= 11.2.1
./install_py.shSee DATASETS.md for download instructions.
python -m pointjepa.datasets.process.check # check if datasets are complete
python -m pointjepa fit -c configs/Point-JEPA/pretraining/shapenet.yaml| Type | Dataset | Evaluation | Config | Checkpoint |
|---|---|---|---|---|
| Point-JEPA pre-trained | ShapeNet | - | config | checkpoint |
| Point-JEPA SVM Linear | ShapeNet | 93.7±0.2 | - | checkpoint |
| Classification fine-tuned | ModelNet40 | 93.8±0.2 / 94.1±0.1 (OA / Voting) | config | checkpoint |
| Classification fine-tuned | ScanObjectNN | 86.6±0.3 (OA) | config | checkpoint |
| Part segmentation fine-tuned | ShapeNetPart | 85.8±0.1 (Cat. mIoU) | config | checkpoint |
Please note that weights that are attached above are the ones that yields the best results out of 10 independent runs (details mentioned in the paper).
The scripts in this section use Weights & Biases for logging, so it's important to log in once with wandb login before running them.
Checkpoints will be saved to the artifacts directory.
A note on reproducibility: While reproducing our results on most datasets is straightforward, achieving the same test accuracy on ModelNet40 is more complicated due to the high variance between runs (see also Pang-Yatian/Point-MAE#5 (comment), ma-xu/pointMLP-pytorch#1 (comment), CVMI-Lab/PAConv#9 (comment)). To obtain comparable results on ModelNet40, you will likely need to experiment with a few different seeds (This is the motivation behind multiple runs in our paper). However, if you would like to precisely replicate our test environment, you can try the following
We used a mixture of two environments (compute canada and the other), Narval server and another. Narval server was used for pretrainng and the other server was used specifically for downstream tasks.
- Python 3.10.2
- CUDA 11.4
- on A100SXM4
- Python 3.10.2
- CUDA 11.8
- on RTX 5500