Pytorch code for the paper: Centroid Based Concept Learning for RGB-D Inddor Scene Classification

This paper contributes a novel cognitively-inspired method for RGB-D indoor scene classification. High intra-class variance and low inter-class variance make indoor scene classification an extremely challenging task. To cope with this problem, we propose a clustering approach inspired by the concept learning model of the hippocampus and the neocortex, to generate clusters and centroids for different scene categories. Test images depicting different scenes are classified by using their distance to the closest centroids (concepts). Modeling of RGB-D scenes as centroids not only leads to state-of-the-art classification performance on benchmark datasets (SUN RGB-D and NYU Depth V2), but also offers a method for inspecting and interpreting the space of centroids. Inspection of the centroids generated by our approach on RGB-D datasets leads us to propose a method for merging conceptually similar categories, resulting in improved accuracy for all approaches.

• torch (Currently working with 1.3.1)
• Scipy (Currently working with 1.2.1)
• Scikit Learn (Currently working with 0.21.2)
• Get the models pretrained on Places365 dataset from https://github.com/CSAILVision/places365
• Download the SUNRGBD and NYU datasets in */data directory

• First run depth_training.py to train VGG16 from scratch on depth data. You can change the dataset from SUN to NYU in the file to train on NYU Depth V2 dataset.
• Run get_features.py twice to get the VGG16 features for all the RGB and depth images in the dataset. Change the model name in the file to get the correct features.
• Features are extracted using an updated version of the img2vec.py file from https://github.com/christiansafka/img2vec repo. Check that repo if you want more details.
• After feature extraction, simply run main_file.py to get the results.
• You can also get the silhouette analysis results from main.py. The helper functions are called from the Functions.py file.
• If you want to run the experiment with updated labels. Change category labels based on the merged categories provided in supplementary file of the paper.

@InProceedings{Ayub_2020_BMVC,  
author = {Ayub, Ali and Wagner, Alan R.},  
title = {Centroid Based Concept Learning for RGB-D Indoor Scene Classification},  
booktitle = {British Machine Vision Conference (BMVC)},  
year = {2020}  
}