Deep learning on 3D point cloud: an exploratory experiment in incorporating spatial-related features

Keywords: Deep learning, deep neural network, point cloud, LiDAR, spatial statistics, semantic segmentation
Abstract Type: Paper Abstract

Abstract

Acquisition of the 3D point cloud data for indoor and outdoor scenes becomes increasingly efficient and productive with the development of LiDAR and drone technologies, spurring the progress of 3D GIS applications (e.g., smart city) by providing essential 3D datasets. Deep learning, representing a cutting-edge machine learning method, may also contribute to the applications of 3D GIS in understanding point process in 3D context. Deep neural networks taking point cloud as input have been becoming prevalent since 2017 due to its state-of-the-art performance in learning and predicting 3D shapes.
There are opportunities for applying GIScience knowledge in the evolution of the 3D deep learning due to its expertise in spatial domain. The input of the point-based deep neural networks are the spatial coordinates, x, y, and z of each point. Furthermore, the color channels (RGB) are sometimes used as additional feature channels to enhance the input information so that to increase the performance of the model. Inspired by this, we will explore whether incorporating spatial-related feature channels (e.g., spatial autocorrelation and spatial point patter) into input can further boost the performance of the 3D deep neural networks. Points are not independently existing in the space. Spatial-related features may further help to represent the relationship between one single point and its local neighbors, and global point cloud. Therefore, we would conduct exploratory experiments to find whether spatial-related features used as additional feature channels can boost the predicting performance of 3D point-based deep neural networks on indoor and outdoor benchmark datasets.