Urban livability evaluation based upon multimodal deep learning

Keywords: Urban livability, multimodal deep learning, satellite images, digital surface model, nighttime light remote sensing images, textual information
Abstract Type: Paper Abstract

Abstract

Urban livability has a large influence on the quality of life. Livable cities can enhance urban economic development, improve physical and mental health, foster well-being, and promote urban sustainability. Evaluating urban livability is important for policymakers to inform urban planning and development strategies aimed at creating more livable cities. Traditional methods relied on survey results, statistical data, and indicators extracted from geospatial data are time-consuming to collect and are irregularly updated. This study investigates using multi-modal remote sensing images and textual geospatial data for urban livability evaluation through deep learning. We use high spatial resolution remote sensing (RS) images, digital surface models (DSM), night light remote sensing (NLRS) images and point of interest (POI) data. To address challenges of aligning different modalities and leveraging both their intrinsic information and interrelationships to enhance evaluation results, we propose a transformer-based multi-task multimodal regression (TMTMR) model to simultaneously evaluate urban livability and its five domain scores by fusing features of different modalities. Our research encompasses 13 Dutch areas, demonstrating that the TMTMR model efficiently evaluates urban livability with correlation coefficients ranging from 0.605 to 0.779, and root mean square error values between 0.070 and 0.112 in four unseen test areas. Furthermore, we analyze the synergy between different modalities, concluding that urban livability can be effectively predicted by aligning RS images, NLRS images, DSM, and POI data, with a descending contribution. We conclude that the proposed TMTMR model is proven capable of effectively predicting urban livability.