Do Foundation Models Learn Geospatial Properties?

Keywords: Foundation Model, AI, Geospatial Analytics, Remote Sensing
Abstract Type: Paper Abstract

Abstract

There is a significant growth in development and utilization of foundation models for remote sensing applications. These models are trained on large scale unlabeled data and commonly evaluated on downstream tasks using labeled datasets. While this approach provides a platform to assess the performance of the model for specific downstream tasks, there has been limited effort to quantify the characteristics of the foundation model after pre-training. Specifically, we are interested to examine if the model has learned the spectral, spatial and temporal properties of remote sensing data.
In this presentation, I will introduce GFM-Bench as a global benchmark for geospatial foundation models using multispectral remote sensing imagery. GFM-Bench contains separate datasets that allow the user to evaluate a foundation model’s properties in the embedding space, and demonstrate whether the model has learned spectral, spatial and temporal features. For spectral assessment, we generate a set of chips with homogeneous spatial patterns from all major land cover classes. For spatial assessment, the same data used for spectral assessment are utilized but the spatial patterns are replaced with heterogeneous features representative of their true distribution. Finally, for temporal assessment a set of chips containing time series imagery of pre- and post-event for disturbances such as wildfire and flood are curated.