Map Understanding Model: Generating GeoSpatial Linked Data from Map Images

Keywords: geospatial linked data, visual document understanding, self-supervised learning
Abstract Type: Paper Abstract

Abstract

Extracting information from semi-structured document images (e.g., receipts) to create semantic-rich knowledge datasets has achieved excellent performance using self-supervised pretraining techniques. Existing work exploits image segments and text sequences from a document image in pretraining with various masking objectives to jointly understand the document layout and contents. However, these existing approaches do not work on spatial documents, such as historical maps. The main challenges of understanding maps are to capture the spatial relations among non-horizontal map text and their unique cartographic principles. Thus, we develop an end-to-end map understanding model (MUM) to generate structured geospatial linked data from text labels on historical map images. Our MUM's self-supervised pretraining objectives encourage the model to exploit the cartographic patterns of map text and their neighboring spatial context to infer their semantic types and corresponding geo-entities from external knowledge bases (KBs). MUM also employs spatial context embeddings to incorporate spatial relations among bounding polygons of map text. Specifically, map text with the same semantic types often shares similar cartographic patterns such as curvature, font size, and font spacing. For example, river names are often curved following the river feature, while county names are placed on a large area with a large capitalized font. Moreover, as we often can represent a place name with surrounding geo-entities, the neighboring spatial context (i.e., map text and their semantic types) provides an important cue to predict linked geo-entities from KBs. The generated geospatial linked data enriches extensive semantic information and broadens the scope of map search.