Moving Beyond Computation: Reproducing Geographical Analyses of COVID-19 to Assess and Improve the Validity of Research

Keywords: reproducibility; R&R; COVID-19
Abstract Type: Paper Abstract

Abstract

Despite recent calls to make geographical analyses more reproducible, formal attempts to reproduce or replicate published work remain largely absent from the geographic literature. The reproductions of geographic research that do exist typically focus on computational reproducibility–whether results can be recreated using data and code provided by the authors–rather than on the validity and evidential value of the original analysis. However, knowing if a study is computationally reproducible is insufficient if the goal of a reproduction is to identify and correct errors in our knowledge. We argue that reproductions of geographic work should focus on assessing the validity of existing empirical studies. We present three model reproductions of geographical analyses of COVID-19, including bivariate Moran's I, global spatial regression, and multiscale geographically weighted regression analyses, that demonstrate how to achieve this goal. Each reproduction is based on a common, open access template and is published as an open access repository, complete with pre-analysis plan, data, code, and final report. Although we find each study to be partially reproducible, moving past computational reproducibility, our assessments reveal conceptual and methodological concerns that raise questions about the predictive value and the magnitude of the associations presented in each study. Collectively, these reproductions and our template materials offer a practical framework others can use to reproduce and replicate empirical spatial analyses and ultimately remove errors from the geographic knowledge base.