Spatial Regularization for Analysis of Text and Epidemiological Data

Abstract

Use of spatial data has become an important aspect of data analysis. Use of location information can provide useful insight into the dataset. Advancement of sensor technologies and improved data connectivity have made it possible to the generation of large amounts of passively generated user location data. Apart from passively generated data from users, explicit effort has been made by commercial vendors to curate large amounts of location related data such as residential histories from a variety of sources such as credit records, litigation data, driving license records etc. Such spatial data, when linked with other datasets can provide useful insights. In this dissertation, we show that spatial information of data enables us to derive useful insights in domains of text analysis and epidemiology. We investigated primarily two types of data having spatial information - text data with location information and disease related data having residential address information. We show that in the case of text data, spatial information helps us find spatially informative topics. In the case of epidemiological data, we show residential information can be used to identify high risk spatial regions. There are instances where a primary analysis is not sufficient to establish a statistically robust conclusion. For instance, in domains such as epidemiology, where a finding is not considered to be relevant unless some statistical significance is established. We proposed techniques for significant tests which can be applied to text analysis, topic modelling, and disease mapping tasks in order to establish significance of the findings.