A transformer model for learning spatiotemporal contextual representation in fMRI data
Citations
Altmetric:
Genre
Journal article
Date
2023-11-23
Advisor
Committee member
Group
Department
Computer and Information Sciences
Psychology and Neuroscience
Psychology and Neuroscience
Permanent link to this record
Collections
Research Projects
Organizational Units
Journal Issue
DOI
https://doi.org/10.1162/netn_a_00281
Abstract
Representation learning is a core component in data-driven modeling of various complex phenomena. Learning a contextually informative representation can especially benefit the analysis of fMRI data because of the complexities and dynamic dependencies present in such datasets. In this work, we propose a framework based on transformer models to learn an embedding of the fMRI data by taking the spatiotemporal contextual information in the data into account. This approach takes the multivariate BOLD time series of the regions of the brain as well as their functional connectivity network simultaneously as the input to create a set of meaningful features that can in turn be used in various downstream tasks such as classification, feature extraction, and statistical analysis. The proposed spatiotemporal framework uses the attention mechanism as well as the graph convolution neural network to jointly inject the contextual information regarding the dynamics in time series data and their connectivity into the representation. We demonstrate the benefits of this framework by applying it to two resting-state fMRI datasets, and provide further discussion on various aspects and advantages of it over a number of other commonly adopted architectures.
Description
Citation
Nima Asadi, Ingrid R. Olson, Zoran Obradovic; A transformer model for learning spatiotemporal contextual representation in fMRI data. Network Neuroscience 2023; 7 (1): 22–47. doi: https://doi.org/10.1162/netn_a_00281
Citation to related work
MIT Press
Has part
Network Neuroscience, Vol. 7, Iss. 1
ADA compliance
For Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu