Loading...
COMBINING CONVOLUTIONAL NEURAL NETWORKS AND GRAPH NEURAL NETWORKS FOR IMAGE CLASSIFICATION
Trivedy, Vivek
Trivedy, Vivek
Citations
Altmetric:
Genre
Thesis/Dissertation
Date
2021
Advisor
Committee member
Group
Department
Computer and Information Science
Permanent link to this record
Collections
Research Projects
Organizational Units
Journal Issue
DOI
http://dx.doi.org/10.34944/dspace/6430
Abstract
Convolutional Neural Networks (CNNs) have dominated the task of imageclassification since 2012. Some key components of their success are that the
underlying architecture integrates a set inductive biases such as translational
invariance and the training computation can be significantly reduced by employing
weight sharing. CNNs are powerful tools for generating new representations of
images tailored to a particular task such as classification. However, because each
image is passed through the network independent of other images, CNNs are not
able to effectively aggregate information between examples. In this thesis, we
explore the idea of using Graph Neural Networks (GNNs) in conjunction with CNNs
to produce an architecture that has both the representational capacity of a CNN
and the ability to aggregate information between examples.
Graph Neural Networks apply the concept of convolutions directly on graphs. A
result of this is that GNNs are able to learn from the connections between nodes.
However, when working with image datasets, there is no obvious choice on how to
construct a graph. There are certain heuristics such as ensuring homophily that have
empirically been shown to increase the performance of GNNs. In this thesis, we apply
different schemes of constructing a graph from image data for the downstream task of
image classification and experiment with settings such as using multiple feature spaces
and enforcing a bipartite graph structure. We also propose a model that allows for
end to end training using CNNs and GNNs with proxies and attention that improves
classification accuracy in comparison to a regular CNN.
Description
Citation
Citation to related work
Has part
ADA compliance
For Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu