Loading...
Citations
Altmetric:
Genre
Thesis/Dissertation
Date
2022
Advisor
Committee member
Group
Department
Computer and Information Science
Subject
Permanent link to this record
Collections
Research Projects
Organizational Units
Journal Issue
DOI
http://dx.doi.org/10.34944/dspace/7723
Abstract
Personal healthcare has become an emerging research trend in the past few years due to the shortage of public healthcare resources. Smart mobile devices have the potential to enhance healthcare services for both patients and healthcare providers. Human-computer interaction (HCI) is key to realizing a useful and usable connection between the users and these smart healthcare technologies. Meanwhile, it is important to protect sensitive and private personal healthcare data on mobile devices. Moreover, deep learning (DL) technologies bring unique opportunities and challenges to create more robust and smarter models directly on resource-constrained mobile devices that are tailored specifically for personal healthcare applications.Recently, advanced sensing and DL technologies have created many opportunities to enable HCI on mobile devices and facilitate personal healthcare. We extensively explore a broad range of sensing modalities on the state-of-the-art wrist-worn wearable device to enable HCI. We propose a PPG-based gesture recognition system to recognize finger-level gestures using commodity wearables. Unlike existing hand gestures recognition methods, this novel approach enables robust fine-grained human computer interactions (e.g., sign-language interpretation and virtual reality) using low-cost sensors in commercial wearable devices. We further explore the feasibility of using both PPG and motion sensors in wearables to improve the sign language gesture recognition accuracy when there are limited body movements. We develop a gradient boost tree (GBT) model and a deep neural network-based model (i.e., ResNet) for classification. The transfer learning technique is applied to the ResNet-based model to reduce the training effort. We develop a prototype using low-cost PPG and motions sensors and conduct extensive experiments in the static and body-motion scenarios.
Results demonstrate that our system can differentiate nine finger-level gestures from the American Sign Language with an average recognition accuracy of over 98%.
As most health-related information is private, developing practical and effective security mechanisms for protecting personal information and inference models in mobile devices is also highly demanded. We investigate the existing traditional user authentication approaches on mobile devices and their vulnerabilities. To solve them, we develop a continuous user authentication system that exploits users’ pulsatile signals from commercial wearable devices’ PPG sensors. This system does not require active user participation and is feasible in practical scenarios with non-clinical PPG measurements and human motion artifacts. We explore the uniqueness of the human cardiac system and develop adaptive MA filtering methods to mitigate the impacts of transient and continuous activities from daily life. Furthermore, we identify general fiducial features and develop an adaptive classifier that can authenticate users continuously based on their cardiac characteristics with little additional training effort. Experiments with our wrist-worn PPG sensing platform under practical scenarios demonstrate that our system can achieve a high CA accuracy of over 90% and a low false detection rate of 4% in detecting random attacks. We show that our MA mitigation approaches can improve the CA accuracy by around 39% under both transient and continuous daily activity scenarios. Toward this end, we also propose a continuous user verification system, which re-uses the widely deployed WiFi infrastructure to capture the unique physiological characteristics rooted in user’s respiratory motions. A deep learning based user verification scheme is developed to identify legitimate users accurately and detect the existence of spoofing attacks. Extensive experiments involving 20 participants demonstrate that the proposed system can robustly verify/identify users and detect spoofers under various types of attacks.
With the fast advances of DL algorithms and sustained emergence of powerful mobile hardware, bringing deep-learning-empowered smart personal healthcare systems directly on the resource-constrained mobile device attracts a lot of attention. We conduct a comprehensive survey of deep learning optimization techniques on mobile devices. Particularly, we propose a DL model optimization pipeline to bring DL-empowered smart systems to mobile devices. We also point out the current trend and potential direction of research on DL optimization for various mobile applications and resource-constrained mobile devices. In addition, we propose a neuron bonds pruning to effectively reduce the convolution computations and size of weights in CNNs, which could significantly reduce the pruning iterations by only performing a one-shot pruning when the total number of unimportant parameters reaches the compress ratio. It also reduces the computational cost of powerful DNNs so that they can run on low-end mobile devices.
Description
Citation
Citation to related work
Has part
ADA compliance
For Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu