Loading...
Passive sensing and active protection: privacy and security challenges in emerging virtual environments and mobile systems
Ye, Zhengkun
Ye, Zhengkun
Citations
Altmetric:
Genre
Thesis/Dissertation
Date
2025-12
Advisor
Committee member
Group
Department
Computer and Information Science
Permanent link to this record
Collections
Files
Loading...
Ye_temple_0225E_16275.pdf
Adobe PDF, 5.25 MB
Research Projects
Organizational Units
Journal Issue
DOI
Abstract
Emerging virtual environments and mobile systems are increasingly integrated into daily life, offering immersive and ubiquitous experiences. However, these systems also introduce novel privacy and security risks, particularly due to the often-unrestricted access to sensor data. This dissertation investigates the dual role of passive sensing—as both a vector for privacy leakage and a foundation for designing privacy-preserving interaction mechanisms.
We first reveal how passive motion sensing in virtual reality (VR) systems can result in severe biometric privacy breaches. To demonstrate this, we present BPSniff, the first practical system capable of inferring users’ blood pressure (BP) from subtle motion data captured by VR headset built-in motion sensors. BP is a critical health indicator protected under HIPAA, yet BPSniff shows that it can be estimated covertly without user consent. Leveraging the insight that blood-induced microvibrations in the skull propagate to the headset, BPSniff reconstructs fine-grained blood flow patterns using a variational autoencoder (VAE) and continuously estimates BP using an LSTM-based regression model. We evaluate BPSniff through extensive experiments and a longitudinal study of 8 weeks, involving 37 participants and two VR headset models. The results show that BPSniff can achieve low mean errors of 1.75 mmHg for systolic blood pressure (SBP) and 1.34 mmHg for diastolic blood pressure (DBP), which are comparable to commercial BP monitors and satisfy the standard (i.e., mean error 5.0 mmHg) specified by Food and Drug Administration (FDA)'s Association for the Advancement of Medical Instrumentation (AAMI) protocol.
To overcome limitations in real-world datasets, we extend BPSniff with an AI-augmented framework that uses a conditional generative adversarial network (cGAN) to synthesize realistic PPG and accelerometer signals conditioned on physiological and demographic factors. This approach expands the training dataset by 3× while preserving physiological integrity and correcting demographic imbalances. Domain-aligned retraining with quality-gated data integration yields up to 50\% reductions in mean absolute error and root mean squared error, along with improved correlation with ground-truth BP. These enhancements significantly improve model generalization, robustness to motion artifacts, and scalability in real-world VR health monitoring.
We then transition from exposing privacy risks to exploring structural privacy defenses for mobile devices. We propose a novel approach that leverages device enclosures (e.g., smartphone cases or VR headset shells) as functional surfaces for privacy-preserving interaction. By embedding miniature acoustic structures into these enclosures, the physical layer itself can regulate and localize signal propagation, providing secure input zones and mitigating unauthorized sensing.
As an initial demonstration of this concept, we develop CasePad, a smartphone-based prototype that enables fine-grained back-of-device finger activity recognition using passive acoustic sensing without requiring additional sensors or user calibration. By exploiting unique acoustic signatures created by mini-structures during finger interactions, CasePad supports secure mobile interactions in public spaces. A multi-task learning framework with multilayer perceptron (MLP)-based encoding and tailored long short-term memory (LSTM)-based decoders enables accurate gesture detection. Experiments across multiple phone models demonstrate CasePad’s high accuracy (98.76%) and precise tracking capabilities for direction, distance, and speed.
Collectively, this dissertation presents a comprehensive investigation of passive sensing as both a privacy threat and a design opportunity. Through practical systems like BPSniff and CasePad, and the integration of generative AI techniques, we offer foundational insights for building secure, privacy-aware mobile and immersive systems in the age of pervasive sensing.
Description
Citation
Citation to related work
Has part
ADA compliance
For Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu
Embedded videos
License
IN COPYRIGHT- This Rights Statement can be used for an Item that is in copyright. Using this statement implies that the organization making this Item available has determined that the Item is in copyright and either is the rights-holder, has obtained permission from the rights-holder(s) to make their Work(s) available, or makes the Item available under an exception or limitation to copyright (including Fair Use) that entitles it to make the Item available.