People
|
Timeline
- Fall 2015-present
Project Description
This project is a part of the Tustworthy Health & Wellness (THaW) project, which is an NSF-funded project that tackles many of the research challenges to provide trustworthy information systems for health and wellness.
When filling out privacy-related forms in public places such as hospitals or clinics, people usually are not aware that the sound of their handwriting leaks personal information. In this project, we explore the possibility of eavesdropping on handwriting via nearby mobile devices based on audio signal processing and machine learning. We show the usage of mobile devices to collect the sound of victims’ handwriting, and to extract handwriting-specific features for machine learning based analysis. We focus on the situation where the victim’s handwriting follows certain print style. An attacker can keep a mobile device, such as a common smartphone, touching the desk used by the victim to record the audio signals of handwriting. Then our system can provide a word-level estimate for the content of the handwriting. To reduce the impacts of various writing habits and writing locations, the system utilizes the methods of letter clustering and dictionary filtering. Our prototype system’s experimental results reveal the danger of privacy leakage through the sound of handwriting.
Publications
- Tuo Yu, Haiming Jin, Klara Nahrstedt, “Audio based Eavesdropping of Handwriting via Mobile Devices“, in the ACM 2016 Joint International Conference on Pervasive and Ubiquitous Computing (Ubicomp 2016), Heidelberg, Germany, September 2016.
- Tuo Yu, Haiming Jin, Klara Nahrstedt, “Audio based Handwriting Input for Tiny Mobile Devices,” in 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 130-135, 2018.
- Tuo Yu, Haiming Jin, Klara Nahrstedt, “Mobile Devices based Eavesdropping of Handwriting,” in IEEE Transactions on Mobile Computing, 2019.
Funding Agencies
This project is supported by the a collaborative award from the National Science Foundation (NSF award numbers CNS-1329686, 1329737,1330142 and 1330491).