Just-in-Time Human Gesture Recognition Using WiFi Signals

MENG Xianjia; FENG Lin; CHEN Hao; CHEN Ting; MA Jianfeng; WANG Anwen; LIU Dongdong; ZHAO Yanfeng

doi:10.1049/cje.2021.07.022

Volume 30 Issue 6

Nov. 2021

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2021 > 30(6): 1111-1119

MENG Xianjia, FENG Lin, CHEN Hao, et al., “Just-in-Time Human Gesture Recognition Using WiFi Signals,” Chinese Journal of Electronics, vol. 30, no. 6, pp. 1111-1119, 2021, doi: 10.1049/cje.2021.07.022

Citation:

MENG Xianjia, FENG Lin, CHEN Hao, et al., “Just-in-Time Human Gesture Recognition Using WiFi Signals,” Chinese Journal of Electronics, vol. 30, no. 6, pp. 1111-1119, 2021, doi: 10.1049/cje.2021.07.022

Citation:

PDF( 7694 KB)

Just-in-Time Human Gesture Recognition Using WiFi Signals

doi: 10.1049/cje.2021.07.022

1. Shaanxi International Research Center for Passive Internet of Things, Northwest University, Xi'an 710127, China;
2. China University of Labor Pelations, Beijing 100048, China;
3. Chang'an University, Xi'an 710064, China;
4. Xidian University, Xi'an 710071, China;
5. Xi'an University of Finance and Economics, Xi'an 710100, China

Funds:

This work is supported by National Natural Science Foundation of China (No.61702416, No.61602382, No.61802310, No.61672428, No.61772422), the Key R&D Foundation of Shaanxi Province (No.2018SF-369), the ShaanXi Science and Technology Innovation Team Support Project (No.2018TD-O26), the China University of Labor Relations (No.20XYJS007), Shaanxi International Joint Research Centre for the Battery-free Internet of Things (No.2018SD0011), and Research on Target Perception, Recognition and Imaging Based on Low-Cost Commercial Equipment Wireless Signal (No.2019KWZ-05).

Received Date: 2019-12-25
Rev Recd Date: 2020-06-18

Available Online: 2021-09-23

Publish Date: 2021-11-05

Abstract

Abstract

In-air gesture recognition using wireless signals acts as a key enabler for various applications including smart homes, remote healthcare, shared autopilot, etc. Although researchers have conducted extensive research on WiFi-based gesture recognition, it remains an open question of providing accurate, robust, and in-time recognition system with the commodity WiFi infrastructure. We present FaSee, a just-in-time WiFibased gesture recognition system by identifying the fine-grained Channel state information (CSI) features upon off-the-shelf WiFi devices. The core of FaSee is essentially a novel hybrid recognition algorithm, which combines the classical K-Means algorithm with Dynamic time warping (DTW) together, to transform the feature matching in traditional gesture recognition schemes into a hierarchical manner, thereby significantly improving the recognition efficiency. Experimental results show that FaSee recognizes 9 representative gestures with an average accuracy of 94.75% without tedious per-person training, while achieving 30% signal processing delay saving when compared with the state-of-the-arts gesture recognition schemes.
- Gesture recognition,
- Hierarchical matching,
- Just-in-time

FullText(HTML)

References(34)

References

S. Tan and K. Yang, "Wifinger:Leveraging commodity wifi for fine-grained finger gesture recognition", MobiHoc, Paderborn, Germany, pp.201-210, 2016.

T.P. Smith, W.M. Bell, K. Ali, et al., "Recognizing keystrokes using wifi devices",AJ-SAC, Vol.35, No.5, pp.1175-1190, 2017.

Aljaafreh and Ahmad.Falah, "Collaborative classification of multiple ground vehicles in wireless sensor networks based on acoustic signals", Dissertations, Western Michigan University, USA, pp.5-75, 2011.

A.Virmani and M. Shahzad, "Position and orientation agnostic gesture recognition using wifi", MobiSys, Niagara Falls, NY, USA, pp.252-264, 2017.

C. Liu, D. Fang, Z. Yang, et al., "Rss distribution-based passive localization and its application in sensor networks", IEEE Transactions on Wireless Communications, Vol.15, No.4, pp.2883-2895, 2016.

Microsoft, "X-box kinect", https://www.xbox.com/zh-CN/xbox-one/accessories/kinect,2019-07-01.

Leap Motion Corporation, "Leap motion", https://www. leapmotion.com, 2018-12-01.

Q. Pu, S. Gupta, S. Gollakota, et al., "Whole-Home gesture recognition using wireless signals", ACM MobiCom, Miami, Florida, USA, pp.27-38, 2013.

L. Sun, S. Sen, D. Koutsonikolas, et al., "WiDraw:Enabling hands-free drawing in the air on commodity WiFi devices", ACM MobiCom, Paris, France, pp.77-89, 2015.

Y. Wang, J. Liu, Y. Chen, et al., "E-eyes:Device-free location-oriented activity identification using fine-grained WiFi signatures", ACM MobiCom, Maui, Hawaii, USA, pp.617-628, 2014.

W. Wang, A.X. Liu, M. Shahzad, et al., "Understanding and modeling of WiFi signal based human activity recognition", ACM MobiCom, Paris, France, pp.65-76, 2015.

H. Abdelnasser, M. Youssef and K.A. Harras. "WiGest:A ubiquitous WiFi-based gesture recognition system", 2015 IEEE Conference on Computer Communications (INFOCOM), Kowloon, Hong Kong, China, 2015.

M. Zhao, F. Adib and D. Katabi. "Emotion recognition using wireless signals", ACM MobiCom, New York City, New York, pp.95-108, 2016.

Y. Wang, S. Wang, Z. Tian, et al., "Two-Stream fusion neural network approach for hand gesture recognition based on FMCW radar", Acta Electronica Sinica, Vol.47, No.7, pp.1408-1415, 2019.

W. He, K. Wu, Y. Zou, et al., "WiG:WiFi-based gesture recognition system", ICCCN, Las Vegas, NV, USA, pp.1095-2055, 2015.

N. Yu, W. Wang, AX. Liu, et al., "QGesture:Quantifying gesture distance and direction with WiFi signals", Ubicomp, Vol.2, No.1, 2018.

J. Shang and J. Wu. "A robust sign language recognition system with sparsely labeled instances using Wi-Fi signals", 2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems, Orlando, FL, USA, pp.2155-6814, 2017.

M.A.A. Haseeb and R. Parasuraman. "Wisture:Touchless hand gesture classification in unmodified smartphones using Wi-Fi signals", IEEE Sensors Journal, Vol.19, No.1, pp.257-267, 2019.

L. Chen, X. Chen, L. Ni, et al., "Human behavior recognition using Wi-Fi CSI:Challenges and opportunities", IEEE Communications Magazine, Vol.55, No.10, pp.112-117, 2017.

X. Chen, L. Chen, C. Feng, et al., "Sensing our world using wireless signals", IEEE Internet Computing, Vol.23, No.3, pp.38-45, 2019.

T. Zhang, T. Song, D. Chen, et al., "WiGrus:A Wifi-Based gesture recognition system using software-defined radio", IEEE Access, Vol.7, pp.131102-131113, 2019.

A. Parate, M.C. Chiu, C. Chadowitz, et al., "RisQ:Recognizing smoking gestures with inertial sensors on a wristband", Mobisys, Bretton Woods, New Hampshire, USA, pp.149-161, 2014.

C. Zhang, J. He and W.D. Wang, "Visual recognition of chinese traffic police gestures based on spatial context and temporal features", Acta Electronica Sinica, Vol.48, No.5, pp.966-974, 2020. (in Chinese)

B. Kellogg, V. Talla and S. Gollakota, "Bringing gesture recognition to all devices", USENIX NSDI, University of Washington, USA, pp.303-314, 2014.

P. Kodeswaran, R. Kokku and M. Mallick, "Demultiplexing activities of daily living in IoT enabled smarthomes", IEEE INFOCOM, San Francisco, CA, USA, DOI:10.1109/INFOCOM.2016.7524359, 2016.

S. Agrawal, I. Constandache, S Gaonkar, et al., "Using mobile phones to write in air", Mobisys, Bethesda, Maryland, USA, pp.15-28, 2011.

T. Park, J. Lee, I. Hwang, et al., "E-gesture:A collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices", SenSys, Seattle, Washington, pp.260-273, 2011.

B. Chen, V. Yenamandra and K. Srinivasan, "Tracking keystrokes using wireless signals", Mobisys, Florence, Italy, pp.31-44, 2015.

J. Liu, Y. Wang, G. Kar, et al., "Snooping keystrokes with mm-level audio ranging on a single phone", MobiCom, Paris, France, pp.142-154, 2015.

F. Adib and D. Katabi, "See through walls with WiFi!", SIGCOMM, Hong Kong, China, pp.75-86, 2013.

L.B. Jiang, X.L. Zhou and L. Che, "Few-shot learning for human motion recognition based on carrier-free UWB radar", Acta Electronica Sinica, Vol.48, No.3, pp.602-615, 2020. (in Chinese)

Z. Du, J. Pan, S. Chu, et al., "Quasi-affine transformation evolutionary algorithm with communication schemes for application of RSSI in wireless sensor networks", IEEE Access, Vol.8, pp. 8583-8594, 2020.

J. Zhang, Z. Tang, M. Li, et al., "CrossSense:Towards crosssite and large-scaleWiFi sensing", MobiCom 2018, New Delhi, India, pp.305-320, 2018.

Y. Zheng, Y. Zhang, K. Qian, et al., "Zero-effort cross-domain gesture recognition with Wi-Fi", MobiSys'19, Seoul, Republic of Korea, pp.305-320, 2019.

Relative Articles

Supplements(0)

Cited By

Proportional views