Long Short-Term Memory Spiking Neural Networks for Classification of Snoring and Non-snoring Sound Events

ZHANG Rulin; LI Ruixue; LIANG Jiakai; YUE Keqiang; LI Wenjun; LI Yilin

doi:10.23919/cje.2022.00.210

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Corrected proof

Rulin ZHANG, Ruixue LI, Jiakai LIANG, et al., “Long Short-Term Memory Spiking Neural Networks for Classification of Snoring and Non-snoring Sound Events,” Chinese Journal of Electronics, vol. 33, no. 3, pp. 778–787, 2024 doi: 10.23919/cje.2022.00.210

Citation:

Rulin ZHANG, Ruixue LI, Jiakai LIANG, et al., “Long Short-Term Memory Spiking Neural Networks for Classification of Snoring and Non-snoring Sound Events,” Chinese Journal of Electronics, vol. 33, no. 3, pp. 778–787, 2024 doi: 10.23919/cje.2022.00.210

Citation:

PDF( 6812 KB)

Long Short-Term Memory Spiking Neural Networks for Classification of Snoring and Non-snoring Sound Events

doi: 10.23919/cje.2022.00.210

1.
Electronics and Information College, Hangzhou Dianzi University, Hangzhou 310018, China

More Information

Author Bio:
Rulin ZHANG received the B.E. degree from College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao, China, in 2019. She is currently pursuing the M.S. degree in Electronics and Information College, Hangzhou Dianzi University, Hangzhou, China. Her research interests include spike neural networks and voice recognition. (Email: rlzhang_0609@163.com)

Ruixue LI graduated from the School of Electrical and Information Engineering, Tianjin University and received the Ph.D. degree in 2020. She is now a Post-doctor in Electronics and Information College, Hangzhou Dianzi University, Hangzhou, China. Her current research interests include computational neuroscience, pattern recognition, audio analysis and image processing. (Email: rxli2012@163.com)

Jiakai LIANG received the B.E. degree in electronic information engineering from the Wenzhou University, Wenzhou, China, in 2021. He is currently pursuing the M.S. degree in Electronics and Information College, Hangzhou Dianzi University, Hangzhou, China. His research interests include spiking neural networks and image processing. (Email: ljk211040090@hdu.edu.cn)

Keqiang YUE received the Ph.D. degree from College of Electrical Engineering Zhejiang University, Hangzhou, China, in 2014. He is currently an M.S. Supervisor with Electronics and Information College, Hangzhou Dianzi University, Hangzhou, China. His research interest include internet of things system development and intelligent communication. (Email: kqyue@hdu.edu.cn)

Wenjun LI received the Ph. D. from Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghia, Chian, in 2004. He is currently a Professor of Hangzhou Dianzi University, Hangzhou, China. His research interest include integrated circuit design and intelligent computing and hardware. (Email: liwenjun@hdu.edu.cn)

Yilin LI received the Ph. D. from School of Marine Science and Technology, Northwestern Polytechnical University, Xi’an, China, in 2018. He is currently teaching in Electronics and Information College, Hangzhou Dianzi University, Hangzhou, China. His research interests include multidisciplinary design optimization, weak signal detection and AI model compression. (Email: ericlee@hdu.edu.cn)
Corresponding author: Email: liwenjun@hdu.edu.cn
Received Date: 2022-07-10
Accepted Date: 2023-06-21

Available Online: 2023-08-25

Abstract

Abstract

Snoring is a widespread occurrence that impacts human sleep quality. It is also one of the earliest symptoms of many sleep disorders. Snoring is accurately detected, making further screening and diagnosis of sleep problems easier. Snoring is frequently ignored because of its underrated and costly detection costs. As a result, this research offered an alternative method for snoring detection based on a long short-term memory based spiking neural network (LSTM-SNN) that is appropriate for large-scale home detection for snoring. We designed acquisition equipment to collect the sleep recordings of 54 subjects and constructed the sleep sound database in the home environment. And Mel frequency cepstral coefficients (MFCCs) were extracted from these sound signals and encoded into spike trains by a threshold encoding approach. They were classified automatically as non-snoring or snoring sounds by our LSTM-SNN model. We used the backpropagation algorithm based on an alternative gradient in the LSTM-SNN to complete the parameter update. The categorization percentage reached an impressive 93.4%, accompanied by a remarkable 36.9% reduction in computer power compared to the regular LSTM model.

FullText(HTML)

References(34)

References

[1]	W. C. Ni, M. G. Cao, and Y. L. Tang, “Effect of sleep apnea syndrome with nasal continuous positive airway pressure,” Chinese Journal of Ophthalmology and Otorhinolaryn, vol. 6, no. 4, pp. 214–215, 2009.
[2]	P. E. Brockmann, O. Bruni, L. Kheirandish-Gozal, et al., “Reduced sleep spindle activity in children with primary snoring,” Sleep Medicine, vol. 65, pp. 142–146, 2020. doi: 10.1016/j.sleep.2019.10.001
[3]	J. Wang, C. Janson, E. Lindberg, et al., “Dampness and mold at home and at work and onset of insomnia symptoms, snoring and excessive daytime sleepiness,” Environment International, vol. 139, article no. 105691, 2020. doi: 10.1016/j.envint.2020.105691
[4]	D. Pevernagie, R. M. Aarts, and M. De Meyer, “The acoustics of snoring,” Sleep Medicine Reviews, vol. 14, no. 2, pp. 131–144, 2010. doi: 10.1016/j.smrv.2009.06.002
[5]	B. Y. Chen and Q. Y. He, “Systemic damage in obstructive sleep apnea syndrome,” National Medical Journal of China, vol. 92, no. 18 , pp. 1225–1227, 2012. (in Chinese) doi: 10.3760/cma.j.issn.0376-2491.2012.18.001
[6]	X. X. Chen, L. Chen, Z. M. Deng, et al., “Diagnostic significance of dynamic electrocardiogram and synchronous respiration monitoring in snoring patients,” Journal of Guangdong Medical University, vol. 39, no. 3, pp. 274–277, 2021. (in Chinese) doi: 10.3969/j.issn.1005-4057.2021.03.007
[7]	O. Yildirim, U. B. Baloglu, and U. R. Acharya, “A deep learning model for automated sleep stages classification using PSG signals,” International Journal of Environmental Research and Public Health, vol. 16, no. 4, article no. 599, 2019. doi: 10.3390/ijerph16040599
[8]	B. Arsenali, J. van Dijk, O. Ouweltjes, et al., “Recurrent neural network for classification of snoring and non-snoring sound events,” in Proceedings of 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Honolulu, HI, USA, pp. 328–331, 2018.
[9]	M. Camacho, M. Robertson, J. Abdullatif, et al., “Smartphone apps for snoring,” The Journal of Laryngology & Otology, vol. 129, no. 10, pp. 974–979, 2015. doi: 10.1017/S0022215115001978
[10]	Y. Yang, Y. Qin, W. N. Huang, et al., “Acoustic characteristics of snoring sound in patients with obstructive sleep apnea hypopnea syndrome,” Journal of Clinical Otorhinolaryngology Head and Neck Surgery, vol. 26, no. 8, pp. 360–363, 2012. (in Chinese) doi: 10.13201/j.issn.1001-1781.2012.08.012
[11]	M. Dilkes and A. Adams, Stop Snoring The Easy Way: And the Real Reasons You Need To. The Orion Publishing Group Ltd, London, UK, pp. 11–16, 2017.
[12]	H. J. Xu, L. S. Yu, W. N. Huang, et al., “A preliminary study of acoustic characteristics of snoring sound in patients with obstructive sleep apnea/hypopnea syndrome (OSAHS) and with simple snoring,” Journal of Audiology and Speech Pathology, vol. 17, no. 3, pp. 235–238, 2009. (in Chinese) doi: 10.3969/j.issn.1006-7299.2009.03.010
[13]	K. Qian, C. Janott, V. Pandit, et al., “Classification of the excitation location of snore sounds in the upper airway by acoustic multifeature analysis,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 8, pp. 1731–1741, 2017. doi: 10.1109/TBME.2016.2619675
[14]	S. M. Cervera, D. Nikolić, A. Barney, et al., “A model of breathing abnormalities in sleep for development of classification and diagnosis techniques,” in Proceedings of 2010 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies, Rome, Italy, pp. 1–2, 2010.
[15]	Osborne J E, Osman E Z, Hill P D, et al., “A new acoustic method of differentiating palatal from non-palatal snoring,” Clinical Otolaryngology & Allied Sciences, vol. 24, no. 2, pp. 130–133, 1999. doi: 10.1046/j.1365-2273.1999.00229.x
[16]	S. J. Lim, S. J. Jang, J. Y. Lim, et al., “Classification of snoring sound based on a recurrent neural network,” Expert Systems with Applications, vol. 123, pp. 237–245, 2019. doi: 10.1016/j.eswa.2019.01.020
[17]	T. L. Nguyen and Y. Won, “Sleep snoring detection using multi-layer neural networks,” Bio-Medical Materials and Engineering, vol. 26, no. S1, pp. S1749–S1755, 2015. doi: 10.3233/BME-151475
[18]	Y. M. Jiang, J. X. Peng, and X. W. Zhang, “Automatic snoring sounds detection from sleep sounds based on deep learning,” Physical and Engineering Sciences in Medicine, vol. 43, no. 2, pp. 679–689, 2020. doi: 10.1007/s13246-020-00876-1
[19]	T. Khan, “A deep learning model for snoring detection and vibration notification using a smart wearable gadget,” Electronics, vol. 8, no. 9, article no. 987, 2019. doi: 10.3390/electronics8090987
[20]	J. L. Xie, X. Aubert, X. Long, et al., “Audio-based snore detection using deep neural networks,” Computer Methods and Programs in Biomedicine, vol. 200, article no. 105917, 2021. doi: 10.1016/j.cmpb.2020.105917
[21]	M. M. van Gilst, J. P. van Dijk, R. Krijn, et al., “Protocol of the SOMNIA project: An observational study to create a neurophysiological database for advanced clinical sleep monitoring,” BMJ Open, vol. 9, no. 11, article no. e030996, 2019. doi: 10.1136/bmjopen-2019-030996
[22]	J. P. Sun, X. Y. Hu, Y. Y. Zhao, et al., “SnoreNet: Detecting snore events from raw sound recordings,” in Proceedings of 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, pp. 4977–4981, 2019.
[23]	L. Zhang and W. G. Gong, “Improved wiener filtering speech enhancement algorithm,” Computer Engineering and Applications, vol. 46, no. 26, pp. 129–131, 2010. (in Chinese) doi: 10.3778/j.issn.1002-8331.2010.26.040
[24]	M. H. Wang, E. H. Zhang, and M. H. Wang, “Research and improvement on endpoint detection based on dual-threshold algorithm,” Computer & Digital Engineering, vol. 45, no. 11, pp. 2223–2228, 2017. (in Chinese) doi: 10.3969/j.issn.1672-9722.2017.11.030
[25]	A. Gupta and H. Gupta, “Applications of MFCC and vector quantization in speaker recognition,” in Proceedings of 2013 International Conference on Intelligent Systems and Signal Processing, Vallabh Vidyanagar, India, pp. 170–173, 2013.
[26]	S. Y. Cheng, C. Wang, K. Q. Yue, et al., “Automated sleep apnea detection in snoring signal using long short-term memory neural networks,” Biomedical Signal Processing and Control, vol. 71, article no. 103238, 2022. doi: 10.1016/j.bspc.2021.103238
[27]	Q. H. Wang, L. N. Wang, and S. Xu, “Research and application of spiking neural network model based on LSTM structure,” Application Research of Computers, vol. 38, no. 5, pp. 1381–1386, 2021. (in Chinese) doi: 10.19734/j.issn.1001-3695.2020.05.0123
[28]	A. Lotfi Rezaabad and S. Vishwanath, “Long short-term memory spiking networks and their applications,” in Proceedings of the International Conference on Neuromorphic Systems 2020, Oak Ridge, TN, USA, article no. 3, 2020.
[29]	A. Graves and A. Graves, “Long short-term memory,” in Supervised Sequence Labelling with Recurrent Neural Networks, A. Graves, Ed. Springer, Berlin, Heidelberg, Germany, pp. 37–45, 2012.
[30]	G. Van Houdt, C. Mosquera and G. Nápoles, “A review on the long short-term memory model,” Artificial Intelligence Review, vol. 53, no. 8, pp. 5929–5955, 2020. doi: 10.1007/s10462-020-09838-1
[31]	Y. J. Wu, L. Deng, G. Q. Li, et al., “Spatio-temporal backpropagation for training high-performance spiking neural networks,” Frontiers in Neuroscience, vol. 12, article no. 331, 2018. doi: 10.3389/fnins.2018.00331
[32]	E. O. Neftci, H. Mostafa, and F. Zenke, “Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks,” IEEE Signal Processing Magazine, vol. 36, no. 6, pp. 51–63, 2019. doi: 10.1109/MSP.2019.2931595
[33]	I. Sporea and A. Grüning, “Supervised learning in multilayer spiking neural networks,” Neural Computation, vol. 25, no. 2, pp. 473–509, 2013. doi: 10.1162/NECO_a_00396
[34]	Q. Yu, Y. L. Yao, L. B. Wang, et al., “A multi-spike approach for robust sound recognition,” in Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, pp. 890–894, 2019.