A Helmet Detection Algorithm Based on Transformers with Deformable Attention Module

CHEN Songle; SUN Hongbo; WU Yuxin; SHANG Lei; RUAN Xiukai

doi:10.23919/cje.2023.00.346

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Uncorrected proof

Songle CHEN, Hongbo SUN, Yuxin WU, et al., “A Helmet Detection Algorithm Based on Transformers with Deformable Attention Module,” Chinese Journal of Electronics, vol. x, no. x, pp. 1–13, xxxx doi: 10.23919/cje.2023.00.346

Citation:

Songle CHEN, Hongbo SUN, Yuxin WU, et al., “A Helmet Detection Algorithm Based on Transformers with Deformable Attention Module,” Chinese Journal of Electronics, vol. x, no. x, pp. 1–13, xxxx doi: 10.23919/cje.2023.00.346

Citation:

PDF( 9846 KB)

A Helmet Detection Algorithm Based on Transformers with Deformable Attention Module

doi: 10.23919/cje.2023.00.346

CHEN Songle^{1, 2
,},
SUN Hongbo^1
,,
WU Yuxin^1
,,
SHANG Lei^1
,,
RUAN Xiukai^{3
,
,}

1.
Engineering Research Center of Post Big Data Technology and Application of Jiangsu Province, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
2.
State Key Lab for Novel Software Technology, Nanjing University, Nanjing 210023, China
3.
Institute of Intelligent Locks, Wenzhou University, Wenzhou 325035, China

More Information

Author Bio:
Songle CHEN received the Ph.D. degree from the department of Computer Science and Technology of Nanjing University in 2015. Now he is an Associate Professor with the Nanjing University of Posts and Telecommunications. His research interests include computer vision, computer graphics, deep learning etc. (Email: chensongle@njupt.edu.cn)

Hongbo SUN is currently an M.S. candidate from Nanjing University of Posts and Telecommunications, under the supervision of associate professor Songle Chen. His research interests include deep learning and computer vision. (Email: 1222097618@njupt.edu.cn)

Yuxin WU is currently an M.S. candidate from Nanjing University of Posts and Telecommunications, under the supervision of associate professor Songle Chen. Her research interests include deep learning and computer vision. (Email: 1221097413@njupt.edu.cn)

Lei SHANG is currently an M.S. candidate from Nanjing University of Posts and Telecommunications, under the supervision of associate professor Songle Chen. His research interests include deep learning and computer vision. (Email: 1222097617@njupt.edu.cn)

Xiukai RUAN received the Ph.D. degree from Nanjing University of Posts and Telecommunications, China. Now he is a Professor of Wenzhou university, working as the director of the Nationallocal Lock Engineering Laboratory for Digitalized Electrical Design Technology. (Email: ruanxiukai@163.com)
Corresponding author: Email: ruanxiukai@163.com
Received Date: 2023-11-02
Accepted Date: 2024-03-15

Available Online: 2024-06-28

Abstract

Abstract

Wearing a helmet is one of the effective measures to protect workers’ safety. To address the challenges of severe occlusion, multi-scale, and small target issues in helmet detection, this paper proposes a helmet detection algorithm based on deformable attention Transformers. The main contributions of this paper are as follows. A compact end-to-end network architecture for safety helmet detection based on Transformers is proposed. It cancels the computationally intensive Transformer Encoder module in the existing detection transformer (DETR) and uses the Transformer Decoder module directly on the output of feature extraction for query decoding, which effectively improves the efficiency of helmet detection. A novel feature extraction network named DSwin Transformer is proposed. By sparse cross-window attention, it enhances the contextual awareness of multi-scale features extracted by Swin Transformer, and keeps high computational efficiency simultaneously. The proposed method generates the query reference points and query embeddings based on the joint prediction probabilities, and selects an appropriate number of decoding feature maps and sparse sampling points for query decoding, which further enhance the inference capability and processing speed. On the benchmark safety-helmet-wearing-dataset (SHWD), the proposed method achieves the average detection accuracy mAP@0.5 of 95.4% with 133.35G FLOPs and 20 FPS, the state-of-the-art method for safety helmet detection.
- Helmet,
- object detection,
- deformable attention,
- Transformer,
- decoder

FullText(HTML)

References(38)

References

[1]	S. Chi, C. H. Caldas, and D. Y. Kim, “A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques,” Computer-Aided Civil and Infrastructure Engineering, vol. 24, no. 3, pp. 199–211, 2009. doi: 10.1111/j.1467-8667.2008.00580.x
[2]	S. Dong, Q. H. He, H. Li, et al., “Automated PPE misuse identification and assessment for safety performance enhancement,” in Proceedings of 2015 International Conference on Construction and Real Estate Management, Lulea, Sweden, 2015.
[3]	S. H. Kim, C. Wang, S. D. Min, et al, “Safety helmet wearing management system for construction workers using three-axis accelerometer sensor,” Applied Sciences, vol. 8, no. 12, article no. 2400, 2018. doi: 10.3390/app8122400
[4]	A. Kelm, L. Laußat, A. Meins-Becker, et al, “Mobile passive radio frequency identification (RFID) portal for automated and rapid control of personal protective equipment (PPE) on construction sites,” Automation in Construction, vol. 36, pp. 38–52, 2013. doi: 10.1016/j.autcon.2013.08.009
[5]	B. F. Spencer Jr, V. Hoskere, and Y. Narazaki, “Advances in computer vision-based civil infrastructure inspection and monitoring,” Engineering, vol. 5, no. 2, pp. 199–222, 2019. doi: 10.1016/j.eng.2018.11.030
[6]	R. Waranusast, N. Bundon, V. Timtong, et al., “Machine vision techniques for motorcycle safety helmet detection,” in Proceedings of 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013), Wellington, New Zealand, pp. 35–40, 2013.
[7]	K. Li, X. Zhao, J. Bian, et al., “Automatic safety helmet wearing detection,” in Proceedings of the 2017 IEEE 7th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 617–622, 2017.
[8]	R. R. V. e Silva, K. R. T. Aires, and R. de Melo Souza Veras, “Helmet detection on motorcyclists using image descriptors and classifiers,” in Proceedings of the 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images, Rio de Janeiro, Brazil, pp. 141–148, 2014.
[9]	A. H. M. Rubaiyat, T. T. Toma, M. Kalantari-Khandani, et al., “Automatic detection of helmet uses for construction safety,” in Proceedings of 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops (WIW), Omaha, NE, USA, pp. 135–142, 2016.
[10]	T. Cover and P. Hart, “Nearest neighbor pattern classification,” IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 21–27, 1967. doi: 10.1109/TIT.1967.1053964
[11]	M. A. Hearst, S. T. Dumais, E. Osuna, et al, “Support vector machines,” IEEE Intelligent Systems and Their Applications, vol. 13, no. 4, pp. 18–28, 1998. doi: 10.1109/5254.708428
[12]	Q. Fang, H. Li, X. C. Luo, et al, “Detecting non-hardhat-use by a deep learning method from far-field surveillance videos,” Automation in Construction, vol. 85, pp. 1–9, 2018. doi: 10.1016/j.autcon.2017.09.018
[13]	S. Q. Ren, K. M. He, R. Girshick, et al, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, 2017. doi: 10.1109/tpami.2016.2577031
[14]	J. Shen, X. Xiong, Y. Li, et al, “Detecting safety helmet wearing on construction sites with bounding-box regression and deep transfer learning,” Computer-Aided Civil and Infrastructure Engineering, vol. 36, no. 2, pp. 180–196, 2021. doi: 10.1111/mice.12579
[15]	J. Redmon, S. Divvala, R. Girshick, et al., “You only look once: Unified, real-time object detection,” in Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 779–788, 2016.
[16]	W. Liu, D. Anguelov, D. Erhan, et al., “SSD: Single shot MultiBox detector,” in Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 21–37, 2016.
[17]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 6000–6010, 2017.
[18]	N. Carion, F. Massa, G. Synnaeve, et al., “End-to-end object detection with transformers,” in Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, pp. 213–229, 2020.
[19]	X. Z. Zhu, W. J. Su, L. W. Lu, et al., “Deformable DETR: Deformable transformers for end-to-end object detection,” in Proceedings of the 9th International Conference on Learning Representations, virtual event, Austria, 2021.
[20]	“Safety-helmet-wearing-dataset,” Available at: https://github.com/njvisionpower/Safety-Helmet-Wearing-Dataset.
[21]	M. Jin, J. Zhang, X. W. Chen, et al., “Safety helmet detection algorithm based on color and hog features,” in Proceedings of the 2020 IEEE 19th International Conference on Cognitive Informatics & Cognitive Computing (ICCICC)*, Beijing, China, pp. 215–219, 2020.
[22]	S. Q. Yue, Q. Zhang, D. Q. Shao, et al, “Safety helmet wearing status detection based on improved boosted random ferns,” Multimedia Tools and Applications, vol. 81, no. 12, pp. 16783–16796, 2022. doi: 10.1007/s11042-022-12014-y
[23]	R. Girshick, J. Donahue, T. Darrell, et al, “Region-based convolutional networks for accurate object detection and segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 1, pp. 142–158, 2016. doi: 10.1109/TPAMI.2015.2437384
[24]	J. E. Espinosa-Oviedo, S. A. Velastín, and J. W. Branch-Bedoya, “EspiNet V2: A region based deep learning model for detecting motorcycles in urban scenarios,” Dyna, vol. 86, no. 211, pp. 317–326, 2019. doi: 10.15446/dyna.v86n211.81639
[25]	F. T. Wang, L. G. Wang, J. Tang, et al., “Progressive fusion network for safety protection detection,” in Proceedings of the 11th International Conference on Image and Graphics, Haikou, China, pp. 293–304, 2021.
[26]	Z. H. Sun G and Li Chao, “A safety helmet wearing detection method incorporating self-attention mechanism,” Journal of Computer Engineering Applications, in press.
[27]	S. Hao, J. J. He, L. Luo, et al., “Helmet wearing detection algorithm based on improved YOLOX,” in Proceedings of the 2022 5th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), Chengdu, China, pp. 316–321, 2022.
[28]	C. H. Dong, C. X. Pang, Z. W. Li, et al, “PG-YOLO: A novel lightweight object detection method for edge devices in industrial internet of things,” IEEE Access, vol. 10, pp. 123736–123745, 2022. doi: 10.1109/ACCESS.2022.3223997
[29]	Q. Q. Ge, Z. J. Zhang, L. Yuan, et al, “Safety helmet wearing detection method of fusing environmental features and improved YOLOv4,” Journal of Image and Graphics, vol. 26, no. 12, pp. 2904–2917, 2021. doi: 10.11834/jig.200606
[30]	Z. Liu, Y. T. Lin, Y. Cao, et al., “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of 2021 IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, pp. 9992–10002, 2021.
[31]	A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proceedings of the 9th International Conference on Learning Representations, virtual event, Austria, 2021.
[32]	J. F. Dai, H. Z. Qi, Y. W. Xiong, et al., “Deformable convolutional networks,” in Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, Italy, pp. 764–773, 2017.
[33]	L. Liu, W. L. Ouyang, X. G. Wang, et al, “Deep learning for generic object detection: A survey,” International Journal of Computer Vision, vol. 128, no. 2, pp. 261–318, 2020. doi: 10.1007/s11263-019-01247-4
[34]	D. Jia, Y. H. Yuan, H. D. He, et al., “DETRs with hybrid matching,” in Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, pp. 19702–19712, 2023.
[35]	A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “YOLOv4: Optimal speed and accuracy of object detection,” arXiv preprint, arXiv: 2004.10934, 2020.
[36]	“Yolov5,”Available at: https://github.com/ultralytics/yolov5. (查阅网上资料, 未找到本条文献作者、引用日期信息, 请补充)
[37]	C. Y. Wang, A. Bochkovskiy, and H. Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, pp. 7464–7475.
[38]	“YOLOv8,” Available at: https://github.com/ultralytics/ultralytics. (查阅网上资料, 未找到本条文献作者、引用日期信息, 请补充)