A Model for Chinese Named Entity Recognition Based on Global Pointer and Adversarial Learning

ZHANG Yangsen; LI Jianlong; XIN Yonghui; ZHAO Xiquan; LIU Yang

doi:10.23919/cje.2022.00.279

Volume 32 Issue 4

Jul. 2023

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2023 > 32(4): 854-867

ZHANG Yangsen, LI Jianlong, XIN Yonghui, et al., “A Model for Chinese Named Entity Recognition Based on Global Pointer and Adversarial Learning,” Chinese Journal of Electronics, vol. 32, no. 4, pp. 854-867, 2023, doi: 10.23919/cje.2022.00.279

Citation:

ZHANG Yangsen, LI Jianlong, XIN Yonghui, et al., “A Model for Chinese Named Entity Recognition Based on Global Pointer and Adversarial Learning,” Chinese Journal of Electronics, vol. 32, no. 4, pp. 854-867, 2023, doi: 10.23919/cje.2022.00.279

Citation:

PDF( 5357 KB)

A Model for Chinese Named Entity Recognition Based on Global Pointer and Adversarial Learning

doi: 10.23919/cje.2022.00.279

1.
Institute of Intelligent Information Processing, Beijing Information Science and Technology University, Beijing 100192, China
2.
Computer Network Emergency Response Technical Team, Coordination Center of China, Beijing 100029, China

Funds: This work was supported by the National Natural Science Foundation of China (61772081)

More Information

Author Bio:
Yangsen ZHANG was born in Shanxi, China, in 1962, doctor, Doctoral Supervisor. He graduated from the Department of Mathematics, Nankai University in 1983. He received special government allowances from the State Council. His research interests include intelligent information processing and natural language processing. (Email: zhangyangsen@163.com)

Jianlong LI was born in Jiangxi, China, in 1997, postgraduate. His research interests include deep learning and natural language processing. (Email: 1436631592@qq.com)

Yonghui XIN was born in 1990, doctor, graduated from University of Chinese Academy of Sciences in 2018, majoring in signal and information processing. His research interests include information security and machine learning. (Email: xinyh@cert.org.cn)

Xiquan ZHAO received the Ph.D. degree in computer science from University of Chinse Academy of Sciences, China, in 2016. He is a Postdoctoral Researcher at the Beijing Information Science and Technology University. His research interests include parallel computing, deep learning, and natural language processing. (Email: zhaoxiquan@bistu.edu.cn)

Yang LIU was born in 1983, postgraduate, graduated from Beijing University of Posts and Telecommunications in 2009. Her research interests include information security and data processing. (Email: liuyang195753@sina.com)
Received Date: 2022-08-19
Accepted Date: 2023-01-13

Available Online: 2023-03-22

Publish Date: 2023-07-05

Abstract

Abstract

To solve the problem that the Chinese named entity recognition (NER) models have poor anti-interference ability and inaccurate entity boundary recognition, this paper proposes the RGP-with-FGM model which is based on global pointer and adversarial learning. Firstly, the RoBERTa-WWM model is used to optimize the semantic representation of the text, and fast gradient method is used to add perturbation to the word embedding layer to enhance the robustness of the model. Then, BiGRU is used to focus on the timing information of Chinese characters to enhance the semantic connection. Finally, the global pointer is constructed in the decoding layer to obtain more accurate entity boundary recognition results. In order to verify the effectiveness of the model proposed in this paper, we construct Uyghur names dataset (UHND) to train the Chinese NER model, and perform extensive experiments with public Chinese NER data sets. Experimental results show that for UHND, the F1 value is 95.12%, which is 3.09% higher than that of the RoBERTa-WWM-BiGRU-CRF model. For the Resume data set, the Precision and F1 value are 96.28% and 96.10% respectively.
- Chinese named entity recognition (NER),
- Global pointer,
- RoBERTa-WWM model,
- Fast gradient method (FGM),
- BiGRU

FullText(HTML)

References(45)

References

[1]	Y. Y. Deng, C. X. Wu, Y. F. Wei, et al., “A survey on named entity recognition based on deep learning,” Journal of Chinese Information Processing, vol.35, no.9, pp.30–45, 2021. (in Chinese) doi: 10.3969/j.issn.1003-0077.2021.09.003
[2]	S. Z. Yang, Y. X. Liu, K. W. Zhang, et al., “Survey on distantly supervised relation extraction,” Chinese Journal of Computers, vol.44, no.8, pp.1636–1660, 2021. (in Chinese)
[3]	J. X. Zhang, X. S. Zhang, C. X. Wu, et al., “Survey of knowledge graph construction techniques,” Computer Engineering, vol.48, no.3, pp.23–37, 2022. (in Chinese) doi: 10.19678/j.issn.1000-3428.0061803
[4]	Z. Y. Yue, X. Ye, and R. H. Liu, “A review of pretrained technology based on language model,” Journal of Chinese Information Processing, vol.35, no.9, pp.15–29, 2021. (in Chinese) doi: 10.3969/j.issn.1003-0077.2021.09.002
[5]	H. K. Yu, H. P. Zhang, Q. Liu, et al., “Chinese named entity identification using cascaded hidden Markov model,” Journal on Communications, vol.27, no.2, pp.87–94, 2006. (in Chinese) doi: 10.3321/j.issn:1000-436X.2006.02.013
[6]	H. Duan and Y. Zhang, “A study on features of the CRFS-based Chinese named entity recognition,” International Journal of Advanced Intelligence Paradigms, vol.3, no.2, pp.287–294, 2011.
[7]	Z. H. Huang, W. Xu, and K. Yu, Bidirectional LSTM-CRF models for sequence tagging, arXiv preprint, arXiv: 1508.01991, 2015.
[8]	X. Z. Ma and E. Hovy, “End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, pp.1064–1074, 2016.
[9]	Y. Zhang and J. Yang, “Chinese NER using Lattice LSTM,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia, pp.1554–1564, 2018.
[10]	Y. Zhou, X. Q. Zheng, and X. J. Huang, Chinese named entity recognition augmented with lexicon memory, arXiv preprint, arXiv: 1912.08282, 2019.
[11]	N. Tashpolat, K. Wang, H. Askar, et al., “Combination of statistical and rule-based approaches for Uyghur person name recognition,” Acta Automatica Sinica, vol.43, no.4, pp.653–664, 2017. (in Chinese) doi: 10.16383/j.aas.2017.c150769
[12]	A. Mahmoud, H. Yusuf, J. J. Zhang, et al., “Name recognition in the Uyghur language based on fuzzy matching and syllable-character conversion,” Journal of Tsinghua University (Science and Technology), vol.57, no.2, pp.188–196, 2017. (in Chinese) doi: 10.16511/j.cnki.qhdxxb.2017.22.013
[13]	J. C. Guo, G. Wan, X. J. Hu, et al., “Chinese resume named entity recognition based on BERT,” Journal of Computer Applications, vol.41, no.S1, pp.15–19, 2021. (in Chinese) doi: 10.11772/j.issn.1001-9081.2020091404
[14]	L. Xu and J. N. Li, “Biomedical named entity recognition based on BERT and BiLSTM-CRF,” Computer Engineering and Science, vol.43, no.10, pp.1873–1879, 2021. (in Chinese) doi: 10.3969/j.issn.1007-130X.2021.10.022
[15]	J. H. Li, M. M. Chen, H. J. Wang, et al., “Chinese named entity recognition method based on ALBERT-BGRU-CRF,” Computer Engineering, vol.48, no.6, pp.89–94,106, 2022. (in Chinese) doi: 10.19678/j.issn.1000-3428.0061630
[16]	W. H. Wang, H. Y. Lv, Y. H. Cao, et al., “A hybrid neural network entity recognition method based on BERT model,” Computer Technology and Development, vol.31, no.8, pp.100–105, 2021. (in Chinese) doi: 10.3969/j.issn.1673-629X.2021.08.017
[17]	J. T. Li and K. Meng, MFE-NER: Multi-feature fusion embedding for Chinese named entity recognition, arXiv preprint, arXiv: 2109.07877, 2021.
[18]	J. Hu, Y. Hu, M. C. Liu, et al., “Chinese named entity recognition based on knowledge base entity enhanced BERT model,” Journal of Computer Applications, vol.42, no.9, pp.2680–2685, 2022. (in Chinese) doi: 10.11772/j.issn.1001-9081.2021071209
[19]	W. Liu, X. Y. Fu, Y. Zhang, et al., “Lexicon enhanced Chinese sequence labeling using BERT adapter,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual, Online, pp.5847–5858, 2021.
[20]	Y. H. Liu, M. Ott, N. Goyal, et al., RoBERTa: A robustly optimized BERT pretraining approach, arXiv preprint, arXiv: 1907.11692, 2019.
[21]	J. Devlin, M. W. Chang, K. Lee, et al., “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, pp.4171–4186, 2019.
[22]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp.6000–6010, 2017.
[23]	Y. M. Cui, W. X. Che, T. Liu, et al., “Revisiting pre-trained models for Chinese natural language processing,” in Proceedings of Findings of the Association for Computational Linguistics: EMNLP 2020, Virtual, Online, pp.657–668, 2020.
[24]	K. Cho, B. van Merriënboer, C. Gulcehre, et al., “Learning phrase representations using RNN encoder–decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp.1724–1724, 2014.
[25]	Z. L. Zeng, X. Yan, G. Y. Xu, et al., “Query-oriented news multi-document extractive summarization method based on hierarchical BIGRU+attention,” Journal of Chinese Computer Systems, vol.44, no.1, pp.185–192, 2023. (in Chinese) doi: 10.20009/j.cnki.21-1106/TP.2021-0457
[26]	P. Shaw, J. Uszkoreit, and A. Vaswani, “Self-attention with relative position representations,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, LA, USA, pp.464–468, 2018.
[27]	T. Miyato, A. M. Dai, and I. J. Goodfellow, “Adversarial training methods for semi-supervised text classification,” in Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 2017.
[28]	Y. X. Meng, W. Wei, F. Wang, et al., “Glyce: Glyph-vectors for Chinese character representations,” arXiv preprint:1901.10125, 2020.
[29]	D. Hu and L. W. We, “SLK-NER: Exploiting second-order lexicon knowledge for Chinese NER,” in Proceedings of the 32nd International Conference on Software Engineering and Knowledge Engineering, pp.413–417, 2020.
[30]	X. N. Li, H. Yan, X. P. Qiu, et al., “FLAT: Chinese NER using flat-lattice transformer,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Virtual, Online, pp.6836–6842, 2020.
[31]	H. Yan, B. C. Deng, X. N. Li, et al., TENER: Adapting transformer encoder for named entity recognition, arXiv preprint, arXiv: 1911.04474, 2019.
[32]	L. Xu, Y. Tong, Q. Q. Dong, et al., CLUENER2020: fine-grained named entity recognition dataset and benchmark for Chinese, arXiv preprint, arXiv: 2001.04351, 2020.
[33]	G. A. Levow, “The third international Chinese language processing bakeoff: Word segmentation and named entity recognition,” in Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing, Sydney, Australia, pp.108–117, 2006.
[34]	S. W. Yu, H. M. Duan, and Y. F. Wu, “Corpus of multi-level processing for modern Chinese,” Available at: https://doi.org/10.18170/DVN/SEYRX5, 2018.
[35]	K. R. Min, C. G. Ma, T. M. Zhao, et al., “BosonNLP: An ensemble approach for word segmentation and POS tagging,” in Proceedings of the 4th CCF Conference on Natural Language Processing and Chinese Computing, Nanchang, China, pp.520–526, 2015.
[36]	M. H. Tong, S. Wang, B. Xu, et al., “Learning from miscellaneous other-class words for few-shot named entity recognition,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual, Online, pp.6236–6247, 2021.
[37]	D. Y. Li, L. Yan, J. Z. Yang, et al., “Dependency syntax guided BERT-BiLSTM-GAM-CRF for Chinese NER,” Expert Systems with Applications, vol.196, article no.116682, 2022. doi: 10.1016/j.eswa.2022.116682
[38]	X. C. Guo, H. Zhou, J. Su, et al., “Chinese agricultural diseases and pests named entity recognition with multi-scale local context features and self-attention mechanism,” Computers and Electronics in Agriculture, vol.179, article no.105830, 2020. doi: 10.1016/j.compag.2020.105830
[39]	S. Wu, X. N. Song, Z. H. Feng, et al., NFLAT: non-flat-lattice transformer for Chinese named entity recognition, arXiv preprint, arXiv: 2205.05832, 2022.
[40]	Y. Y. Shen, H. Yun, Z. Lipton, et al., “Deep active learning for named entity recognition,” in Proceedings of the 2nd Workshop on Representation Learning for NLP, Vancouver, Canada, pp.252–256, 2017.
[41]	J. P. C. Chiu and E. Nichols, “Named entity recognition with bidirectional LSTM-CNNs,” Transactions of the Association for Computational Linguistics, vol.4, pp.357–370, 2016. doi: 10.1162/tacl_a_00104
[42]	L. F. He, X. F. Zhang, Z. W. Li, et al., “A Chinese named entity recognition model of maintenance records for power primary equipment based on progressive multitype feature fusion,” Complexity, vol.2022, article no.8114217, 2022. doi: 10.1155/2022/8114217
[43]	X. M. Han, F. Zhou, Z. Y. Hao, et al., “MAF-CNER: a Chinese named entity recognition model based on multifeature adaptive fusion,” Complexity, vol.2021, article no.6696064, 2021. doi: 10.1155/2021/6696064
[44]	X. H. Song, H. T. Yu, and S. M. Li, “Chinese named entity recognition based on word fusion of graph attention network,” Computer Engineering, vol.48, no.10, pp.298–305, 2022. (in Chinese) doi: 10.19678/j.issn.1000-3428.0063055
[45]	X. B. Hu, X. Q. Yu, S. M. Li, et al., “Chinese named entity recognition based on knowledge enhancement,” Computer Engineering, vol.47, no.11, pp.84–92, 2021. (in Chinese) doi: 10.19678/j.issn.1000-3428.0059810