Word-Based Method for Chinese Part-of-Speech via Parallel and Adversarial Network

HUANG Kaiyu; CAO Jingxiang; LIU Zhuang; HUANG Degen

doi:10.1049/cje.2020.00.411

Volume 31 Issue 2

Mar. 2022

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2022 > 31(2): 337-344

HUANG Kaiyu, CAO Jingxiang, LIU Zhuang, et al. “Word-Based Method for Chinese Part-of-Speech via Parallel and Adversarial Network”. Chinese Journal of Electronics, vol. 31 no. 2. doi: 10.1049/cje.2020.00.411

Citation:

HUANG Kaiyu, CAO Jingxiang, LIU Zhuang, et al. “Word-Based Method for Chinese Part-of-Speech via Parallel and Adversarial Network”. Chinese Journal of Electronics, vol. 31 no. 2. doi: 10.1049/cje.2020.00.411

Citation:

PDF( 2956 KB)

Word-Based Method for Chinese Part-of-Speech via Parallel and Adversarial Network

doi: 10.1049/cje.2020.00.411

1.
School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China
2.
School of Foreign Languages, Dalian University of Technology, Dalian 116024, China
3.
School of Applied Finance, Dongbei University of Finance and Economics, Dalian 116025, China

Funds: This work was supported by the National Key Research and Development Program of China (2020AAA0108004) and the National Natural Science Foundation of China (U1936109, 61672127)

More Information

Author Bio:
obtained the Ph.D. degree in computer application technology at Dalian University of Technology, China, in 2021. And he received the B.S. degree in computer science and the B.A. degree in Japanese from Dalian University of Technology, China, in 2016. He has published at highly-ranked conferences and journals, such as ACL, EMNLP, IJCAI, and ACM TALLIP. Specifically, he favors the research perspectives on natural language processing, including pre-trained language models, document-level (discourse) neural machine translation, conversational question and answering, and text sequence labeling (i.e., CWS, POS, NER). Moreover, he have joined multiple research projects and foundations. (Email: kaiyuhuang@mail.dlut.edu.cn)

received the B.A. degree in English for science and technology from Dalian University of Technology (DUT), China, in 1995, and M.A degree in linguistics and applied linguistics from Dalian Maritime University, China, in 2000, and Ph.D. degree in computer application technology from DUT in 2013. She made a one-year visit to University Centre for Computer Corpus Research on Language (UCREL), Lancaster University, U.K., in 2011. She is currently an Associate Professor of linguistics at Dalian University of Technology. Her research interests include corpus linguistics, natural language processing, and machine translation. She is now a Member of CCF and ACL. (Email: caojx@dlut.edu.cn)

received the Ph.D. degree in computer science from Dalian University of Technology. Currently, he is a Lecturer at School of Applied Finance, Dongbei University of Finance and Economics. His research covers areas of natural language processing (e.g. machine comprehension, question answering, and dialogue generation), graph neural networks, financial text mining, and block chain. He has published at highly-ranked journals such as ACM TIST, and leading international conferences such as IJCAI, ECAI, CIKM, and EMNLP. He served as the Architect and Deputy Technical Director of financial technology companies including Alipay before returning to university. He served as Session Chair for data mining in IJCAI 2020, as Guest Editor of Information Extraction and NLP, and as Program Committee Members and/or Reviewers regularly at numerous journals and conferences such as IEEE TNNLS, IEEE Access, IEEE Signal Proc. Let., AAAI, ACL, NAACL, EMNLP, etc. (Email: liuzhuang@dufe.edu.cn)

(corresponding author) was born in 1965. He received the Ph.D. degree in computer science from the Dalian University of Technology, China, in 2004. He is currently a Professor with the School of Computer Science, Dalian University of Technology. His research interests include machine translation and knowledge graph. He is now a Senior Member of CIPS, ACM, CAAI. (Email: huangdg@dlut.edu.cn)
Received Date: 2020-12-11
Accepted Date: 2021-02-24

Available Online: 2021-10-18

Publish Date: 2022-03-05

Abstract

Abstract

Chinese part-of-speech (POS) tagging is an essential task for Chinese downstream natural language processing tasks. The accuracy of the Chinese POS task will drop dramatically by word-based methods because of the segmentation errors and the word sparsity. Also, there are several Chinese POS tagging sets with different criteria. Some of them only have a small-scale annotated corpus and are hard to train. To this end, we propose a modified word-based transformer neural network architecture. Meanwhile, we utilize an adversarial transfer learning method that splits the architecture into shared and private parts. This work directly improves the ability of the word-based model, instead of adopting a joint character-based method. Extensive experiments show that our method achieves state-of-the-art performance on all datasets, and more importantly, our method improves performance effectively for the word-based Chinese sequence labeling task.
- Natural language processing,
- Part-of-speech tagging,
- Word-based model,
- Adversarial network

http://www.cncorpus.org; https://github.com/UniversalDependencies/UD_Chinese-GSDSimp

FullText(HTML)

References(37)

References

[1]	Z. Q. Huang, M. P. Harper, and W. Wang, “Mandarin part-of-speech tagging and discriminative reranking,” in Proc. of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic, pp.1093–1102, 2007.
[2]	T. Brants, “TnT: A statistical part-of-speech tagger,” in Proc. of the Sixth Conference on Applied Natural Language Processing, Seattle, Washington, USA, pp.224–231, 2000.
[3]	W. B. Jiang, L. Huang, Q. Liu, et al., “A cascaded linear model for joint Chinese word segmentation and part-of-speech tagging,” in Proc. of the 46th Annual Meeting of the Association for Computational Linguistics, Columbus, Ohio, USA, pp.897–904, 2008.
[4]	J. Devlin, M.W. Chang, K. Lee, et al., “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proc. of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, USA, pp.4171–4186, 2019.
[5]	H. Zhou, Z. T. Yu, Y. Zhang, et al., “Word-context character embeddings for Chinese word segmentation,” in Proc. of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp.760–766, 2017.
[6]	L. Yang, M. S. Zhang, Y. Liu, et al., “Joint POS tagging and dependence parsing with transition-based neural networks,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol.26, no.8, pp.1352–1358, 2017.
[7]	X. C. Chen, X. P. Qiu, and X. J. Huang, “A feature-enriched neural model for joint Chinese word segmentation and part-of-speech tagging,” in Proc. of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia, pp.3960–3966, 2017.
[8]	Y. Shao, C. Hardmeier, J. Tiedemann, et al., “Character-based joint segmentation and POS tagging for Chinese using bidirectional RNN-CRF,” in Proc. of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Taipei, China, pp.173–183, 2017.
[9]	X. Y. Li, Y. X. Meng, S. F. Sun, et al., “Is word segmentation necessary for deep learning of Chinese representations?” in Proc. of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.3242–3252, 2019.
[10]	J. Yang, Y. Zhang, and S. L. Liang, “Subword encoding in lattice LSTM for Chinese word segmentation,” in Proc. of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers), Minneapolis, Minnesota, USA, pp.2720–2725, 2019.
[11]	Y. X. Meng, W. Wu, F. Wang, et al., “Glyce: Glyph-vectors for Chinese character representations,” in Proc. of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, pp.2742–2753, 2019.
[12]	D. Cai, H. Zhao, Z. S. Zhang, et al., “Fast and accurate neural word segmentation for Chinese,” in Proc. of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Vancouver, Canada, pp.608–615, 2019.
[13]	X. P. Qiu, H. Z. Pei, H. Yan, et al., “Multi-criteria Chinese word segmentation with transformer,” arXiv preprint, arXiv: 1906.12035, 2019.
[14]	G. J. Jin and X. Chen, “The fourth international Chinese language processing bakeoff: Chinese word segmentation, named entity recognition and Chinese pos tagging,” in Proc. of the Sixth SIGHAN workshop on Chinese Language Processing, pp.69–81, 2008.
[15]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proc. of 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, pp.5998–6008, 2017.
[16]	Y. Kim, “Convolutional neural networks for sentence classification,” in Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp.1746–1751, 2014.
[17]	H. Yan, B. C. Deng, X. N. Li, et al., “Tener: Adapting transformer encoder for name entity recognition,” arXiv preprint, arXiv: 1911.04474, 2019.
[18]	Z. H. Huang, W. Xu and K. Yu, “Bidirectional LSTM-CRF models for sequence tagging,” arXiv preprint, arXiv: 1508.01991, 2015..
[19]	P. F. Cao, Y. B. Chen, K. Liu, et al., “Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism,” in Proc. of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp.182–192, 2018.
[20]	A. Paszke, S. Gross, F. Massa, et al., “PyTorch: An imperative style, high-performance deep learning library,” in Proc. of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, pp.8024–8035, 2019.
[21]	S. Li, Z. Zhao, R. F. Hu, et al., “Analogical reasoning on Chinese morphological and semantic relations,” in Proc. of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, Australia, pp.138–143, 2018.
[22]	M. S. Zhang, N. Yu and G. H. Fu, “A simple and effective neural model for joint word segmentation and POS tagging,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol.26, no.9, pp.1528–1538, 2018.
[23]	Z. Q. Huang, V. Eidelman and M. Harper, “Improving a simple bigram hmm part-of-speech tagger by latent annotation and self-training,” in Proc. of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Paper, Boulder, Colorado, USA, pp.213–216, 2009.
[24]	X. P. Qiu, J. Y. Zhao, and X. J. Huang, “Joint Chinese word segmentation and POS tagging on heterogeneous annotated corpora with multiple task learning,” in Proc. of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, pp.658–668, 2013.
[25]	W. W. Sun and X. J Wan, “Towards accurate and efficient Chinese part-of-speech tagging,” Computational Linguistics, vol.42, no.3, pp.391–419, 2016. doi: 10.1162/COLI_a_00253
[26]	R. Collobert, J. Weston, L. Bottou, et al., “Natural language processing (almost) from scratch,” Journal of Machine Learning Research, vol.12, pp.2493–2537, 2011.
[27]	W. Z. Pei, T. Ge, and B. B. Chang, “Max-margin tensor neural network for Chinese word segmentation,” in Proc. of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, Maryland, USA, pp.293–303, 2014.
[28]	X. C. Chen, X. P. Qiu, C. X. Zhu, et al., “Long short-term memory neural networks for Chinese word segmentation,” in Proc. of the 2015 Conf. on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp.1197–1206, 2015.
[29]	D. G. Huang, J. Zhang, and K. Y. Huang, “Automatic microblog-oriented unknown word recognition with unsupervised method,” Chinese Journal of Electronics, vol.27, no.1, pp.1–8, 2018. doi: 10.1049/cje.2017.11.004
[30]	N. Xi, X. Y Dai, S. J. Huang, et al., “Discriminative word alignment over multiple word segmenations,” Chinese Journal of Electronics, vol.23, no.2, pp.263–279, 2014.
[31]	X. C. Chen, Z. Shi, X. P. Qiu, et al., “Adversarial multi-criteria learning for Chinese word segmentation,” in Proc. of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, pp.1193–1203, 2017.
[32]	P. Bhatia, K. Arumae, and E. B. Celikkaya, “Dynamic transfer learning for named entity recognition,” Int. Workshop on Health Intelligence, Hawaii, USA, pp.69–81, 2019.
[33]	Y. Ganin, E. Ustinova, H. Ajakan, et al., “Domain-adversarial training of neural networks,” The Journal of Machine Learning Research, vol.17, no.1, pp.2096–2030, 2016.
[34]	Z. Y. Pei, Z. J. Cao, M. S. Long, et al., “Multi-adversarial domain adaptation,” in Proc. of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, pp.3934–3941, 2018.
[35]	J. Y. Yi, J. H. Tao, Y. Bai, et al., “Adversarial transfer learning for punctuation restoration,” arXiv preprint, arXiv: 2004.00248, 2020.
[36]	D. S. Luo, M. X. Nie, and X. H. Wu, “Generating basic unit movements with conditional generative adversarial networks,” Chinese Journal of Electronics, vol.28, no.6, pp.1099–1107, 2019. doi: 10.1049/cje.2019.07.013
[37]	C. Qin and X. G. Gao, “Spatio-temporal generative adversarial networks,” Chinese Journal of Electronics, vol.29, no.4, pp.623–631, 2020. doi: 10.1049/cje.2020.04.001