WU Yujia, LI Jing, SONG Chengfang, CHANG Jun. Words in Pairs Neural Networks for Text Classification[J]. Chinese Journal of Electronics, 2020, 29(3): 491-500. doi: 10.1049/cje.2020.03.005
Citation: WU Yujia, LI Jing, SONG Chengfang, CHANG Jun. Words in Pairs Neural Networks for Text Classification[J]. Chinese Journal of Electronics, 2020, 29(3): 491-500. doi: 10.1049/cje.2020.03.005

Words in Pairs Neural Networks for Text Classification

doi: 10.1049/cje.2020.03.005
Funds:  This work is supported by National Natural Science Foundation of China (No.61772180, No.41201404) and Fundamental Research Funds for the Central Universities of China (No.2042015gf0009).
More Information
  • Corresponding author: LI Jing (corresponding author) was born in 1967. He received the Ph.D. degree from Wuhan University, Wuhan, China, in 2006. He is currently a Professor in Computer School of Wuhan University, Wuhan, China. His research interests include data mining and multimedia technology. (Email:leejingcn@whu.edu.cn)
  • Received Date: 2018-12-25
  • Rev Recd Date: 2020-01-09
  • Publish Date: 2020-05-10
  • Existing methods utilized single words as text features. Some words contain multiple meanings, and it is difficult to distinguish its specific classification according to a single word, which probably affects the accuracy of the text classification. Propose a framework based on Words in pairs neural networks (WPNN) for text classification. Words in pairs include all single word combinations which have a high mutual association. Mine the crucial explicit and implicit Words in pairs as text features. These words in pairs as a text feature are easily classified. The words in pairs are utilized as the input of the neural network, which provides a better classification ability to the model, because they are more recognizable than the single word. Experimental results show that our model outperforms five benchmark algorithms.
  • loading
  • WU Lianwei,RAO Yuan,YU Hualei,et al., “A multi-semantics classification method based on deep learning for incredible messages on social media”, Chinese Journal of Electronics, Vol.28, No.4, pp.754-763, 2019.
    MENG Deyu,SUN Lina, “Some new trends of deep learning research”, Chinese Journal of Electronics, Vol.28, No.6, pp.1087-1091, 2019.
    Y. Kim, “Convolutional neural networks for sentence classification”, Proc. of Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, pp.1746-1751,2014.
    W. Yih, X. He and C. Meek, “Semantic parsing for singlerelation question answering”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, USA, pp.643-648,2014.
    Y. Shen, X. He, J. Gao, et al., “Learning semantic representations using convolutional neural networks for web search”, Proc. of International Conference on World Wide Web, Seoul, Korea, pp.373-374, 2014.
    R. Johnson and T. Zhang, “Effective use of word order for text categorization with convolutional neural networks”, Proc. of Conference on North American Chapter of the Association for Computational Linguistics-Human Language Technologies, Denver, Colorado, USAA, pp.103-112,2015.
    X. Zhang, J. Zhao and Y. LeCun, “Character-level convolutional networks for text classification”, Proc. of Conference on Advances in Neural Information Processing Systems, Montreal, Canada, pp.649-657,2015.
    D. Tang, B. Qin and T. Liu, “Learning semantic representations of users and products for document level sentiment classification”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Beijing, China, pp.1014-1023,2015.
    P. Liu, X. Qiu and X. Huang, “Recurrent neural network for text classification with multi-task learning”, Proc. of Conference on International Joint Conference on Artificial Intelligence, Melbourne, Australia, pp.1480-1489,2017.
    S. Lai, L. Xu, K. Liu, et al., “Recurrent convolutional neural networks for text classification”, Proc. of Conference on Association for the Advancement of Artificial Intelligence,Austin, Texas, USA, pp.2267-2273, 2015.
    Z. Yang, D. Yang, C. Dyer, et al., “Hierarchical attention networks for document classification”, Proc. of Conference on North American Chapter of the Association for Computational Linguistics-Human Language Technologies, San Diego, California, USA, pp.1480-1489, 2016.
    B. Wang, “Disconnected recurrent neural networks for text categorization”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, pp.2311-2320,2018.
    J. Howard and S. Ruder, “Universal language model finetuning for text classification”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, pp.328-339,2018.
    J. Zeng, J. Li, Y. Song, et al., “Topic memory networks for short text classification”, Proc. of Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp.3120-3131, 2018.
    T. Mikolov, I. Sutskever, K. Chen, et al., “Distributed representations of words and phrases and their compositionality”, Proc. of Conference on Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, USA, pp.3111-3119, 2013.
    J. Pennington, R. Socher and C. Manning, “Glove: Global vectors for word representation”, Proc. of Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, pp.1532-1543,2014.
    G. Wang, C. Li, W. Wang, et al., “Joint embedding of words and labels for text classification”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, pp.2321-2331, 2018.
    Z. H. Yahia, A. Sieg, L. A. Deleris, “Towards unsupervised text classification leveraging experts and word embeddings”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.371-379,2019.
    B. Trstenjak, S. Mikac and D. Donko, “KNN with TFIDF based framework for text categorization”, Procedia Engineering, Vol.69, No.1, pp.1356-1364, 2014.
    R. Johnson and T. Zhang, “Deep pyramid convolutional neural networks for text categorization”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, pp.562-570,2017.
    L. Zhang, S. Wang, B. Liu, “Deep learning for sentiment analysis: A survey”, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol.8, No.4, pp.1-34, 2018.
    M. Yang, W. Zhao, L. Chen,et al., “Investigating the transferring capability of capsule networks for text classification”, Neural Networks, Vol.118, No.6, pp.247-261, 2019.
    W. Zhao, H. Peng, S. Eger, et al., “Towards scalable and reliable capsule networks for challenging NLP applications”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.1549-1559, 2019.
    Z. Chen and T. Qian, “Transfer capsule network for aspect level sentiment classification,”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.547-556, 2019.
    F. Liu and B. Avci, “Incorporating priors with feature attribution on text classification,”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.6274-6283, 2019.
    T. Hiraoka, H. Shindo and Y. Matsumoto, “Stochastic tokenization with a language model for neural text classification”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.1620-1629, 2019.
    K. Shimura, J. Li, F. Fukumoto, “Text categorization by learning predominant sense of words as auxiliary task,”, Proc. of Conference on Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.1109-1119, 2019.
    C. F. Ahmed, S. K. Tanbeer, B.S. Jeong,et al., “Efficient tree structures for high utility pattern mining in incremental databases”, IEEE Transactions on Knowledge and Data Engineering, Vol.21, No.12, pp.1708-1721, 2009.
    V. S. Tseng, C.W. Wu, B.E. Shie, et al., “Up-growth: An efficient algorithm for high utility itemset mining”, Proc. of Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, pp.253-262, 2010.
    V. S. Tseng, B.E. Shie, C.W. Wu, et al., “Efficient algorithms for mining high utility item-sets from transactional databases”, IEEE Transactions on Knowledge and Data Engineering, Vol.25, No.8, pp.1772-1786, 2013.
    C.W Wu, P. Fournier-Viger, P. S. Yu, et al., “Efficient mining of a concise and lossless representation of high utility itemsets”, Proc. of Conference on IEEE International Conference on Data Mining, Vancouver, Canada, pp.824-833, 2011.
    V. S. Tseng, C.W Wu, P. Fournier-Viger, et al., “Efficient algorithms for mining the concise and lossless representation of high utility itemsets”, IEEE Transactions on Knowledge and Data Engineering, Vol.27, No.3, pp.726-739, 2015.
    M.C. Liu and J.F. Qu, “Mining high utility itemsets without candidate generation”, Proc. of Conference on EMNLP,Maui, Hawaii, USA, pp.55-64,2012.
    P.Fournier-Viger, C. W.Wu, S.Zida, et al., “FHM: Faster highutility itemset mining using estimated utility co-occurrence pruning”, Proc. of Conference on International Symposium on Methodologies for Intelligent Systems, Roskilde, Danmark, pp.83-92, 2014.
    S. Krishnamoorthy, “Pruning strategies for mining high utility itemsets”, Expert Systems with Applications, Vol.42, No.5, pp.2371-2381, 2015.
    J. Sahoo, A.K. Das and A. Goswami, “An efficient fast algorithm for discovering closed+ high utility itemsets”, Applied Intelligence, Vol.45, No.1, pp.44-74, 2016.
    C.W. Wu, P. Fournier-Viger, J.Y. Gu, et al., “Mining closed+ high utility itemsets without candidate generation”, Proc. of Conference on Technologies and Applications of Artificial Intelligence, Taiwan, China, pp.1-8, 2015.
    S.Guo and H.Gao, “HUITWU: An efficient algorithm for highutility itemset mining in transaction databases”, Journal of Computer Science and Technology, Vol.31, No.4, pp.776-786, 2016.
    A. Joulin, E. Grave, P. Bojanowski, et al., “Bag of tricks for efficient text classification”, Proc. of Conference on European Chapter of the Association for Computational Linguistics, Valencia, Spain, pp.427-431, 2017.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (210) PDF downloads(143) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return