ZHANG Yangsen, ZHENG Jia, JIANG Yuru, HUANG Gaijuan, CHEN Ruoyu. A Text Sentiment Classification Modeling Method Based on Coordinated CNN-LSTM-Attention Model[J]. Chinese Journal of Electronics, 2019, 28(1): 120-126. doi: 10.1049/cje.2018.11.004
Citation: ZHANG Yangsen, ZHENG Jia, JIANG Yuru, HUANG Gaijuan, CHEN Ruoyu. A Text Sentiment Classification Modeling Method Based on Coordinated CNN-LSTM-Attention Model[J]. Chinese Journal of Electronics, 2019, 28(1): 120-126. doi: 10.1049/cje.2018.11.004

A Text Sentiment Classification Modeling Method Based on Coordinated CNN-LSTM-Attention Model

doi: 10.1049/cje.2018.11.004
Funds:  This work is supported by the National Natural Science Foundation of China (No.61772081, No.61602044) and the Science and Technology Development Project of Beijing Municipal Education Commission(No.KM201711232014).
  • Received Date: 2017-08-04
  • Rev Recd Date: 2018-05-29
  • Publish Date: 2019-01-10
  • The major challenge that text sentiment classification modeling faces is how to capture the intrinsic semantic, emotional dependence information and the key part of the emotional expression of text. To solve this problem, we proposed a Coordinated CNNLSTM-Attention(CCLA) model. We learned the vector representations of sentence with CCLA unit. Semantic and emotional information of sentences and their relations are adaptively encoded to vector representations of document. We used softmax regression classifier to identify the sentiment tendencies in the text. Compared with other methods, the CCLA model can well capture the local and long distance semantic and emotional information. Experimental results demonstrated the effectiveness of CCLA model. It shows superior performances over several state-of-the-art baseline methods.
  • loading
  • CAO Jiuxin, CHEN Gaojun, WU Jianglin, et al., "Multifeature based opinion mining in social networks", Act Electronica Sinica, Vol.44, No.4, pp.898-905, 2016. (in Chinese)
    LU Pin, ZHONG Luo and TANG Kunhao, "Customer satisfaction degree evaluation of online product review", Act Electronica Sinica, Vol.42, No.4, pp.740-746, 2014. (in Chinese)
    T.K. Landauer and S.T. Dumais, "A solution to Plato's problem:The latent semantic analysis theory of acquisition, induction, and representation of knowledge", Psychological Review, Vol.104, No.2, pp.211-240, 1997.
    HUANG Faliang, LI Chaoxiong, YUAN Changan, et al., "Mining sentiment for web short texts based on TSCM model", Act Electronica Sinica, Vol.44, No.8, pp.1887-1891, 2016. (in Chinese)
    R. Socher, E.H. Huang, J. Pennin, et al., "Dynamic pooling and unfolding recursive autoencoders for paraphrase detection", In Advances in Neural Information Processing Systems, Vol.24, pp.801-809, 2011.
    R. Socher, J. Pennington, E.H. Huang, et al., "Semisupervised recursive autoencoders for predicting sentiment distributions", Proc. of the Conference on Empirical Methods in Natural Language Processing, Edinburgh, UK, pp.151-161, 2011.
    R. Socher, A. Perelygin, J.Y. Wu, et al., "Recursive deep models for semantic compositionality over a sentiment treebank", Proc. of the Conference on Empirical Methods in Natural Language Processing, Washington, USA, pp.1631-1642, 2013.
    J.L. Elman, "Finding structure in time", Cognitive Science, Vol.14, No.2, pp.179-211, 1990.
    N. Kalchbrenner and P. Blunsom, "Recurrent convolutional neural networks for discourse compositionality", Proc. of Workshop on Continuous Vector Space Models and their Compositionality, Sofia, Bulgaria, pp.119-126, 2013.
    K.S. Tai, R. Socher and C.D. Manning, "Improved semantic representations from tree-structured long short-term memory networks", Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, pp.1556-1566, 2015.
    J. Wang, L.C. Yu, K.R. Lai, et al., "Dimensional sentiment analysis using a regional CNN-LSTM Model", Proc. of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, pp.225-230, 2016.
    D. Bahdanau, K. Cho and Y. Bengio, "Neural machine translation by jointly learning to align and translate", Proc. of International Conference on Learning Representations, San Diego, CA, USA, 2015.
    T. Mikolov, I. Sutskever, K. Chen, et al., "Distributed representations of words and phrases and their compositionality", In Advances in Neural Information Processing Systems, Vol. 26, pp.3111-3119, 2013.
    N. Kalchbrenner, E. Grefenstette and P. Blunsom. "A convolutional neural network for modelling sentences", Proc. of the 52rd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, USA, pp.655-665, 2014.
    A.L. Maas, R.E. Daly, P.T. Pham, et al., "Learning word vectors for sentiment analysis", Proc. of the 49th Annual Meeting of the Association for Computational Linguistic, Portland, Oregon, USA, pp.142-150, 2011.
    T. Mikolov, K. Chen, G. Corrado, et al., "Efficient estimation of word representations in vector space", arXiv preprint, arXiv:1301.3781, 2013.
    J. Pennington, R. Socher and C.D. Manning, "Glove:Global vectors for word representation", Proc. of the Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, pp.1532-1543, 2014.
    Q. Le and T. Mikolov, "Distributed represenations of sentences and documents", Proc. of the 31th International Conference on Machine Learning, Beijing, China, pp.1188-1196, 2014.
    ZHANG Yangsen, JIANG Yuruo and TONG Yuxuan, "Study of sentiment classification for chinese microblog based on recurrent neural network", Chinese Journal of Electronics, Vol.25, No.4, pp.601-607, 2016.
    Y. Bengio, R. Ducharme, P. Vincent, et al., "A neural probabilistic language model", Journal of Machine Learning Research, Vol.3, pp.1137-1155, 2003.
    S. Sukhbaatar, A. Szlam, J. Weston, et al., "End-To-end memory networks", Proc. of Advances in Neural Information Processing Systems, Montreal, Canada, 2015.
    J. Li, M.T. Luong and D. Jurafsky, "A hierarchical neural autoencoder for paragraphs and documents", Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, pp.1106-1115, 2015.
    B. Pang and L. Lee, "Seeing stars:Exploiting class relationships for sentiment categorization with respect to rating scales", Proc. of of the 43rd Annual Meeting of the Association for Computational Linguistics, Ann Arbor, USA, pp.115-124, 2005.
    X. Li and D. Roth, "Learning question classifiers", Proc. of 19th International Conference on Computational Linguistics, Taipei, Taiwan, China, pp.556-562, 2002.
    B. Pang and L. Lee, "A sentimental education:Sentiment analysis using subjectivity summarization based on minimum cuts", Proc. of the 42nd Annual Meeting on Association for Computational Linguistics, Stroudsburg, PA, USA, pp.271-279, 2004.
    M.Xouvier, "LZA at semeval-2017 task 4:An ensemble of neural network for sentiment classification", Proc. of the 11th International Workshop on Semantic Evaluation, Vancouver, Canada. pp.760-765, 2017.
    N. Srivastava, G. Hinton, A. Knizhevsky,et al., "Dropont:A simple way to prevent networks from overfitting", The Journal of Machine Learning Research, Vol.15, No.1, pp.1929-1958, 2014.
    J. Silva, L. Coheur, A.C. Mendes, et al., "From symbolic to sub-symbolic information in question classification", Artificial Intelligence Review, Vol.35, No.2, pp.137-154, 2011.
    S. Wang and C.D. Manning, "Baselines and Bigrams:Simple, good sentiment and topic classification", Proc. of 50th Annual Meeting of the Association for Computational Linguistics, Stroudsburg, PA, USA, pp.90-94, 2012.
    G.E. Dahl, R.P. Adams and H. Larochelle, "Training restricted boltzmann machines on word observations", Proc. of the 29th International Conference on Machine Learning, Edinburgh, Scotland, UK, 2012.
    Y. Kim, "Convolutional neural networks for sentence classification", Proc. of the Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, pp.1746-1751, 2014.
    A.M. Dai and Q.V. Le, "Semi-supervised sequence learning", Proc. of the 28th International Conference on Neural Information Processing Systems, Massachusetts, USA, pp.3079-3087, 2015.
    M. Ma, L. Huang, B. Xiang, et al., "Dependency-based convolutional neural networks for sentence embedding", Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, pp.174-179, 2015.
    R. Zhang, H. Lee and D. Radev, "Dependency sensitive convolutional neural networks for modeling sentences and documents", Proc. of 14th Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,San Diego, California, USA, pp.1512-1521, 2016.
    B. Hu, Z. Lu, H. Li, et al., "Convolutional neural network architectures for matching natural language sentences", Proc. of the 27th International Conference on Neural Information Processing Systems, Massachusetts, USA, pp.2042-2050, 2014.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (434) PDF downloads(1072) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return