Abstract
Existing text classification models based on graph convolutional networks usually update node representations simply by fusing neighborhood information of different orders through adjacency matrices, resulting in an insufficient representation of node semantic information. In addition, models based on conventional attention mechanisms are only Word vectors which are forward-weighted representations, ignoring the impact of negative words on the final classification. This paper proposes a model based on a bidirectional attention mechanism and a gated graph convolutional network to solve the above problems. The model first uses the gated graph convolutional network, selectively fuses the multi-order neighbourhood information of the nodes in the graph, retains the information of the previous order, and enriches the feature representation of the nodes. Secondly, the influence of different words on the classification results is learned through a two-way attention mechanism. While giving positive weights to terms that play a positive role in classification, negative consequences are given to words that have adverse effects of weakening their influence in vector representation, thereby improving the model's ability to discriminate nodes with different properties in documents. Finally, maximum pooling and average pooling are used to fuse the vector representation of the word, and the document representation is obtained for the final classification. Experiments are carried out on four benchmark data sets, and the results show that the method is significantly better than the baseline model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Tang H, Mi Y, Xue F, Cao Y (2020) An integration model based on graph convolutional network for text classification. IEEE Access 8:148865–148876. https://doi.org/10.1109/ACCESS.2020.3015770
Jati WK, Kemas Muslim L (2020) Optimization of decision tree algorithm in text classification of job applicants using particle swarm optimization. In: 3rd international conference on information and communications technology (ICOIACT), Yogyakarta, Indonesia, pp 201–205. https://doi.org/10.1109/ICOIACT50329.2020.9332101
Wang Z, Liu J, Sun G, Zhao J, Ding Z, Guan X (2020) An ensemble classification algorithm for text data stream based on feature selection and topic model. In: IEEE international conference on artificial intelligence and computer applications (ICAICA), Dalian, China, pp 1377–1380. https://doi.org/10.1109/ICAICA50127.2020.9181903
Buldin ID, Ivanov NS (2020) Text classification of illegal activities on onion sites. In: IEEE conference of russian young researchers in electrical and electronic engineering (EIConRus), St. Petersburg and Moscow, Russia, pp 245–247. https://doi.org/10.1109/EIConRus49466.2020.9039341
Wang P et al (2020) Classification of proactive personality: text mining based on weibo text and short-answer questions text. IEEE Access 8:97370–97382. https://doi.org/10.1109/ACCESS.2020.2995905
Zhang Y, Wang Y, Gu H, Liu L, Zhang J, Lin H (2020) Defect diagnosis method of main transformer based on operation and maintenance text mining. In: IEEE international conference on high voltage engineering and application (ICHVE), Beijing, China, pp 1–4. https://doi.org/10.1109/ICHVE49031.2020.9280086
Rui Z, Yutai H (2020) Research on short text classification based on Word2Vec microblog. In: International conference on computer science and management technology (ICCSMT), Shanghai, China, pp 178–182. https://doi.org/10.1109/ICCSMT51754.2020.00042
Kalcheva N, Karova M, Penev I (2020) Comparison of the accuracy of SVM kemel functions in text classification. In: 2020 International conference on biomedical innovations and applications (BIA), Varna, Bulgaria, pp 141–145. https://doi.org/10.1109/BIA50171.2020.9244278
Meng X, Yu H, Cao H (2020) Tibetan text classification algorithm based on syllables. In: IEEE 3rd international conference on information systems and computer aided education (ICISCAE), Dalian, China, pp 622–625. https://doi.org/10.1109/ICISCAE51034.2020.9236833
Wang Q, Xu C, Zhang W, Li J (2021) GraphTTE: travel time estimation based on attention-spatiotemporal graphs. IEEE Signal Process Lett 28:239–243. https://doi.org/10.1109/LSP.2020.3048849
Xie R, Yin J, Han J (2021) DyGA: a hardware-efficient accelerator with traffic-aware dynamic scheduling for graph convolutional networks. IEEE Trans Circuits Syst I Regul Pap 68(12):5095–5107. https://doi.org/10.1109/TCSI.2021.3112826
Gao Q, Zeng H, Li G, Tong T (2021) Graph reasoning-based emotion recognition network. IEEE Access 9:6488–6497. https://doi.org/10.1109/ACCESS.2020.3048693
Sun B, Zhao D, Shi X, He Y (2021) Modeling global spatial-temporal graph attention network for traffic prediction. IEEE Access 9:8581–8594. https://doi.org/10.1109/ACCESS.2021.3049556
Yu L et al (2021) STEP: aspatio-temporal fine-granular user traffic prediction system for cellular networks. IEEE Trans Mobile Comput 20(12):3453–3466. https://doi.org/10.1109/TMC.2020.3001225
Wang Y, Yan P, Gai M (2021) Dynamic soft sensor for anaerobic digestion of kitchen waste based on SGSTGAT. IEEE Sensors J 21(17):19198–19208. https://doi.org/10.1109/JSEN.2021.3090524
Buroni G, Lebichot B, Bontempi G (2021) AST-MTL: an attention-based multi-task learning strategy for traffic forecasting. IEEE Access 9:77359–77370. https://doi.org/10.1109/ACCESS.2021.3083412
Yang S, Li G, Yu Y (2021) Relationship-embedded representation learning for grounding referring expressions. IEEE Trans Pattern Analysis Mach Intell 43(8):2765–2779. https://doi.org/10.1109/TPAMI.2020.2973983
Luo J, Zhou D, Han Z, Xiao G, Tan Y (2021) Predicting travel demand of a docked bikesharing system based on LSGC-LSTM networks. IEEE Access 9:92189–92203. https://doi.org/10.1109/ACCESS.2021.3062778
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Pande, S.D., Kumaresan, T., Lanke, G.R., Degadwala, S., Dhiman, G., Soni, M. (2023). Bidirectional Attention Mechanism-Based Deep Learning Model for Text Classification Under Natural Language Processing. In: Balas, V.E., Semwal, V.B., Khandare, A. (eds) Intelligent Computing and Networking. IC-ICN 2023. Lecture Notes in Networks and Systems, vol 699. Springer, Singapore. https://doi.org/10.1007/978-981-99-3177-4_34
Download citation
DOI: https://doi.org/10.1007/978-981-99-3177-4_34
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-3176-7
Online ISBN: 978-981-99-3177-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)