Abstract
In this paper we present a novel neural networks ensemble to solve the task of coreference resolution in Russian texts. The ensemble consists of several neural networks, each based on recurrent Bidirectional long short-term memory layers (BiLSTM), attention mechanism, consistent scoring with selection of probable mentions and antecedents. The applied neural network topology has already shown state-of-the-art results in English for this task, and is now adapted for the Russian language. The resulting coreference markup is obtained by aggregating output scores from several blocks of independently trained neural network models. To represent an input source text, a combination of word vectors from two language models is used. We study the dependence of the coreference detection accuracy on various combinations of models of vector representation of words along with two tokenization approaches: gold markup or UDpipe tools. Finally, to show the improvement made by our ensemble approach, we present the results of experiments with both RuCor and AnCor datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The metric is UAS-F1, dataset: “en\_ewt”, link: www.universaldependencies.org/conll18/results-uas.html.
- 2.
Baseline implementation were taken from www.github.com/kentonl/e2e-coref.
- 3.
- 4.
- 5.
- 6.
- 7.
References
Lee, K., He, L., Zettlemoyer, L.: Higher-order coreference resolution with coarse-to- fine inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 2, pp. 687–692. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Che, W., Liu, Y., Wang, Y., Zheng, B., Liu, T.: Towards better UD parsing: Deep contextualized word embeddings, ensemble, and treebank concatenation. In: Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 55–64. Association for Computational Linguistics, Brussels, Belgium (2018)
Lim, K.T, Park, C., Lee, C., Poibeau, T.: SEx BiST: A multi-source trainable parser with deep contextualized lexical representations. In: Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 143–152. Association for Computational Linguistics, Brussels, Belgium (2018)
Zeman, D., Hajič, J. (edt.).: The SIGNLL conference on computational natural language learning. In: Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics, Brussels, Belgium (2018)
Moosavi, N.S., Strube, M.: Which coreference evaluation metric do you trust? A proposal for a link-based entity aware metric. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 632–642. Association for Computational Linguistics, Berlin, Germany (2016)
Toldova, S., Ionov, M.: Coreference resolution for Russian: the impact of semantic features. In: Proceedings of the 2017 International Conference Computational Linguistics and Intellectual Technologies, Dialogue-2017, pp. 339–348. Russian State University for the Humanitie, Moscow, Russia (2017)
Toldova, S., Roytberg, A., Ladygina, A.A., Vasilyeva, M.D., Azerkovich, I.L., Kurzukov, M., Sim, G., Gorshkov, D.V., Ivanova, A., Nedoluzhko, A., Grishina, Y.: Evaluating anaphora and coreference resolution for Russian. In: Proceedings of the 2014 International Conference Computational Linguistics and Intellectual Technologies, Dialogue-2014, pp. 681–695. Russian State University for the Humanitie, Moscow, Russia (2014)
Kutuzov, A., Andreev, I.: Texts in, meaning out: neural language models in semantic similarity task for Russian. In: Proceedings of the 2015 International Conference Computational Linguistics and Intellectual Technologies, Dialogue-2015, pp. 143–154. Russian State University for the Humanitie, Moscow, Russia (2015)
Loukachevitch, N., Dobrov, B., Chetviorkin, I.: RuThes-lite, a publicly available version of thesaurus of Russian language RuThes. In: Proceedings of the 2014 International Conference Computational Linguistics and Intellectual Technologies, Dialogue-2014, pp. 340–350. Russian State University for the Humanitie, Moscow, Russia (2014)
Sysoev, A., Andrianov, I., Khadzhiiskaia, A.: Coreference resolution in Russian: state-of-the-art approaches application and evolvement. In: Computational Linguistics and Intellectual Technologies, Dialogue-2017, pp. 327–338. Russian State University for the Humanitie, Moscow, Russia (2017)
Segond, M., Borgelt, C.: Item set mining based on cover similarity. In: Huang, J.Z., Cao, L., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2011. LNCS, vol. 6635, pp. 493–505. Springer, Berlin, Heidelberg (2011)
Pradhan, S., Moschitti, A., Xue, N., Uryupina, O., Zhang, Y.: CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In: Proceedings of the Joint Conference on EMNLP and CoNLL: Shared Task, CoNLL 2012, pp. 1–40. Association for Computational Linguistics, Jeju, Korea (2012)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)
Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. In: Proceedings of NAACL-HLT 2018, 2227–2237. Association for Computational Linguistics, New Orleans, Louisiana, USA (2018)
Straka, M., Straková, J.: Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe. In: Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 88–99. Association for Computational Linguistics, Vancouver, Canada (2017)
Acknowledgements
The work was supported by the NRC “Kurchatov Institute” (No. 1359 by the 25.06.2019) and carried out using computing resources of the federal collective usage center Complex for Simulation and Data Processing for Mega-science Facilities at NRC “Kurchatov Institute”, http://ckp.nrcki.ru/.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Sboev, A., Rybka, R., Gryaznov, A. (2020). Deep Neural Networks Ensemble with Word Vector Representation Models to Resolve Coreference Resolution in Russian. In: Misyurin, S., Arakelian, V., Avetisyan, A. (eds) Advanced Technologies in Robotics and Intelligent Systems. Mechanisms and Machine Science, vol 80. Springer, Cham. https://doi.org/10.1007/978-3-030-33491-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-33491-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33490-1
Online ISBN: 978-3-030-33491-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)