Abstract
Determining the semantic similarity of any two arbitrary sentences requires two steps, i.e. sentence encoding and semantic similarity measure. The most important step is to encode a set of sentences into a set of equal-length vectors for similarity measure in forms of classification. Two practical encoding schemes had been proposed, statistical-based direct encoding and pretrained encoding. The first approach lacks considering word correlation and the dimension of encoded vector is very large. For the second approach, it requires an extra training time prior to the classification process. This study compromises the previous approaches by considering shallow neural networks for encoding sentences and classifying entailment relations between two sentences. A set of y-x-y encoder models is proposed where y can be greater or less than x depending on given dataset. Neither encoder models nor their corresponding classifiers are built upon big and complex structure, and hence is suitable for carrying out such task. The encoding scheme is tested with SICK 2014 dataset [1], specially designed for neutral, entailment, and contradiction sentence pairs. Comparison results (neutral 97.1%, entailment 91.1%, contradiction 94.6%) support the possibility of the proposed scheme to sentence similarity measure.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Marco, M., et al.: SemEval-2014 task 1: evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In: Proceedings of the SemEval (2014)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Devlin, J., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. Association for Computational Linguistics, Minneapolis, Minnesota (2019)
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)
Jiao, X., et al.: TinyBERT: distilling BERT for natural language understanding. arXiv preprint arXiv:1909.10351 (2019)
Sanh, V., et al.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)
Li, B., et al.: On the sentence embeddings from pre-trained language models. arXiv preprint arXiv:2011.05864 (2020)
Mikolov, T., et al.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Peters, M.E., et al.: Deep contextualized word representations. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Mikolov, T., et al.: Advances in pre-training distributed word representations. arXiv preprint arXiv:1712.09405 (2017)
Bowman, S.R., et al.: A large annotated corpus for learning natural language inference. Association for Computational Linguistics, Portugal (2015)
Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)
Chen, Z., Gao, Q., Moss, L.S.: NeuralLog: natural language inference with joint neural and logical reasoning. Association for Computational Linguistics (2021)
Lai, A., Hockenmaier, J.: Illinois-LH: a denotational and distributional approach to semantics. Association for Computational Linguistics, Dublin, Ireland (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Jinnovart, T., Lursinsap, C. (2023). Y-X-Y Encoding for Identifying Types of Sentence Similarity. In: Wah, Y.B., Berry, M.W., Mohamed, A., Al-Jumeily, D. (eds) Data Science and Emerging Technologies. DaSET 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 165. Springer, Singapore. https://doi.org/10.1007/978-981-99-0741-0_37
Download citation
DOI: https://doi.org/10.1007/978-981-99-0741-0_37
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0740-3
Online ISBN: 978-981-99-0741-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)