Abstract
Morphosyntactic processing of natural languages is mainly restricted by the lack of labelled data sets. Deep Learning methods proved their efficiency in domains such as imaging or acoustic process. Part-of-speech tagging is an important preprocessing step in many natural language processing applications. Despite much work already carried out in this field, there is still room for improvement, especially in Amazigh language. We propose here architectures based on neural networks and word embeddings, and that has achieved promising results in English. Furthermore, instead of extracting from the sentence a rich set of hand-crafted features which are the fed to a standard classification algorithm, we drew our inspiration from recent papers about the automatic extraction of word embeddings from large unlabelled data sets. On such embeddings, we expect to benefit from linearity and compositionality properties to improve our Amazigh POS Tagging system performances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Goldberg, Y., Levy, O.: Word2vec explained: Deriving Mikilov et al.’s negative-sampling word embedding metod. arXiv prepreint arXiv:1402.3722 (2014)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Manning, C.D.: Part-of-speech tagging from 97% to 100%: Is it time for some linguistics? In: Proceedings of the 12th International Conference on Computational Linguistics and Intelligent Text Processing, CICLing’11, pp. 171–189 (2011)
Bengio, Y.: Practical recommendations for gradient based training of deep architectures. In: Montavon, G., Orr, G.B., Müller, KR. (eds.) Neural Networks: Tricks of the Trade, volume 7700 of Lecture Notes in Computer Science, pp. 437–478. Springer Berlin Heidelberg, (2012) ISBN 978-3- 642-35288-1
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Martin, J.H, Jurafsky, D.: Speech and Language Processing, International Edition (2010)
Van Guilder, L.: Automated Part of Speech Tagging: A Brief Overview. Handout for LING361, Georgetown University, (1995)
Nakagawa, T., Uchimoto, K.: A hybrid approach to word segmentation and pos tagging. In: The 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pp 217–220
Charniak, E.: Statistical Language Learning. MIT Press, Cambridge (1993)
Brill, E.: Transformation-Based Error-Driven Learning and Natural Language Processing: A Case Study in Part-of-Speech Tagging (1995)
Schmid, H.: Improvements in Part-of-speech tagging with an application to German. In: Proceedings of the ACL SIGDAT-Workshop, pp. 13–26. Academic Publishers, Dordrecht (1999)
Ratnaparkhi, A.: A Maximum entropy model for part-of-speech tagging. In: Proceedings of EMNLP, Philadelphia, USA (1996)
Kudo, T., Matsumoto, Y.: Use of Support Vector Learning for Chunk Identification (2000)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of ICML 2001, pp. 282–289 (2001)
Dahl, G.E., Yu, D., et al.: Context-dependent pre-trained deep neural networks for large vocabulary speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(1):30–42 (2012)
Boukhris, F., Boumalk, A., Elmoujahid, E., Souifi, H.: La nouvelle grammaire de l’amazighe. Rabat, Maroc: IRCAM, (2008)
Yoshua, B.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8):1798–1828 (2013)
Bengio, Y.: Learning Deep Architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781 (2013)
Collobert, R., Weston, J., et al.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 2493–2537 (2011)
Mikolov, T., Deoras, A., et al.: Empirical evaluation and combination of advanced language modeling techniques. INTERSPEECH (2011)
Chafiq, M.: [Forty four lessons in Amazigh]. éd. Arabo-africaines (1991)
Chaker, S.: Textes en linguistique berbère -introduction au domaine berbère, éditions du CNRS, pp 232–242 (1984)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Amri, S., Zenkouar, L. (2019). Neural Networks Architecture for Amazigh POS Tagging. In: Ezziyyani, M. (eds) Advanced Intelligent Systems for Sustainable Development (AI2SD’2018). AI2SD 2018. Advances in Intelligent Systems and Computing, vol 915. Springer, Cham. https://doi.org/10.1007/978-3-030-11928-7_86
Download citation
DOI: https://doi.org/10.1007/978-3-030-11928-7_86
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11927-0
Online ISBN: 978-3-030-11928-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)