Abstract
Generative Pre-trained Transformer 2 (GPT-2) has shown awe-inspiring effectiveness of pre-trained language models on several tasks, especially the generation of coherent text. Automatic stories generation represents a great research area that has rarely been studied in the past and presents a unique issue for artificial intelligence. For this reason, we present in this work a novel approach to automatic children’s stories generation based on Generative Pre-trained Transformer 2 to increase reading skills. In our implementation, we identified the Simple Transformers library, which is built like a wrapper around the famous Trans-formers library by Hugging Face. This allows us to train and evaluate Transformer models very quickly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Peters, M.E., et al.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Wang, A., Cho, K.: BERT has a mouth, and it must speak: Bert as a markov random field language model. arXiv preprint arXiv:1902.04094 (2019)
Lee, J.S., Hsiang, J.: Patent claim generation by fine-tuning OpenAI GPT-2. World Patent Inf. 62, 101983 (2020)
OpenAI, GPT-2 source code (n.d.). https://github.com/openai/gpt-2. Accessed 02 June 2019
Sutskever, I., Martens, J., Hinton, G.E.: Generating text with recurrent neural networks. In: ICML (2011)
Zhang, X., Lapata, M.: Chinese poetry generation with recurrent neural networks. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 670–680, October 2014
Santhanam, S.: Context based text-generation using LSTM networks. arXiv preprint arXiv:2005.00048 (2020)
Tran, V.K., Nguyen, L.M.: Semantic Refinement GRU-based neural language generation for spoken dialogue systems. In: Hasida, K., Pa, W. (eds.) PACLING 2017. CCIS, vol. 781, pp. 63–75. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-8438-6_6
Rönnqvist, S., Kanerva, J., Salakoski, T., Ginter, F.: Is multilingual BERT fluent in language generation? arXiv preprint arXiv:1910.03806 (2019)
Wu, X., Lv, S., Zang, L., Han, J., Hu, S.: Conditional BERT contextual augmentation. In: Rodrigues, J.M.F., et al. (eds.) ICCS 2019. LNCS, vol. 11539, pp. 84–95. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22747-0_7
How to Build OpenAI’s GPT-2: The AI That Was Too Dangerous to Release. https://blog.floydhub.com/gpt2/. Accessed 24 June 2021
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318, July 2002
Munkova, D., Hajek, P., Munk, M., Skalka, J.: Evaluation of machine translation quality through the metrics of error rate and accuracy. Procedia Comput. Sci. 171, 1327–1336 (2020)
Manaswini, S., Deepak, G., Santhanavijayan, A.: Knowledge driven paradigm for anomaly detection from tweets using gated recurrent units. In: Motahhir, S., Bossoufi, B. (eds.) ICDTA 2021. LNNS, vol. 211, pp. 145–154. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73882-2_14
Timmi, M., Jeghal, A., EL Garouani, S., Yahyaouy, A.: The review of objectives, methods, tools, and algorithms for educational data mining. In: Motahhir, S., Bossoufi, B. (eds.) ICDTA 2021. LNNS, vol. 211, pp. 177–188. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73882-2_17
Berrajaa, A., Ettifouri, E.H.: The recurrent neural network for program synthesis. In: Motahhir, S., Bossoufi, B. (eds.) ICDTA 2021. LNNS, vol. 211, pp. 77–86. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73882-2_8
Garouani, M., Chrita, H., Kharroubi, J.: Sentiment analysis of moroccan tweets using text mining. In: Motahhir, S., Bossoufi, B. (eds.) ICDTA 2021. LNNS, vol. 211, pp. 597–608. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73882-2_54
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fagroud, F.Z., Rachdi, M., Ben Lahmar, E.H. (2022). Automatic Story Generation: Case Study of English Children’s Story Generation Using GPT-2. In: Motahhir, S., Bossoufi, B. (eds) Digital Technologies and Applications. ICDTA 2022. Lecture Notes in Networks and Systems, vol 455. Springer, Cham. https://doi.org/10.1007/978-3-031-02447-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-02447-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02446-7
Online ISBN: 978-3-031-02447-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)