Abstract
In this work, we attempt to extract Deterministic Finite Automata (DFA) for a set of regular grammars from sequential Recurrent Neural Networks (RNNs). We have considered Long Short-Term Memory (LSTM) architecture, which is a variant of RNN. We have classified a set of regular grammars by considering their imbalances in terms of strings they accept and the strings they reject by using an LSTM architecture. We have formulated a set of the extended Tomita Grammar by adding a few more regular grammars. The different imbalance classes we introduce are Nearly Balanced (NB), Mildly Imbalanced (MI), Highly Imbalanced (HI), Extremely Imbalanced (EI). We have used L* algorithm for DFA extraction from LSTM networks. As a result, we have shown the performance of training an LSTM architecture for extraction of DFA in the context of the imbalances for a set of so formed regular grammars. We were able to extract correct minimal DFA for various imbalanced classes of regular grammar, though in some cases, we could not extract minimal DFA from the Network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Angluin D (1987) Learning regular sets from queries and counterexamples. Inf Comput 75(2):87–106
Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166
Cechin AL, Regina D, Simon P, Stertz K (2003) State automata extraction from recurrent neural nets using \(k\)-means and fuzzy clustering. In: Proceedings of 23rd international conference Chilean Computer Science Society (SCCC), IEEE, pp 73–78
Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555
Elman JL (1990) Finding structure in time. Cognitive Sci 14(2):179–211
Giles C, Sun GZ, Chen HH, Lee YC, Chen D (1989) Higher order recurrent networks and grammatical inference
Goudreau MW, Giles CL, Chakradhar ST, Chen D (1994) First-order versus second-order single-layer recurrent neural networks. IEEE Trans Neural Netw 5(3):511–513
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Jacobsson H (2005) Rule extraction from recurrent neural networks: a taxonomy and review. Neural Comput 17(6):1223–1263
Omlin CW, Giles CL (1996) Extraction of rules from discrete-time recurrent neural networks. Neural Netw 9(1):41–52
Omlin C, Giles C, Miller C (1992) Heuristics for the extraction of rules from discrete-time recurrent neural networks. In: Proceedings of International Joint Conf Neural Networks (IJCNN), vol 1. IEEE, pp 33–38
Tomita M (1982) Dynamic construction of finite-state automata from examples using hill-climbing. In: Proceedings of 4th annual conference cognitive science society, pp 105–108
Weiss G, Goldberg Y, Yahav E (2018) Extracting automata from recurrent neural networks using queries and counterexamples. In: Proceedings of International Conference Machine Learning (ICML), PMLR, pp 5247–5256
Acknowledgements
We thank the anonymous reviewers for their valuable feedback by which the readability of the paper is improved.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sharma, A., Kumar, R. (2023). Imbalanced Learning of Regular Grammar for DFA Extraction from LSTM Architecture. In: Thakur, M., Agnihotri, S., Rajpurohit, B.S., Pant, M., Deep, K., Nagar, A.K. (eds) Soft Computing for Problem Solving. Lecture Notes in Networks and Systems, vol 547. Springer, Singapore. https://doi.org/10.1007/978-981-19-6525-8_8
Download citation
DOI: https://doi.org/10.1007/978-981-19-6525-8_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-6524-1
Online ISBN: 978-981-19-6525-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)