Abstract
Test-based model generation by classical automata learning is very expensive. It requires an impractically large number of queries to the system, each of which must be implemented as a system-level test case. Key in the tractability of observation-based model generation are powerful optimizations exploiting different kinds of expert knowledge in order to drastically reduce the number of required queries, and thus the testing effort. In this paper, we present a thorough experimental analysis of the second-order effects between such optimizations in order to maximize their combined impact.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Angluin D (1987) Learning regular sets from queries and counterexamples. Inf Comput 2(75):87–106
Berg T, Grinchtein O, Jonsson B, Leucker M, Raffelt H, Steffen B (2005) On the correspondence between conformance testing and regular inference. In: Proceedings of FASE 2005, 8th international conference on fundamental approaches to software engineering. Edinburgh, UK, April 2005. Lecture notes in computer science, vol 3442. Springer, Berlin Heidelberg New York, pp 175–189
Brun Y, Ernst MD (2004) Finding latent code errors via machine learning over program executions. In: Proceedings of the 26th international conference on software engineering (ICSE’04), pp 480–490
Chow TS (1978) Testing software design modeled by finite-state machines. IEEE Trans Softw Eng 4(3):178–187
Cook JE, Du Z, Liu C, Wolf AL (2002) Discovering models of behavior for concurrent systems. Technical report, Dept of Computer Science, New Mexico State University, Las Cruces, NM
Cook JE, Wolf AL (1998) Discovering models of software processes from event-based data. ACM Trans Softw Eng Methodol pp 215–249
Ernst MD, Czeisler A, Griswold WG, Notkin D (2000) Quickly detecting relevant program invariants. In: Proceedings of the 22nd international conference on software engineering (ICSE 2000), pp 449–458
Fujiwara S, Bochmann Gv, Khendek F, Amalou M, Ghedamsi A (1991) Test selection based on finite state models. IEEE Trans Softw Eng 17(6):591–603
Groce A, Peled D, Yannakakis M (2002) Adaptive model checking. In: Proceedings of the 8th conference on tools and algorithms for the construction and analysis of systems (TACAS 2002). Lecture notes in computer science, vol 2280. Springer, Berlin Heidelberg New York, pp 357–370
Hagerer A, Hungar H, Niese O, Steffen B (2002) Model generation by moderated regular extrapolation. In: Proceedings of the 5th international conference on fundamental approaches to software engineering (FASE 2002). Lecture notes in computer science, vol 2306. Springer, Berlin Heidelberg New York, pp 80–95
Hungar H, Steffen B (2004) Behavior-based model construction. In: Int J Softw Tools Technol Transfer 6(1):4–14
Margaria T, Raffelt H, Steffen, B (2005) Analyzing second-order effects between optimizations for system-level test-based model generation. In: Proceedings of the IEEE international test conference (ITC), 8–10 November 2005. IEEE Press, Austin, TX
Niese O, Margaria T, Hagerer A, Brune G, Goerigk W, Ide H-D, Steffen B (2001) Automated regression testing of CTI-Systems. In: Proceedings of the IEEE European test workshop (ETW2001), May 2001, Stockholm, Sweden
Hagerer A, Margaria T, Niese O, Steffen B, Brune G, Ide H (2001) Efficient regression testing of CTI-systems: testing a complex call-center solution. In: Annual review of communication, vol 55. International Engineering Consortium (IEC)
Hungar H, Margaria T, Steffen B (2003) Test-based model generation for legacy systems. In: IEEE international test conference (ITC), Charlotte, NC, 30 September–2 October 2003
Hungar H, Niese O, Steffen B (2003) Domain-specific optimization in automata learning. In: Proceedings of the 15th international conference on computer aided verification. Lecture notes in computer science, vol 2725. Springer, Berlin Heidelberg New York, pp 315–327
Lee D, Yannakakis M (1996) Principles and methods of testing finite state machines – a survey. Proc IEEE 84(8):1090–1126
Margaria T, Niese O, Steffen B, Erochok A (2002) System level testing of virtual switch (Re-)configuration over IP. In: Proceedings of the IEEE European test workshop. IEEE Press, Corfu, Greece
Mariani L, Pezzè M (2004) A technique for verifying component-based software. In: Proceedings of the international workshop on test and analysis of component based systems (TACOS 2004), Barcelona
Nimmer JW, Ernst MD (2002) Automatic generation of program specifications. In: Proceedings of the 2002 international symposium on software testing and analysis (ISSTA 2002), pp 232–242
Peled D, Vardi MY, Yannakakis M (1999) Black box checking formal methods for protocol engineering and distributed systems, (FORTE/PSTV). Kluwer, Dordrecht, pp 225–240
Rivest RL, Schapire RE (1993) Inference of finite automata using homing sequences. Inf Comput 103:299–347
Sabnani K, Dahbura A (1988) A protocol test generation procedure. Comput Netw ISDN Syst 15(4):285–297
Steffen B, Hungar H (2003) Behavior-based model construction. In: Mukhopadhyay S, Zuck L (eds) Proceedings of the 4th international conference on verification, model checking and abstract interpretation. Lecture notes in computer science, vol 2575. Springer, Berlin Heidelberg New York
Vuong ST, Chan WYL, Ito MR (1990) The UIOv-method for protocol test sequence generation. In: Proceedings of the 2nd international workshop on protocol test systems. North-Holland, Amsterdam, pp 161–176
Xie T, Notkin D (2003) Mutually enhancing test generation and specification inference. In: Proceedings of the 3rd international workshop on formal approaches to testing of software (FATES 2003) Lecture notes in computer science, vol 2931. Springer, Berlin Heidelberg New York, pp 60–69
Margaria T, Niese O, Raffelt H, Steffen B (2004) Efficient test-based model generationfor legacy reactive systems. In: Proceedings of international high level design validation and test workshop, Sonoma, CA (in press)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Margaria, T., Raffelt, H. & Steffen, B. Knowledge-based relevance filtering for efficient system-level test-based model generation. Innovations Syst Softw Eng 1, 147–156 (2005). https://doi.org/10.1007/s11334-005-0016-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11334-005-0016-y