Abstract
With testing, a system is executed with a set of selected stimuli, and observed to determine whether its behavior conforms to the specification. Therefore, testing is a strategic activity at the heart of software quality assurance, and is today the principal validation activity in industrial context to increase the confidence in the quality of systems. This paper, summarizing the six hours lesson taught during the Summer School FOSAD’12, gives an overview of the test data selection techniques and provides a state-of-the-art about Model-Based approaches for security testing.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Linear Temporal Logic
- Equivalent Class
- Coverage Criterion
- System Under Test
- Security Assertion Markup Language
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
IEEE: IEEE Standard for Software and System Test Documentation. IEEE Std 829-2008 (2008)
Myers, G., Sandler, C., Badgett, T., Thomas, T.: The Art of Software Testing, 2nd edn. Wiley (2004) ISBN: 978-0-4714-6912-4
Dijkstra, E.: Notes on structured programming. Technical Report EWD249, Eindhoven University of Technology (1970)
Tretmans, J.: Model Based Testing with Labelled Transition Systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008)
Clarke, L.A., Podgurski, A., Richardson, D.J., Zeil, S.J.: A formal evaluation of data flow path selection criteria. IEEE Transactions on Software Engineering 15(11), 1318–1332 (1989)
Zhu, H., Hall, P., May, J.: Software Unit Test Coverage and Adequacy. ACM Computing Surveys 29(4), 366–427 (1997)
Vilkomir, S., Bowen, J.: Formalization of software testing criteria using the Z notation. In: Proceedings of the 25th International Conference on Computer Software and Applications (COMPSAC 2001), Chicago, USA. IEEE Computer Society Press (October 2001)
Offutt, A., Xiong, Y., Liu, S.: Criteria for generating specification-based tests. In: Proceedings of the 5th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS 1999), pp. 119–131. IEEE Computer Society Press, Las Vegas (1999)
Chilenski, J., Miller, S.: Applicability of modified condition/decision coverage to software testing. Software Engineering Journal 9(5), 193–200 (1994)
RTCA Committee SC-167: Software considerations in airborne systems and equipment certification, 7th draft to Do-178B/ED-12A (July 1992)
Beizer, B.: Black-Box Testing: Techniques for Functional Testing of Software and Systems, 2nd edn. John Wiley & Sons, New York (1995)
Legeard, B., Kosmatov, N., Peureux, F., Utting, M.: Boundary Coverage Criteria for Test Generation from Formal Models. In: Proceedings of the 15th International Symposium on Software Reliability Engineering (ISSRE 2004), Saint-Malo, France, pp. 139–150. IEEE Computer Society Press (November 2004)
Prowell, S.J.: Jumbl: A tool for model-based statistical testing. In: HICSS, p. 337 (2003)
Bauer, T., Bohr, F., Landmann, D., Beletski, T., Eschbach, R., Poore, J.: From requirements to statistical testing of embedded systems. In: SEAS 2007: Proceedings of the 4th International Workshop on Software Engineering for Automotive Systems, p. 3. IEEE Computer Society, Washington, DC (2007)
le Guen, H., Marie, R.A., Thelin, T.: Reliability estimation for statistical usage testing using markov chains. In: ISSRE, pp. 54–65. Computer Society (2004)
Offutt, A., Liu, S., Abdurazik, A., Ammann, P.: Generating test data from state-based specifications. The Journal of Software Testing, Verification and Reliability 13(1), 25–53 (2003)
Bernard, E., Legeard, B., Luck, X., Peureux, F.: Generation of test sequences from formal specifications: GSM 11-11 standard case study. Software: Practice and Experience 34(10), 915–948 (2004)
Zhu, H., Belli, F.: Advancing test automation technology to meet the challenges of model-based software testing. Journal of Information and Software Technology 51(11), 1485–1486 (2009)
Utting, M., Legeard, B.: Practical Model-Based Testing - A tools approach. Elsevier Science (2006) ISBN 0 12 372501 1
Dias-Neto, A., Travassos, G.: A Picture from the Model-Based Testing Area: Concepts, Techniques, and Challenges. Advances in Computers 80, 45–120 (2010) ISSN: 0065-2458
Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Software Testing, Verification and Reliability 22(5), 297–312 (2012)
OMG: Sysml documentation, http://www.omgsysml.org/
Spivey, J.M.: The Z notation: a reference manual. Prentice Hall International (UK) Ltd., Hertfordshire (1992)
Abrial, J.R.: The B-Book. Cambridge University Press (1996)
Halbwachs, N., Caspi, P., Raymond, P., Pilaud, D.: The synchronous data flow programming language lustre. Proceedings of the IEEE 79(9), 1305–1320 (1991)
Bouquet, F., Grandpierre, C., Legeard, B., Peureux, F., Vacelet, N., Utting, M.: A subset of precise UML for model-based testing. In: A-MOST 2007, 3rd Int. Workshop on Advances in Model Based Testing, pp. 95–104. ACM Press (2007)
Bouquet, F., Grandpierre, C., Legeard, B., Peureux, F.: A test generation solution to automate software testing. In: 3rd Int. Workshop on Automation of Software Test, AST 2008, Leipzig, Germany, pp. 45–48. ACM Press (May 2008)
Schieferdecker, I., Großmann, J., Schneider, M.: Model-Based Security Testing. In: Proceedings of the 7th Int. Workshop on Model-Based Testing (MBT 2012), Tallinn, Estonia. EPTCS, vol. 80, pp. 1–12 (March 2012)
Tian-yang, G., Yin-sheng, S., You-yuan, F.: Research on Software Security Testing. World Academy of Science, Engineering and Technology 4(9), 572–576 (2010)
Wichers, D.: Owasp top 10 (October 2013), https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project (last visited: May 2014)
MITRE: Common weakness enumeration (October 2013), http://cwe.mitre.org/ (last visited: May 2014)
Whitehat: Website security statistics report (October 2013), https://www.whitehatsec.com/assets/WPstatsReport_052013.pdf (last visited: May 2014)
Felderer, M., Agreiter, B., Zech, P., Breu, R.: A classification for model-based security testing. In: The Third International Conference on Advances in System Testing and Validation Lifecycle, VALID 2011, pp. 109–114 (2011)
Lund, M.S., Solhaug, B., Stølen, K.: Model-Driven Risk Analysis: The CORAS Approach, 1st edn. Springer Publishing Company, Incorporated (2010)
Jürjens, J.: UMLsec: Extending UML for secure systems development. In: Jézéquel, J.-M., Hussmann, H., Cook, S. (eds.) UML 2002. LNCS, vol. 2460, pp. 412–425. Springer, Heidelberg (2002)
Jürjens, J.: Model-based security testing using UMLsec. Electron. Notes Theor. Comput. Sci. 220(1), 93–104 (2008)
Lodderstedt, T., Basin, D., Doser, J.: SecureUML: A UML-based modeling language for model-driven security. In: Jézéquel, J.-M., Hussmann, H., Cook, S. (eds.) UML 2002. LNCS, vol. 2460, pp. 426–441. Springer, Heidelberg (2002)
Bosik, B.S., Uyar, M.U.: Finite state machine based formal methods in protocol conformance testing: from theory to implementation. Computer Networks and ISDN Systems 22(1), 7–33 (1991); 9th IFIP TC-6 International Symposium on Protocol Specification, Testing and Verification
Fernandez, J.-C., Jard, C., Jeron, T., Viho, C.: An experiment in automatic generation of test suites for protocols with verification technology. Science of Computer Programming 29(1), 123–146 (1997)
Dadeau, F., Héam, P.C., Kheddam, R.: Mutation-based test generation from security protocols in HLPSL. In: Harman, M., Korel, B. (eds.) 4th Int. Conf. on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, pp. 240–248. IEEE Computer Society Press (March 2011)
Sutton, M., Greene, A., Amini, P.: Fuzzing: brute force vulnerability discovery. Pearson Education (2007)
Godefroid, P.: Random testing for security: blackbox vs. whitebox fuzzing. In: Proceedings of the 2nd International Workshop on Random Testing: Co-located with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007), pp. 1. ACM (2007)
Miller, B.P., Fredriksen, L., So, B.: An Empirical Study of the Reliability of UNIX Utilities. Commun. ACM 33(12), 32–44 (1990)
Takanen, A., DeMott, J., Miller, C.: Fuzzing for Software Security Testing and Quality Assurance. Artech House, Inc., Norwood (2008)
Duchene, F., Groz, R., Rawat, S., Richier, J.L.: XSS Vulnerability Detection Using Model Inference Assisted Evolutionary Fuzzing. In: Proc. of the 5th Int. Conference on Software Testing, Verification and Validation (ICST 2012), Montreal, Canada, pp. 815–817. IEEE CS (April 2012)
Schieferdecker, I.: Model-Based Fuzzing for Security Testing. Keynote talk at the 3rd International Workshop on Security Testing (SECTEST 2012), Montreal, Canada (April 2012)
Schneider, M., Großmann, J., Tcholtchev, N., Schieferdecker, I., Pietschker, A.: Behavioral Fuzzing Operators for UML Sequence Diagrams. In: Haugen, Ø., Reed, R., Gotzhein, R. (eds.) SAM 2012. LNCS, vol. 7744, pp. 88–104. Springer, Heidelberg (2013)
Abou El Kalam, A., El Baida, R., Balbiani, P., Benferhat, S., et al.: Organization based access control. In: Lutfiyya, H., Moffett, J., Garcia, F. (eds.) Policies for Distributed Systems and Networks (POLICY 2003), Como, January 01-December 31, pp. 120–131. Institute of Electrical and Electronics Engineers (2003)
Ribeiro, C., Zuquete, A., Ferreira, P., Guedes, P.: Spl: An access control language for security policies and complex constraints. In: NDSS, vol. 1 (2001)
Pnueli, A.: The temporal semantics of concurrent programs. Theoretical Computer Science 13, 45–60 (1981)
Tan, L., Sokolsky, O., Lee, I.: Specification-based testing with linear temporal logic. In: IEEE Int. Conf. on Information Reuse and Integration, IRI 2004, pp. 413–498 (November 2004)
Gargantini, A., Heitmeyer, C.: Using model checking to generate tests from requirements specifications. SIGSOFT Softw. Eng. Notes 24(6), 146–162 (1999)
Ammann, P.E., Black, P.E., Majurski, W.: Using model checking to generate tests from specifications. In: 2nd IEEE Int. Conf. on Formal Engineering Methods, ICFEM 1998, pp. 46–54. IEEE Computer Society Press (December 1998)
Jard, C., Jéron, T.: Tgv: theory, principles and algorithms: A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems. Int. J. Softw. Tools Technol. Transf. 7(4), 297–315 (2005)
Frantzen, L., Tretmans, J., Willemse, T.A.C.: Test generation based on symbolic specifications. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 1–15. Springer, Heidelberg (2005)
Clarke, D., Jéron, T., Rusu, V., Zinovieva, E.: STG: A symbolic test generation tool. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 151–173. Springer, Heidelberg (2002)
Tretmans, G.J., Brinksma, H.: TorX: Automated model-based testing. In: First European Conference on Model-Driven Software Engineering, Nuremberg, Germany, pp. 31–43 (December 2003)
Bigot, C., Faivre, A., Gallois, J.-P., Lapitre, A., Lugato, D., Pierron, J.-Y., Rapin, N.: Automatic test generation with AGATHA. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 591–596. Springer, Heidelberg (2003)
Aichernig, B.K., Weiglhofer, M., Wotawa, F.: Improving fault-based conformance testing. Electron. Notes Theor. Comput. Sci. 220, 63–77 (2008)
Bertolino, A., Marchetti, E., Muccini, H.: Introducing a reasonably complete and coherent approach for model-based testing. Electron. Notes Theor. Comput. Sci. 116, 85–97 (2005)
Basanieri, F., Bertolino, A., Marchetti, E.: The Cow_Suite approach to planning and deriving test suites in UML projects. In: Jézéquel, J.-M., Hussmann, H., Cook, S. (eds.) UML 2002. LNCS, vol. 2460, pp. 383–397. Springer, Heidelberg (2002)
Felderer, M., Breu, R., Chimiak-Opoka, J., Breu, M., Schupp, F.: Concepts for Model-based Requirements Testing of Service Oriented Systems. In: Proceedings of the IASTED International Conference, vol. 642, p. 018 (2009)
Fourneret, E., Ochoa, M., Bouquet, F., Botella, J., Jurjens, J., Yousefi, P.: Model-based security verification and testing for smart-cards. In: 6th International Conference on Availability, Reliability and Security, ARES 2011, pp. 272–279. IEEE (2011)
Ledru, Y., du Bousquet, L., Maury, O., Bontron, P.: Filtering TOBIAS combinatorial test suites. In: Wermelinger, M., Margaria-Steffen, T. (eds.) FASE 2004. LNCS, vol. 2984, pp. 281–294. Springer, Heidelberg (2004)
Ledru, Y., Dadeau, F., Du Bousquet, L., Ville, S., Rose, E.: Mastering combinatorial explosion with the TOBIAS-2 test generator. In: ASE 2007: Procs of the 22nd IEEE/ACM Int. Conf. on Automated Software Engineering, pp. 535–536 (2007)
Maury, O., Ledru, Y., du Bousquet, L.: Intégration de TOBIAS et UCASTING pour la génération des tests. In: 16th Int. Conf. on Software and Systems Engineering and their Applications, ICSSEA 2003, Paris, France (2003)
Van Aertryck, L., Jensen, T.: UML-CASTING: Test synthesis from UML models using constraint resolution. In: AFADL 2003 (2003)
Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: 21st International Conference on Software Engineering, ICSE 1999, Los Angeles, California, United States, pp. 411–420 (1999)
Castillos, K.C., Dadeau, F., Julliand, J., Kanso, B., Taha, S.: A compositional automata-based semantics for property patterns. In: Johnsen, E.B., Petre, L. (eds.) IFM 2013. LNCS, vol. 7940, pp. 316–330. Springer, Heidelberg (2013)
Botella, J., Cao, P., Civeit, C., Gidoin, D., Peureux, F.: Model-Based Test Generation of Aircraft Traffic Attack Scenarios using ADS-B Standard Signals. In: 1-st User Conference on Advanced Automated Testing, UCAAT 2013, Paris, France (October 2013)
Botella, J., Bouquet, F., Capuron, J.F., Lebeau, F., Legeard, B., Schadle, F.: Model-Based Testing of Cryptographic Components – Lessons Learned from Experience. In: Proc. of the 6th Int. Conference on Software Testing, Verification and Validation (ICST 2013), Luxembourg, pp. 192–201. IEEE CS (March 2013)
Lebeau, F., Legeard, B., Peureux, F., Vernotte, A.: Model-Based Vulnerability Testing for Web Applications. In: Proc. of the 4th Int. Workshop on Security Testing (SECTEST 2013), Luxembourg, pp. 445–452. IEEE CS Press (March 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Bouquet, F., Peureux, F., Ambert, F. (2014). Model-Based Testing for Functional and Security Test Generation. In: Aldini, A., Lopez, J., Martinelli, F. (eds) Foundations of Security Analysis and Design VII. FOSAD FOSAD 2013 2012. Lecture Notes in Computer Science, vol 8604. Springer, Cham. https://doi.org/10.1007/978-3-319-10082-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-10082-1_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10081-4
Online ISBN: 978-3-319-10082-1
eBook Packages: Computer ScienceComputer Science (R0)