Abstract
Software testing is an important and expensive activity to the software industry, with testing accounting for over 50% of the cost of software. To ease this problem, test automation is very critical to the process of software testing. One important issue in this automation is to automatically determine whether a program under test (PUT) responds the correct(expected) output for an arbitrary input. In this paper, we model PUTs in black-box way, i.e. processing and responding a list of numbers, and design input/output list relation language(IOLRL) to formally describe the relations between the input and output lists. Given several labelled test cases(test verdicts are set), we use genetic programming to evolve the most distinguishing relations of these test cases in IOLRL and encode the test cases into bit patterns to build a classifier with support vector machine as the constructed test oracle. This classifier can be used to automatically verify if a program output list is the expected one in processing a program input list. The main contribution of this work are the designed IOLRL and the approach to construct test oracle with evolve relations in IOLRL. The experiments show the constructed test oracle has good performance even when few labelled test cases are supplied.
Chapter PDF
Similar content being viewed by others
Keywords
References
Brown, D.B., Roggio, R.F., Cross II, J.H., McCreary, C.L.: An automated oracle for software testing. IEEE Transactions on Reliability 41(2), 272–280 (1992)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2001), Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Chen, T.Y., Ho, J.W.K., Liu, H., Xie, X.: An innovative approach for testing bioinformatics programs using metamorphic testing. BMC Bioinformatics 10 (2009)
Dustin, E., Rashka, J., Paul, J.: Automated software testing: introduction, management, and performance. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)
Edvardsson, J.: A survey on automatic test data generation. In: Proceedings of the Second Conference on Computer Science and Engineering in Linköping, October 1999. ECSEL, pp. 21–28 (1999)
Jones, E.L., Chatmon, C.L.: A perspective on teaching software testing. In: Proceedings of the seventh annual consortium for computing in small colleges central plains conference on The journal of computing in small colleges, USA, pp. 92–100. Consortium for Computing Sciences in Colleges (2001)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
Peters, D.K., Parnas, D.L.: Using test oracles generated from program documentation. IEEE Transactions on Software Engineering 24(3), 161–173 (1998)
Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming (2008), Published via, http://lulu.com , and freely available at http://www.gp-field-guide.org.uk (With contributions by J. R. Koza)
Vanmali, M., Last, M., Kandel, A.: Using a neural network in the software testing process. Int. J. Intell. Syst. 17(1), 45–62 (2002)
Vapnik, V.N.: Statistical Learning Theory. Wiley Interscience, Hoboken (1998)
Walker, M.: Introduction to genetic programming (2001)
Wang, Y.-C.F., Casasent, D.: A hierarchical classifier using new support vector machine. In: ICDAR, pp. 851–855. IEEE Computer Society Press, Los Alamitos (2005)
Weinbrenner, T.: Genetic programming techniques applied to measurement data. Diploma Thesis (February 1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, F., Wu, JH., Huang, CH., Chang, KH. (2011). Evolving a Test Oracle in Black-Box Testing. In: Giannakopoulou, D., Orejas, F. (eds) Fundamental Approaches to Software Engineering. FASE 2011. Lecture Notes in Computer Science, vol 6603. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19811-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-19811-3_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19810-6
Online ISBN: 978-3-642-19811-3
eBook Packages: Computer ScienceComputer Science (R0)