Abstract
Test case prioritization (TCP) is a widely accepted and extensively used strategy during regression testing. TCP is the permutation of test cases to enhance efficiency in achieving performance goals. These goals can belong to the category of single objective problem or multi-objective problem. This empirical study focuses on three objectives wherein two objectives are to be maximized and the remaining one minimized. During this study, three websites and various versions were created on which non-dominated sorting genetic algorithm-II and variant of non-dominated sorting artificial bee colony algorithm were applied to prioritize sequence of test cases. The problem size varies from small-size fault matrix \((34\times 27)\) to mid-size fault matrix \((157\times 128)\). Performance of the two algorithms was measured on various parameters and also verified on the basis of statistical testing. An alternate approach for solving this multi-objective problem, based on dynamic programming, is also proposed in this study, and it is concluded that performance of this algorithm is at par with other suggested ones.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Mathur, A.: Foundations of Software Testing, Seventh Impression. Pearson Education, London (2012)
Chauhan, N.: Software Testing Principles and Practices, 1st edn. Oxford University Press, Oxford (2010)
Singh, Y.: Software Testing, 1st edn. Cambridge University Press, Cambridge (2012)
Rothermal, G.; Untch, R.; Harrold, M.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng. 27(10), 929–948 (2001)
Malishevsky, A.G.; Ruthruff, J.R.; Rothermel, G.; Elbaum, S.: Cost-cognizant test case prioritization. Technical Report TR-UNL-CSE-2006-0004, Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln (2006)
Harman, M.; Li, Z.; Hierons, R.: Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)
Elbaum, S.; Malishevsky, A.; Rothermel, G.: Incorporating varying test costs and fault severities into test case prioritization. In: Proceedings of the 23rd International Conference on Software Engineering, pp. 329–338 (2001)
Jiang, B.; Zhang, Z.; Chan, W.K.; Tse, T.H.: Adaptive random test case prioritization. In: ASE2009, 24th IEEE/ACM International Conference on Automated Software Engineering, pp. 233–244 (2009)
Noor, T.B.; Hemmati, H.: A similarity-based approach for test case prioritization using historical failure data. In: 26th International Symposium on Software Reliability Engineering ISSRE, pp. 58–68 (2015)
Arafeen, M.J.; Do, H.: Test case prioritization using requirements-based clustering. In: 6th International Conference on Software Testing, Verification and Validation, pp. 312–321 (2013)
Laali, M.; Liu, H.; Hamilton, M.; Spichkova, M.; Schmidt, H.: Test case prioritization using online fault detection information. In: Ada-Europe International Conference on Reliable Software Technologies, pp. 78–93 (2016)
Wang, S.; Nam, J.; Tan, L.: QTEP: quality-aware test case prioritization. In: Foundations of Software Engineering, pp. 523–534. ACM, New York (2017)
Mei, H.; Hao, D.; Zhang, L.; Zhou, J.; Rothermel, G.: A static approach to prioritizing Junit test cases. IEEE Trans. Softw. Eng. 38(6), 1258–1275 (2012)
Thomas, S.W.; Hemmati, H.; Hassan, A.E.; Blostein, D.: Static test case prioritization using topic models. Empir. Softw. Eng. 19(1), 182–212 (2012)
Mohanty, R.; Suman, S.; Das, S.K.: Modelling the pull-out capacity of ground anchors using multi-objective feature selection. Arab. J. Sci. Eng. 42(3), 1231–1241 (2017)
Shapiai, M.I.; Ibrahim, Z.; Adam, A.: Pareto optimality concept for incorporating prior knowledge for system identification problem with insufficient samples. Arab. J. Sci. Eng. 42(7), 2697–2710 (2017)
Zhang, Y.; Harman, M.; Mansouri, S.A.: The multi-objective next release problem. In: GECCO’07, pp. 1129–1137. ACM, London (2007)
Ruiz, M.; Roderiguez, D.; Riquelme, J.; Harrison, R.: Multi-objective Simulation Optimization in Software Project Management. Oxford Brookes University, Oxford (2011)
Wang, Z.; Tang, K.; Yao, X.: Multi-objective approaches to optimal testing resource allocation in modular software systems. IEEE Trans. Reliab. 59(3), 563–575 (2000)
Choudhary, K.; Purohit, G.: A Multi-objective optimization algorithm for uniformly distributed generation of test cases. In: IEEE International Conference on Computing for Sustainable Global Development, pp. 455–457 (2014)
Mondal, D.; Hemmati, H.; Durocher, S.: Exploring test suite diversification and code coverage in multi-objective test case selection. In: IEEE Conference on Software Testing, Verification and Validation, pp. 1–10 (2015)
Yoo, S.; Harman, M.: Pareto efficient multi-objective test case selection. In: ISSTA 2007, pp. 140–150. ACM, London (2007)
Marchetto, A.; Islam, M.; Scanniello, G.; Susi, A.: A multi-objective technique for test suite reduction. In: The 8th International Conference on Software Engineering Advances. IARIA (2013)
Zheng, W.; Hierons, R.; Li, M.; Liu, X.; Vinciotti, V.: Multi-objective optimization for regression testing. Inf. Sci. 334, 1–16 (2015)
Canfora, G.; Lucia, A.D.; Penta, M.D.; Oliveto, R.; Panichella, A.; Panichella, S.: Defect prediction as a multi-objective optimization problem. Softw. Test. Verif. Reliab. 25(4), 426–459 (2015)
Marchetto, A.; Islam, M.; Scanniello, G.; Asghar, W.; Susi, A.: A multi-objective technique to prioritize test cases. IEEE Trans. Softw. Eng. 42(10), 918–940 (2016)
Karaboga, D.; Goremli, B.; Ozturk, C.; Karaboga, N.: A comprehensive survey: artificial bee colony (ABC) and applications. Artif. Intell. Rev. 42(1), 21–57 (2014)
Karaboga, D.; Beyza, G.: A combinatorial artificial bee colony algorithm for travelling salesman problem. In: INISTA IEEE International Symposium, pp. 50–53 (2011)
Lam, S.S.B.; Raju, M.LH.P.; Kiran, U.M.; Swaraj, C.; Srivastava, P.R.: Automated generations of independent paths and test suite optimization using artificial bee colony. In: ICCTSD2011, Procedia Engineering 30, pp. 191–200 (2012)
Chong, C.S.; Low, M.Y.H.; Sivakumar, A.I, Lay.: A bee colony optimization for job shop scheduling. In: IEEE Proceedings of the 38th Conference on Winter Simulation, pp. 1954–1961 (2006)
Kaur, A.; Goyal, S.: A bee colony optimization algorithm for code coverage test suite prioritization. IJEST 3(4), 2786–2795 (2011)
Srikanth; Kulkarni, N.J.; Naveen, K.V.; Singh, P.; Srivastava, P.R.: Test case optimization using artificial bee colony algorithm. In: International Conference on Advances in Computing and Communication, pp. 570–579 (2011)
Joseph, A.K.; RadhaMani, G.: A hybrid model of particle swarm optimization and artificial bee colony algorithm for test case optimization. IJCSE 3(5), 459–471 (2011)
Mala, D.J.; Mohan, V.; kamalapriya, M.: Automated software test optimization framework and artificial bee colony optimization based approach. IET Softw. 4(5), 334–348 (2010)
Dahiya, S.K.; Chhabra, J.K.; Kumar, S.: Application of artificial bee colony algorithm to software testing. In: 21st IEEE Australian Software Engineering Conference, pp. 149–154 (2010)
Konsaard, P.; Ramingwong, L.: Using artificial bee colony for code coverage based test suite prioritization. In: 2nd International Conference on Information Science and Security, pp. 1–4 (2015)
Aghdam, Z.K.; Arasteh, B.: An efficient method to generate test data for software structural testing using artificial bee colony optimization algorithm. Int. J. Softw. Eng. Knowl. Eng. 27(6), 951–966 (2017)
Li, X.; Li, Z.; Lin, L.: An artificial bee colony algorithm for multi-objective optimization. In: 2nd International Conference on Intelligent Systems Design and Engineering Application, pp. 153–156 (2012)
Amarjeet, P.; Chhabra, J.K.: Many-objective artificial bee colony algorithm for large-scale software module clustering problem. Soft Comput. 22(19), 6341–6361 (2017). https://doi.org/10.1007/s00500-017-2687-3
Mann, M.; Tomar, P.; Sangwan, O.P.: Bio-inspired meta heuristics: evolving and prioritizing software test data. Appl. Intell. 48(3), 687–702 (2017). https://doi.org/10.1007/s10489-017-1003-3
Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)
Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms, 1st edn. Wiley, London (2010)
Coreman, T.H.: Introduction to Algorithms, 2nd edn. PHI Printing Press, New York (2001)
Elbaum, S.; Rothermal, G.; Karre, S.; FisherII, M.: Leveraging user-session data to support web application testing. IEEE Trans. Softw. Eng. 31(3), 187–202 (2005)
Elbaum, S.; Malishevsky, A.G.; Rothermal, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)
Hutchins, M.; Foster, H.; Goradia, T.; Ostrand, T.: Experiments on the effectiveness of dataflow and control flow based test adequacy criteria. In: International Conference Software Engineering, pp. 191–200 (1994)
Wong, W.; Horgan, J.; London, S.; Mathur, A.: Effect of test set minimization on fault detection effectiveness. In: Proceedings 17th International Conference on Software Engineering, pp. 41–50 (1995)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Khanna, M., Chaudhary, A., Toofani, A. et al. Performance Comparison of Multi-objective Algorithms for Test Case Prioritization During Web Application Testing. Arab J Sci Eng 44, 9599–9625 (2019). https://doi.org/10.1007/s13369-019-03817-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13369-019-03817-7