Abstract
In the professional field of software development, unit tests are often used as a verification mechanism of the code produced. With the design and execution of test cases, it is possible to verify new or modified code, as well as to verify existing versions of it. In order to prevent new defects from spreading over existing versions of the code, it is necessary to strategically re-run different test cases. This activity is known as software regression testing and can constitute a high percentage of the total cost of the verification process. Researchers continuously are dealing with making software regression testing efficient. In order to shed light on the application of software regression testing in industry, this paper presents a literature review on different aspects of regression testing applied in industrial settings. As a result, 40 primary studies that report the use of regression testing in the industry were identified. We observe that the main regression testing technique used is the selection of test cases followed by the prioritization of them. The use of combinations of metrics based on coverage, requirements, risk, defects, cost-efficiency searches is also observed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Rothermel, G., Harrold, M.J.: A safe, efficient algorithm for regression test selection. In: Proceedings of the Conference on Software Maintenance, 1993, CSM-1993, pp. 358–367 (1993)
Leung, H.K.N., White, L.: Insights into regression testing [software testing]. In: Proceedings Conference on Software Maintenance - 1989. pp. 60–69 (1989)
Beizer, B.: Software Testing Techniques, 2Nd edn. Van Nostrand Reinhold Co., New York (1990)
Humble, J., Farley, D.: Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Pearson Education, Boston (2010)
Rosero, R.H., Gómez, O.S., Rodríguez, G.: 15 years of software regression testing techniques — a survey. Int. J. Softw. Eng. Knowl. Eng.. 26, 675–689 (2016)
Rosero, R.H., Gómez, O.S., Rodríguez, G.: Regression testing of database applications under an incremental software development setting. IEEE Access. 5, 18419–18428 (2017)
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering – a systematic literature review. Inf. Softw. Technol. 51, 7–15 (2009)
Rothermel, G., Harrold, M.J.: Empirical studies of a safe regression test selection technique. IEEE Trans. Softw. Eng. 24, 401–419 (1998)
Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verif. Reliab. 22, 67–120 (2012)
Rothermel, G., Harrold, M.J.: Analyzing regression test selection techniques. IEEE Trans. Softw. Eng. 22, 529–551 (1996)
Rothermel, G., Untch, R.H., Chu, C., Harrold, M.J.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng.. 27, 929–948 (2001)
Mohanty, R., Ravi, V., Patra, M.R.: The application of intelligent and soft-computing techniques to software engineering problems: a review. Int. J. Inf. Decis. Sci. 2, 233–272 (2010)
Santos, A., Gómez, O., Juristo, N.: Analyzing families of experiments in SE: a systematic mapping study. IEEE Trans. Softw. Eng. 46, 566–583 (2020)
Dieste, O., Grimán, A., Juristo, N.: Developing search strategies for detecting relevant experiments. Empir. Softw. Eng. 14, 513–539 (2009)
Lübke, D.: Selecting and prioritizing regression test suites by production usage risk in time-constrained environments. In: Winkler, D., Biffl, S., Mendez, D., Bergsmann, J. (eds.) SWQD 2020. LNBIP, vol. 371, pp. 31–50. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-35510-4_3
Magalhães, C., Andrade, J., Perrusi, L., Mota, A., Barros, F., Maia, E.: HSP: a hybrid selection and prioritisation of regression test cases based on information retrieval and code coverage applied on an industrial case study. J. Syst. Softw. 159, 110430 (2020)
Wu, Z., Yang, Y., Li, Z., Zhao, R.: A time window based reinforcement learning reward for test case prioritization in continuous integration. In: Proceedings of the 11th Asia-Pacific Symposium on Internetware. pp. 1–6. Association for Computing Machinery, New York (2019)
Pal, D., Vain, J.: A systematic approach on modeling refinement and regression testing of real-time distributed systems. IFAC-PapersOnLine. 52, 1091–1096 (2019)
Correia, D.: An industrial application of test selection using test suite diagnosability. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1214–1216. Association for Computing Machinery, New York (2019)
Marijan, D., Gotlieb, A., Liaaen, M.: A learning algorithm for optimizing continuous integration development and testing practice. Softw. Pract. Exp. 49, 192–213 (2019)
Ali, S., Hafeez, Y., Hussain, S., Yang, S.: Enhanced regression testing technique for agile software development and continuous integration strategies. Softw. Qual. J. 28(2), 397–423 (2019). https://doi.org/10.1007/s11219-019-09463-4
Wang, X., Zeng, H., Gao, H., Miao, H., Lin, W.: Location-based test case prioritization for software embedded in mobile devices using the law of gravitation. Mobile Inf. Syst. 2019, e9083956 (2019)
Tulasiraman, M., Vivekanandan, N., Kalimuthu, V.: Multi-objective test case prioritization using improved Pareto-optimal clonal selection algorithm. 3D Res. 9(3), 1–13 (2018). https://doi.org/10.1007/s13319-018-0182-y
Aman, H., Nakano, T., Ogasawara, H., Kawahara, M.: A topic model and test history-based test case recommendation method for regression testing. In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). pp. 392–397 (2018)
Akin, A., Sentürk, S., Garousi, V.: Transitioning from manual to automated software regression testing: experience from the banking domain. In: 2018 25th Asia-Pacific Software Engineering Conference (APSEC), pp. 591–597 (2018)
Marijan, D., Liaaen, M.: Practical selective regression testing with effective redundancy in interleaved tests. In: Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice, pp. 153–162. Association for Computing Machinery, New York (2018)
Araújo, J., Araújo, J., Magalhães, C., Andrade, J., Mota, A.: Feasibility of using source code changes on the selection of text-based regression test cases. In: Proceedings of the 2nd Brazilian Symposium on Systematic and Automated Software Testing, pp. 1–6. Association for Computing Machinery, New York (2017)
Aman, H., Nakano, T., Ogasawara, H., Kawahara, M.: A test case recommendation method based on morphological analysis, clustering and the Mahalanobis-Taguchi method. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 29–35 (2017)
Ramler, R., Salomon, C., Buchgeher, G., Lusser, M.: Tool support for change-based regression testing: an industry experience report. In: Winkler, D., Biffl, S., Bergsmann, J. (eds.) SWQD 2017. LNBIP, vol. 269, pp. 133–152. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49421-0_10
EmaShankari, K.H., ThirumalaiSelvi, R., Balasubramanian, N.V.: Industry based regression testing using IIGRTCP algorithm and RFT tool. In: Lecture Notes in Engineering and Computer Science, pp. 473–478. Newswood Limited (2016)
de Oliveira Neto, F.G., Torkar, R., Machado, P.D.L.: Full modification coverage through automatic similarity-based test case selection. Inf. Softw. Techn.. 80, 124–137 (2016)
Anderson, J., Salem, S., Do, H.: Striving for failure: an industrial case study about test failure prediction. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, pp. 49–58 (2015)
Horváth, F., et al.: Test suite evaluation using code coverage based metrics. Presented at the Proceedings of the 14th Symposium on Programming Languages and Software Tools (SPLST’15) October (2015)
Marijan, D.: Multi-perspective regression test prioritization for time-constrained environments. In: 2015 IEEE International Conference on Software Quality, Reliability and Security, pp. 157–162 (2015)
Arora, A., Chauhan, N.: A regression test selection technique by optimizing user stories in an agile environment. In: 2014 IEEE International Advance Computing Conference (IACC), pp. 1454–1458 (2014)
Fourneret, E., Cantenot, J., Bouquet, F., Legeard, B., Botella, J.: SeTGaM: Generalized technique for regression testing based on UML/OCL Models. In: 2014 Eighth International Conference on Software Security and Reliability (SERE), pp. 147–156 (2014)
Gligoric, M., Negara, S., Legunsen, O., Marinov, D.: An empirical evaluation and comparison of manual and automated test selection. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering, pp. 361–372. Association for Computing Machinery, New York (2014)
Anderson, J., Salem, S., Do, H.: Improving the effectiveness of test suite through mining historical data. In: Proceedings of the 11th Working Conference on Mining Software Repositories, pp. 142–151. ACM, New York (2014)
Marijan, D., Gotlieb, A., Sen, S.: Test case prioritization for continuous regression testing: an industrial case study. In: 2013 IEEE International Conference on Software Maintenance, pp. 540–543 (2013)
Xu, Z., Liu, Y., Gao, K.: A novel fuzzy classification to enhance software regression testing. In: 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), pp. 53–58 (2013)
Rogstad, E., Briand, L., Torkar, R.: Test case selection for black-box regression testing of database applications. Inf. Softw. Technol. 55, 1781–1795 (2013)
Buchgeher, G., Ernstbrunner, C., Ramler, R., Lusser, M.: Towards tool-support for test case selection in manual regression testing. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, pp. 74–79 (2013)
Prakash, N., Rangaswamy, T.R.: Modular based multiple test case prioritization. In: 2012 IEEE International Conference on Computational Intelligence and Computing Research, pp. 1–7 (2012)
Chen, W., et al.: Change impact analysis for large-scale enterprise systems. In: ICEIS (2012)
Srikanth, H., Cohen, M.B.: Regression testing in software as a service: an industrial case study. In: 2011 27th IEEE International Conference on Software Maintenance (ICSM), pp. 372–381 (2011)
Rogstad, E., Briand, L., Dalberg, R., Rynning, M., Arisholm, E.: Industrial experiences with automated regression testing of a legacy database application. In: 2011 27th IEEE International Conference on Software Maintenance (ICSM), pp. 362–371 (2011)
Engström, E., Runeson, P., Ljung, A.: Improving regression testing transparency and efficiency with history-based prioritization – an industrial case study. In: Verification and Validation 2011 Fourth IEEE International Conference on Software Testing, pp. 367–376 (2011)
Juergens, E., Hummel, B., Deissenboeck, F., Feilkas, M., Schlögel, C., Wübbeke, A.: Regression test selection of manual system tests in practice. In: 2011 15th European Conference on Software Maintenance and Reengineering, pp. 309–312 (2011)
Wikstrand, G., Feldt, R., Gorantla, J.K., Zhe, W., White, C.: Dynamic regression test selection based on a file cache an industrial evaluation. In: 2009 International Conference on Software Testing Verification and Validation, pp. 299–302 (2009)
Ramasamy, K., Arul Mary, S.: Incorporating varying requirement priorities and costs in test case prioritization for new and regression testing. In: 2008 International Conference on Computing, Communication and Networking, pp. 1–9 (2008)
Sneed, H.M.: Selective Regression testing of a host to DotNet migration. In: 2006 22nd IEEE International Conference on Software Maintenance, pp. 104–112 (2006)
Skoglund, M., Runeson, P.: A case study of the class firewall regression test selection technique on a large scale distributed software system. In: 2005 International Symposium on Empirical Software Engineering, 2005, p. 10 (2005)
White, L., Robinson, B.: Industrial real-time regression testing and analysis using firewalls. In: 20th IEEE International Conference on Software Maintenance, 2004. Proceedings, pp. 18–27 (2004)
Briand, L.C., Labiche, Y., Soccar, G.: Automating impact analysis and regression test selection based on UML designs. In: International Conference on Software Maintenance, 2002. Proceedings, pp. 252–261 (2002)
Acknowledgments
The authors thank the Escuela Superior Politécnica de Chimborazo, in particular to the Faculty of Informatics and Electronics for their support in carrying out this work. Also thank anonymous reviewers. César Pardo acknowledges the contribution of the Universidad del Cauca, where he works as an associate professor.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Annex A
Annex A
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rosero, R.H., Gómez, O.S., Villa, E.R., Aguilar, R.A., Pardo, C.J. (2022). Software Regression Testing in Industrial Settings: Preliminary Findings from a Literature Review. In: Botto-Tobar, M., S. Gómez, O., Rosero Miranda, R., Díaz Cadena, A., Montes León, S., Luna-Encalada, W. (eds) Trends in Artificial Intelligence and Computer Engineering. ICAETT 2021. Lecture Notes in Networks and Systems, vol 407. Springer, Cham. https://doi.org/10.1007/978-3-030-96147-3_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-96147-3_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96146-6
Online ISBN: 978-3-030-96147-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)