Abstract
Debugging is one of the most expensive and challenging phases in the software development life-cycle. One important cost factor in the debugging process is the time required to analyze failures and find underlying faults. Two types of techniques that can help developers to reduce this analysis time are Failure Clustering and Automated Fault Localization. Although there is a plethora of these techniques in the literature, there are still some gaps that prevent their operationalization in real-world contexts. Besides, the abundance of these techniques confuses the developers in selecting a suitable method for their specific domain. In order to help developers in reducing analysis time, we propose methodologies and techniques that can be used standalone or in a form of a tool-chain. Utilizing this tool-chain, developers (1) know which data they need for further analysis, (2) are able to group failures based on their root causes, and (3) are able to find more information about the root causes of each failing group. Our tool-chain was initially developed based on state-of-the-art failure diagnosis techniques. We implemented and evaluated existing techniques. We built on and improved them where the results were promising and proposed new solutions where needed. The overarching goal of this study has been the applicability of techniques in practice.
Chapter PDF
Similar content being viewed by others
References
Abreu, R., Zoeteweij, P., Van Gemund, A.J.C.: On the accuracy of spectrum-based fault localization. In: Proceedings—Testing: Academic and Industrial Conference Practice and Research Techniques, TAIC PART-Mutation 2007 (2007)
Bundschuh Manfred, D.C.: Object-Oriented Metrics, pp. 241–255. Springer, Berlin (2008)
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 13(1), 21–27 (1967)
Defects4J Dissection. http://program-repair.org/defects4j-dissection/#!/bug/Lang/26. Accessed: 2021-10-05
DiGiuseppe, N., Jones, J.A.: Fault density, fault types, and spectra-based fault localization. Empir. Softw. Eng. 20, 928–967 (2015)
Golagha, M.: A framework for failure diagnosis. Ph.D. Thesis, Technical University of Munich, Germany (2020)
Golagha, M., Lehnhoff, C., Pretschner, A., Ilmberger, H.: Failure clustering without coverage. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2019 (2019)
Golagha, M., Pretschner, A., Briand, L.: Can we predict the quality of spectrum-based fault localization? pp. 4–15 (2020)
Golagha, M., Pretschner, A., Fisch, D., Nagy, R.: Reducing failure analysis time: an industrial evaluation. In: Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP ’17, pp. 293–302. IEEE Press, Piscataway (2017)
Golagha, M., Raisuddin, A.M., Mittag, L., Hellhake, D., Pretschner, A.: Aletheia: a failure diagnosis toolchain. In: 2018 IEEE/ACM 40th International Conference on Software Engineering Companion (2018)
Hatton, L.: Characterising the diagnosis of software failure. IEEE Software—SOFTWARE (2008)
Hogerle, W., Steimann, F., Frenkel, M.: More debugging in parallel. In: Proceedings—International Symposium on Software Reliability Engineering, ISSRE, pp. 133–143 (2014)
Jones, J.A., Bowring, J.F., Harrold, M.J.: Debugging in parallel. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis—ISSTA ’07, p. 16 (2007)
Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, ASE ’05, pp. 273–282. ACM, New York (2005)
Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are mutants a valid substitute for real faults in software testing? In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering—FSE 2014, pp. 654–665 (2014)
Khuri, A.I.: Introduction to Linear Regression Analysis, 5th edn. In: Montgomery, D.C., Peck, E.A., Vining, G.G. (eds.) International Statistical Review (2013)
Kochhar, P.S., Xia, X., Lo, D., Li, S.: Practitioners’ expectations on automated fault localization. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, pp. 165–176. ACM, New York (2016)
Lehnhoff, C.: Reducing failure analysis time: a data-driven approach. Master’s Thesis, Technical University of Munich (2019)
Liu, C., Han, J.: Failure proximity: a fault localization-based approach. In: Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering, SIGSOFT ’06/FSE-14, pp. 46–56. ACM, New York (2006)
Luce, R.D., Perry, A.D.: A method of matrix analysis of group structure. Psychometrika 14(2), 95–116 (1949)
Masri, W.: Chapter three—automated fault localization: advances and challenges, pp. 103–156. Elsevier, Amsterdam (2015)
McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. SE-2(4), 308–320 (1976)
Ochiai, A.: Zoogeographical studies on the soleoid fishes found in Japan and its neighbouring regions-II. Nippon Suisan Gakkaishi 22, 526–530 (1957)
Pang-Ning, T., Steinbach, M., Kumar, V.: Introduction to Data Mining (2006)
Papadimitriou, C.H., Steiglitz, K.: Combinatorial Optimization: Algorithms and Complexity. Prentice-Hall, Upper Saddle River (1982)
Parnin, C., Orso, A.: Are automated debugging techniques actually helping programmers? In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA ’11, pp. 199–209. ACM, New York (2011)
Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge (2000)
Pearson, S., Campos, J., Just, R., Fraser, G., Abreu, R., Ernst, M.D., Pang, D., Keller, B.: Evaluating and improving fault localization. In: Proceedings of the 39th International Conference on Software Engineering (2017)
Perez, A., Abreu, R., van Deursen, A.: A test-suite diagnosability metric for spectrum-based fault localization approaches. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pp. 654–664 (2017)
Podgurski, A., Leon, D., Francis, P., Masri, W., Minch, M., Sun, J.S.J., Wang, B.W.B.: Automated support for classifying software failure reports. In: 25th International Conference on Software Engineering, 2003. Proceedings, vol. 6, pp. 465–475 (2003)
Rogstad, E., Briand, L.C.: Clustering deviations for black box regression testing of database applications. IEEE Trans. Reliab. 65(1), 4–18 (2016)
Rokach, L., Maimon, O.: Chapter 15—Clustering Methods. The Data Mining and Knowledge Discovery Handbook, p. 32 (2010)
Sobreira, V., Durieux, T., Delfim, F.M., Monperrus, M., de Almeida Maia, M.: Dissection of a bug dataset: anatomy of 395 patches from defects4j. CoRR abs/1801.06393 (2018)
Sohn, J., Yoo, S.: Fluccs: using code and change metrics to improve fault localization. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2017, pp. 273–283. ACM, New York (2017)
Trembly, A.C.: Software bugs cost billions annually. Nat. Underwriter/Life Health Financ. Serv. 106(31), 43 (2002)
Utting, M., Legeard, B., Pretschner, A.: A taxonomy of model-based testing. Softw. Testing Verif. Reliab. 22(April) (2006)
Wong, W.E., Debroy, V., Gao, R., Li, Y.: The DStar method for effective software fault localization. IEEE Trans. Reliab. 63(1), 290–308 (2014)
Wong, W.E., Gao, R., Li, Y., Abreu, R., Wotawa, F.: A survey on software fault localization. IEEE Trans. Softw. Eng. PP(99) (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this chapter
Cite this chapter
Golagha, M. (2022). How to Effectively Reduce Failure Analysis Time?. In: Felderer, M., et al. Ernst Denert Award for Software Engineering 2020. Springer, Cham. https://doi.org/10.1007/978-3-030-83128-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-83128-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-83127-1
Online ISBN: 978-3-030-83128-8
eBook Packages: Computer ScienceComputer Science (R0)