Skip to main content
SpringerLink
Log in

Comparison and Validation of Mutation Testing Tools Based on Java Language

  • Chapter
  • First Online:
Optimization of Automated Software Testing Using Meta-Heuristic Techniques

Part of the book series: EAI/Springer Innovations in Communication and Computing ((EAISICC))

  • 295 Accesses

Abstract

Mutation testing is the fault-based software testing approach that is widely applicable for assessing the effectiveness of a test suite. The test suite effectiveness is measured through artificial seeding of faults into the programs under test. Mutation testing tools (MTTs) are significant enablers of the conversion of mutation testing from the research perspective into the real life and mostly applicable testing process. Without using the automatic MTT, mutation testing can’t be really connected in this present reality and is unrealistic to be acknowledged by the industry. Authors analyze six open-source JAVA-based MTT (Jester, JavaMut, MuJava, Jumble, Judy, and Javalanche). The results are based on the performance of various JAVA programs and two real-life applications. Benchmark comparison among the MTT is presented in terms of mutants, mutation operator, mutation score, and quality output. On the basis of comparative analysis, the performance of each tool is explained with the protocol for finding the appropriate tool among the six MTTs. The results show that the MuJava performs best compared to the others.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978). Hints on test data selection: Help for the practicing programmer. Computer, 11(4), 34–41.

    Article  Google Scholar 

  2. Jeevarathinam, R., & Thanamani, A. S. (2011). A survey on mutation testing methods, fault classifications and automatic test cases generation.

    Google Scholar 

  3. Irvine, S. A., Pavlinic, T., Trigg, L., Cleary, J. G., Inglis, S., & Utting, M. (2007, September). Jumble java byte code to measure the effectiveness of unit tests. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007) (pp. 169–175). IEEE.

    Google Scholar 

  4. Madeyski, L., & Radyk, N. (2010). Judy – A mutation testing tool for Java. IET Software, 4(1), 32–42.

    Article  Google Scholar 

  5. Offutt, A. J., Pan, J., Tewary, K., & Zhang, T. (1996). An experimental evaluation of data flow and mutation testing. Software: Practice and Experience, 26(2), 165–176.

    Google Scholar 

  6. Moore, I. (2001). Jester-a JUnit test tester. Proceedings of 2nd XP, 84–87.

    Google Scholar 

  7. Untch, R. H., Offutt, A. J., & Harrold, M. J. (1993, July). Mutation analysis using mutant schemata. In Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis (pp. 139–148).

    Google Scholar 

  8. DeMillo, R. A., & Offutt, A. J. (1991). Constraint-based automatic test data generation. IEEE Transactions on Software Engineering, 17(9), 900–910.

    Article  Google Scholar 

  9. King, J. C. (1976). Symbolic execution and program testing. Communications of the ACM, 19(7), 385–394.

    Article  MathSciNet  Google Scholar 

  10. Sen, K., Marinov, D., & Agha, G. (2005). CUTE: A concolic unit testing engine for C. ACM SIGSOFT Software Engineering Notes, 30(5), 263–272.

    Article  Google Scholar 

  11. Harman, M., & McMinn, P. (2007, July). A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation. In Proceedings of the 2007 international symposium on software testing and analysis (pp. 73–83).

    Google Scholar 

  12. Zapf, C. N. (1993). MedusaMothra-A distributed interpreter for the Mothra mutation testing system (Master’s thesis, Clemson University).

    Google Scholar 

  13. Maldonado, J. C., Delamaro, M. E., Fabbri, S. C., da Silva Simão, A., Sugeta, T., Vincenzi, A. M. R., & Masiero, P. C. (2001). Proteum: A family of tools to support specification and program testing based on mutation. In Mutation testing for the new century (pp. 113–116). Springer.

    Chapter  Google Scholar 

  14. Ma, Y. S., Offutt, J., & Kwon, Y. R. (2006, May). MuJava: A mutation system for Java. In Proceedings of the 28th international conference on Software engineering (pp. 827–830).

    Google Scholar 

  15. Schuler, D., & Zeller, A. (2009, August). Javalanche: Efficient mutation testing for Java. In Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (pp. 297–298).

    Google Scholar 

  16. Offutt, A. J., & King, K. N. (1987, July). A Fortran 77 interpreter for mutation analysis. In Papers of the symposium on interpreters and interpretive techniques (pp. 177–188).

    Google Scholar 

  17. Agrawal, H., DeMillo, R., Hathaway, R., Hsu, W., Hsu, W., Krauser, E. W., … Spafford, E. (1989). Design of mutant operators for the C programming language (Technical report SERC-TR-41-P). Software Engineering Research Center, Purdue University.

    Google Scholar 

  18. Papadakis, M., & Malevris, N. (2012). Mutation based test case generation via a path selection strategy. Information and Software Technology, 54(9), 915–932.

    Article  Google Scholar 

  19. Delamaro, M. E., Maldonado, J. C., Pasquini, A., & Mathur, A. P. (2001). Interface mutation test adequacy criterion: An empirical evaluation. Empirical Software Engineering, 6(2), 111–142.

    Article  Google Scholar 

  20. Chevalley, P., & Thevenod-Fosse, P. (2003). A mutation analysis tool for Java programs. International Journal on Software Tools for Technology Transfer, 5(1), 90–103.

    Article  Google Scholar 

  21. Guderlei, R., Just, R., Schneckenburger, C., & Schweiggert, F. (2008, April). Benchmarking testing strategies with tools from mutation analysis. In 2008 IEEE international conference on software testing verification and validation workshop (pp. 360–364). IEEE.

    Google Scholar 

  22. Alexander, R. T., Bieman, J. M., Ghosh, S., & Ji, B. (2002, November). Mutation of Java objects. In 13th international symposium on software reliability engineering, 2002. Proceedings (pp. 341–351). IEEE.

    Google Scholar 

  23. Tuya, J., Suárez-Cabal, M. J., & De La Riva, C. (2007). Mutating database queries. Information and Software Technology, 49(4), 398–417.

    Article  Google Scholar 

  24. Serrestou, Y., Beroulle, V., & Robach, C. (2006, April). How to improve a set of design validation data by using mutation-based test. In 2006 IEEE design and diagnostics of electronic circuits and systems (pp. 75–76). IEEE.

    Google Scholar 

  25. Ferrari, F. C., Nakagawa, E. Y., Maldonado, J. C., & Rashid, A. (2011, March). Proteum/AJ: A mutation system for AspectJ programs. In Proceedings of the tenth international conference on Aspect-oriented software development companion (pp. 73–74).

    Google Scholar 

  26. Ma, Y. S., Harrold, M. J., & Kwon, Y. R. (2006, May). Evaluation of mutation testing for object-oriented programs. In Proceedings of the 28th international conference on software engineering (pp. 869–872).

    Google Scholar 

  27. Jia, Y., & Harman, M. (2010). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5), 649–678.

    Article  Google Scholar 

  28. Nanavati, J., Wu, F., Harman, M., Jia, Y., & Krinke, J. (2015). Mutation testing of memory-related operators. In 2015 IEEE eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 1–10). IEEE.

    Google Scholar 

  29. Li, N., West, M., Escalona, A., & Durelli, V. H. (2015, April). Mutation testing in practice using ruby. In 2015 IEEE eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 1–6). IEEE.

    Google Scholar 

  30. Aichernig, B. K., Brandl, H., Jöbstl, E., Krenn, W., Schlick, R., & Tiran, S. (2015). Killing strategies for model-based mutation testing. Software Testing, Verification and Reliability, 25(8), 716–748.

    Article  Google Scholar 

  31. Just, R. (2014, July). The major mutation framework: Efficient and scalable mutation analysis for Java. In Proceedings of the 2014 international symposium on software testing and analysis (pp. 433–436).

    Google Scholar 

  32. http://www.ise.gmo.edu:8080/ofut/jsp/stis

  33. https://adrive.com/

  34. Khari, M., Dalal, R., & Rohilla, P. (2020). Extended paradigms for botnets with WoT applications: A review. Smart Innovation of Web of Things, 105.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer and Systems Sciences, Jawaharlal Nehru University, Delhi, India

    Manju Khari

Authors
  1. Manju Khari
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer & Systems Sciences, Jawaharlal Nehru University, New Delhi, Delhi, India

    Manju Khari

  2. Department of MCA, GITA Autonomous College, Bhubaneswar, India

    Deepti Bala Mishra

  3. Department of Computer Engineering-AI, Marwadi University, Rajkot, Gujarat, India

    Biswaranjan Acharya

  4. Computer Science and Technology, Universidad Internacional de La Rioja, La Rioja, Spain

    Ruben Gonzalez Crespo

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Khari, M. (2022). Comparison and Validation of Mutation Testing Tools Based on Java Language. In: Khari, M., Mishra, D.B., Acharya, B., Gonzalez Crespo, R. (eds) Optimization of Automated Software Testing Using Meta-Heuristic Techniques. EAI/Springer Innovations in Communication and Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-07297-0_2

Download citation

Publish with us

Policies and ethics

Search

Navigation