Abstract
Several decades ago, inspired by other knowledge areas, simulation was introduced as a research method to Software Engineering. Motivated by potential benefits achieved in other areas, the software engineering community has used simulation-based studies for planning, controlling, and improving software development. However, unclear expectations from simulation-based studies, a lack of methodological support, as well as dispersed knowledge to support model building and calibration have hindered widespread adoption of simulation-based investigations. In this chapter, we delineate the role of simulation in software engineering research and compile processes and guidelines into a comprehensive life cycle. This chapter aims to guide software engineering researchers to conduct effective simulation-based studies in real-world settings.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Abdel-Hamid TK (1988) Understanding the “90% syndrome” in software project management: a simulation-based case study. J Syst Softw 8(4):319–330
Ahmed R, Robinson S (2007) Simulation in business and industry: how simulation context can affect simulation practice? In: Proceedings of the 2007 spring simulation multiconference, vol 3. Society for Computer Simulation International, pp 152–159
Ahmed R, Hall T, Wernick P, Robinson S (2005) Evaluating a rapid simulation modelling process (RSMP) through controlled experiments. In: 2005 International symposium on empirical software engineering. IEEE, Piscataway, p 10
Ahmed R, Hall T, Wernick P, Robinson S, Shah M (2008) Software process simulation modelling: a survey of practice. J Simul 2(2):91–102
Alexopoulos C, Seila AF (1998) Output data analysis. In: Banks J (ed) Handbook of simulation. Wiley, New York, pp 225–272
Ali NB (2016) Is effectiveness sufficient to choose an intervention?: considering resource use in empirical software engineering. In: Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement, ESEM 2016, Ciudad Real, September 8–9, 2016, pp 54:1–54:6
Ali NB, Petersen K (2012) A consolidated process for software process simulation: state of the art and industry experience. In: 2012 38th Euromicro conference on software engineering and advanced applications. IEEE, Piscataway, pp 327–336
Ali NB, Unterkalmsteiner M (2014) Use and evaluation of simulation for software process education: a case study. In: European conference software engineering education (ECSEE), Shaker Verlag, Herzogenrath, pp 59–73
Ali NB, Petersen K, Wohlin C (2014) A systematic literature review on the industrial use of software process simulation. J Syst Softw 97:65–85
Ali NB, Petersen K, de França BBN (2015) Evaluation of simulation-assisted value stream mapping for software product development: two industrial cases. Inf Softw Technol 68:45–61
Babuska I, Oden JT (2004) Verification and validation in computational engineering and science: basic concepts. Comput Methods Appl Mech Eng 193(36):4057–4066
Balci O (1990) Guidelines for successful simulation studies. In: Proceedings of the winter simulation conference. IEEE Press, New Jersey, pp 25–32
Banks J (1999) Introduction to simulation. In: Proceedings of the 31st conference on Winter simulation: simulation – a bridge to the future, WSC 1999, Phoenix, December 05–08, 1999, vol 1, pp 7–13
Barlas Y (1989) Multiple tests for validation of system dynamics type of simulation models. Eur J Oper Res 42(1):59–87. https://doi.org/10.1016/0377-2217(89)90059-3
Barros M, Werner C, Travassos G (2000) Applying system dynamics to scenario based software project management. In: Proceedings of the 2000 international system dynamics conference
Basili VR, Rombach HD (1988) The TAME project: towards improvement-oriented software environments. IEEE Trans Softw Eng 14(6):758–773
Burton A, Altman DG, Royston P, Holder RL (2006) The design of simulation studies in medical statistics. Stat Med 25(24):4279–4292
Christie AM (1999) Simulation: an enabling technology in software engineering. CROSSTALK J Def Softw Eng 12(4):25–30
Davis JP, Eisenhardt KM, Bingham CB (2007) Developing theory through simulation methods. Acad Manag Rev 32(2):480–499
De França BBN, Travassos GH (2012) Reporting guidelines for simulation-based studies in software engineering. In: Proceedings of the 16th international conference on evaluation assessment in software engineering (EASE), pp 156–160
De França BBN, Travassos GH (2013) Are we prepared for simulation based studies in software engineering yet? CLEI Electron J 16(1):9
De França BBN, Travassos GH (2015) Simulation based studies in software engineering: a matter of validity. CLEI Electron J 18:5
De França BBN, Travassos GH (2016) Experimentation with dynamic simulation models in software engineering: planning and reporting guidelines. Empir Softw Eng 21(3):1302–1345
Dybå T, Sjøberg DI, Cruzes DS (2012) What works for whom, where, when, and why?: on the role of context in empirical software engineering. In: Proceedings of the ACM-IEEE international symposium on empirical software engineering and measurement. ACM, New York, pp 19–28
Eck JE, Liu L (2008) Contrasting simulated and empirical experiments in crime prevention. J Exp Criminol 4(3):195–213
Feldt R, Zimmermann T, Bergersen GR, Falessi D, Jedlitschka A, Juristo N, Münch J, Oivo M, Runeson P, Shepperd M, Sjøberg DIK, Turhan B (2018) Four commentaries on the use of students and professionals in empirical software engineering experiments. Empir Softw Eng 23(6):3801–3820. https://doi.org/10.1007/s10664-018-9655-0
Garousi V, Khosrovian K, Pfahl D (2009) A customizable pattern-based software process simulation model: design, calibration and application. Softw Process Improv Pract 14(3):165–180. https://doi.org/10.1002/spip.411
Gorschek T, Wohlin C, Garre P, Larsson S (2006) A model for technology transfer in practice. IEEE Softw 23(6):88–95
Houston DX, Ferreira S, Collofello JS, Montgomery DC, Mackulak GT, Shunk DL (2001) Behavioral characterization: finding and using the influential factors in software process simulation models. J Syst Softw 59(3):259–270. https://doi.org/10.1016/S0164-1212(01)00067-X
Jørgensen M, Kitchenham B (2012) Interpretation problems related to the use of regression models to decide on economy of scale in software development. J Syst Softw 85(11):2494–2503
Kellner MI, Madachy RJ, Raffo DM (1999) Software process simulation modeling: Why? What? How? J Syst Softw 46:91–105
Kitchenham B (2010) What’s up with software metrics? – a preliminary mapping study. J Syst Softw 83(1):37–51
Kleijnen JPC, Sanchez SM, Lucas TW, Cioppa TM (2005) State-of-the-art review: a user’s guide to the brave new world of designing simulation experiments. INFORMS J Comput 17(3):263–289. https://doi.org/10.1287/ijoc.1050.0136
Law AM (2007) Simulation modeling and analysis, vol 4. McGraw-Hill, New York
Law AM, Kelton WD (2000) Simulation modeling and analysis, vol 3. McGraw-Hill, New York
Madachy RJ (2002) Simulation. Wiley, New York
Madachy RJ (2008) Software process dynamics. Wiley-IEEE Press, New York
Maria A (1997) Introduction to modeling and simulation. In: Proceedings of the 29th conference on Winter simulation, WSC, pp 7–13
McCall JA, Wong G, Stone A (1979) A simulation modeling approach to understanding the software development process. In: Fourth annual software engineering workshop. Goddard Space Flight Center, Greenbelt
Melis M, Turnu I, Cau A, Concas G (2006) Evaluating the impact of test-first programming and pair programming through software process simulation. Softw Process Improv Pract 11(4):345–360
Monks T, Currie CS, Onggo BS, Robinson S, Kunc M, Taylor SJ (2019) Strengthening the reporting of empirical simulation studies: introducing the stress guidelines. J Simul 13(1):55–67
Montgomery DC (2017) Design and analysis of experiments. Wiley, Hoboken
Müller M, Pfahl D (2008) Simulation methods. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer, London, pp 117–152
Münch J, Rombach D, Rus I (2003) Creating an advanced software engineering laboratory by combining empirical studies with process simulation. In: Proceedings of the international workshop on software process simulation and modeling (ProSim), pp 3–4
Murphy S, Perera T (2001) Key enablers in the development of simulation. In: Proceedings of the Winter simulation conference, pp 1429–1437
Olsen NC (1993) The software rush hour (software engineering). IEEE Softw 10(5):29–37
Ören TI (1981) Concepts and criteria to assess acceptability of simulation studies: a frame of reference. Commun ACM 24(4):180–189
Park S, Kim H, Kang D, Bae DH (2008) Developing a software process simulation model using SPEM and analytical models. Int J Simul Process Model 4(3–4):223–236. https://doi.org/10.1504/IJSPM.2008.023684
Petersen K, Gencel C (2013) Worldviews, research methods, and their relationship to validity in empirical software engineering research. In: 2013 Joint conference of the 23rd international workshop on software measurement and the 8th international conference on software process and product measurement. IEEE, Piscataway, pp 81–89
Petersen K, Wohlin C (2009) Context in industrial software engineering research. In: Proceedings of the 3rd international symposium on empirical software engineering and measurement (ESEM), pp 401–404
Pfahl D (2001) An integrated approach to simulation based learning in support of strategic and project management in software organisations. Fraunhofer-IRB-Verlag, Stuttgart
Pfahl D (2014) Process simulation: a tool for software project managers? In: Software project management in a changing world. Springer, Berlin, pp 425–446
Pfahl D, Lebsanft K (1999) Integration of system dynamics modelling with descriptive process modelling and goal-oriented measurement. J Syst Softw 46(2):135–150. https://doi.org/10.1016/S0164-1212(99)00007-2
Pfahl D, Lebsanft K (2000) Knowledge acquisition and process guidance for building system dynamics simulation models: an experience report from software industry. Int J Softw Eng Knowl Eng 10(04):487–510
Pfahl D, Ruhe G (2002) IMMoS: a methodology for integrated measurement, modelling and simulation. Softw Process Improv Practice 7(3–4):189–210
Pfahl D, Stupperich M, Krivobokova T (2004) PL-SIM: a generic simulation model for studying strategic SPI in the automotive industry. In: IET conference proceedings. Citeseer, pp 149–158. https://doi.org/10.1049/ic:20040454
Raffo DM (2005) Software project management using prompt: a hybrid metrics, modeling and utility framework. Inf Softw Technol 47(15):1009–1017
Raffo DM, Kellner MI (2000) Empirical analysis in software process simulation modeling. J Syst Softw 53(1):31–41. https://doi.org/10.1016/S0164-1212(00)00006-6
Rus I, Neu H, Münch J (2003) A systematic methodology for developing discrete event simulation models of software development processes. In: Proceedings of the 4th international workshop on software process simulation and modeling (ProSim 2003), Portland, May 3–4
Sargent R (1999) Validation and verification of simulation models. In: WSC’99. 1999 Winter simulation conference proceedings. ‘Simulation – a bridge to the future’ (Cat. No. 99CH37038), vol 1. IEEE, Piscataway, pp 39–48
Shannon RE (1986) Intelligent simulation environments. In: Proceedings of the conference on intelligent simulation environments, pp 150–156
Travassos GH, Barros MO (2003) Contributions of in virtuo and in silico experiments for the future of empirical studies in software engineering. In: 2nd Workshop on empirical software engineering the future of empirical studies in software engineering, pp 117–130
Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer, New York
Wu M, Yan H (2009) Simulation in software engineering with system dynamics: a case study. J Softw 4(10):1127–1135
Zhang H (2012) Special panel: software process simulation – at a crossroads? In: Proceedings of the international conference on software and system process (ICSSP). IEEE, Piscataway, pp 215–216
Zhang H, Kitchenham B, Pfahl D (2010) Software process simulation modeling: an extended systematic review. In: Proceedings of the international conference on software process (ICSP). Springer, Berlin, pp 309–320
Zhang H, Jeffery R, Houston D, Huang L, Zhu L (2011) Impact of process simulation on software practice: an initial report. In: Proceedings of the 33rd international conference on software engineering (ICSE), pp 1046–1056
Acknowledgements
Breno has been supported by CNPq-Brazil (Grant 141152/2010-9) during the development of part of this work. Nauman has been supported by a research grant for the VITS project (reference number 20180127) by the Knowledge Foundation in Sweden and by ELLIIT, a Strategic Area within IT and Mobile Communications, funded by the Swedish Government. We would also like to thank the reviewers for their feedback that has helped us to improve the presentation and the contents of the chapter significantly.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
de França, B.B.N., Ali, N.B. (2020). The Role of Simulation-Based Studies in Software Engineering Research. In: Felderer, M., Travassos, G. (eds) Contemporary Empirical Methods in Software Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-32489-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-32489-6_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32488-9
Online ISBN: 978-3-030-32489-6
eBook Packages: Computer ScienceComputer Science (R0)