Abstract
In this paper we 1) review industry acceptance testing practices and 2) present a systematic approach to scenario analysis and its application to acceptance testing with the aim to improve the current practice. It summarizes the existing practice into categories and identifies the serious weakness. Then, a new approach based on the formal scenario analysis is presented. It is systematic, and easily applicable to any software or system. A simple, yet realistic example is used to illustrate its effectiveness. Finally, its benefits and its applicability are summarized.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Alford, M. (1977), “A Requirements Engineering Methodology for Real-Time Systems,” IEEE Transactions on Software Engineering 3,1, 60–69.
Andriole, S.J. (1989), Storyboard Prototyping — A New Approach to User Requirements Analysis, QED Information Systems, Wellesley, MA.
Bauer, J., S. Faasse, A. Finger, and W. Goodhue (1978), “The Automatic Generation and Execution of Function Test Plans for Electronic Switching Systems,” In Proceedings of the Software Quality Assurance Workshop, ACM, New York, NY, pp. 92–100.
Bauer, J. and A. Finger (1979), “Test Plan Generation Using Formal Grammars,” In Proceedings of the Fourth International Conference on Software Engineering, IEEE Computer Society Press, Los Alamitos, CA, pp. 425–432.
Beizer, B. (1984), Software System Testing and Quality Assurance, Van Nostrand Reinhold, New York, NY.
Chandrasekharan, M., B. Dasarathy, and Z. Kishimoto (1989), “Requirements-Based Testing of Real-Time Systems: Modeling for Testability,” IEEE Computer 18,4, 71–80.
Chow, T. (1977), “Testing Software Design Modeled by Finite State Machines,” In Proceedings COMPSAC'77, IEEE Computer Society Press, Washington, DC, pp. 58–64; also in IEEE Transactions on Software Engineering 4, 3, 178-187.
Compton, H., P. Giloth, M. Haverty, and B. Niedfeldt (1977), “System Integration and Early Office Experience,” The Bell System Technical Journal 56,7, 1279–1294.
Dasarathy, B. and M. Chandrasekharan (1982), “Test Generation for Functional Validation of Real-Time Systems,” Sixth IEEE International Conference on Software Engineering, Unpublished poster session.
Davis, A. (1980), “Automating the Requirements Phase: Benefits to Later Phases of the Software Life Cycle,” In Proceedings COMPSAC'80, IEEE Computer Society Press, Washington, DC, pp. 42–48.
Davis, A. (1990), “System Testing: Implications of Requirements Specifications,” Information and Software Technology 32,6, 407–414.
Davis, A. (1993), Software Requirements, Objects, Function & States, Prentice Hall, Englewood Cliffs, NJ.
Davis, A. and T. Ranscher (1978), “Formal Techniques and Automatic Processing to Ensure Correctness in Requirements Specifications,” In Proceedings of the Specifications of Reliable Software Conference, IEEE Computer Society Press, Los Alamitos, CA, pp. 15–35.
Deutsch, M. (1982), Software Verification and Validation, Prentice Hall, Englewood Cliffs, NJ.
Dunn, R. and R. Ullman (1982), Quality Assurance for Computer Software, McGraw-Hill, New York, NY.
Ferguson, R. and B. Korel (1996), “The Chaining Approach for Software Test Data Generation,” ACM Transactions on Software Engineering and Methodology 5,1, 63–86.
Gomaa, H. and D. Scott (1981), “Prototype as a Tool in the Specification of User Requirements,” In Proceedings of the 5th International Conference Software Engineering, IEEE Computer Society Press, Los Alamitos, CA, pp. 333–342.
Hetzel, W. (1984), The Complete Guide to Software Testing, QED Information Systems, Wellesley, MA.
Hsia, P., J. Gao, J. Samuel, D. Kung, Y. Toyoshima, and C. Chen (1994a), “A Formal Approach to Scenario Analysis,” IEEE Software 11,2, 33–41.
Hsia, P., J. Gao, J. Samuel, D. Kung, Y. Toyoshima, and C. Chen (1994b), “Behavior-Based Integration Testing of Software Systems: A Formal Scenario Approach,” In Proceedings of the Third International Conference on System Integration, IEEE Computer Society Press, Los Alamitos, CA, pp. 1138–1147.
Hsia, P., J. Gao, J. Samuel, D. Kung, Y. Toyoshima, and C. Chen (1994c), “Behavior-Based Acceptance Testing of Software Systems: A Formal Scenario Approach,” In Proceedings of COMPSAC'94, IEEE Computer Society Press, Los Alamitos, CA, pp. 293–298.
Hsia, P., J. Samuel, D. Kung, L. Li, C.T. Hsu, C. Chen, and Y. Toyoshima (1995), “A Usage-Model Based Approach to Test the Therac-25,” Safety and Reliability in Emerging Control Technologies, IFAC Workshop Report, Pergamon Press, Daytona Beach, FL, pp. 55–63.
Huebner, D. (1979), “Systems Validation through Automated Requirements Verification,” In Proceedings of COMPSAC'79, IEEE Computer Society Press, Washington, DC, pp. 222–227.
Janowiak, T. (1990), “A Simulation for Combat Systems Development and Acceptance Testing,” In Proceedings of the 1990 Winter Simulation Conference, IEEE, New York, NY, pp. 210–213.
Kaner, C., J. Falk, and H. Nguyen (1988), Testing Computer Software, Second Edition, Van Nostrand Reinhold, New York, NY.
Kleinstein, D. (1988), “System Acceptance Testing in the Real World: A Case Study,” In Proceedings of the Third Israel Conference on Computer Systems and Software Engineering, Tel-Aviv, Israel, pp. 40–46.
Krause, K. and L. Diamant (1978), “A Management Methodology for Testing Software Requirements,” In Proceedings of COMPSAC'78, IEEE Computer Society Press, Washington, DC, pp. 749–754.
Lea, R., S. Chen, and C. Chung (1991), “On Generating Test Data From Prototypes,” In Proceedings of the 15th International Computer Software and Applications Conference, IEEE Computer Society Press, Los Alamitos, CA, pp. 345–350.
Linehan, T., Jr. (1988), “Application Software Configuration Management and Testing in a Pharmaceutical Laboratory Automation Environment,” In Proceedings of the Conference on Software Maintenance, IEEE Computer Society Press, Los Alamitos, CA, pp. 178–182.
Lipow, M. and T. A. Thayer (1977), “Prediction of Software Failures,” In Proceedings of the Annual Symposium on Reliability and Maintainability, IEEE, New York, NY, pp. 489–494.
McCabe, T. (1983), “Structured Requirements and Testing,” Software Outlook Newsletter, McCabe and Associates, Columbia, MD.
McCabe, T. and G. Schulmeyer (1985), “System Testing aided by Structured Analysis — A Practical Experience,” IEEE Transactions on Software Engineering 11,9, 917–921.
Meyers, G. (1979), The Art of Software Testing, Wiley, New York, NY.
Musa, J. and A. Ackerman (1989), “Quantifying Software Validation: When to Stop Testing,” IEEE Software, 19–27.
Nardi, B. (1992), “The Use of Scenarios in Design,” SIGCHI Bulletin 24,4, 13–14.
NBS (1984), Guideline for Lifecycle Validation, Verification, and Testing of Computer Software (1984), US Department of Commerce, National Bureau of Standards, Springfield, VA.
Osder, S. and D. Decker (1982), “Software Validation for Flight-Critical Systems,” In Proceedings of the Phoenix Conference on Computers and Communications, IEEE Computer Society Press, Washington, DC, pp. 193–198.
Protzel, P. (1988), “Automatically Generated Acceptance Test: A Software Reliability Experiment,” In Proceedings of the 2nd Workshop on Software Testing, Verification, and Analysis, IEEE Computer Society Press, Washington, DC, pp. 196–203.
Quirk, W., Ed. (1985), Verification and Validation of Real-Time Software, Springer, Berlin, Germany.
Ramamoorthy, C., S. Ho, and W. Chen (1976), “On the Automated Generation of Program Test Data,” IEEE Transactions on Software Engineering 2,4, 293–300.
Ray, S. (1989), “Acceptance Tests of Distributed Processing Oriented Supervisory Control and Data Acquisition (SCADA) Systems for Offshore Platforms,” In Proceedings of the First European Test Conference, IEEE Computer Society Press, Washington, DC, pp. 221–228.
Rumbaugh, J. (1991), Object-Oriented Modeling and Design, Prentice-Hall, Englewood Cliffs, NJ.
Samson, D. (1990), “Test Planning from System Requirements: A Classification Approach,” Jet Propulsion Laboratory Quarterly Report 12, 33–37.
Szaboky, G. (1996), “A Systematic Approach for Testing GUI Applications,” Masters Thesis, Computer Science and Engineering Department, University of Texas at Arlington, Arlington, TX.
Tausworthe, R. (1979), Standardized Development of Computer Software, Part II, Standards, Prentice-Hall, Englewood Cliffs, NJ.
Wallace, D. and J. Cherniavsky (1986), Report on the NBS Software Acceptance Test Workshop April 1–2, 1986, Natl. Bur. Stand. US Department of Commerce, Washington, DC.
Wallace, D. (1990), Guide to Software Acceptance, US Department of Commerce, National Institute of Standards and Technology, Gaithersburg, MD.
Wardle, P. (1991), “Methodology and Tools for Requirements Capture, Traceability, and Verification,” In Proceedings of the Third International Conference on Software Engineering for Real Time Systems (Conf. Publication No. 344), The Institute, London, UK, pp. 46–50.
Worrest, R. (1982), “Using an Automatic Test Executor to Support Parallel System and Test Development,” In Proceedings of the Phoenix Conference on Computers and Communications, IEEE Computer Society Press, Washington, DC, pp. 204–208.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hsia, P., Kung, D. & Sell, C. Software requirements and acceptance testing. Annals of Software Engineering 3, 291–317 (1997). https://doi.org/10.1023/A:1018938021528
Issue Date:
DOI: https://doi.org/10.1023/A:1018938021528