Introduction

Abstract

Software has become an indispensable part of our daily lives, as our dependence on software is constantly increasing. Software fails, and these failures cost money, time, resources, and sometimes even lives. An organization’s reputation and success depends on its ability to create and deliver high-quality software.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Software has become an indispensable part of our daily lives, as our dependence on software is constantly increasing. Software fails, and these failures cost money, time, resources, and sometimes even lives. An organization’s reputation and success depends on its ability to create and deliver high-quality software. Software quality is thus critically important. This chapter introduces the concept of software quality assurance, discusses software inspection (the most frequently performed software quality assurance activity other than testing), and outlines the contents of this book.

1.1 Software Quality, Software Reviews and Inspections

The software creation process requires a lot of manpower, from requirements and design to implementation and testing. As human beings, we all make mistakes, which are manifested as defects or bugs in software products. Software development is a complex process and tools are frequently used to assist development. People’s familiarity with tools and their interaction with tools are the other complicating factors that can potentially inject defects into software products. The nontrivial software product development nowadays may involve many people and teams across the globe. Coordinating the distributed team members and tasks remains a management challenge. Despite many years of software engineering efforts, we are still unable to deliver defect-free software products to the end users. Software defects seem to be inevitable.

Software quality assurance is a structured approach to improve software quality and involves defect prevention, detection and removal, and defect containment activities. Defect detection and removal is mostly emphasized in industry practice, particularly in non-safety critical systems. If it is hard or impossible not to inject defects into software, the next best thing to do is to remove them before they reach end users. It is highly desirable to detect and remove defects as soon as they are injected into software, since defects can be several times more expensive to fix at latter stages of a software project. The escaped or residual defects in field deployment may cause interruption to the user’s normal operations, resulting in substantial costs including reduced productivity, data loss or corruption, security vulnerabilities, or even physical harm.

Testing and inspection are the two most effective and commonly used methods to detect defects in software. In software testing, a tester runs a software system or its components, executes its functions, observes system behaviors or responses, and determines if the system behaviors conform to its requirements, specifications, or expectations. To test software, one needs a running executable, which is not feasible all the time, particularly for artifacts rather than code. The effectiveness of testing also depends on test cases, and if there is no test case to exercise a particular path of the code, there is no way to tell if there is a defect in that path or not. Dijkstra once famously said that testing can only confirm the presence of a defect, not the absence of a defect ( https://en.wikiquote.org/wiki/Edsger_W_Dijkstra ).

There are many other quality assurance alternatives besides testing. Software review or inspection is one of these alternatives that is widely practiced. The terms “software review” and “software inspection” are not used consistently in literature. IEEE Standard 1028 defines five types of software reviews: management reviews, technical reviews, inspections, walkthroughs, and audits (IEEE 1028); it treats software inspection as a type of software review. In general software engineering literature, inspection is specific and has a defined process, and review is a more general term. We will use these two terms interchangeably in this book.

Software inspection is a formalized peer review process applicable to any software artifact. It is a static analysis method and was first introduced by Fagan based on his practical experience at IBM in the 1970’s (Fagan, 1976). Fagan inspection has been influential ever since it was published. In fact, the IEEE Standard 1028 is largely based on Fagan inspection.

Based on decades of research and industry practices, it is widely established that software inspection is effective (it finds many defects), efficient (low cost per defect), and practical (easy to carry out). We will review the status of software review and inspection in Chapter 2. It has been reported that inspection can detect (and correct) 20% to 90% of defects (Gilb & Graham, 1993, Laitenberger, 2002). The benefits and cost-effectiveness of software inspection have long been recognized. Doolan (1992) reported that every hour invested in inspection has a payback of 30 hours and Russell (1991) claimed that in his organization, every inspection hour saved 33 hours of maintenance work. Although the earlier benefits might be overstated (Porter et al., 1996; Laitenberger, 2002), more recent simulation results have revealed that code inspection saves 39% of defect costs compared to testing alone, and design inspection saves 44% of defect costs compared to testing alone (Briand et al.,1998). Software inspection nowadays is easy to conduct. Books have been published about it (e.g., Gilb, 1993) and standards on review procedures are also available (e.g., IEEE 1028).

During software development, many documents are generated by humans, e.g., requirements specifications, design, code, test plans, test cases, and user documentation. If these documents are used to guide subsequent activities, any errors in these documents may propagate to the downstream activities and artifacts. It is critical to catch and fix errors early to prevent them from propagating down the stream. It is much cheaper for the development organization to fix an error when it is introduced. It is believed that, in general, it would be 10 times more expensive to fix an error if it slips to the next phase. Defects in the released product may result in substantial costs to end users as well.

Table 1-1 shows the cost of fixing a defect depending on when the defect is injected into the software and when the defect is fixed (McConnell, 2004). The first column shows when the defect is injected and the first row shows when the defect is fixed. It assumes the cost to fix the defect injected in the same development phase is 1x. Although the real cost varies and depends on the project size and industry, the general trend is clear.

Table 1-1. The Cost of Fixing a Defect (adapted from McConnell, 2004)

Requirements, design, and other non-executable artifacts cannot be tested by machines and have to be reviewed by humans. Requirements not implemented yet cannot be machine tested but can be analyzed by humans to see if they meet user needs. Design, particularly early design sketches, can be abstract, imprecise, and incomplete. It is not possible to test designs by machine. With inspection, both requirements and design can be read and analyzed, which does not require the artifacts to be complete or executable. Even for program code, which can be machine-tested, testing and inspection are complementary and uncover different kinds of issues (Basili & Selby, 1987; Juristo & Vegas, 2003). Issues hard to uncover via testing often can be detected by inspection relatively easily. Mantyla and Lassenius (2009) affirmed that code reviews are a good tool for detecting code evolvability defects (documentation, visual representation, and structure) that cannot be found in later phase testing, since the defects do not affect the software’s visible functionality. Inspection can be applied to test plan and test cases as well, which further improves the defect detection efficiency of testing.

The immediate benefit of software review is to detect and fix errors or problems in software artifacts. As a side effect of inspection, the software artifacts become more readable and maintainable. There are other derived benefits to individuals who participate in the review and to the organization as a whole. The author who created the artifacts can use the review outcome as a learning instrument, e.g., he can avoid similar errors in his future work. To other participants, they can also learn from another person’s mistakes, particularly if they did not catch the mistakes in their own reviews. The software review can be used to cross-train team members and build a stronger team. Inspection enables a team to share technical expertise. To the organization, fewer mistakes lead to higher quality and better customer satisfaction and less rework and higher development productivity.

If the organization builds a knowledge database of past errors and problems uncovered during software inspection, they can use it to their competitive advantage. In the case of code review, one can identify the most error-prone modules and decide whether redesign or recoding is worthwhile. Certainly more review and more testing in those problematic areas are warranted before the software is released. The organization can also look at the distribution of error types via the Pareto chart and perform root cause analysis of the most common errors. For common systematic errors, the organization can then decide if additional training, better tooling, or an improved development process could help.

1.2 About This Book

In Fagan’s inspection and its derivatives, individuals typically read software artifacts, then come together to discuss their findings and hopefully uncover new ones. The defect collection meeting is considered critical. There is convincing evidence, however, that defect detection in software artifacts is primarily an individual effort and happens during individual preparation and reading (Johnson & Tjahjono, 1998; Votta, 1993), and the purpose of the meeting is mostly to agree on true defects and dismiss false positives. In the state-of-industry practice, the meeting is either removed or not the emphasis anymore. Instead, how the inspectors examine the software artifacts becomes important. To improve an individual’s effectiveness, various reading techniques have been proposed and tested. Reading is a key technical activity for verifying and validating software (Basili, 1997). This book is devoted to software reading techniques.

1.2.1 Organization of This Book

Many reading techniques have been developed and tested. This book categorizes them based on their characteristics. Some reading techniques are generic and can be used for many, if not all, software artifacts, while other techniques are only applicable to specific artifacts. This book discusses general reading techniques first, then specific techniques for software artifacts such as requirements, design, and code. The following lists the contents of each chapter.

Chapter 2 discusses the software review procedure. In particular, we discuss Fagan inspection and its extensions. Active design review is discussed in detail as well, since it is the basis of some recent reading techniques such as scenario-based reading.

Chapter 3 defines the terms of software reading and software reading techniques. It discusses reading purposes and classifies existing reading techniques. Ad hoc reading and checklist-based reading are discussed in detail, including various extensions to checklist-based reading, since they are frequently used as baselines and other reading techniques are compared to them. Chapter 3 also discusses differential reading, which can be used to inspect artifacts under evolution.

Chapter 4 focuses on scenario-based reading. Although many reading techniques are classified as scenario-based reading, we discuss defect-based reading, perspective-based reading, and function-point-based reading, as examples of scenario-based reading techniques. Perspective-based reading has been adapted to inspect requirements specifications, design documents, source code files, and user interface usability. The cognitive process of perspective-based reading is also discussed. A simple analytical model is used to shed light on when perspective-based reading could outperform other reading techniques.

Chapter 5 discusses reading techniques specific to requirements specifications. We present a combined reading technique which takes advantages of the strengths of individual reading techniques, while compensating for their weaknesses. Test-driven reading is introduced as an economic reading technique for organizations with limited resources.

Chapter 6 discusses specific reading techniques for design. While most reading techniques gear to find as many defects as possible, not all defects have the same impact on end users. Usage-based reading uses prioritized use-cases as a guiding light to focus readers’ attention on defects that matter to users most. This chapter also discusses traceability-based reading that checks for consistency among all design artifacts and between design and requirement artifacts. It is a family of techniques organized as horizontal and vertical readings. Traceability-based reading is also applied to architecture review. Although most of the time we read software artifacts to detect defects, we read software for construction under some circumstances. Scope-based reading is designed to enhance a reader’s ability to understand object-oriented application frameworks, which the reader can then use to design and implement their own applications.

Chapter 7 is devoted to specific code reading techniques. It covers reading by stepwise abstraction first and then illustrates how it is extended as abstraction-driven reading for object-oriented code. There are unique challenges in object-oriented code reading. While abstraction-driven reading focuses on the static behavior of an object-oriented system, use-case-driven reading is concerned with the dynamic behavior. Legacy software applications are abundant without much documentation. Task-directed reading can be used to improve code quality and complete the necessary documentation on code and design.

Chapter 8 concludes the book and encourages readers to apply reading techniques to their practice.

1.2.2 Intended Audience and How to Use This Book

This book is intended for software engineering practitioners. We suggest that readers start with Chapters 2 and 3 to get an overview of software inspection and preliminary reading techniques.

• If you are interested in inspecting software requirements specifications, you can continue with Chapters 4 and 5. You may skip non-requirements applications of perspective-based reading in Chapter 4, however.

• If you are interested in inspecting design documents, you can read part of Chapter 4 (Section 4.3.3) and Chapter 6.

• If you are interested in inspecting source code, you can read part of Chapter 4 (Section 4.3.4) and Chapter 7.

• If you are interested in usability inspection, you can read Chapter 4 (Section 4.3.5).

This book is also intended for software engineering students and can be used to supplement courses such as software engineering, software quality, and software testing. As students are preparing themselves to join the software industry as software engineers, it is beneficial for students to read the entire book.

This book can be used by software engineering researchers, particularly if you are interested in software quality assurance, software inspection, and software reading. This book has the most comprehensive material on software reading. You can find references to the original articles where particular reading techniques were first proposed, as well as information on the most recent developments and experiences with those reading techniques.

1.3 References

(Basili, 1987) V.R. Basili and R.W. Selby, Comparing the effectiveness of software testing strategies, IEEE Transactions on Software Engineering, vol.13, no,12m pp.1278-1296, 1987.

(Basili, 1997) V.R. Basili, Evolving and packaging reading technologies, Journal of Systems and Software, vol.38, no.1, pp.3-12, 1997.

(Briand, 1998) L. Briand, K. EI Emam, O. Laitenberger, and T. Fussbroich, Using simulation to build inspection efficiency benchmarks for development projects, Proceedings of the International Conference on Software Engineering, pp.340-349, 1998.

(Doolan, 1992) E. Doolan, Experience with Fagan’s inspection method, Software Practice and Experience, vol.22, o.2, pp.173-182, 1992.

(Fagan, 1976) M.E. Fagan, Design and code inspections to reduce errors in program development, IBM Systems Journal, vol.15, no.3, pp.182-211, 1976.

(Gilb, 1993) T. Gilb and D. Graham, Software Inspection, Addison-Wesley, 1993.

(IEEE 1028) IEEE Std 1028-2008, IEEE Standard for Software Reviews and Audits, 2008.

(Johnson, 1998) P.M. Johnson and D. Tjahjono, Does every inspection really need a meeting? Empirical Software Engineering, vol.3, pp.9-35, 1998.

(Juristo, 2003) N. Juristo and S. Vegas, Functional testing, structural testing and code reading: what fault type do they each detect? Lecture Notes in Computer Science, vol.2765, pp.208-232, 2003.

(Laitenberger, 2002) O. Laitenberger, A survey of software inspection technologies, in Handbook on Software Engineering ad Knowledge Engineering, vol.2, pp.517-555, 2002.

(Mantyla, 2009) M.V. Mantyla and C. Lassenius, What types of defects are really discovered in code reviews? IEEE Transactions on Software Engineering, vol.35, no.3, pp.430-448, 2009.

(McConnell, 2004) S. McConnell, Code Complete, 2^nd ed., Microsoft Press, 2004.

(Porter, 1996) A. Porter, H. Siy, and L. Votta, A review of software inspections, Advances in Computers, vol.42, pp.39-76, 1996.

(Russell, 1991) G.W. Russell, Experience with inspection in ultra large-scale developments, IEEE Software, vol.8, no.1, pp.25-31, 1991.

(Votta, 1993) L.G. Votta Jr., Does every inspection need a meeting? Proceedings of the ACM SIGSOFT Symposium on Foundations of Software Engineering, pp.107-114, 1993.