Good requirements practices are neither necessary nor sufficient

The title of this essay was selected to create controversy among the readers and perhaps cause some of you to become defensive. However, we will try to go about demonstrating why we sincerely believe that good requirements practices are indeed neither necessary nor sufficient, even though we are devoted to the field of requirements as active researchers and practitioners. In fact, after you read this article, we trust that you too will say, “Well, of course, I agree with you.”

To argue about the necessity or sufficiency of good requirements practices, we must first agree to (1) what is a requirements practice, (2) what is a good requirements practice, (3) what is the purpose of a requirements practice, and (4) what is it necessary and sufficient for? Only then can we argue why “good” requirements practices are neither necessary nor sufficient to achieve the intended purpose for requirements.

There is little uniformity in terminology concerning the classes of activities that make up requirements. However, we all seem to agree that the following activities are included:

• Gathering/eliciting/ascertaining/uncovering requirements from stakeholders

• Analyzing (and perhaps modeling and/or refining) those requirements for consistency, completeness, appropriateness, and so on

• Determining what subset of those requirements should actually be addressed given the constraining budgets and schedules

• Documenting the selected requirements

• Verifying that the specified requirements conform to all the quality standards

• Managing changes to requirements

We consider a requirements practice to be the use of a principle, tool, notation, and/or method in order to perform any or all of the above activities. Many sources of requirements practices exist, e.g., Davis [1], Gause and Weinberg [2], Gottesdiener [4], IEEE Standard 830 [6], Laueson [7], Leffingwell and Widrig [8], the Roberstons [9], Sommerville and Sawyer [10], and Weigers [11].

We consider a good requirements practice to be a requirements practice that either reduces the cost of the development project or increases the quality of the resulting product when used in specific situations. Few requirements practices have been validated as “good” in practice, and those that have rarely, if ever, describe the specific situations in which they are effective. However, we do have a variety of sources of requirements practices for which the authors do claim goodness. For example, Sommerville and Sawyer wrote a book titled Requirements Engineering: A Good Practice Guide [10], Weigers [11] provides a chapter on good practices, Young has written a book titled Effective Requirements Practices [12] and the Robertsons’ book discusses Mastering [9] the requirements process. One can think of a set of indicators that you can check your RE practice against to decide if your practices are “good” or not. Table 1 provides three examples of requirements practices that all of the above sources seem to agree are good.

Table 1 Consensus on some “good” requirements practices

The purpose of requirements is to raise the likelihood that the right system will be built, i.e., the system when built satisfies its intended customers and addresses their needs to an acceptable degree. Some may argue that the purpose is more short term, e.g., that its purpose is to ensure communication among all stakeholders, provide designers and testers with an oracle, guide project managers in allocation of resources, serve as a basis for requirements evolution, and so on. However, we argue that each of these short-term objectives are important only because of their strong correlation to the real purpose of requirements, i.e., to raise the likelihood that the right system will be built.

When we state that good requirements practices are neither necessary nor sufficient, we need to ask, “necessary and sufficient for what?” The answer is, “necessary and sufficient for success,” where success means some combination of on time, within budget, fulfilling the needs of customers, socially acceptable to the users, and (if the development organization is within a company different from the customer’s) fulfilling the targeted business goals of the development organization.

Now let us explore why good requirements practices are neither necessary nor sufficient for product success. Figure 1 shows the possible outcomes of a project with respect to whether or not it uses good requirements practices and whether or not it results in a successful product. Note that the abundance of projects in regions A and D are not surprising; clearly, we all believe that there is some correlation between performing good practices and product success. However, no data exist to indicate the relative percentages of projects that fall into region A vs. region B, nor the relative percentages of projects that fall into region C vs. region D. It is just our collective acts of faith that cause us to consider project managers who consciously decide to place their projects in row A–B to be acting responsibly, and those who select row C–D to be excessive risk takers. In any case, to prove that good requirements practices are neither necessary nor sufficient, it suffices to demonstrate the existence of just one project in each of regions B and C.

Fig. 1

Relationship of good requirements practices to success.

1 Not sufficient

To prove the insufficiency of good requirements practices, we need to disprove the statement that “good requirements practices always yield product success.” In other words, we need to prove the existence of at least one project in region B. This should not present difficulty for any of us. After all, if you do a perfect job of requirements but the subsequent design and coding stages introduce millions of errors, the resultant product will clearly not be successful. Even if the subsequent stages of software development are perfect, the resultant product could still fail miserably due to poor pricing strategy, ineffective sales and marketing, poor customer support, and so on.

To make our argument even more convincing, good requirements practices are highly dependent on project characteristics [3, 5]. Thus, what is a good practice for one project may not be a good practice for another. Furthermore, what does it even mean to perform good requirements practices on a project (i.e., to claim that a project is in row A–B of Fig. 1)? Does it mean that all the requirements practices employed are good for this project (i.e., that region C of Fig. 2 is empty)? Or does it mean that all requirements practices that are good for this project were employed (i.e., that region A of Fig. 2 is empty)? Or both (i.e., that the two circles of Fig. 2 are coincident)? We raise these questions to highlight the fact that our industry does not even know what it means to “apply good practices” to a project, let alone be able to assess whether such practices are beneficial or detrimental!

Fig. 2

Relationship of good requirements practices to success.

2 Not necessary

To prove the unnecessary nature of good requirements practices, we need to disprove the statement “product success implies that good requirements practices were employed.” ¹ In other words, we need to prove the existence of at least one project in region C. This may be more challenging to prove. However, look at this situation: Suppose you are about to start a new software development project. You either fully understand your customers’ needs (which will exist at product delivery time) or you do not. If you ignore requirements altogether (and thus fail to use good requirements practices), you will end up constructing a software system that reflects your understanding. If you were wrong in your understanding, your product will fail. If you were correct, your product might just succeed. So, now the question becomes “is it possible that you could already know your customers’ needs, that will exist at product delivery time?” We must assume that the answer is “yes, but highly unlikely.” Thus, success is purely a stroke of luck. What is the probability that you will be equally lucky next time?

3 Conclusions

You might ask why we wrote this article in the first place. The answer is to make sure that none of us (as requirements researchers or practitioners) get too overconfident in our quest to develop or apply good requirements practices. Remember,

• Do not discard a good requirements practice just because your project failed. We have seen too many practitioners adopt some “good” requirements practice only to discover that the resulting product failed, and the practice was subsequently abandoned. Without careful and objective post-mortem analyses, you have no way of knowing that the practice in question is the cause of the failure.

• Long-term results are more important than short-term results, even though the short-term results are easier to measure. Do not fall victim to goals displacement. Do not define a successful requirements practice as being one that provides only short-term results (e.g., one that increases the number of requirements agreed to per hour). This could motivate you to permanently adopt a practice that helps in the short-term but causes product failure in the long term.

• Requirements are but a small piece of a large whole ². Applying good requirements practices will not guarantee you success, nor will applying bad requirements practices guarantee failure.

“The race is not always to the swift nor the battle to the strong, but that’s the way to bet.” (Damon Runyon)