Keywords

1 Introduction

Is risk management worth practicing in software projects? The initial reaction is probably a conditioned: “Yes!” Upon reflection, however, there may be equivocal answers. Conceptually, we know that risk management is a good, indeed, necessary, thing to do. However, actual experience with risk management in practice may be less forthright. Also, the movement towards Agile practices has tended to mitigate some risks through the iterative nature of the development process itself, seemingly obviating the need for an explicit add-on risk management process.

This chapter re-examines software project risk management from research and practice perspectives (the scope is limited to commercial software and systems projects, excluding specialist domains such as safety critical systems). It builds on prior work in Bannerman (2008c). It aims to highlight key aspects of what we currently know from empirical research and uncover opportunities for developing further knowledge through future research. A central contribution of the chapter is the description of six current and emerging risk management perspectives—and their underlying theoretical schools of thought—that frame current research approaches and promising areas of future work. The central argument is that software project risk management is still in its infancy, but evidence suggests that benefits are real. Specific opportunities are identified to extend risk management in practice and diversify research.

First, the next section reviews the status quo of software project risk management in research and practice, focusing on evidence relating to its impact on project performance, adoption in practice, and implementation barriers and enablers. Then the perspectives and foundational schools of thought that frame current approaches to risk management are outlined. Finally, implications of this research-practice landscape and conclusions are drawn for the future.

2 State of Play

Prior study has found a research-practice anomaly: risk management research lags the needs of practice and risk management practice lags the prescriptions of research (Bannerman 2008c). In this (and the next) section we examine the current state of risk management research and practice in software projects to determine if progress has been made and seek to uncover roadblocks that need to be cleared by researchers and practitioners going forward. First, this section considers evidence of risk management’s contribution to project outcomes; trends in the adoption/non-adoption of risk management in practice; and barriers and enablers to risk management utilization in practice.

2.1 Does Risk Management Improve Project Performance ?

We usually take for granted that risk management is a good thing; that it helps to remove or reduce potential problems that could affect the successful outcome of a project. Considering the significant cost and effort in applying risk management in software projects, a pivotal question, therefore, is whether it does actually make a difference: does empirical research show that use of risk management is positively related to project performance?

Overall, evidence suggests that risk management is related to project success. However, the answer to the question is complicated by the multidimensionality of the concept of project success (Bannerman 2008b), the multiple categories of risks that can impact a project, and the range of available risk response treatments (Bannerman 2008c). As a consequence, achievement of a particular type of project success (such as process success or product success) will likely require appropriate management of particular types of project risks that are contingent on the focal success criterion. Several research examples illustrate this finding (see also Na et al. 2007 for further background).

In a study of 86 experienced information systems (IS) project managers, Jiang and Klein (1999, 2000) found that three risk categories (lack of clarity of role definitions among team members, application complexity, and lack of user experience on applications) were most significantly related to system success. Further, in a study of 194 IS project managers, it was found that success also requires matching risk reduction strategies to the risks identified in the project (Jiang et al. 2001). System success was significantly related to behavioral-related risks, behavioral-based strategies (most influential), and technology-based strategies.

Similarly, Wallace et al. (2004b) found that social subsystem risk influences technical subsystem risk, which influences the level of project management risk and, subsequently, project performance. The latter was viewed separately as product performance and process performance. Product success was critically dependent on managing customer/user risks, scope and requirements, and project execution, while process success was dependent upon managing scope, requirements and project execution—the risks over which project managers feel that they have the most control (Wallace and Keil 2004).

Further, in a study of over 100 projects, Raz et al. (2002) found that risk management practices are more correlated with success in meeting time and budget goals than with product performance success. Overall, however, they conclude that higher levels of risk management are associated with higher levels of project success.

Finally, recent case study research supports this conclusion with the finding that individual risk management activities contribute to project success (as defined by stakeholder opinion) via direct instrumental effects and communicative effects that create a shared view of the project situation (de Bakker et al. 2012).

2.2 Risk Management Adoption /Non-adoption in Practice

Few studies have specifically focused on the adoption of risk management in software projects. Most evidence is provided as secondary observations in studies with other foci. Overall, this evidence suggests that risk management awareness is high but practices are not widely or consistently used; adoption of early process model practices is quite common (such as risk identification and assessment); adoption occurs early in the project and risk management life cycles but then tends to drop away; and integration with other project management and software development practices is varied but generally under-developed. In general, risk management practice adoption lags the prescriptions of research. Several examples of research illustrate support for these conclusions.

First, a study of 83 project managers found that while a large number were engaged in some form of risk management activity, 75 % of the project managers did not follow any detailed risk management approach, and only vaguely understood the software risk concept and its managerial implications (Ropponen and Lyytinen 1997; Ropponen 1999).

Further, a multi-industry study of project management maturity based on PMI’s knowledge areas found that the risk knowledge area had the lowest maturity of all knowledge areas in the IS industry, and the risk maturity level of the IS industry was the lowest of the four industries in the study (Ibbs and Kwak 2000).

More recently, a study of 37 software organizations compared the industrial risk models used against a best-practice model synthesized from the literature (Nyfjord and Kajko-Mattsson 2008a). In support of the earlier findings, it was found that most of the organizations had defined a risk management process model; most used the risk identification and analysis phases but use dropped off for the other phases; and only some of the core activities in the comparison model were used in the phases that were used. Process integration was found to be still in its infancy. Most organizations had integrated their risk management processes with the software processes to some degree but in an ad hoc manner, without a defined integrated process model (Nyfjord and Kajko-Mattsson 2008b).

2.3 Risk Management Barriers/Enablers in Practice

If research evidence suggests that risk management is worthwhile, contributing to successful project outcomes, why is adoption of risk management practices so limited? The answer may lie, at least in part, in an array of barriers that can reduce or make implementing risk management unattractive in practice (Gemmer 1997; McGrew and Bilotta 2000; Dedolph 2003; Kwak and Stoddard 2004; Bannerman 2008c; Kutsch and Hall 2009; Kutsch et al. 2012). Risk management implementation barriers include:

  • Cost: Formally and explicitly practicing risk management can add a significant process overhead to software projects and detract attention from the main game of developing or implementing quality software-based systems. Applying risk mitigation strategies and developing/testing contingency plans can be expensive in resource effort and schedule time, as well as financial cost. Projects may also be considered too small to justify these overheads. Further, some software development methods, such as Spiral and Scrum, may be considered to have sufficient inherent risk management features that an additional process is not justified. Combined with other barriers, stakeholders may conclude that the costs of risk management outweigh the benefits.

  • Culture: Companies vary in their position on the risk acceptance–risk aversion scale, which can influence the extent to which risk management is practiced (if at all). Some companies view risk management as negative thinking, while some managers view practicing risk management as impugning their ability to manage organizational activities through to successful completion. Others methodically manage risks because it is a company practice standard to do so. Organizational culture can work for or against effective risk management.

  • Uncertain benefits: This is also referred to as causal ambiguity or the ‘Y2K effect’. As noted above, overall, it is difficult to show that risk management improves project performance (most empirical evidence points to specific risk factor, risk strategy or project component performance effects). Risk management interventions may change the course of a project but it is difficult to determine whether these changes actually reduced the threat and avoided a negative impact on project outcomes and whether the risk response was appropriate (or an overkill) for each treated risk. This causal ambiguity can greatly dampen enthusiasm for risk management in practice.

  • Capability: While risk management awareness is high, expertise in practicing risk management processes and techniques is often underdeveloped and a secondary priority to other project-related activities. Consequently, adoption and effectiveness of risk management practices can be constrained by limited individual and organizational capabilities in applying risk management.

  • Ownership: In software projects, responsibility and process ownership of risk management are often not explicitly defined, resulting in an absence of proprietorship of the process. Project managers tend to view risk management as the responsibility of project governance while governance bodies see themselves as too remote from the action to accept full accountability. Ultimately, however, it is the role of governance to ensure that the operational responsibility for risk management is assigned and overseen.

  • Reviews: Post-project reviews are one of the least performed project management activities and, when they are held, the role and contribution of risk management to the project is even less likely to be considered. Without a review, the value of risk management is unlikely to be understood and the process is unlikely to be continuously improved to deliver increasing benefits. Compounding this problem, project success is usually claimed by the project manager or sponsor as the result of good project management rather than attributed to good risk management. Monitoring and reviewing the process can provide visibility into its value and effectiveness.

  • Reward structures: Typically, project reward structures are directed towards ‘on-time, within-budget, to-specification’ delivery, not identification and mitigation of potential barriers to success. The view tends to be that if risk management contributes to project success then well and good, but it is not an objective in itself.

Key enablers of risk management adoption and utilization avoid these barriers and build a climate that encourages proactive risk management thinking and practice. Enablers include:

  • Senior management commitment and support: Executive support provides a strong signal to the project team that managing risks is aligned to organizational priorities, practices and expectations.

  • Governance: Holding governance bodies ultimately accountable for process improvement and benefits realization from project and risk management activities ensures that project practices are aligned to expected standards.

  • Empowerment: Empowering and developing individuals and project teams to deliver and defend project objectives fosters risk awareness and buy-in to risk management processes.

  • Culture: Institutionalizing risk management into the culture of how software projects are run makes risk management the normal way of smoothing and securing business-as-usual and project activity outcomes in the organization.

  • Continuous improvement: Monitoring, evaluating and continuously improving risk management practices and techniques sharpens skills and increases returns on investment in practices over time.

  • Highlighting successes: Acknowledging and celebrating risk management successes communicates and reinforces the benefits derived from risk management practices throughout the project and organizational communities.

3 Risk Management Perspectives

This section overviews the research foundations of six risk management perspectives, namely, risk management as: factor analysis; a rational process; modeling; a social process; a capability; and data analytics. The first three reflect common current practice, while the last three are emergent approaches that offer potential additional contributions in diversifying risk management in the future. The aims of this analysis are to highlight the current and possible potential scope of risk management in software projects, and provide a framework within which to encourage and promote further research and practice development.

3.1 Risk Management as Factor Analysis (Contingency School )

Arguably the most recognizable approach to risk management in software projects seeks to identify and mitigate contextual variables that might threaten a successful project outcome. The perspective reflects the logic of the contingency school of management which argues that situational factors—inherent to the activity and its environmental context—may influence the outcome of the activity (in this case, a software project). Therefore, from a risk management perspective, to ensure successful project performance, it is necessary to identify the specific factors that might impact the project and manage them away to avoid or minimize their impact on outcomes.

Within this perspective, three broad research approaches are evident from the literature. The first and most common approach aims to identify and mitigate individual risk factors that might impact a particular project or project type. These factors are commonly cited in ‘top ten’ checklists of risk/success/failure factors that must be managed to ensure the project succeeds. Example lists are provided by Alter and Ginzberg (1978), Keil et al. (1998), Boehm (1991), Heemstra and Kusters (1996), Sumner (2000), Houston et al. (2001), Johnson et al. (2001), Schmidt et al. (2001), Addison and Vallabh (2002), and Han and Huang (2007). Risk factors include, for example, clear objectives, firm requirements, user involvement, senior management commitment, and skilled resources. According to this approach, each factor is individually assessed to determine its probability of occurrence, likely impact, and relative priority (in comparison to other identified risk factors), and is individually mitigated to manage the threat. Depending on the complexity of the project and the organization’s risk tolerance (or aversion), risk management under this approach can carry a substantial effort overhead and cost for the project.

The second factor analysis approach seeks to identify categories of sources of risk factors that could impact the project. The rationale here is that risk factors often arise from common sources of exposure or threat (Barki et al. 1993), and may be treated in common by one or more specific control measures, rather than treating each factor separately. This approach can leverage risk management effort and reduce cost. Risk identification, therefore, seeks to uncover the critical categories of threats that might affect the project. Examples of source categories are provided by Boehm and Ross (1989), Barki et al. (1993), Carr et al. (1993), Chittister and Haimes (1993), Heemstra and Kusters (1996), Keil et al. (1998), Cule et al. (2000), Ropponen and Lyytinen (2000), Jiang et al. (2002), Tiwana and Keil (2004), Wallace and Keil (2004), Wallace et al. (2004b), and Han and Huang (2007). Examples of source categories include requirements, technology, client, people, organization, and environment.

The third factor analysis approach aims to identify and apply higher level project dimensions that might have risk implications for the project, particularly in contexts characterized by high levels of change and/or uncertainty (hence the focus on broader dimensions of risk). This typically involves an explicit contingency approach to factor analysis that identifies and applies environmental dimensions (high level factors) to determine an overall risk profile for each project. Each dimension is usually scaled to reflect various levels of risk exposure. Assessing the project against the scale of each dimension—often in the form of a radar (or spider) chart—enables the risk profile of the project to be plotted for analysis of where risk treatments might be best applied. Examples of this approach are provided by McFarlan (1981), Ropponen and Lyytinen (2000), Wallace et al. (2004a), Shenhar and Dvir (2004), Han and Huang (2007), Howell et al. (2010), and Taylor et al. (2012). Shenhar and Dvir, for example, identify four key project dimensions against which project risk can be assessed: product novelty, technological uncertainty, complexity, and pace.

The main benefit of risk management as factor analysis is that it focuses attention on potential threats to a particular project. Its key limitation, however, is maintaining the effort and discipline in identifying, treating, and tracking relevant factors throughout the course of the project.

3.2 Risk Management as a Rational Process (Process School )

Closely associated with factor analysis is risk management as a rational process, which prescribes processes for managing project risk. Processes typically specify predefined (planned) stepwise tasks to follow throughout the risk management life cycle. Identifying, assessing and treating risk factors (the focus of the previous approach) are common steps within such processes. This approach draws on the logic of the process school of management which argues that organizational performance (including in project organizations) can be improved by performing work according to defined, efficient, repeatable and managed processes. Processes can incorporate accepted ‘best practices’ to improve efficiency and effectiveness in completing routine operational tasks. Such processes can be embedded into the routines of the organization as operational capabilities, improving the prospects of consistent, strong performance outcomes. According to this view, risk management is best approached as a managed process.

Risk management processes tend to lie on a continuum that reflects the level of certainty/uncertainty (or ease of identifying and responding to risks) in the project environment. The most common processes are plan-based, which are positioned at the certainty end of the continuum where it is assumed that risk factors can be explicitly identified, assessed and treated. Conceptually, these processes share similar steps that are executed iteratively throughout the project. These typically include: risk strategy; risk identification; risk analysis (or assessment); risk response; and risk control (or monitoring and review). Examples of such process approaches include: Boehm (1991), ISO 31000 (2009), PRINCE2 (2009), PRAM (2010), CMMI (2010), and PMBOK (2013).

At the other end of the continuum, where knowledge of the project and its environment is incomplete or highly uncertain due to high levels of change, complexity and/or ambiguity, the assumptions of plan-based processes no longer apply. In particular, relevant contingent factors cannot be readily identified within sufficient time for mitigations to be determined and applied to manage the risks. Two alternative process solutions that draw from the innovation management literature are relevant here: trial and error learning and selectionism. Trial and error learning refers to iterative incremental adjustments to project activities and targets in response to emerging new information. Selectionism refers to trying out several different solution strategies in parallel and selecting the best variation ex post. Examples of this approach are provided by: Pender (2001), Pich et al. (2002), Sommer and Loch (2004), and Loch et al. (2006). A related third approach, scenario analysis, is drawn from the strategy management literature. It confronts uncertainty by considering alternative possible futures (scenarios), the likelihood and consequences of each scenario, and the risk mitigation strategies that might be applied to avoid the negative effects of each scenario (for example, Ahn and Skudlark 2002; Bahi and Rivard 2003).

The major benefit of risk management as a rational process is that it provides a defined map to follow to manage risks under different environmental conditions. The main limitation, however, is the potential for complacency in executing the process mechanically and/or assuming that it will guarantee a successful outcome.

3.3 Risk Management as Modeling (Management Science School )

Less commonly used in software projects (other than in software-based safety critical systems and hazard analysis applications), risk management as modeling aims to represent the risk management context as an abstraction of a phenomenon that can be analyzed to determine and/or assess threats to its successful operation. In essence, this perspective represents a loosely related collection of analysis techniques that are unified by a modeling approach to managing risk. It is influenced by the traditions of management science which apply advanced analytical methods—usually but not always quantitatively-based—to help make better decisions and produce better outcomes. Applied to software projects, risk management modeling can help ‘flesh-out’ complex project and/or system dynamics and dependencies to clarify risks, responses, decision options and/or paths forward based on probabilistic and non-probabilistic analysis techniques in a structured manner. Examples of analysis techniques used in modeling risk include: fault tree analysis; event tree analysis; root cause analysis; cause-consequence analysis; failure mode and effects analysis; decision analysis; expected value analysis; Bayesian network analysis; system dynamics, Markov analysis; Petri net analysis; Monte Carlo simulations; fuzzy logic; and genetic algorithms (Prichard 2005; Borison and Hamm 2010; Kwan and Leung 2011; Li, Chen and Feng 2012; Xiao et al. 2013). Chapter 49 of this handbook, for example, proposes a framework for modeling complex project risk interactions for analysis in a network structure diagram. When used in software projects, because of the overheads involved in these techniques, risk management as modeling tends to be used in high stakes projects involving complex software-based systems.

A particular benefit of risk management as modeling is in contexts that permit and require precise mathematical analysis to manage risk. By contrast, the risk analysis techniques used in the risk management as a rational process perspective tend to be less sophisticated and more qualitative. This, however, does not make them inferior. The ‘best’ risk management approach is likely to be from the perspective that aligns closest to the needs of the project and its context. The main limitations of risk management as modeling are the measurement/data, analysis overhead, and analytical expertise required. Careful justification of the chosen approach is needed to ensure beneficial returns.

The following three perspectives are emergent, with isolated support in the literature. They are profiled here as promising software project risk management perspectives that may attract wider acceptance in research and practice as their respective benefits are confirmed.

3.4 Risk Management as a Social Process (Behavioral School )

Project management has a long history of recognizing and balancing the key roles of technical elements and people’s behavior in projects (Slevin and Pinto 2004). Similarly, behavior in social (team, project and organizational) contexts is both an important enabler of risk management and a source category of potential project risks. Therefore, it is not surprising that risk management as a social process has attracted research attention. This perspective is influenced by the organizational behavioral school of thought which seeks to improve organizational effectiveness by understanding the impact of individual and group behaviors.

March and Shapira (1987) set the stage for a behavioral perspective on risk management by arguing that managerial approaches to risk follow a different process to that assumed by classical conceptions. By contrast, managers are quite insensitive to probability estimates of possible outcomes; their decisions are driven by a focus on current performance targets; and they sharply distinguish between taking risks and taking a bet on a particular course of action. Lyytinen et al. (1998) extend the departure from a “rational calculus” approach to risk management by integrating a behavioral view embodying organizational attention shaping routines in a socio-technical model of organizational change. According to this approach, “risk management forms a continuous exercise where project managers engage in multiple maneuvers to master their environment” (page 236).

Other research within this perspective includes a focus on desirable functional behavior (Gemmer 1997); organizational culture and human behavior (Kwak and Stoddard 2004; Krivkovich and Levy 2013); project risk as a subjective social construction (Zhang 2011; Lim et al. 2011); and how project managers define and react to risk (Moeini and Rivard 2012).

Research on behavioral approaches to organizational work has a strong foundation in other domains (such as organizational theory) as well as in software project management more generally. Application and development of this knowledge could be beneficially extended to the software project risk management domain. Its key contribution is in highlighting the role of people, as individuals and in socio-cultural settings, as the primary actors in identifying and managing project risks. Its key challenge is the tendency for IS practitioners to fixate on technical tasks and solutions at the expense of human agency.

3.5 Risk Management as a Capability (Learning School )

This perspective focuses on identifying and developing personal skills and organizational capabilities that are important in successfully managing projects. It has attracted research attention since the turn of the century (Bannerman 2013). It recognizes that managing risk is about the ability to ‘do it’, not just ‘plan it’—particularly in dynamic, uncertain and complex environments. We have seen that while there is high awareness of risk management in practice, the level of understanding and application of risk practices is low. This capability gap limits a project’s ability to practice effective risk management as well as to be responsive to unforeseen disruptive events. This perspective is influenced by resource-based theory and organizational learning theory which argues that an organization’s ability to perform well (and better than industry competitors) is critically dependent on its ability to learn from experience and accumulate distinctive capabilities (Levitt and March 1988; Barney and Clark 2007).

Examples of research from this perspective include contributions on: real time management of risks (Jaafari 2001); learning as a strategy to cope with information inadequacy (Pich et al. 2002); trial and error learning as a strategy to manage innovation under uncertainty and complexity (Sommer and Loch 2004); capability-based project performance (Bannerman 2008c, 2012, 2013); and managing unforeseen project contingencies (Thamhain 2013).

Underpinning this perspective is recognition that risk response strategies cannot always be planned. Sometimes project threats arise quickly, clouded in an information vacuum. What is needed in these situations is a risk response capability, similar to that used in crisis management, which cannot follow traditional processes or permit time for detailed analyses (see Bannerman 2008a, 2008c for further discussion on this perspective). The key benefits of this perspective are that it focuses attention on the ability to respond to project threats quickly and do risk management well. The main limitation is that it involves complex intangibles (such as individual and organizational learning and capability development) that are difficult to examine in research and measure in practice.

3.6 Risk Management as Data Analytics (Business Analytics School )

For software projects, this perspective is more speculative and may ultimately emerge as a subset of the modeling perspective. However, its scope is potentially broader than modeling so it is considered as a separate, emergent perspective. The approach reflects the increased interest in business intelligence, business analytics, and big data that has emerged over the last decade (Davenport et al. 2010). The underpinning logic of this approach is that current and accumulated organizational and project data can be a rich source of information to support risk identification, risk strategy formulation and decision making. Data analytics can help understand what happened in the past that needs to be changed (historical view), what problems exist now that need action (current view), and what is likely to happen in the future that needs to be managed (future view).

Of course, adoption of this approach assumes the availability of, and ability to generate, relevant project-related data and risk-related metrics, as well as skilled analysts and a tool-box of data analytics techniques. These are not insignificant hurdles to overcome. As such, they may limit application of the perspective to large, high stakes projects and complex software-based systems.

4 Conclusions

This chapter has briefly reassessed the state of risk management in software projects with the aims of highlighting what we currently know from empirical research and uncovering opportunities for further knowledge development through future research. Key implications and conclusions from the reassessment include:

  • Development of risk management in software projects remains slow, in both research and practice. Risk management research still lags the needs of practice and risk management practice still lags the prescriptions of research. Opportunities exist to both extend the application of existing research-based knowledge in practice and broaden how risk management is viewed and studied in research.

  • Further research is needed into the causal linkages between risk(s), risk strategies, and project performance. Understanding risk-related paths to project performance and the boundaries of risk management are foundational to legitimizing a role for risk management in software projects and achieving demonstrable benefits from risk management practices.

  • Current perspective-based approaches to risk management in software projects are narrow, reflecting a domain discipline still in its infancy. Opportunities exist for further development within the core current approaches as well as in the alternative emergent perspectives. In particular, to further apply risk management as modeling in software projects and diversify research into the role of people and social interactions in risk management; develop capability-based as well as plan-based risk management; and business intelligence/data analytics to identify source categories of risks.

The path forward would likely benefit from closer interaction between the research and practice communities, perhaps via joint participation in focused research-industry conferences and workshops, as well as in industry-based studies. Such interaction could mutually reinforce development of the interests of both parties.

As the use of software-based systems continues to expand, software development methods evolve, and project management improves, the need for a strong and effective risk management capability is likely to remain. The most capable organizations are likely to adopt multiple risk management perspectives and match them to the project context, rather than default to a single approach. When this is common practice in industry, benefits and progress will likely be palpable.