Keywords

1 Introduction

The SPI Manifesto [1] proposes applying risk management as a principle (Principle 7), as any improvement effort may go wrong or not work as expected. Risk management must be a proactive effort as it gives a chance to avoid or prevent problems that may lead to bad results in the future [1].

Several risks are inherent to software projects, such as those related to deadlines, budget and schedule estimation, technology evolution and stakeholder expectations, among others [2, 3]. Although the software projects risks cannot be completely eliminated, the impact of these risks can be reduced through its proper management [4]. Several research in the Software Engineering area have proposed solutions for risk management in software projects [5], increasing the importance of risk management in software development processes over time [6].

Risk planning and monitoring processes are typically included in plan-driven software development approaches [7], following practices defined in process reference models or standards both generic or specific for a software engineering domain, such as ISO 14971 [8], PMBOK [9], ISO 31000 [10], IEC 80001 [11] or IEC 62304 [12].

As an alternative to the plan-driven approaches [13,14,15,16], in the agile methods, the risk management is usually performed implicitly [17, 18], seeking to optimize predictability and, thus, keeping risks under control [19]. Common practices of agile methods, such as small increments, work visibility and expectations management, tend to be good risk mitigation strategies [17].

However, software projects that use agile methods also fail [22] for several reasons, such as: team capacity, customer involvement [23], inadequate size of the organization, lack of project management competence [24] and also the absence of explicit risk management [25]. In many ways, agile methods do not differ significantly from other traditional software development methods [20, 21].

In this sense, in certain scenarios the implicit risk management of the agile methods may not be sufficient [26,27,28]. Risk management has been carried out in software development projects to manage the impact of risks, especially in some highly regulated software development domains, such as healthcare or automotive [7], for example. In software development projects that use agile methods in other domains, risk management is often overlooked, despite its possible impacts [2, 3].

Some initiatives emerged proposing explicit risk management practices to the context of agile software development [27, 29,30,31]. However, to the best of our knowledge, none of the proposed approaches offers a comprehensive guide that provides elements for explicitly integrating agile risk management practices with different agile approaches [32].

Thus, our research question is: “How to integrate explicit risk management practices with agile methods?”. To answer this question, a state-of-the-art analysis was carried out, followed by the development and initial evaluation of a Guide for Risk Management in Agile software projects. The Guide is structured according to elements proposed by ISO/IEC 24774 [33] and the SPEM 2.0 meta-model [34]. The content of the Guide is based on the state of the art on risk management in agile methods [32] and in the classic project risk management literature [9,10,11,12, 20]. An initial evaluation of the Guide's content was carried out through an Expert Panel [35, 36]. The Guide is available in the form of a technical report and also with the support of a web tool.

The main contribution of this paper consists in the research and development of a guide for the integration of agile risk management in software projects that can be used by organizations to improve their risk management processes without losing the agile methods benefits. In addition, another important scientific contribution of this article is the research, systematization, and adaptation of risk management practices applicable to agile methods, collected from the traditional literature on risk management and on the state of the art of risk management in agile methods.

This paper is organized as follows: in Sect. 2 a theoretical background on risk management is presented, followed by Sect. 3 where the methodological approach is presented. In Sect. 4 the related works are presented and in Sect. 5 the development of the Guide is presented. Finally, in Sect. 6 an initial evaluation of the guide is presented, followed by Sect. 7, where the conclusions are presented.

2 Risk Management

In Software Engineering, project risk management typically include identifying risks, estimating them, mitigating its impacts, and monitoring. Risk management practices tend to lead to a disciplined scenario for decision making to manage problems in software development [4].

There are several concepts for risks. For project management, risks are events or uncertain conditions that, if materialized, affect at least one of the project's objectives [9]. Several standards and guidelines propose risk management practices applicable to software development, both generic (ISO 14971 [8], ISO 31000 [10, 52], IEC 80001 [11]) and for specific domains of software development, such as healthcare (IEC 62304 [12]) or automotive (Automotive SPICE [7]). One of the most widely accepted approaches to risk management in project management is proposed in PMBOK [9], where risk management processes are defined as:

  • Planning: definition of the schedule, budget, and resources to conduct risk management activities

  • Identification: identification and documentation of risks that may affect the project

  • Qualitative and Quantitative analysis: assessment of the risk exposure to prioritize risks

  • Plan risk responses: definition of the strategies and mitigation plans

  • Implement risks responses: execution of the planned responses to risks

  • Monitoring risks: monitor and control risks over the project's life cycle

Agile methods, however, in general do not include explicit risk management practices [30]. Agile methods propose a way to develop software that aims at frequent deliveries and adaptability to changes [37]. The Agile Manifesto [38] formalized these aspirations by defining a set of values and principles that characterize these agile methods [39]. Since then, several new agile methods have been proposed, both for managerial and technical aspects of software development, varying in practices and techniques, but sharing the common values described in the manifest [16] and bringing several benefits, such as: cost reduction, improved teamwork, improved confidence [47], increased productivity [48], improved project management [49], work visibility, better communication [50], among others. Agile methods claim to be risk-oriented and usually address implicit risk management [17, 18], given the nature of its iterative and incremental practices that tend to mitigate risks [17, 19].

3 Methods

To develop the Guide for Agile Risk Management in software projects, a multi-method approach was adopted, involving the analysis of the state of the art, research and development of the guide's content, development of a web support tool and the initial evaluation of the guide.

To analyze the state-of-the-art in relation to the integration of risk management in agile methods, a Systematic Literature Mapping (SLM) [40, 41] was carried out. An SLM allows for the identification, analysis and interpretation of relevant results from primary studies on a given research question. The SLM approach used followed the process proposed by Petersen, Vakkalanka & Kuzniarz [40] and Petersen et al. [41]. The most relevant results of this SLM are presented in Sect. 4.

The content of the Guide was developed based on the traditional literature on risk management and the results identified in the state of the art. The structure of the Guide is defined based on standards and models related to the documentation of software processes, such as ISO/IEC 24774 [33] and the SPEM 2.0 meta-model [34]. A web tool was also developed, in order to facilitate access and understanding of the Guide's content, following an agile development approach in an iterative and incremental life cycle [42]. Section 5 details the development of the Guide and the tool.

The initial evaluation of the Guide was conducted through an Expert Panel, which consists of compiling individual opinions of experts on a given topic, to analyze and evaluate a proposition [35, 36]. The Expert Panel was carried out following the Goal-Question-Metric (GQM) approach [43], which is based on the definition of goals, questions, and metrics. The GQM approach can be applied to measure and evaluate system and software processes, including risk management. The initial assessment is presented in Sect. 6.

4 Related Works

Given the importance of the risk management in agile approaches, there are some secondary studies that address the subject from different perspectives.

Tavares, da Silva & de Souza [4] identified and classified 127 risk management practices in 34 selected studies. Rafeek, Arbain & Sudarmilah [44] performed a Systematic Literature Review (SLR) on risk mitigation techniques in Agile Methods for Global Software Development (GSD), identifying 40 studies related to risk management. Hossain, Babar & Paik [45] also performed a SLR to identify challenges in risk management in Scrum, finding 20 studies and selecting and compiling best practices [46].

About the selection of appropriated risk-related practices for agile methods, Gasca-Hurtado [51] identified 28 techniques and tools and proposed a gamified approach for risk analysis in agile methods. First results of a case study applied to an undergraduate course discipline indicate that the approach is enjoyable, close to reality and promotes the active participation of team members. Although interesting, the approach is more focused on the learning of risk analysis practices than on a guide to support risk management in agile software development contexts.

As no secondary studies were found specifically looking for experiences of explicitly integrating risk management activities with agile methods in general, we performed a Systematic Literature Mapping (SLM), following the process proposed by Petersen, Vakkalanka & Kuzniarz [40] and Petersen et al. [41]. The detailed description of the performed SLM, including the research protocol and the analysis of the results, can be found in [32]. As a result of our SLM, 18 primary studies were found reporting experiences of explicitly integrating risk management practices into agile methods [32]. Among the primary studies, it was observed that the most used agile method to integrate explicit risk management practices is Scrum. Regarding the observed agile development practices, 10 of the studies presented evidence for all PMBOK risk management processes, with risk identification being the only process covered by all studies. In general, the selected studies indicated that the results of the integration of risk management practices with agile methods are positive, including improved communication, improved product quality, increased risk visibility, reduced costs, improved team efficiency and reduction of time-to-market [32].

Summarizing what was found in related works and in our SLM, it is possible to affirm that several efforts have been made in an attempt to establish risk management practices in agile methods, indicating the perception of an important gap of explicit risk management in agile that needs to be filled.

However, it was not possible to find in the related works the development of a comprehensive guide that allows the integration of adequate risk management practices to different agile methods so that can be used by an organization [32].

5 Guide Development

This section presents the development of the Guide for Agile Risk Management in organizations that do software development using agile methods. Based on a study of the literature and a survey of the state of the art, the Guide was structured and developed, following an iterative and incremental approach.

In addition to the development of the Guide in the format of a textual technical report, a web tool was also developed to facilitate access to the Guide content. In the following sections, technical aspects of this web tool to support the Guide are also presented.

5.1 Guide Structure

The structure of the Guide was defined based on the elements proposed in ISO/IEC 24774 [33] and the SPEM 2.0 meta-model [34], which provide a base structure for documentation of software processes.

Thus, the structure of the guide consists of the following elements: roles, events/ceremonies, activities, tasks, techniques, tools and work products. The core of the guide focuses mainly on activities, which include agile risk management tasks, implementing techniques and being carried out by roles, sometimes in the context of events/ceremonies, using tools and generating work products. The details of each component of the Guide are described below:

  • Roles: represents the occurrence of one or more people performing the tasks described in the activities and may have specific or multiple responsibilities.

  • Events/ceremonies: meetings with periodicity and exclusive characteristics, in which aspects of the project are discussed with the stakeholders and/or internal team. The term “ceremonies” is used in the Scrum framework.

  • Activities: lists of actions that can be performed to achieve predetermined results. Each activity can be designed as a group of small, related actions or individually.

  • Tasks: specific requirements, recommendations, possibilities or actions, which have a clear objective, in which steps are defined that represent the work necessary to achieve the defined objectives. Generally, an activity contemplates one or more tasks, while a task implements one or more techniques.

  • Techniques: sets of steps or systematic procedures to reach a certain result. While each task is exclusive and may have dependencies between them, the techniques can be reused and there is no interdependence between them.

  • Tools: definition of one or more resources (physical or virtual) that are useful, optional, or necessary for the completion of a task or technique.

  • Work products: the results, tangible or intangible, of an activity, task, or technique. A work product must enable measurement and quantification by the organization that owns the project.

5.2 Guide Content

The content of the Guide was developed based on the risk management practices reported in the state of the art [4, 29,30,31,32, 44, 45] and in the traditional project risk management literature [9,10,11,12, 20]. Initially taking as a reference the traditional risk management literature, the agile practices identified in the state of the art have been grouped in order to contemplate, at least in general, all the processes typically defined for risk management [9, 32]. Thus, the selection and adaptation of risk management practices was carried out without compromising the values and principles of the agile methods.

A process itself was not defined to the Guide, but only the components that can be used to complement virtually any existing agile process, avoiding being unnecessarily prescriptive. Instead, implementation alternatives were proposed, to allow adaptation to different organizations as needed.

As a result, 6 roles, 5 ceremonies, 7 activities, 16 techniques, 13 work products and 6 tools were defined (see Table 1). All Guide components were proposed as alternatives and can be used independently or together, depending on the organization's scenario.

Table 1. Content of the guide.
Full size table

The Technical Report containing the Agile Risk Management Guide is available at: http://www.incod.ufsc.br/wp-content/uploads/2020/05/Relatorio_Tecnico_GQS_GuiaGestaoAgilDeRiscos.pdf.

5.3 Guide Tool

In order to facilitate access to the content of the Guide, a web tool was also developed, named RM-Agile (Risk Management - Agile). The tool aims to present the complete content of the Guide, allowing the search and navigation of the content and providing other forms of support for its understanding, with better usability of the Guide compared to the Technical Report document (Fig. 1). A video tutorial was also included in the tool to facilitate understanding of the Guide and the tool.

Based on the structure and content developed for the Guide, the requirements of the Guide support tool were collected and analyzed, with the main objective of facilitating access to its content. The tool main features were then defined, including: the storage of all Guide components, navigation between its different parts, the possibility of sending feedback on the Guide and the inclusion of a tutorial, among others.

From the initial list of requirements, low-fidelity prototypes of the tool screens were then elaborated through wireframes. The prototypes were refined and cross-validated by the authors.

Fig. 1.
figure 1

RM-Agile tool presenting part of the guide.

Full size image

The tool was then implemented and tested, following an iterative and incremental agile process. The technologies used for the development of the tool were selected based on the authors' previous experience, in order to facilitate its development. The following technologies were used: Javascript, PHP, H5P, HTML and CSS, allowing the implementation of the requirements.

The RM-Agile tool can be accessed at: https://rm-agile.herokuapp.com. The source code is available at: https://github.com/vieiramarcel/ferramentaRisco.

6 Initial Evaluation

After the development of the first version of the Guide and the tool, an initial evaluation was carried out through an Expert Panel. An Expert Panel consists of compiling individual opinions of experts on a given topic to analyze and evaluate a proposition [35] and can be applied in multiple contexts that require expert consensus opinions, having been used in several areas, such as medicine and software engineering [36].

To define the objectives of the evaluation and systematize the conduct of the evaluation process of the Guide, the GQM approach [43] was used. Using the GQM approach, the following goals have been defined for the evaluation of the Guide:

  • Goal 1: Evaluate the scope and applicability of the Guide for Risk Management in Agile Software Projects, from the point of view of an expert panel.

  • Goal 2: Evaluate the coverage of the content of the Guide for Risk Management in Agile Software Projects, from the point of view of an expert panel.

Following the GQM approach, the evaluation goals were derived in questions. As a data collection instrument, a questionnaire was elaborated containing 11 questions, 9 of which were questions with a Likert scale and a space for comments, and 2 were open-ended questions. The questionnaire was then reviewed by the authors and made available on the Google Forms tool. Table 2 presents the questions.

Table 2. Evaluation questions.
Full size table

Professionals with proven training and experience in software project management using agile methods were invited as experts. From a LinkedIn search for professionals with the desired profile, among the invited, four independent specialists from four different companies accepted the invitation and carried out the complete evaluation of the Guide. The participating experts were project managers in companies of different sizes, have an average of 7 years of experience in software project management with agile methods and have degrees in Computer Science, Information Systems, Administration or Logistics.

The initial evaluation of the Guide was carried out though interviews with the selected experts during December/2020. The Guide was made available to participants both in the Technical Report format and through the web tool. Instructions were also provided to the participants on the objectives of the Guide, how to access and use the content of the Guide and how to conduct the evaluation.

6.1 Analysis

The analysis of the results collected from the responses is presented in this section grouped by each Evaluation Goal and its respective questions. Figure 2 presents an overview of the evaluation results.

Fig. 2.
figure 2

Evaluation results.

Full size image

Goal 1: Evaluate the Scope and Applicability of the Guide for Risk Management in Agile Software Projects, From the Point of View of an Expert Panel.

As can be seen in Fig. 2, for the Q1 question two experts fully agreed that the Guide is adaptable to different domains, such as finance, education, health and e-commerce. The other two experts agreed on the adaptability of the Guide. In the question Q2, three experts fully agreed that the Guide is adaptable to different agile methods, such as Kanban, Scrum and XP. An expert agreed on the adaptability of the Guide in this regard.

A small disagreement was observed in question Q3. For this question, one expert fully agreed on the Guide's adaptability to organizations of different sizes, two experts agreed that the Agile Risk Management Guide is adaptable. An expert disagreed with this point, commenting that in the Guide there were no clear demonstrations of how to adapt to the reality of different organizational sizes.

For question Q4, three experts agreed that the Agile Risk Management Guide addresses activities, techniques, and tools that can effectively be used in practice. An expert totally agreed with this question. In question Q5, two experts fully agreed that the Agile Risk Management Guide addresses ceremonies and roles that can be effectively applied in practice. The other two experts agreed on this aspect of the Guide.

Goal 2: Evaluate the Coverage of the Content of the Guide for Risk Management in Agile Software Projects, From the Point of View of an Expert Panel.

In question Q6, two experts fully agreed that the rationale for the Guide provides the necessary background for understanding the Guide. The other two experts agreed with this question. No further comments were made. For Q7, three experts fully agreed that the content of the Guide is in line with the agile principles and values defined in the agile manifest. An expert agreed with this question. No further comments were made.

About the content scope, for the question Q8, two experts totally agreed that the Guide has enough activities, techniques, tools, and other components for its practical application. The other two experts agreed. In Q9, three experts fully agreed that the Agile Risk Management Guide has sufficient details of the elements that comprise it, such as activities, techniques, tools etc. An expert agreed with this question.

In relation to open-ended questions, the four experts did not identify any other activities, techniques or tools needed for agile risk management in question Q10. Also, the four experts did not identify any error or need for improvement in the content or format of the Guide in Q11.

Even though it is only an initial assessment, with a small group of participants, general analysis of the results, a preliminary insight can be identified that the Guide has good applicability and scope, with the potential to be applied in various types of projects and organizations that use agile models. However, the applicability of the Guide to projects of different company sizes needs to be made more explicit.

Regarding the coverage of the content of the Agile Risk Management Guide, the preliminary results indicate that the Guide includes the content necessary for its practical application.

6.2 Threats to Validity

Possibly the most relevant threat to validity refers to the number of participants of the Expert Panel, consisting of only 4 participants. However, considering that this is an initial evaluation and that a small sample size is common in this type of study [36], we consider that this was sufficient to raise initial indications about the scope, applicability and coverage of the Guide. In addition, the authors sought to mitigate this threat by selecting independent and experienced specialists, with different backgrounds, from different companies and directly involved in software development projects that use agile methods, seeking to diversify and bring different views to the evaluation panel.

Other threats to the validity identified were in relation to the responses, as there were few comments and suggestions for improvements in the Guide for Agile Risk Management, which hindered a deeper analysis of the results, about the theoretical content developed.

7 Conclusion

This article presents the research and development of a Guide for Risk Management in Agile software projects, designed to support software development organizations on managing risks in projects that use agile methods. The Guide was developed starting by analyzing the traditional literature on project management and the state of the art in risk management integrated with agile methods, and then carrying out the selection and adaptation of risk management practices to form the content of the Guide. The Guide was made available as a technical report and can also be accessed through a web tool.

The definition of the Guide's structure was based on standards and models for formalizing software processes. Thus, the content of the Guide was structured in ceremonies, roles, activities, techniques, tools and work products, with the intention that it can be adaptable to the reality of different types of projects, organizations and agile methods. To facilitate access to the content of the Guide, a web tool was developed, named RM-Agile, which presents the content of the Guide, in addition to other features for its understanding and use, including a video tutorial and a module for sending feedback.

An initial evaluation of the Guide was carried out through an Expert Panel in order to assess the applicability, scope and coverage of the Guide's content. The results of the initial evaluation raise indications that the Guide can be considered applicable to different contexts and its content is sufficiently comprehensive to be used by software development organizations.

As a future work, it is planned to apply the Guide in case studies to observe its use in practice and collect data regarding the possible impacts of its use. Another future work includes providing pre-defined profiles and instructions for adapting the Guide's components to different types of organizations’ profiles, as the Guide currently provides several activities, techniques and tools, but does not specifically address different organizational profiles, depending on the user of the Guide to interpret these needs and make these adaptations.