Keywords

1 Introduction

Currently the different Agile approaches are used worldwide to develop software, with distributed Agile teams becoming more and more common. A recent study performed at the end of 2019 by VersionOne on “The State of Agile” [1] reports that 95% of interviewed companies use agile development methods with 51% respondents stating that it is used in more than half of their teams.

The study performed by Sharma et al. [2] indicates that the Scrum framework is the most popular of all Agile frameworks and methodologies both in industrial use and in scientific research. According to Sharma’s research Scrum is constantly gaining popularity in the industry. Many teams have adopted the Scrum framework. This naturally led to the scaling up of Scrum or other frameworks, as well as their adaptation to distributed teams. However, introducing Agile practices to a distributed team requires overcoming multiple communication obstacles [3] and creating a transformation strategy [4]. The initial agile frameworks like Scrum or Extreme Programing (XP) were created for small, co-located teams. Teams small size and co-location facilitates communication, cooperation, self-organization and allows for quick reactions to rapid changes on the market. However, as globalization progressed, distributed teams started becoming a worldwide standard. In order to still benefit from the advantages provided by Agile frameworks and methodologies the practices needed to be adjusted to the new characteristic of distributed teams. Such transformations were already successful in the past [5, 6], usually Agile Software Development Teams (ASDT) were using a mixed approach in order to facilitate communication, increase transparency and reinforce feedback loops in distributed environment.

The core of these mixed strategies were online tools. Their introduction allowed for maintaining communication and knowledge sharing between distributed team members [7]. However, communication facilitation is not sufficient on its own in distributed teams. The transparency of teams’ work is significantly reduced in distributed environment. The initial solution was to introduce tools to visualize tasks and to track everyone’s work [8, 9]. Such solutions worked, however they turned out to be insufficient for more matured ASDT. These teams and organizations started to introduce different metrics [10], customized to the individual characteristic of the team and organization. Today, due to automatization and the use of online tools, such metrics sourced additional information from already existing data, without impacting team members’ every-day work.

Therefore, the best results can be achieved by the use of both communication tools and metrics, as they complement each other. By using both, the team can easily communicate, visualize current work and observe their progress, effectiveness, quality of the product and distribution of effort. This enables the ASDT to make data driven decisions at any time.

1.1 Problem Statement

In early 2020 the global Covid-19 pandemic started. Multiple countrywide lockdowns and market uncertainty forced small [11] and large [12] organizations to reevaluate their business plans. Moreover, all the software development teams were forced to start working from home, creating an additional challenge for management and teams to organize remote work in a very short time. This was an unprecedented situation. All the teams, almost instantaneously had to start working remotely, making every team distributed at least within a single country.

1.2 Objective

The work presented in this paper aims to build an initial understanding of Covid-19’s influence on ASDT’ organization of work. The objective of this work is to determine what metrics and tools are used by ASDT and how the Covid-19 pandemic impacted the work organization of ASDT. The following research questions were created:

  • How the ASDT responded to the circumstances of the Covid-19 pandemic?

  • What tools and metrics are used by ASDT?

1.3 Contribution

For the purpose of this study a total of 120 answers from different Agile software development practitioners were examined. The respondents fulfilled different roles from regular team members to C-level management and came from a wide spread of industries and organization sizes. The survey consisted of questions investigating the characteristic of the organization, the impact of Covid-19 on the teams’ work and what tools and metrics are used.

1.4 Overview

In the second chapter an overview of related works was presented. The third chapter describes the research design and methodology. The forth chapter presents the survey results and was divided into three subsections. The first subsection presents the respondents characteristics, the second section describes the Covid-19 pandemic impact on ASDT, the last subsection presents the tools and metrics used by ASDT. The fifth chapter presents the discussion of the survey results and indicates possible future work. The last chapter contains the conclusions of the survey study.

2 Related Work

Not much research has been published describing the Covid-19 pandemic’s influence on ASDT as the issue is new. In the history of software development there is no precedent for such a forced, rapid, global, industrywide move to remote work. A recent survey performed by Raišienė et al. [13] pictures the influence of rapid introduction of remote work, also known as telework, on Lithuanian workers in many different occupations. However, these interesting findings don’t shed much light on the situation of ASDT and how the tools and practices from distributed teams helped with the rapid transition to remote work.

The possible impact of Covid-19 on Agile was discussed by Mancl et al. [14] in his article based on a panel discussion during the XP2020 conference. Based on their experience they emphasize the importance of carefully selected online tools facilitating the communication and self-organization of the ASDT. The possibility of simulating conditions similar to an in-person meeting with a whiteboard is described as critical for the success of an Agile team. The importance of online telecommunication tools: text, audio and visual in distributed ASDT was brought up in an article by Robinson [6]. As described by Mancl et al. [14] proper use of tools turned out to be crucial when all teams became distributed.

The use of metrics in software development has been a subject of research for a long time. A few years before the Agile Manifesto was signed Schwaber [15] puts emphasis on the importance of measurements in empirical process, the base of Scrum framework. In this work the need for the development of metrics for empirical processes was indicated. Later Hartmann et al. [16] stressed the benefits of measuring Velocity in Scrum projects and proposed a set of additional useful metrics. Metrics can deliver additional information for decision making and monitoring without putting a constrain of ASDT work, therefore Downey et al. [10] proposed a set of metrics for fast working ASDT. Ladas in his book “Scrumban” [17] proposed to use elements of Kanban in ASDT using Scrum as a way to support the software development process and enable ASDT to transition to Kanban in the future. Anderson in his book [18] describes a set of Agile metrics inspired by Toyota Production System [19] as a core of the Kanban Method. Literature studies performed in recent years by Kupiainen et al. [20] and Kurnia et al. [21] indicate that ASDT use different metrics in their work and measuring different aspects of Agile software development is becoming a standard practice.

The state of Agile practices before the Covid-19 pandemic in different teams was well described in the “State of Agile” industry survey performed by VersionOne [1]. This survey was performed between August and December 2019 and gathered 1121 responses from around the world. The resulting report allows for a better understanding of the Agile practices in use, including the use of frameworks, tools and metrics. However only 63% of respondents work in Software Development or IT. Therefore, it provides an insight to all types of Agile teams, not specifically the ASDT.

3 Research Design and Methodology

For the purpose of the empiric study a survey was designed. The initial pool of questions was created by the authors, then the first version was reviewed by 4 independent Agile practitioners working as experts in international software development companies. The remarks to the first version were included in the final version. The final survey, composed of 22 questions with 18 closed-ended and 4 open-ended questions, was divided into four parts. The first nine questions characterized the participant by asking about their country of origin, role in their organization, level of teams’ distribution, used frameworks, remote work pre and post the Covid-19 pandemic, as well as their organization’s size, industry and type. The second group of questions investigated the impact of the Covid-19 pandemic on Product Backlog and Vision, changes in: stakeholders’ involvement; release frequency and Definition of Done. The third group of questions collects information about used metrics and reasons behind their use. It also asks if any new metrics were introduced during the Covid-19 pandemic. The last questions ask about tools used by the teams.

The anonymous survey was created in Google Forms and distributed through a direct approach and social media channels including Facebook and LinkedIn researchers professional networks, Agile software development practitioners groups and pages. The responses were gathered from 01.09.2020 to 11.09.2020. A total of 120 answers were submitted during this period. No partial answer was submitted, because all close-ended questions were obligatory. During the answers inspection no obviously biased or fake answer was detected, therefore no answer was deleted or omitted. The results were exported from Google Forms and imported to Excel. With the use of a spreadsheet tool the data was explored and visual figures were generated.

4 Results

In this section the 120 results of the survey are presented. The first subsection presents an overview of respondents teams. The next subsection presents the influence of Covid-19 on ASDT work. The last subsection presents tools and metrics used by ASDT.

4.1 Teams Characteristic

The first group of questions was designed to characterize the surveyed organization and team. The first question asked about the frameworks and methodologies used in the project. Respondents could select multiple options, with many choosing to do so. As shown in Fig. 1 the most common framework was Scrum (108 answers, 90% of respondents), followed by Kanban (50; 41.7%), DSDM or AgilePM (10; 8.3%), SAFe (8; 6.7%), Nexus and LeSS (4; 3.3% each), XP (3; 2.5%) Scrum@Scale, LeanSD and Waterfall (2; 1.7% each). There were 4 other responses (3.3% of respondents) mentioning self-developed frameworks. The most commonly combined frameworks were Scrum and Kanban with 38 concurrent occurrences (31.7% of respondents). Kanban, despite being the second most popular framework, is mostly used together with other frameworks. Only 8 respondents used Kanban exclusively (6.7% of respondents, 16% of Kanban practitioners). On the other hand Scrum, the most popular framework, is used on its own by 48 respondents (40% of respondents, 44.4% of Scrum practitioners). Moreover, we can also divide Scrum into two categories: Scaled and Nonscaled Scrum. If we count scaled Scrum frameworks (Nexus, LeSS, SAFe, Scrum@Scale) as one it shows that 18 respondents scale Scrum (15% of respondents, 16.7% of Scrum practitioners). On the other hand Scrum is not scaled by 90 respondents (75% of respondents, 83.3% of Scrum practitioners).

Fig. 1.
figure 1

Usage of different methodologies and frameworks in ASDT

Full size image

The second question asked about the participant’s country of origin. Respondents were from 14 different countries: Bulgaria, Canada, China, Denmark, France, Gibraltar, Hong Kong, India, Ireland, Poland, Singapore, Spain, the United Kingdom and the United States. Most of the respondents (85% of all results) were from Poland, the country where the research team was based.

The third question investigated the distribution of the team. As shown in Fig. 2, the most common continent was Europe with 94 answers (78.3% of all answers), then Asia with 39 answers (32.5%) and North America with 38 answers (31.7%). A total of 8 respondents had team members in Australia (6.7%), 2 in South America (1.7%) and 1 in Africa (0.8%). Off all the polled teams 21 (17.5%) were not distributed, 50 (41.7%) were distributed within a single continent, 23 (19.2%) were distributed across two continents, 19 (15.8%) were distributed across three continents and 7 (6%) were distributed across four or more continents as shown in Fig. 3.

Fig. 2.
figure 2

Distribution of surveyed teams between different continents

Full size image
Fig. 3.
figure 3

Levels of surveyed teams’ distribution

Full size image

The fourth and fifth question asked about remote work before and after the start of the Covid-19 pandemic. The results have been presented in Fig. 4. Before the pandemic exactly half of respondents were working in a mixed model, a few days remotely, a few days onsite. Only 10 (8.3%) of the respondents were working fully remotely and 50 (41.7%) of the respondents were working fully onsite. After the start of the pandemic no one was working fully onsite. The majority, in total 103 (85.8%) of the respondents, was working fully remotely. Only 17 (14.2%) of the respondents were working in a mixed model, 8 of these 17 people used to work in a mixed model and 9 used to work fully onsite before Covid-19 pandemic. Therefore, from the 60 people that used to work in the mixed model 86.6% were able to transition into fully remote work. From the 50 people working only onsite, 82% were able to transition into full remote work, with the remaining 18% transitioning to a mixed model. All fully remote workers stayed fully remote.

Fig. 4.
figure 4

Remote and onsite work in ASDT before and during the Covid-19 pandemic

Full size image

In Fig. 5 the industries of the respondents have been presented. Respondents could select multiple answers. A wide spread of different industries can be observed. The most popular industries were “Financial Services, Banking & Insurance” and “High-tech, Electronics & Industrial Engineering” with 33 representatives each. The respondent’s organizations size is also diverse. A total of 39 (32.5%) respondents work in an organization with more than 5000 employees. The other four categories were: 1–50, 51–300, 301–1000, 1001–5000. They each contained between 15.8% and 18.3% of respondents. Moreover, 16 respondents (13.3%) identify their organization as a start-up, with one employing over 5000, one 1001–5000, two 301–1000, two 51–300 and ten 1–50.

Fig. 5.
figure 5

Industries in which surveyed ASDT work

Full size image

In the ninth question participants were asked to select roles they fulfil in the team. They could select multiple options. As shown in Fig. 6, the most common role was a Team Member with 47 answers (39.2%). The next two were Team Leader (23 answers, 19.2%) and Consultant (22, 18.3%), then Product Owner with 15 representatives, Scrum Master/Agile Coach with 14 representatives, C-level with 12, Portfolio/Program Manager with 10, Release Manager with 2 and a single Stakeholder.

Fig. 6.
figure 6

Participants’ roles in their organizations

Full size image

4.2 Pandemic Impact on ASDT Work

To measure the impact of the Covid-19 pandemic on the ASDT, the respondents were asked if the Covid-19 pandemic impacted the content of the Product Backlog or the Product Vision. Every respondent stated that they have easy access to the Product Backlog while working remotely, and therefore should be aware of any Covid-19 influence. As shown in Fig. 7, 59 of the respondents (49.2%) stated that both the Product Backlog and the Product Vision were not impacted. In 16 cases the Product Backlog was not impacted, despite the Product Vision being influenced by the pandemic. In a single case it was a significant impact, in the other 15 cases Product Vision was only slightly impacted. In 37 cases the Product Backlog was slightly impacted, in 12 of these cases the Product Vision was not impacted and in the other 25 cases the Product Vision was slightly impacted. A drastic change in Product Backlog happened in only 8 cases of which 4 cases also reported a significant impact on the Product Vision, 2 reported a slight impact and the other 2 reported no impact on Product Vision.

Fig. 7.
figure 7

The Covid-19 pandemic’s impact on product backlog and vision

Full size image

Figure 8 shows the change in stakeholders’ involvement during the Covid-19 pandemic. In over half the cases the involvement remained the same. Stakeholders involvement increased in 25 cases (20.8%). Only in 6 of the 25 cases, where the stakeholders’ involvement increased, did Product Vision and Product Backlog stay the same. On the other hand in 10 of 25 cases, where stakeholders’ involvement increased, both the Product Vision and Backlog were impacted. The stakeholder’s involvement decreased in 20 cases (16.7%). In 7 of these cases no impact on Product Backlog or Product Vision was reported. In 5 of these 20 cases both the Product Vision and Backlog were impacted.

Fig. 8.
figure 8

Change in stakeholders’ involvement during the Covid-19 pandemic

Full size image

The release frequencies of surveyed ASDT have been shown in Fig. 9. There is no dominant release frequency. Almost three quarters of the teams release at least every month. Almost half of the ASDT is releasing every 2 weeks or more often. Only 16% of the respondents are releasing every quarter.

Fig. 9.
figure 9

Release frequencies of surveyed ASDT

Full size image

Figure 10 shows the change in release frequency during the Covid-19 pandemic. Only 8 (6.6%) of respondents, state that they started releasing more frequently during the Covid-19 pandemic. In all of these cases the Product Backlog was changed, though only slightly in all cases but one (in which it changed significantly). In 6 of these cases the Product Vision changed slightly, only in 2 did it remained the same. Moreover 4 of these 8 cases where the release frequency was increased report that the Definition of Done (DoD) was made more liberal, in 3 cases it didn’t change and in the last case the team didn’t have a DoD. Only 3 respondents report that during the Covid-19 pandemic they are releasing less frequently. In 2 of these cases they used to release more often than every 2 weeks and the third team was releasing every 2 weeks. In all of these 3 teams the DoD was not changed.

Fig. 10.
figure 10

Change in release frequencies because of the Covid-19 pandemic

Full size image

In Fig. 11 changes in DoD have been presented. The DoD was changed in only 14 cases. In 6 of these cases it became more liberal and was accompanied by a change in either Product Vision or Product Backlog. The DoD became more strict in 8 teams. All of these 8 teams didn’t work fully remotely before the Covid-19 pandemic and changed to fully remote work. As many as 18 teams don’t have a DoD, all of these teams except one use the Scrum framework.

Fig. 11.
figure 11

Change in definition of done because of the Covid-19 pandemic

Full size image

In an open question respondents were asked what was the best change introduced in their work because of the Covid-19 pandemic. This question was not obligatory, consequently only 42 meaningful answers were gathered. Most of the respondents (30 from 42) indicated the introduction or maturing of remote work as the best change. From the rest 5 respondents see an increase in communication as the biggest positive and another 4 indicated increased productivity. The remaining mentioned positive aspects were: reduced number of meetings, increased accountability, more automatization and more pair programing. According to the gathered answers, the communication increase was caused by moving all communication to online tools. Therefore, everyone had access to every discussion, while before people were omitted because they were remote at that moment or just not in the room where the discussion took place.

4.3 Metrics and Tools Used by ASDT

In the survey participants were asked to mark metrics used by their team and add any missing metrics. The total number of users for each metric has been presented in Fig. 12. The most popular metric was Velocity, with over half of the teams using it. The next most popular metrics were Quality, Work in Progress, Sprint Goal success Rate and Value Delivered. Only 13 out of 120 respondents did not report using any metrics, 8 of these 13 work in pure Scrum, 2 work in SAFe, 2 in pure Kanban and the last one uses both Scrum and Kanban.

Fig. 12.
figure 12

Popularity of different metrics among respondents

Full size image

In the next question respondents were asked if their team introduced any new metrics because of Covid-19. Only 5 participants reported that a new metric was introduced. These new metrics were:

  • Focus Factor;

  • Vanity Metrics;

  • Daily resolved defects per team member;

  • Skills gained and shared with the team;

  • Weekly work hours reporting instead of monthly reporting.

The next question investigated what collaboration tools are used by the team. The answers have been presented in Fig. 13. Every ASDT uses at least one collaboration tool. The most popular tool is Jira, a task management tool, used by 77.5% of respondents. The second most popular tool is Confluence, a knowledge management tool closely integrated with Jira. The most popular communication tool is Teams (50%) with Slack (42.5%) being a close second. Another common tool is GitHub with its alternative GitLab behind it. These tools also have simple task and knowledge management functionalities in their primary feature of being a code repository. Next is less popular tool Azure DevOps which is both a task and knowledge management tool and a code repository. Later with 18 users there is Trello, a simple task management tool, and online whiteboard like Miro and Conceptboard. The last of the commonly used tools is Mural, also an online whiteboard.

Fig. 13.
figure 13

Popularity of collaboration tools in ASDT

Full size image

5 Discussion and Future Work

Obtained results may indicate that the ASDT were able to adapt to the new circumstances caused by Covid-19. The most common change that was observed is a shift towards remote work for almost all of the ASDT (85% of teams currently work fully remotely in response to Covid-19). The cause of such good adaptation can be found in the tools already used by the teams. The popularity of distributed teams, that have a similar characteristic to remote teams, resulted in the adaptation of commonly used tools and the development of features supporting work in distributed teams. Consequently, non-distributed non-local teams were already using tools supporting remote work prior to the Covid-19 pandemic. This minimalized the need to implement new tools or practices while transitioning to full remote work.

The popularity of different tools among ASDT indicates a significant need for more advanced knowledge sharing. The most popular tool was Jira, a task management tool. However, the second most popular tool was Confluence, used by over a half of the surveyed ASDT. This knowledge management tool, closely integrated with Jira, provides more advanced means of communication, supplementing simpler forms of direct messages and e-mails. Knowledge management functionalities can also be found in other tools popular among the participants including GitHub, GitLab and Azure DevOps. All of these tools offer multiple features including code repository, task management and knowledge management. Therefore, they can be used for different purposes in the ASDT. Exact use of these tools was not addressed in this study and should be further investigated in the future. Investigating which exact functionalities are used by the ASDT should allow for a better understanding of what ASDT need from their tools and provide input for their further improvement.

The disturbance in the business sphere of ASDT was surprisingly low. Almost half of the teams didn’t change either their Product Backlog or their Product Vision. The release frequency and the DoD in most cases remained unchanged. The few ASDT that changed the workflow probably did so to address changes to the Product Backlog and Vision that were also reported by those teams. Despite the turbulence caused by the transition to remote work, most of the ASDT continued to deliver at the same rate and with the same quality. This may indicate that most of the products developed by the ASDT were not impacted by the pandemic and that the ASDT were able to transition to remote work without significant interference to their work. The lack of change in the business aspect of ASDT work can have multiple causes. The simplest one is a lack of influence of the pandemic on the products. The other reason may be the insufficient maturity of the Agile mindset among project managers and stakeholders. It is possible that ASDT have adapted the Agile frameworks but the formal restrictions don’t allow for or don’t require the ASDT to adjust the Product Vision or Backlog. Therefore ASDT are developing a product accordingly to the pre-pandemic circumstances. Investigating the cause for such small changes is a matter for future research.

Most of the ASDT didn’t introduce any new metrics during the Covid-19 pandemic. Most of the teams used at least one metric, therefore they should be aware of the value metrics bring to the team and transparency of work. The lack of new metric introduction can be related to rather small changes in the organization of ASDT work during the pandemic. The other explanation can be a lack of understanding and recognition of metric usefulness in ASDT. Many of the respondents answered in the open question about why they use such metrics that they were chosen by the management or the organization, not by the ASDT themself. Few of the answers suggested a deeper understanding of the motivation behind the usage of metrics. The ASDT understanding and appreciation of metrics in software development should be further investigated in a future study.

6 Conclusion

The main objective of our research was to identify the impact of the Covid-19 pandemic and a rapid transition to remote work on the ASDT. Survey results prove the work of ASDT was not significantly impacted in most cases. The ASDT were able to transition to remote work without much turbulence. This smooth transition was possible due to the popularity of distributed and remote ASDT prior to the Covid-19 pandemic. Only 9% of all surveyed teams didn’t work in a distributed team and didn’t work remotely at all. The prevalence of distributed teams and remote work resulted in the popularity of online tools supporting it. Even the non-distributed, non-remote teams were already using online tools which support distributed teams. Therefore, the need to implement new tools in the ASDT was limited, which led to an easier transition to fully remote work. Accordingly, as not much was changed in the organization of ASDT work, most of the teams didn’t feel a need to introduce new metrics. The business sphere of ASDT work was also not significantly impacted. In few cases the surveyed ASDT responded to the changes in Product Backlog and Vision by accelerating the work flow. They increased the release frequency and in a few cases lowered the overall quality for a short term speed gain. Such behavior could help with a quicker response to new market opportunities.

Results indicate that the transition to remote work didn’t disrupt ASDT’ communication. Rather, respondents state that fully remote work reduced the amount of unnecessary meetings, which were reducing their productivity. Moreover, fully remote work prevented the exclusion of remote or distributed team members from on-site, in-person discussions and meeting. Therefore, fully remote work improved communication in teams that were distributed and non-remote before the pandemic by moving all communication to online tools. Co-located team members couldn’t exclude their distributed colleagues by discussing issues in person.

Over 89% of surveyed ASDT use at least one metric. The use of metrics supports their software development process and allows for making data driven decisions. The most commonly used metric was Velocity, used by over half of surveyed ASDT.

Results show that every surveyed ASDT uses at least one cooperation tool, including all the non-distributed, co-located ASDT. The most commonly used tool is Jira, a task management tool, and Confluence, a knowledge management tool. This shows that a need for more advanced cooperation tools is well known among ASDT and they are using them even when working in a non-distributed, co-located environment. The use of online tools allows each of the 120 respondents to have easy access to the Product Backlog while working remotely.