Information Sharing Among Disaster Responders - An Interactive Spreadsheet-Based Collaboration Approach

Abstract

Recent natural disasters have led crisis management organizations to revise their protocols so as to rely on the contribution of a wider range of actors, including simple citizens as well as expert operators, to support decision making activities. Reliable and timely information sharing among members of distributed teams of disaster responders has become paramount for the success of the overall crisis management process. In this paper we propose a crisis management system based on spreadsheet-mediated collaboration among on-site responders and decision makers. To share data a common spreadsheet artifact has been developed by using a participatory design approach which is accessed through mobile user interfaces. The evaluation results showed that the use of the spreadsheet artifact has resulted in more effective decision making relating to set of earthquake management scenarios in high-risk areas located in Italy.

1 Introduction

In the past decade, catastrophes and disasters as a result of natural hazards have emphasized the crucial role of information and communication technology (ICT) research in the field of crisis management.

The diversity of emergency situations which originated from those events and the variety of users and tools have in fact led crisis management organizations to face specific research issues covering different areas, including data visualization, (geo)visual analytics, advanced (mobile) interfaces, communication technology and collaborative environments. This awareness has then transformed the crisis management domain into a wide and pervasive research field whose results are essential to support decision making activities (Carver and Turoff 2007; Convertino et al. 2011; Sagun et al. 2009).

Researchers agree that reliable information and good communications are crucial requirements to achieve effectiveness in all phases of disaster management, which includes the mitigation, preparedness, response, and recovery phases.

In particular, the emergency preparedness and response phases include actions taken prior as well as during and after a disaster event in order to reduce human and property losses. An overall view of the evolving situation is therefore crucial in order to provide optimum response when managing a disaster. Such a view can be achieved only if it is possible to overlay real time information supplied by people sited in the disaster zone, on top of relevant static information such as maps, details about buildings, population distribution in the affected area, location of key services etc. On the other hand, this aggregated information should be available through a reliable communication infrastructure both to the people in a Command and Control Center or Centro Operativo Comunale (COC), who make effective global strategic decisions, and to the people on the ground who perform actions. Thus, reliable and timely information sharing among members of distributed teams of disaster responders has become paramount for the success of the overall response process.

Our collaboration with people from the Italy Civil Defense Agency highlighted the need to further support cooperation within distributed teams of responders, providing tools to acquire, aggregate and share timely and updated information. Primary requirements were to leverage pervasiveness of mobile technology, to design easy to use collaborative interfaces and, last but not least, to customize data visualization to the needs of the different actors involved in crisis management.

As a result, in this paper a spreadsheet-mediated collaborative system is proposed that improves information sharing between on-site responders and central decision makers and enhances situation awareness as perceived by all the actors. The system allows users to cooperate through mobile devices, share portions of data, apply for resources in a concurrent and reliable manner, and obtain real-time status updates from decision makers.

A participatory design methodology has been adopted to build a collaborative system prototype, in the domain of earthquake management, that combines the advantages of mobile phones with the high potentials of spreadsheets for supporting teams of collaborators acting on a wide geographic area and requiring advanced tools for geodata collection and management.

A usability evaluation was performed on the prototype to validate the proposed solution against the identified requirements. The results showed that the use of the spreadsheet-mediated collaborative system may contribute positively to the decision making process during crisis management. The well-established spreadsheet technology proved to be a key factor for a timely and effective collaboration, much leveraging people familiarity with those artifacts. As a secondary outcome these findings also highlighted the suitability of using participatory design approach for developing such systems where real time information sharing and situation awareness is paramount.

The remainder of the paper is organized as follows. Section 2 presents a comprehensive literature review on research in crisis management, useful to identify the main features of existing ICT solutions. This is the first step of the research methodology described in Section 3. The second step is the contextual inquiry conducted in collaboration with the Civil Defence Agency of the town of Montemiletto, in the South of Italy, as described in Section 4. The major requirements emerged from that field study are illustrated in Section 5. The consequent design steps are then discussed. In particular, Section 6 describes the design of representative activity scenarios that the proposed system supports and provides a discussion on the derived design claims. Section 7 explicates how mobile interface design challenges were addressed in the prototype. A usability study meant to validate the system represents the final phase of the research methodology and is illustrated in Section 8 together with a discussion on the results. Section 9 concludes the paper.

2 Literature review

Emergency management is a critical and continuously evolving research area, where each single step to improve either methods or tools make a significant contribution towards reducing human lives and resource losses. The awareness about this stimulates professionals and researchers from the crisis management field to devote much effort to define future research directions, whose results are in fact essential for drawing up an agenda by public institutions and Civil Defence agencies to identify sectors where investments could produce effective solutions. Geology, construction science, structural engineering, material science and technology are some examples of individual sectors where improvements can be made, with possible advantages in the crisis management field. Information Communication Technology (ICT) represents an across-the-board sector that would contribute to enhancement in all aspects of crisis management. Currently, practitioners are already using ICT tools for some relevant phases. However, further ICT advances are required which could enable teams of practitioners to quickly take the appropriate actions so as to further decrease losses both in terms of people and damages (Carver and Turoff 2007). Several research directions have been identified, all sharing the observation that experiences of different actors and contributions from relevant domains represent the only means to achieve stable and reliable solutions for the crisis management (Petak 1985).

The current state-of-the-art of crisis management research has been analyzed in this paper, with special attention on aspects related to the communication and coordination among different actors, the ICT support for geographic information (Geo-ICT) and the citizen involvement.

Effective communication and coordination among parties represents a crucial aspect relevant to crisis management. Achieving this is a major challenge and, as stated by Carver and Turoff (Carver and Turoff 2007), it is fundamental to provide accurate and timely exchange of information. In particular, in (Convertino et al. 2011) the authors deepen this aspect and discuss the importance to improve the knowledge-sharing and activity-awareness so as to enhance the coordination and the cooperation. Rather than emphasizing information only within a geospatial context, they include different kinds of information coming from meetings and social ties, financial transactions, and so on. An interesting relationship is also investigated in (Yu and Cai 2009), where the lack of communication and coordination is linked to problems raised by political, geographical, or organizational boundaries during disaster response. As for the information flow among different actors, in (Sagun et al. 2009), the authors recognize four channels of information flow during an emergency event that ICT solutions must support in order to facilitate the coordination during the whole process. The four channels include communication within each participating organisation, between organisations, from people to organisations and from organisations to people. A conceptual model is also described, which integrates ICT into emergency management so as to improve collaboration. A formal approach to the analysis of coordination is also provided in (Chen et al. 2008), where the authors state that coordination in the context of emergency is still an understudied research issue. To achieve this, they propose a formal framework aimed at analysing coordination patterns in the different phases of emergency response. Basically, the idea is that the framework can be exploited in order to allow researchers to study the coordination efforts for managing response to an emergency. In (Aedo et al. 2010) an emergency management system is proposed that is meant to support the coordination among geographically distributed and functionally independent agencies which may need to cooperate during a large-scale crisis management. In (Careem et al. 2006) the Sahana system is presented, an open-source Web application coming from large-scale disaster experiences. It can support cross-boundary emergency management processes providing for a platform for inter-organizational data sharing during a disaster.

Researchers have also studied the relationship between knowledge management systems (KMS) and emergency information systems (EIS). In (Turoff et al. 2004) authors identify some design requirements to extend EIS capabilities in terms of decision making and communication and collaboration between emergency response participants. In (Murphy and Jennex 2006) the authors have shown that the use of KMS improves the speed and quality of response actions and that by embedding basic KM considerations within EIS their capabilities of communication and collaboration are notably enhanced. This observation is also shared in (Raman et al. 2010), where the authors discuss the use of wiki technology to facilitate KM for emergency response systems, in terms of connectivity improvement, knowledge sharing, and communications between diverse groups of experts. In particular, they describe the design and implementation of a wiki-based knowledge management system for improving emergency response, where notes, history of notes and the context information are brought together, thus entailing a system based on shared documents.

The aforementioned papers address different aspects of the general issue of communication and coordination. When dealing with coordination issues, the Geo-ICT support could represent a solution to satisfy specific requirements, such as the visualization of shared information and the collaborative work management. The GI and ICT communities are indeed strongly involved in this context. They devote their efforts to the definition and design of methods and techniques which best guarantee the achievement of solutions based on a unique, shared and collaborative platform, where the information collected by different users through several sources, can be conveyed, queried, managed and analysed in a seamless manner.

Within this scenario, maps play a key role, both for their indisputable inner capability of representing data and phenomena related to a territory, and, above all in this case, for being a uniform and common language on which the cooperation can be based. In (Yu and Cai 2009), the authors argue that the collaboration mediated through maps is a promising means to allow emergency responders to access the same information, share and update it and interactively cooperate by analysing all individual inputs.

Along this line, Schafer et al. then highlight the importance of geospatial tools for map management (Schafer et al. 2007). Indeed, geo-visual representation makes information tangible and it is exploited in particular to share information, coordinate actions and support discussions among involved users. The authors present a software architecture that provides a starting point for developing software tools aimed at allowing geo-collaboration. In (Cai et al. 2004) the authors describe an ethnographic field study revealing that maps are an essential collaboration tool in crisis situations and propose a GeoCollaborative tool for crisis management, named GeoAnnotator. The paper described in (Widdis et al. 2009) presents another collaborative tool named Black Coral LIVETM, an information sharing software for emergency crisis support. It is a desktop software application that allows teams to communicate in real-time on top of a digital map.

Another emerging issue related to the ICT support for crisis management is the visualization of data coming from multiple sources. Recently, Gupta and Knoblock have proposed the use of mashups for information visualization based on spreadsheet files to collect data, aggregate information, and calculate statistics about the rising of the fatalities and injuries (Gupta and Knoblock 2010). Although in (Panko 2007) it is argued that spreadsheet applications can lead users in making mistakes due to the lack of a control system for the omissions, which is a common human mistake, spreadsheet based systems have been adopted in quite a few emergency situations. As an example, during the emergency response after the Boston Marathon bombing, tools like Google Spreadsheets or Forms were used in order to coordinate the two thousand volunteers that got involved in the crisis management process (Meier 2013). As a matter of fact, the usage of common daily tools such as spreadsheets to deal with the emergency situations can enhance the management process, because the responders’ familiarity with filling out traditional paper forms help them reduce stress associated with a crisis. Unfamiliarity can notably impact the effective use of a new technology in crisis situations. This has been highlighted in (Jennex 2004), where the author reviews different emergency information systems and concludes that without an adequate training these systems should not be used during a real situation where people are under stress.

Alternative approaches to paper forms and spreadsheets are discussed in (Durbin et al. 2010) where the authors suggest to combine several technologies, such as blogs, cloud computing, wikis and the use of request forms rendered on smart phones of personnel assigned to the location as the points of contact.

Finally, one of the major technological issues addressed by researchers in the field is scalability. A system should be available on multiple platforms to different responders, including both people making decisions and coordinating activities and also people working on the emergency sites. The former group should be equipped with computers with very large screens for visualizing the emergency sites and monitoring and analysing the evolution of the crisis. The latter should rely on small and handheld devices like smart- phones, on which they should be able to visualize information about the area of interest and, at the same time, have an idea of what is happening all around. The use of mobile devices introduces several benefits to support first responders and more generally users on the move. In (Bello et al. 2007) authors remark that during an emergency the organization of the response action and the interaction with civilians may be significantly improved by using ubiquitous mobile communication infrastructures. This necessity of mobile devices is also presented in (Kim et al. 2007), where the authors suggest the use of mobile devices during a crisis management for visualization tasks in order to provide users with information everywhere and for supporting effective decision making. In (Landgren and Nulden 2007), a discussion about patterns of mobile phone interaction in emergency response work is presented. Finally, in (Sung et al. 2007) authors present a software application based on visual analytics techniques. It is aimed to allow first responders to exploit a rapid on-site decision-making tool. The application holds the history of different status and users may use a slide-bar to navigate and visually analyze the previous ones. However, this system only collects data from sensors embedded in an environment and it is mainly oriented to work in closed environments monitored by sensors. It is not intended to make use of information humans can provide by using mobile devices.

The last aspect discussed in this section concerns the citizen involvement. After the 2009 earthquake in Abruzzo, Italy, citizens were asked to contribute and share information about injured people and damaged buildings through Google Maps. After the 2012 earthquake in Emilia Romagna, Italy, an Ushahidi crowd-mapping instance was activated to collect and share information on a common platform using Crowdmap (Ushahidi 2013).

Besides the innovation in terms of methods, such initiatives prove that the direct involvement of citizens in disaster response activities contributes to acquire a complete view of the real situation in a faster and effective manner. Generally speaking, the citizen participation is a relevant aspect for any activity for which the knowledge of a territory and of its changes is crucial. Indeed, as stated in (Goodchild 2007), citizens can act as intelligent sensors that provide feedback on a situation, as it has been done in some local community response grids (Jaeger et al. 2007). Also in (Palen et al. 2010) and (Turoff et al. 2009) the authors give special emphasis to the direct involvement of citizens in disaster preparedness and planning as well as in crisis response activities. In particular, (Palen et al. 2010) view the citizens as a powerful, self-organizing, and collectively intelligent resource that can be exploited by ICT to face a crisis. They state that innovation in the emergency field could greatly benefit from framing the disaster response as a set of socially-distributed activities supporting the processing of emergency problems in times of change and disruption. In (Turoff et al. 2009) the authors discuss the high potentials of social computing, community involvement and citizen participation for emergency preparedness and management. They argue that the opportunities offered by the Web evolution in the new Information Society pushes for the adoption of dedicated social networks and Web-based collaboration tools in the domain of emergency management. This statement has been recognized also in (Diaz et al. 2013), where the authors state that social media and IT technologies support a more active role of citizens in each phase of emergency management process by treating them as agents capable of contributing to get a better solution. In particular, as White writes in (White 2012), citizens can track alerts and the evolution of hazards, coordinate and collaborate in the local response, support community preparation or provide information and knowledge on how to recover from a crisis. Social networks have been successfully exploited in different events wherein citizens came across dangerous situations (Beaumont 2008) and (Boodhoo 2010). The work presented in (Howe et al. 2011) shows an experiment called X24 aimed at showing how emergency organizations can improve their strategy by incorporating social media into their activities, thus proving that social media are relevant even in the area of the emergency response.

Finally, in (Palen et al. 2010) the authors highlight that one of the derived challenges is the alignment of informal and precious sources of information coming from local communities with the official sources of information that emergency agencies deal with. This issue entails the need for mechanisms able to provide trustworthiness and security of information, which is addressed as one of the main obstacles to the exploitation of relevant, unofficial information as further support to the decision making process.

3 The methodology

The aim of this section is to describe the methodology followed to extract a set of requirements that should be met in order to enhance collaboration in crisis management processes. The goal of the present research on crisis management in fact was to identify new opportunities for IT based collaboration activities and real time information sharing that could be suitably integrated with existing decision support systems.

The literature review on the current state-of-the-art of crisis management research, combined with HCI theory for collaboration systems, had the twofold effect of laying the basis for our work and helping us discover the research contribution gained from the proposed solution in the field of emergency management.

A participatory scenario-based design approach has been adopted (Rosson and Carroll 2002), which involved representative users and stakeholders since the initial phases of problem analysis and requirements specification and throughout the whole design/evaluation process.

In order to gain insight into current emergency management activities, a contextual inquiry was conducted in collaboration with the Civil Defence Agency of the town of Montemiletto, in the South of Italy, struck hard in 1980 by an earthquake of the magnitude of 6.9 on the Richter scale. The Agency is using a software platform called Sirio (3DGis 2014), which is part of a set of integrated modules meant to support domain experts and workers during the planning phase and the crisis management. During one of the simulation events which take place every 6 months, the authors observed the emergency management activities in order to identify the actors, the physical environment where they operate, and how they communicate when following the established protocols. Follow-up interviews were conducted with the different actors involved in the process, aimed to understand which factors may hinder success in response operations, especially in case of exceptionally evolving situations. Annotations gathered during the observation and data collected from the interviews were analyzed and used to derive scenarios of current practices, on which an extensive brainstorming activity was then conducted. In agreement with the participatory approach, three people recruited from the Italy Civil Defence Agency took part in the analysis. Relevant claims about the engaged problem were thus stated, from which a set of user requirements could be distilled and reviewed with the users.

Next, relying again on users’ participation to address the interface design challenges, appropriate scenarios of activity, interaction and information design were carried out, with associated claims, from which the prototype of the target collaborative system evolved. In the paper, in order to explicate the rationale behind the design of the prototype, the major interface design challenges faced are discussed, especially focusing on the use of mobile interfaces and on the adoption of spreadsheets as a shared artifact among disaster responders in a distributed environment.

A usability evaluation of the prototype was performed in order to validate the solution against the identified requirements. It started from the design of a reference scenario enabling for the study of the emergency management processes in a controlled laboratory setting. While retaining the benefits of a controlled environment, the ecological validity of the experiment was guaranteed by working in collaboration with the Civil Defense experts. In the study, a comparative evaluation was performed between the spreadsheet-mediated collaboration system and existing procedures for collaboration and information sharing between local responders and central decision makers. The results of the study are discussed in the paper along with their implications on the design strategies adopted in crisis management.

4 The contextual inquiry

The research work illustrated in this paper evolved from the field of earthquake management in Italy high-risk areas. It was based on a participatory scenario-based design approach, which allowed for gaining insight into current emergency management activities and derived new interactive collaboration scenarios. These could form the basis for the future development of a collaborative system. In the present section the contextual inquiry conducted in collaboration with the Civil Defence Agency of the town of Montemiletto is described. It was performed during a simulation event and was addressed to observe the emergency management activities to identify:

• the different actors involved in the event, ranging from the chief of the Civil Defence Agency to the on site operators and volunteering citizens,

• the physical environment where they operate and

• the way they communicate when following the established protocols.

The interviews conducted with the different actors involved in the process helped researchers to understand which factors were considered as obstacles to the success of the response operations, especially in the case of exceptionally evolving situations. Moreover, it was possible to understand the extent to which common citizens can contribute to response operations.

4.1 The observed intervention model

Montemiletto is a town located in Campania, South of Italy (Geographic coordinates 41° 0′ 41.44″ N - 14° 54′ 28.56″ E). The land surface is 21.47 km², residents are 5,462, and the density of population is 253 residents/km².

In Figure  1 the whole territory is captured and in Figure  2 the corresponding seismic hazard map is depicted.

Figure 1

The GoogleEarth view of Montemiletto.

Figure 2

The seismic hazard map for Montemiletto.

In agreement with the emergency plan of the Civil Defence Agency, the territory is divided into zones based on the density of their population and the number of buildings. Each zone includes residents belonging to a set of contained or overlapped roads. A zone is classified as a gathering area if it contains a wide open space where people can converge and wait for assistance. If such a zone can also be reached through a large accessible road, it becomes an area for gathering reinforcements. Finally, a zone is transformed into a sheltering area if some facilities are also available, such as power and water connections. Figure  3 shows a zone associated with a gathering and a sheltering area.

Figure 3

A zone associated with a gathering and a sheltering area.

The development of an emergency plan ends with a validation phase aiming at facing possible exceptions caused by both human factors and temporary objective impediments, such as work in progress on a road network. During this phase targeted training activities are scheduled which may contribute to tune the involved parameters (residents, personnel and tools) of the underlying protocol, by taking into account both general requirements set by national regulations and local availability and supply. Final results consist of modifications and instructions to be integrated within the initial intervention model.

Data useful to handle the whole emergency plan are collected through a set of available technical and thematic maps, which include, e.g., the road network, the land use and the geological risk layers. Moreover, a large dataset is accessible which provides professionals with information useful to complete the scenario under investigation, such as demographic, statistics, building risk level/type, and services data. The Civil Defence Agency for Montemiletto is using a software platform called Sirio (3DGis 2014), which is part of a set of integrated modules meant to support domain experts and workers during the planning phase and the crisis management. In particular, it is designed to allow different authorities to collect and visualize data, to automatically aggregate and send them to a central authority. Sirio also allows operators to schedule actions when an event occurs, such as parameterized messages via fax, e-mail or phone in order to give immediate commands for operators without the necessity of waiting for human decisions. Finally, Sirio allows users to save all actions performed during a crisis management and to temporally visualize them in order to analyze the crisis evolution. A cartographic module is also integrated with Sirio, which allows to add data about the crisis management plan directly on the map.

Each gathering area is associated with the information needed during the initial step of an emergency plan, namely, the number of residents living in the associated roads/buildings, of families, of impaired people, of mobility disabled people, the number of buildings with the corresponding risk level, the number of critical sites (schools and hospitals), and of roads leading to and crossing the area.

In the following tables, examples of useful data for the local activities management are shown. In particular, information about the zone no.1 of Figure  3 is given in Table  1 , Table  2 details data about the number of residents, buildings and roads referring to the gathering area no.1, finally Table  3 describes basic features of the sheltering area no.1.

Table 1 Information about the zone no. 1.

Table 2 Information about the gathering area no. 1.

Table 3 Information about the sheltering area no. 1.

Each zone is associated with a set of on-site emergency responders, playing different roles, including reporters from the health office, the facility supply office and the registry office, along with a traffic policeman and a fireman. These professionals are in charge of monitoring the gathering area and of interacting with the COC in order to communicate how the emergency plan evolves and receive possible commands on the next required actions.

The COC is chaired by the municipal mayor who is assisted by a local functional foreman appointed by the Civil Defence Agency. These two roles are supported by a team of Technological Services operators, a team of Public Work operators, a team of traffic policemen, and 9 municipal employees supporting specific tasks, namely:

• 1. technical/scientific tasks and planning activity

• 2. health, social and veterinary security

• 3. voluntary service

• 4. material goods and means

• 5. basic services (electricity, water supply, etc.), school services

• 6. survey on damage to people and things

• 7. local operating structures, road network

• 8. telecommunication

• 9. people assistance

Finally, the association of volunteers (ANPAS - Associazioni Nazionali di Pubblica Assistenza) plays an important role in assisting COC teams and personnel involved in gathering and sheltering areas. Indeed, this association has members whose expertise varies from a high level, such as medical and fireman staff, to low level, such as pedestrian flow controllers. Based on their skills, such volunteers support both operators on site and COC employees.

Common citizens usually take part in the simulation event and are instructed on the closest gathering area they have to reach and on the corresponding road connections. Apart from those who join some volunteering association and undergo a specific disaster recovery training, common citizens are not expected to play any active role in the process. People from the Civil Defence Agency explained that no deeper involvement could be allowed according to their intervention model and asked researchers to keep this constraint in mind when designing the new system.

In the following section, a scenario is provided which describes how the intervention model of the emergency plan works in case of a seismic event. Specific actors are described and their tasks are analyzed in order to evaluate possible improvements through advanced tools of interaction and collaboration.

4.2 A scenario of current practices

In order to capitalize the knowledge gained from the fieldwork and the interviews, a scenario of emergency management practices was envisaged, from which a brainstorming activity started meant to the formalization of initial requirements and the design of a possible solution. Based on that knowledge, three personas were specified, representing three possible stakeholders, who could be the actors of the problem scenario.

5 Deriving requirements from problem scenarios

The working scenarios of the aforementioned intervention model gave the authors the opportunity to reason about the major user requirements emerged from the field observation performed during one of the crisis management simulation events.

5.1 Claims about the problem scenarios

After the event, most of the interviewees recognized that one of the problems with the current protocols was the lack of real-time information sharing among people involved in the crisis management process, including on site responders and volunteering citizens. They were aware that a higher degree of efficiency could be achieved by allowing local responders to collaborate with the center by sharing information through some software artifact. Yet, they wanted the overall crisis management process to be handled by the COC, which could rely on a team of expert operators and on the advanced analysis and decision support functionality offered by the Sirio platform. Indeed, one of the shortcomings of the observed activities, also illustrated in the problem scenarios was the time required to collect data from the emergency areas and to communicate decisions on the appropriate actions to on site responders, especially when some exceptional circumstances happened, which required changes to the original emergency plan. Therefore, some innovative techniques that facilitate data sharing and collaborative tasks were considered highly desirable to improve the processes.

The following table summarizes the claims analyzed while developing the problem scenarios and lists the positive and negative consequences of the features characterizing the envisaged situations.

Summarizing the results of the field study, researchers were able to understand participants’ main expectations for the improvement of current response processes:

• A collaboration system should be built upon the current emergency management processes, not affecting the existing centralized decision making policies, but with the goal to increase on site responders’ awareness of the current situation and to let them play an active role in sharing real time data on the local situation.

• Migration towards a collaboration system should be as smooth as possible to on site responders.

  Keeping those points in mind and reasoning about the claims derived from scenarios, researchers were able to elicit an initial set of 5 user requirements:

• UReq 1 Learnability. Little training effort to move from asynchronous to synchronous information sharing.

• UReq 2 Preservation of existing emergency management protocols. The collaboration system should guarantee suitable separation of concerns and privileges between central and local disaster responders.

• UReq 3 Appropriate synchronization policy among users. Concurrent accesses and modifications to the same (portions of) data should be managed through proper access control.

• UReq 4 Collaboration awareness and version control. Detailed modification loggings (who modifies what) should be guaranteed, so as to keep track of data evolution and allow the central operators for historical analyses.

• UReq 5 Real-time information about the crisis evolution provided to local responders by means of mobile interfaces.

  Two functional requirements were also soon apparent from the observation and from the interviews:

• FReq 1 The system should be able to have a dialogue with Sirio information system so as to be integrated with the existing platform, which supports decision making activities by means of powerful statistical and geographical analysis functionalities.

• FReq 2 On site responders should be equipped with cheap mobile technology (smartphones were considered the best choice), by which they may interact with the collaboration system and exploit advanced cartographic solutions to receive visual summaries of data of interest.

6 Identifying design claims

The first design issue researchers had to face was concerned with the choice of an appropriate software artifact to support collaboration through information sharing in the observed context. Spreadsheets were considered a favorable choice, as the most natural digital counterpart of printed data sheets that are currently being used. Since the introduction of the first computerized spreadsheet, VisiCalc, in 1979, the adoption of spreadsheet applications has been continuously increasing in different domains, and is today recognized to play a central role in the evolution of practical activities. Spreadsheets offer a flexible way to aggregate and analyze large amounts of (possibly geo-referenced) data coming from different locations and can form the basis for a collaborative system for business processes (Ginige et al. 2010; Paolino et al. 2008; Paolino et al. 2010). Most of their functions can be used by any basic user with very little training and multiple real time visualizations of aggregated data can be created in a seamless way. In the specific case under study, researchers also considered that spreadsheets could be easily connected to Sirio centralized information systems, which would hence allow for the integration of the collaboration system in the deployed decision support system.

The choice of spreadsheets inspired the subsequent design phases and guided the scenario transformation activity. In Figure  4 three relevant actors of an intervention model are shown, namely the manager of a gathering area, the manager of an Emergency Station and the Chief of the local Civil Defence Agency. Both the managers interact with the COC personnel and contribute to the plan execution, whereas the COC Chief supervises activities in progress and intervenes on critical situations. In particular, they interact through an application based on collaborative modules which allow them to synchronously update information and reserve on site facilities.

Figure 4

Actors of an intervention model and their interaction with the shared spreadsheet.

The activity scenarios, enriched with interaction and information features, allowed researchers to engage in interesting discussions with the 3 users participating in the design. From these discussions they were able to derive important claims about the shared spreadsheet artifact and the corresponding interface design issues.

The first important issue was concerned with how the (possibly large) shared spreadsheet should be visualized to view on the on-site responders’ mobile interfaces.

Reasoning about the described scenarios researchers agreed that only the information relevant to the specific mobile user should be visualized, whereas the overall view of the shared artifact was left to the COC central operators, for monitoring and analysis tasks.

• Design Claim 1. The onsite responder accesses only the information about the current situation at the associated shelter. This allows to dedicate the small screen of the mobile device to the user-relevant information. He may read the list of recovery requests made so far and the number of vacancies left. He may also use the spreadsheet to place a new request for the same shelter.

• Design Claim 2. Whenever an on-site user is editing a cell to place a request, on the interface of any other user concurring for the same shelter the input request cell should appear ‘locked’, to prevent undesirable overlapping. This choice supports multiple users conflict resolution by means of appropriate collaboration feedthrough.

• Design Claim 3. On site responders from other gathering areas concurring for beds at the same shelter should be visible to each other. This design choice ensures collaboration awareness among participants sharing the same resources. However, better support can be achieved considering the next claim.

• Design Claim 4. The awareness of who is accessing the common resource should be provided to all the related on-site responders. This allows the single user to perceive how the crisis is evolving around him, also perceiving whether some unexpected evolution of the crisis is happening in the areas around him.

  Similar claims were derived for the second part of the scenarios, concerning the request for medical care from shared hospitals.

  As for the COC users, different claims could be derived:

• Design Claim 5. For each of the supported activities by possibly different local responders the complete shared spreadsheet should be visualized on the screen. This provides COC users with real-time aggregated information on the ongoing local activities.

• Design Claim 6. Whenever an exceptional event occurs, requiring intervention by COC decision makers, notification should be triggered and some persistence cue (e.g. a flashing cell, a flag, etc.) should be provided to COC operators. This choice ensures that in case the notification message is missed, the pending centralized decision is perceived on the interface until a solution is provided.

  In the following section it is explained how some of the interface design challenges raised by the previous claims are addressed in the current prototype.

7 Interface design challenges

This section describes the main features of the system prototype developed using the participatory scenario-based design process. Again considering the representative interactive collaboration scenarios, the interface design choices made to address the derived claims are explained. Such choices were also based on the adaptation of a set of interactive collaboration patterns, described by (Martin and Sommerville 2004). The patterns, which were derived from other, quite different, settings, were used here to gain a better understanding in the domain of spreadsheet based collaboration. In fact, combining the analysis of such patterns with the considerations drawn from the fieldwork researchers were able to fruitfully exploit those patterns for the design of our spreadsheet-mediated collaboration prototype. Moreover, key principles of graphic design were followed during the collaborative interface design, that also drove decisions on layout, on content display and on the use of colors as a further hint for collaboration awareness and feedthrough. These key principles were based on Cooper’s work on interaction design (Cooper et al. 2007).

The prototype was developed on Apple™ IOS 4.1 for iPhone™, with 480 × 320 pixel screen resolution. The web interface of the collaborative spreadsheet was developed as a RIA solution exploiting the AJAX framework. Finally, the communication between the central application Sirio and the mobile application exploits web services based on Apache Axis 2 and Apache Tomcat. Figure  5 describes a representative interaction scenario, showing spreadsheet-mediated collaboration and its effects on situation awareness, both locally and centrally.

Figure 5

An interaction scenario.

The first interaction scenario is concerned with the mobile interface design issues. In Figure  6a a user-customized interface is visualized, where the relevant portion of a spreadsheet is displayed with appropriate resource synchronization among competing users.

Figure 6

a the GA1 spreadsheet interface which shows the reservation request, and b the GA2 interface which depicts the consequential locking of the cell involved in the collaboration.

During the event, people located at a gathering area, who are in charge to manage evacuation are provided with a mobile device that has the proposed application. On the main window, the manager can read information of interest and start organizing evacuation. The spreadsheet interfaces displayed in Figure  6 address the design challenges raised in Claim 1 of the previous section so that only the user-relevant customized information is visualized. On the top-left corner, the manager reads the ID of the gathering area he is managing. The contour of this cell is yellow, which uniquely identifies this user as well as the changes he makes to the spreadsheet. Next to this cell, the name of the shelter where citizens from GA1 should be directed is displayed. In the corresponding column, main information about the shelter is highlighted. In particular, the second row indicates the current number of vacancies at that shelter, namely the number of people who can still be accommodated there. The third row specifies how many people have been identified and sent so far to the shelter by the manager of the gathering area (200 people in Figure  6a). These are values aggregated from the list located below, which indicate the requests that the GA manager has performed so far. Each request corresponds to a shelter reservation and has to be written by filling the first empty cell in the column (in Figure  6a, the user is setting the last cell to 50). This value is then subtracted from Vacancies and, once selected the people to rescue, added to the Processed.

The scenario also illustrates how the issue of multiple users conflict resolution were addressed (Claim 2 above). When GA1 user makes his request, he enters in competition with other managers for vacancies. In this case, the collaborative system must provide for appropriate collaboration feedthrough among the competitors and ensure that the number of visualized vacancies is consistent across the different user interfaces. To achieve that, the system locks the first empty request cell of the other managers. The image shown in Figure  6b depicts the view of the GA2 interface at the time when GA1 user makes his request.

As further support to collaboration awareness and in response to Claim 3 above, the locked cell is associated the color of the updating user (yellow for GA1 user in Figure  6b), so that all the competing users know who is currently updating the vacancy value. On the right side of the spreadsheet, it is possible to see two rectangles which indicate the manager who currently concur to the reservation of the vacancies of Shelter 1. The two rectangles are highlighted by a color which indicates who is the responsible for locking and contain the number of people processed at the moment.

On the bottom side of the interface, researchers designed a common-summarizing part where some important information is always visible even if the sheet changes. As an example, the two icons in Figure  6 indicate the shelter vacancies and the number of people who have not been recognized yet. Incidentally, those icons were adopted in the original emergency management system and, being associated with standard meanings in that context, researchers decided to adopt them directly.

The reservation operation automatically opens the gathering area census sheet (Figure  7 ) on the manager’s mobile interface. Here, he may select the identified people who, only at this moment, can be directed to the shelter. The application allows to select up to the number of people corresponding to the reserved vacancies. Once the manager completes this operation he can go back to the main window and start the process again.

Figure 7

The GA1 sheet concerning the census information.

The information visible to each GA manager is visualized on the screen of the complete shared spreadsheet on the COC side (Claim 5). As an example, the complete spreadsheet shown in Figure  8 summarizes the information of Shelter 1 collected from the different GA manager devices. In the Figure, the data coming from GA1 and GA2 are visualized on two separate columns. According to the local execution, the operations which lock cells are automatically shown on the COC interface. In this way, the COC operator is able to gain a view of the shared data as well as of the operations which are currently performed.

Figure 8

The COC view of the shelter information.

As the shelter (Shelter 1) is filled up, the number of vacancies goes down to 0, as shown in Figure  9a. In this case, the COC may decide to assign a new shelter to the GA managers who require it.

Figure 9

a the sheet showing the relevant information of the gathering area GA1 when vacancies goes down to 0, and b the summarizing COC sheet visualizing the shelter assignment.

The COC user checks the current receptivity situation at the shelters as well as the current shelter assignments (Figure  9b) which are highlighted by a tick corresponding to the associations. To address the design challenge raised by Claim 6, concerning notification and persistence on the interface, in case a shelter vacancies turn out to be 0, both the shelter and the gathering areas associated are highlighted by a red rectangle. Then, the COC user may decide to assign a new shelter to the GA managers by simply changing the association ticks as shown in Figure  10 .

Figure 10

The new shelter assignment.

The result of this operation is automatically viewed by all the involved GA managers on their interfaces (as immediate collaboration feedthrough). As a matter of fact, a column indicating the information concerning the newly assigned Shelter 3 appears on the main window of the application while the information about the previous shelter is moved next. As shown in Figure  11 , the total vacancies of the shelter is shown in the first cell and the total processed in the second one.

Figure 11

The new view of the main sheet of the GA1 user interface after the assignment of Shelter 3.

A further change with respect to the previous view concerns the managers who are currently concurring for beds at the Shelter 3. In fact, as shown in Figure  11 , new concurrent managers may appear.

A final consideration concerns the scenario for the management of injured people. In this case the mobile application user is the health assistant (HA), who is responsible for triage operations at a given gathering area. If a group of people need quick transportation to hospital, the HA can gain a general overview of the vacancies and the distances from hospitals enabling HA to make rapid decision. This information is summarized in graphical form and presented on a map by exploiting Framy (Paolino et al. 2010), a visual analytic technique for mobile interfaces. Figure  12a and b illustrate the use of this technique. The map is surrounded by two frames, each divided into sectors, which graphically represent the number of vacancies and the distance from the hospitals located within each sector. The color intensity of each sector of the yellow frame is proportional to the number of beds in the hospitals located off-screen, whereas the color intensity of each sector of the blue frame is inversely proportional to their distance from the gathering area were the user is located. Thus the user is able to perceive that the most appropriate hospital is located besides the top-right corner of the screen. Once selected, the HA taps on the corner and the shared spreadsheet devised for hospital receptivity analysis and reservation is automatically displayed (Figure  12c). Similarly to the reservation of vacancies at the shelter, the HA can reserve recovery for a group of injured people in the hospital by editing the first empty cell of the corresponding hospital, provided that it is not locked by other users.

Figure 12

a and b the Framy view of the information concerning the hospital vacancies and distances, and c the corresponding spreadsheet view where making requests for vacancies.

8 Usability evaluation

This section describes the usability study performed in order to validate the system prototype. Researchers designed a reference scenario that enabled them to study the emergency management activities in a controlled laboratory setting. The scenario was conceived so that they could validate the rationale behind the main design choices, including visualization of user-relevant customized information, use of colors to uniquely identify other users accessing the shared spreadsheet, cell locking/unlocking for multiple users conflict resolution, and any other design choice made to support collaboration, situation awareness and real-time information sharing. With the help of the Civil Defence experts, researchers devised a laboratory setting which would guarantee ecological validity of the experiment, while retaining the benefits of a controlled environment.

A comparative study was conducted between groups adopting the proposed prototype (Software based approach) and groups following the traditional approach (Paper form based approach).

8.1 Research question and hypotheses

The research question that underlies the formulation of the hypotheses is

Does an enhancement occur in terms of performance between the traditional and the software-based approaches?

The experiments were conceived to verify whether the proposed collaborative tool is able to address existing problems related to the phases of communication and information sharing.

As usual, the null hypotheses are first formulated

• H01. Time to complete decisional processes with the Software based approach is longer than time required by the Paper form based approach.

• H02. There is no improvement in the quality of the decisions with the Software-based approach with respect to the Paper form based approach.

If both the hypotheses can be rejected, the experiment succeeds and it can be concluded that an improvement is achieved with the system, in terms of both time and quality of decisions.

8.2 Participants and groups

Two different groups of participants were recruited. For convenience, in the following sections they are referred as the PF and SW groups. The difference between PF and SW groups was that participants of the former group used the traditional paper form based approach, whereas participants from the latter group used the collaborative software tool to carry out their tasks within the given scenario.

Participants were 20 people recruited from the Civil Defence Agency. They were chosen considering the strategic roles identified by Italy law as crucial for crisis management activities, namely the Decision-Maker (DM), the Medical Doctor (MD), the Environment and Territory expert (ET) and the Public Employee(PE), the latter covering also the role of gathering area manager. Such roles were equally distributed so as to form three teams of responders who used the paper form based approach and three teams who used the software based approach, respectively. Therefore, each of the two experimental groups, PF and SW, was composed of 1 DM, 3 MDs, 3 ETs and 3 PEs.

Some information about the participants. Basically, their ages ranged between 35 and 62 years. Most of them had graduate degree while a minority had undergraduate degrees. All of them were familiar with the activities described in the reference scenario, were equally comfortable in handling maps on paper as well as on computers and extensively used smartphones in their everyday lives.

8.3 Experimental scenario

The reference scenario was conceived to fully represent a real emergency scenario. The city where the scenario is based is again Montemiletto. Before giving detailed instructions on the experimental tasks participants are provided with a common background.

During the experiment, all materials needed to carry out the described activities, including thematic maps and paper forms, were facsimiles of the official ones. Each group was assigned 10 different emergency configurations and was required to create the corresponding evacuation plans. In our experimental design, these were considered repeated trials referred to as runs (R1, …, R10). Participants did not change their roles across the runs. The task scenarios for each run were similar in that there were groups of people in need of rescue after an earthquake event, there were three possible shelters, only one route to each, and the same amount of information was provided, as explained in the background session.

8.4 Procedure

For each experimental group (PF and SW), participants were seated in four adjoining rooms. The DM in one room, representing the COC center, and 3 rooms with teams of 3 people each, namely 1 ET, 1 MD and 1 PE. These represented local teams, who collected data on the situation at a given GA and communicated remotely to the DM in the other room. Participants of the PF group were provided with a microphone, and set of speakers for verbal communication among group members and with the experimenters (this configuration was meant to simulate the communication happening among Civil Defence members using low frequency radio links), while SW group participants were provided with an iPhone 4 s installed with prototype application. For all the runs, at the beginning of the procedure, the participants read descriptions of the scenario and read the role-specific constraint sheet relating each piece of information to their map. At that point, the participants began to collaborate on the planning task. In case of PF group, a messenger (external to the group) was passing every 5 min to collect requests and bring them to DM.

In order to collect and distribute information, participants from PF group had to share it following the traditional procedure, namely by writing down their requests onto the emergency sheets, then waiting for the messenger to collect and send them to DM (it is important to notice that every time the messenger arrived they had to send some requests, even if partial). For both experimental groups, when DM made a decision, s/he had to write down the final plan and share it with the three local teams. Groups were given about 20 min to complete each task. This process was repeated ten times (also named ten repeated runs), with new constraints and information presented each time.

8.5 Dependent variables

Quality of decision and efficiency of the decision task were chosen as dependent variables. The former was measured in terms of number of errors, as the distance between decision-maker’s solution (DMS) and the optimal solution (OS), the latter was measured as the mean time to complete the decisional process.

Basically, for each configuration of the reference scenario, researchers chose one optimal solution expressed in terms of distribution of evacuees across the shelters. Analyzing decision maker’s solution, the number of errors were reported. Table  4 presents an example of errors derived from wrong or missing assignments. In the table only 2 evacuees are not correctly assigned, namely the one assigned to the wrong shelter and the one not assigned at all.

Table 4 An example of errors derived from the correct solution and the provided solution.

8.6 Data collected and discussion of results

The data collected during the experiment is shown in Table  5 . It contains the time and the errors recorded for each of the two experimental groups and for each run.

Table 5 Summary of the collected data and averages.

The column indices correspond to the 10 runs (R1, R2, …, R10) and the row indices correspond to the groups. The first two rows report the mean time for task completion, expressed in minutes. The third row reports the benefit gained in efficiency for each run, namely for each completed plan. The other three rows indicate the number of errors made and the benefits in terms of accuracy.

Analyzing SW averages for efficiency and number of errors, 7.5 and 4.2, respectively, it was noticed that in both cases they are lower than the average values from PF group (9.9 and 6, respectively). In particular, analyzing the differences of the single runs corresponding to the third and the sixth rows, it is possible to notice that an improvement does exist both in terms of efficiency and accuracy. The average improvement in terms of time is 2.3 min, whereas the average difference in number of errors is 1.8.

The purpose of the experiment has then been to verify the validity of the above assertions, proving that the improvements were not casually derived. To achieve that, a one-tail t-test was applied with a significance p-value <0.05 on the collected results for each pair of groups to be compared.

Results shown in Table  6 allowed researchers to conclude that both the null hypotheses can be rejected. In particular, the null hypotheses H₀₁ is rejected at 95 % of confidence, that is to say at this significance level, from the collected data it is evident a time enhancement is achieved using the software prototype compared to the paper form based approach. Also in the case of the second null hypothesis H₀₂, stating that quality of decision decreases with the software prototype with respect to paper form based approach, the t-test returns T ≈ 8.4 and pH2 ≈ 0.009. Therefore, it should be rejected in favor of the alternative one, at a 95 % confidence level: quality of decision does improve with the software prototype.

Table 6 T-test results.

9 Final remarks

During a crisis, several strongly related elements are in running, namely human resources, normative protocols, methods and technologies. All of them are addressed to guide the response and support decision making in order to reduce loss of human lives and resource. In the Literature Review Section it has been widely described the fundamental role of the communication and cooperation activities in crisis management, whose low efficacy represents one of the major reasons of failure of a well-established emergency information systems.

The goal of the research described in this paper was to design a tool for the improvement of cooperation among distributed teams of responders during a crisis management. Some primary requirements were identified through the collaboration with people from the Italy Civil Defense Agency, namely to exploit the pervasiveness of well-established technologies and take advantage as much as possible of users’ familiarity with customized tools. As a result, a spreadsheet-mediated collaborative system is proposed which allows users to cooperate through mobile devices, whose widespread diffusion overcomes the possible initial difficulties usually encountered when using a new technology. Moreover, the graphical similarity between a spreadsheet and a data sheet card allowed for exploiting the users’ experience of filling out traditional paper forms thus requiring a very little training by any basic user and making the computer system more likely to be used when under stress. In particular, the proposed system is capable to adapt output data visualization to the different actors involved in crisis management by offering different reduced views of the whole document to different users, different permissions on single cells depending on the user’s role and clear and understandable visual masks to guide operators in filling out the form. Finally, because of its collaborative nature, the whole document is constantly under the supervision of the command center operators that can exploit the communicational channels provided by the system to require possible corrections or omissions.

Another positive aspect of the presented solution is that it integrates seamlessly with the technologies and protocols already in use, because of the interoperable modules on which it is based. This feature also reduces the impact on the current well-established and validated intervention model, satisfying one of the general requirements derived from our interviews.

Finally, the integration of the visual analytic technique Framy for a qualitative visualization of data, provided impactful visual cues and enhanced the acquired knowledge. Indeed, it is possible to interact with spreadsheets and maps to show both analytical and synthesized data referring to a geographic area, and obtain a summarized global view of a specific phenomenon.

In order to satisfy the aforementioned requirements and achieve the expected results a design approach combining participatory and scenario based was used and concluded with a usability study meant to validate solutions proposed to the mobile interface design challenges.

A major lesson learnt from this research is that the concerns associated with real time information sharing and situation awareness should be central to any design activity related to disaster response systems. In fact, a low level of cooperation and lack of context perception by users may represent the weak points of any disaster management process, even when a sophisticated decision support system is available to decision makers

Another useful insight was the success of combining participatory design approach with the scenario based design approach to arrive at an optimal solution.

The involvement of the users throughout the design process greatly assisted to validate the design at every stage of the process against the users’ requirements. The goal of our future work is to contribute to the specification of guidelines which may support designers in developing mobile collaborative applications useful in both ordinary and critical management processes.