Human Computation for Disaster Response

Abstract

Human computation methods involving the use of real-time social media data have been used successfully to support humanitarian efforts for disaster-affected communities. Through an examination of various case studies, this chapter describes specific crowdsourcing methodologies applied to disaster relief, with attention to the challenges, benefits, and outcomes. Furthermore, consideration is given to potential methods that might combine more effectively the roles of machines and humans, such as adaptive systems, gamification, and high volume analytic techniques. A “call to action” concludes the chapter, endorsing a policy by which existing volume limitations on social media data access are suspended temporarily for humanitarian aid organizations during emergent crises.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

Disaster-affected communities are increasingly using social media to communicate during major disasters. One consequence of this is the rise of Big (Crisis) Data. Recent empirical studies reveal that a small but critical-and-growing fraction of tweets posted during a disaster contain important information for disaster response.¹ Finding the proverbial needle in this growing “haystack” of crisis information has rapidly become a major challenge for the international humanitarian community. Social media use during Hurricane Sandy in 2012 produced a “haystack” of half-a-million Instagram photos and 20 million tweets over just a few days. The year before, over 300,000 tweets were posted every minute following Japan’s devastating earthquake and Tsunami. There are at least two ways to manage this volume and velocity of data: (1) Artificial Intelligence and (2) Artificial Artificial Intelligence, or Human Computation.² The purpose of this chapter is to analyze the use of human computation for disaster response.

The chapter is structured as follows: the first section describes the use of human computation in response to six major humanitarian crises: Haiti Earthquake (2010), Libya Revolution (2011), Somali Crisis (2011), Hurricane Sandy (2012), Typhoon Pablo (2012) and Mali Crisis (2013). The human computation technologies used to support these disaster response efforts include CrowdCrafting, CrowdFlower, Humanitarian OpenStreetMap’s Tasking Server, MapMill, Tomnod and Ushahidi. The groups engaged in deploying and using these technologies include the Standby Volunteer Task Force (SBTF), the Humanitarian OpenStreetMap Team (HOT), the UN Office for the Coordination of Humanitarian Affairs (UN OCHA), the UN High Commissioner for Refugees (UNHCR) and the US Federal Emergency Management Agency (FEMA). The second section builds on these case studies to outline what the future of human computation for disaster response will look like. This section also highlights the use of mobile solutions, gamification and massively multiplayer online games to process humanitarian microtasks. The chapter concludes with a call to action—namely the launch of Big (Crisis) Data Philanthropy for Humanitarian Response in order to grant humanitarian organizations full access to social media data during major disasters.

Haiti Earthquake

Human computation was first used for disaster response following the devastating earthquake that struck Port-au-Prince on January 12, 2010. Graduate students at The Fletcher School (Tufts University) launched a live crisis map within hours of the earthquake to document both the extent of the damage and the disaster-affected population’s urgent needs.³ This information was initially sourced from social media such as Twitter and quickly complemented with reports from the mainstream media. In order to cope with the extensive live coverage of the disaster, Fletcher School students decided to crowdsource the real-time monitoring and processing of several hundred online sources. Within days, several hundred volunteers from Boston, Montreal, New York, London and Geneva answered the call. Together, they manually triaged and geo-referenced over 1,500 reports that were mapped using the Ushahidi platform. Ushahidi is a free and open source mapping software.

Several days after the earthquake, an SMS short code was set up and integrated with the Ushahidi platform. This short code (4636) enabled anyone in Haiti to text in his or her location and urgent needs.⁴ Information about the short code was disseminated via community radio stations in Haiti and via Haitian Diaspora news channels. The team behind the Ushahidi software quickly developed a platform to crowdsource the translation of incoming text messages since the vast majority of these were written in Haitian Creole. Volunteers from the Haitian Diaspora were recruited via social media. Together, they translated some 10,000 text messages during the entire search and rescue phase. Two weeks later, the translation efforts were relocated to Haiti thanks to a partnership with the microtasking company CrowdFlower. This enabled Haitians to earn money for their translation work.

These volunteer-based efforts in response to the Haiti Earthquake marked a watershed moment for the international humanitarian community and the new field of Humanitarian Technology. One first responder, the US Marine Corps, publicly stated that the live crisis map enabled them to save hundreds of lives.⁵ Craig Fugate, the Administrator of the US Federal Emergency Management Agency (FEMA), referred to the crisis map as the most comprehensive and up-to-date information available to the humanitarian community.⁶ As a result of these efforts, the Fletcher student who spearheaded the Haiti response proposed the launch of a global volunteer community for digital humanitarian response.⁷ Together with several colleagues, he co-founded the Standby Volunteer Task Force (SBTF) in October 2010. Today, the SBTF includes over 1,000 digital volunteers based in over 80 countries around the world. Together, this award-winning network of pro-active volunteers have managed some of the most important live crisis mapping operations that have supported both humanitarian and human rights organizations over the past 3 years.⁸

Libya Revolution

One of the most important SBTF deployments remains the response to the Libya Crisis. The United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA) officially activated the SBTF to request a live, crowdsourced social-media crisis map of the escalating situation in the country.⁹ The SBTF launched the crisis map within an hour of the request. The volunteer network was able to do this because they had designed specific criteria and workflows beforehand to manage live crisis mapping requests. For example, the SBTF has specific activation criteria that must be met by the activating organization. In addition, the SBTF is composed of multiple teams each of which is responsible for the human computation of the information processing cycle. For example, the Media Monitoring Team is responsible for monitoring both social and mainstream media for the type of information requested by the activating organization. The Geo-Location Team is tasked with identifying the GPS coordinates for relevant reports identified by the Media Monitoring Team. The Mapping Team adds the tagged reports to the crisis map while the Analysis Team produces regular trends analyses.

Thanks to these pre-designed human computation workflows and the use of Skype, SBTF volunteers were able to monitor well over 300 online sources and map thousands of relevant reports for an entire month, maintaining live coverage of the situation throughout. The fact that volunteers are also based in multiple time zones also meant that the map was updated around the clock. Because OCHA did not initially have any information officers on the ground in Libya and could obviously not rely on Libyan state media for accurate information, the live social media crisis map provided them with critical situational awareness during the early weeks of the crisis. Moreover, “OCHA did not have the idle capacity to gather, verify and process the enormous amount of available online information.”¹⁰ In an email to SBTV volunteers, OCHA wrote “The dedication and professionalism of the Task Force is commendable. Your efforts at tackling a difficult problem have definitely reduced the information overload; sorting through the multitude of signals is no easy task. The Task Force has given us an output that is manageable and digestible, which in turn contributes to better situational awareness and decision making.”¹¹

Somali Crisis

“Having a real-time map complete with satellite photos, of where everyone is at any one moment is almost as good as having your own helicopter.”¹² The United Nations High Commissioner for Refugees (UNHCR) was in desperate need of such a map when the crisis in Somalia began to escalate in October 2011. A massive number of people had been displaced to the “Afgooye Corridor” just West of Mogadishu due to the worsening famine and Al Shabab’s terrorist activities. While UNHCR had a couple estimates for the number of displaced individuals, they needed another way to validate these estimates. Getting an accurate figure for the number of Internally Displaced People (IDPs) is critical for disaster response. However, due to the volatile security situation brought about by Al Shabab, humanitarian organizations could not directly access IDPs in order to carry out on-the-ground surveys.

Live crisis maps, like helicopters, can provide a “bird’s eye view” of an unfolding situation in real-time. So the SBTF recommended that UNHCR “take to the skies” and use satellite imagery to estimate the number of IDPs in the “Afgooye Corridor.” The SBTF partnered with the satellite-imagery provider DigitalGlobe and Tomnod (c.f. Chapter by Luke Barrington) to microtask the analysis of satellite imagery of Somalia. Tomnod is a microtasking platform specifically designed for the tagging satellite imagery. The imagery is sliced up into smaller pictures each of which is then displayed to volunteers on the Tomnod platform. Users were asked to tag any informal and permanent shelters they could see in each satellite image. Within 120 h, volunteers created over a quarter million tags after analyzing close to 4,000 images.¹³ One of the advantages of microtasking platforms like Tomnod is the built-in quality control mechanisms that ensure a relatively high quality of output data. In the case of the Somalia project, each unique image was viewed by at least three different volunteers. Only when there was consensus between three volunteers vis-à-vis the type and location of a given shelter was that data point pushed to UNHCR. This triangulation mechanism yielded a count of 47,000 shelters in the Afgooye Corridor—a figure that the UN was able to use to estimate the approximate number of IDPs in the area.

After the completion if this human computation project for disaster response, the Deputy High Commissioner of UNHCR Alex Aleinikoff thanks SBTF volunteers via video.¹⁴ The transcript: “[…] I’ve just learned about the wonderful work done by the Standby Task Force which has permitted us to count shelters in the Afgooye Corridor in Somalia through the volunteer work of folks like you around the world. This is such a wonderful project for us it provides enormously important information to UNHCR and helps to create a worldwide virtual community involved in helping refugees and internally displaced people. So I salute you for your work and for the time you have devoted to this project, it’s important to us, it’s important to people who have been forced from their homes and who are trying to create a new home and a new beginning, thank you.”

Hurricane Sandy

Hurricane Sandy caused extensive damage along the Northeastern United States in October 2012. Within hours of the damage, the US Civil Air Patrol (CAP) flew a number of aircraft along the coastline to capture very high-resolution aerial imagery of the disaster-affected areas. According to a FEMA official working with Air Patrol at the time, “CAP imagery is critical to our decision making as they are able to work around some of the limitations with satellite imagery so that we can get an area of where the worst damage is. Due to the size of this event there is an overwhelming amount of imagery coming in, your assistance will be greatly appreciated and truly aid in response efforts. Thank you all for your willingness to help.”

To rapidly analyze the tens thousands of pictures produced by CAP for damage assessment purposes, the Humanitarian Open Street Map Team (HOT) team customized the MapMill platform to microtask the analysis of the imagery.¹⁵ Volunteers using MapMill would tag each picture as “OK” (no infrastructure damage), “Not OK” (some damage) or “Bad” (significant damage). The result? Nearly 6,000 volunteers analyzed over 30,000 images within the first week and provided almost 150,000 damage assessments in that time. About half of these volunteers produced around 80,000 assessments in the first 48 h alone. On average, every image was tagged or voted on 91 times. The resulting assessments were automatically shared with FEMA via their public GeoPlatform.¹⁶ FEMA subsequently launched a service for people to type in their address and get the CAP image of their house or building.

The HOT network was launched shortly after the remarkable response carried out by OpenStreetMap (OSM) volunteers following the devastating Haiti Earthquake of 2010. Using aerial and satellite imagery provided by the World Bank, volunteers traced the most detailed street map of Port-au-Prince ever created—and they did this within a week. Some 700 volunteers made over 1.4 million edits to the map during the first 30 days following the earthquake.¹⁷

Typhoon Pablo

Typhoon Pablo devastated large regions of the Philippines in December 2012. Twenty-four hours after the typhoon made landfall, the UN Office for the Coordination of Humanitarian Affairs (OCHA) activated the Standby Volunteer Task Force (SBTF) to assess the damage. OCHA requested that the multimedia assessment be based on Twitter and the resulting analysis provided to the UN within 12 h. The SBTF partnered with the Qatar Computing Research Institute’s (QCRI) Crisis Computing Team to collect over 20,000 tweets related to the Typhoon.¹⁸ Next, the SBTF used the CrowdFlower microtasking platform previously employed in response to the Haiti Earthquake. This time, CrowdFlower workers were paid to rapidly identify all tweets that had links to either pictures or video footage. These relevant tweets were then uploaded to the free and open source CrowdCrafting microtasking platform where SBTF volunteers tagged each image and video if they depicted evidence of damage. Volunteers also used CrowdCrafting to microtask the geo-tagging of all relevant pictures and video footage. Twelve hours after OCHA’s activation, the SBTF provided them with a detailed dataset of some 100 georeferenced images and videos depicting the devastation resulting from Typhoon Pablo.¹⁹ Note that like Tomnod, both CrowdFlower and CrowdCrafting also have built-in quality control mechanisms.

The OCHA team in Geneva used this data to create an official UN crisis map of the situation, which they immediately shared with their personnel in the Philippines. The map was also used by the Government of the Philippines and several other UN agencies. This crisis map of the typhoon was the first ever official UN information product based entirely on social media content. Following this deployment, QCRI’s Crisis Computing Team developed a way to automatically identify tweets that link to pictures or videos. The SBTF plans to use this in future deployments to accelerate the processing of tweets. This doesn’t mean that paid microtasking work has no role to play in digital humanitarian response. Microtasking platforms like Amazon Mechanical Turk and CrowdFlower have large, multinational and multi-lingual global workforces that will continue to be relevant for disaster-response human computation.

Mali Crisis

In January 2013, the Humanitarian OpenStreetMap Team (HOT) of volunteers began to map the transportation infrastructure, buildings and populated areas of Northern Mali to produce a basemap for humanitarian organizations monitoring the humanitarian crisis in the country. The volunteer network carries out these mapping assignments by tracing high (and low) resolution satellite imagery. Having access to the resulting map is particularly important for humanitarian logistics—that is, the delivery of goods and services to the disaster-affected population. This explains why open access to satellite imagery (and indeed other relevant data) is so important for disaster response. At the end of January, UN OCHA formally activated the HOT network to encourage volunteers to continue their mapping efforts and also expand them to include airports, health facilities, schools, water points, land use areas, etc.²⁰

To carry out this work, OpenStreetMap volunteers used their own customized microtasking platform.²¹ This tool places a grid of cells on top of the area that needs to be mapped. The platform can prioritize the microtasking work to focus on certain cells if specific areas are of particular importance to humanitarian organizations. For the Mali deployment, the HOT network traced roads, rivers, buildings, contour of residential areas, water wells, health services and other points of interest.²² At the time of writing, over 700,000 points had been added to the OSM database over a 6-week period. Each mapped object—such as a well or house—is represented by one or many points that trace the outline of said object.

The Future

As William Gibson famously noted, “The future is already here—it’s just not evenly distributed.” To get a glimpse of what the future holds for the use of human computation in disaster response, one should look back 2 years at the launch of SyriaTracker.²³ The project combines crowdsourced human intelligence with automated data mining in order to collect relevant information on the crimes and atrocities committed in Syria. The team behind SyriaTracker (all volunteers) use crowdsourcing to collect on the ground eyewitness accounts via email and Twitter. In addition, they repurposed Harvard University’s HealthMap, which used data mining for rapid digital disease detection. SyriaTracker customized the platform to automatically monitor human rights violations in Syria by mining over 20,000 English-based sources of news that regularly cover the crisis. The team cross-references and triangulates the crowdsourced reports with the data mining results in an attempt to further verify the accuracy of the collected information. The US Agency for International Aid (USAID), the Office of US Foreign Disaster Assistance (OFDA) and several other agencies are making direct use of the SyriaTracker data in their own official crisis maps of Syria.²⁴

SyriaTracker is the longest running crisis map ever. Why? Because the project is powered by human computation and data mining. Keeping this map up to date using volunteer-based human computation alone would be a Herculean task. Recall the “haystack” of half-a-million Instagram photos and 20 million tweets posted during Hurricane Sandy. Microtasking is no match for this volume and velocity of Big Crisis Data. Advanced computing techniques such as Artificial Intelligence and Machine Learning are needed to build hybrid approaches that combine the power of the crowd with the speed and scalability of automated algorithms.²⁵ QCRI is developing just such a system, a Twitter Dashboard for Disaster Response.²⁶ The platform enables users such as professional humanitarians to create their own automated classifier on the fly. A classifier is an algorithm that automatically classifies information. For example, if an earthquake were to strike Indonesia, OCHA could create a classifier to automatically detect tweets referring to infrastructure damage. Of course, the algorithm will not accurately tag all tweets, but the use of machine learning will ensure that the classifier improves over time, i.e., learns from its mistakes thanks to human supervision. To create these classifiers on the fly requires the use of microtasking—hence the importance a hybrid approach for disaster response.

The human computation component for disaster response still requires considerable improvement, however. Microtasking needs to become “Smart Microtasking,” which means a system that adapts to the skill set of its users. For example, a user that is particularly adept at geo-tagging should be assigned such tasks whereas a user that is more efficient at the categorization of messages as health, shelter, food, etc., should be given those tasks. These “Smart Microtasking” solutions also need to have mobile solutions—that is, they must be easily accessible via smart phone app. In terms of interface, whether web-based or mobile-based, the microtasking platforms used for disaster response have thus far been devoid of any gamification features. This stands in stark contrast to other microtasking projects in the area of citizen science. Zooniverse, for example, has mastered the development of gamified microtasking platforms, which explains why they have hundreds of thousands of users (See Chapter by Chris Lintott). But Zooniverse’s expertise and savoir faire has yet to crossover into the humanitarian space.

Lastly, there is huge untapped potential in leveraging the “cognitive surplus” available in massively multiplayer online games to process humanitarian microtasks during disasters.²⁷ The online game “League of Legends,” for example, has 32 million players every month and three million on any given day.²⁸ Over 1 billion hours are spent playing League of Legends every month. Riot Games, the company behind League of Legends is even paying salaries to select League of Legend players. Now imagine if users of the game were given the option of completing microtasks in order to acquire additional virtual currency, which can buy better weapons, armor, etc. Imagine further if users were required to complete a microtask in order to pass to the next level of the game. Hundreds of millions of humanitarian microtasks could be embedded in massively multiplayer online games and instantaneously completed. Maybe the day will come when kids whose parents tell them to get off their computer game and do their homework will turn around and say: “Not now, Dad! I’m microtasking crisis information to help save lives in Haiti!”

Conclusion

Human computation has already played an invaluable role in disaster response. The future, however, belongs to hybrid methodologies that combine human computation with advanced computing. The success of these next-generation humanitarian technologies depends on a number of critical factors. The first is the availability of the data. Twitter’s Terms of Service (ToS) restricts the number of downloadable tweets per day to a few thousand. Compare this with the 20 million tweets posted during Hurricane Sandy. Accessing the full Twitter Firehose of ~450 million daily tweets is prohibitively expensive. A possible solution? Big (Crisis) Data Philanthropy for Disaster Response.²⁹ Data philanthropy involves companies sharing proprietary datasets for social good. Call it Corporate Social Responsibility (CRS) for digital humanitarian response. Companies in this Data Philanthropy club would benefit from the publicity of supporting these positive and highly visible efforts. More importantly, their support would help to save lives. All that is needed is an agreed set of protocols that would provide humanitarian organizations with temporary emergency access to Big Crisis Data. The time to act is now. Both UN Secretary General Ban Ki Moon and UN Under-Secretary General for Humanitarian Affairs Valerie Amos have demonstrated the political will to have the humanitarian industry join the digital age. What we need now is the corporate will from Twitter and companies others to help save lives during the next major humanitarian disaster.