Keywords

1 Introduction

The Internet of Things (IoT) is a powerful industrial system of radio-frequency identification (RFID) and wireless (mobile and sensor) devices that have the ability to transfer data over a network without needing human interaction [1]. Mainly, IoT consists of three layers: (1) the sensing layer to gather data from the real world via existing hardware, e.g., sensors and RFID; (2) the network layer to transfer the collected data over wired/wireless network; and (3) the application layer which is responsible for two-way communication between user and systems [2]. IoT applications are rapidly evolving and growing in various fields (e.g., health-care service industry, safer mining production, transportation and logistics, firefighting, and crowd management).

In this chapter, the focus is on crowd management using IoT and smartphone technologies. Examples of crowds that need to be properly managed can be seen in the Kingdom of Saudi Arabia where several millions of pilgrims visit it annually in order to perform the holy rituals of Hajj and Umrah and in Mumbai suburban railway (India) where eight million passengers travel every day in the highest crowded railway in the world [3]. Both cases require a robust crowd management framework (CMF) which is able to manage the huge number of visitors in order to avoid crowd disasters (e.g., stampede and suffocation).

Sensing the crowd by normal sensors and managing it is a challenging problem. Crowd management requires several stages including crowd data acquisition via sensor layers, data transferring via network layers, data analysis for decision-making, and applying crowd control measures via application layer; see Fig. 3.1.

Fig. 3.1
figure 1

IoT architecture for crowd data management [2]

Full size image

Using different sensor devices, it is possible to gather information about visitors’ crowds and determine which areas are overcrowded. This information is then transmitted to the management layer where the administrator can decide to close some doors and roads. The management layer shall be equipped by a smart service that can recommend which doors to be opened/closed to direct the crowd flow. The admins then decide whether to publish this information to the public or not. The visitors receive the publicly available information through a mobile application which informs them about (1) current open roads and doors, (2) how to find noncrowded areas, or (3) how to locate their groups and friends. These services will lead to great save in time and efforts in managing the crowd flow.

The proposed framework shall be highly usable and reliable. In order to increase the user-end application usability, the application provides different languages and a user-friendly GUI. Moreover, the application shall be accessible not only by mobile devices but also by distributed devices throughout the crowded region with a touch screen (and/or visual impairment enabled device) interface for visitors. The collected data shall be reliable, as it is collected from different resources as monochrome cameras, infrared cameras, and RFID tags. The framework shall be always available with high performance. These features are enabled via replicated web services of the middleware layer on various servers. In the case of service failure or overload, the framework adaptively selects another replica to respond to visitor’s requests.

The rest of the chapter is organized as follows. Crowd information acquisition and management are covered in Sect. 3.2. Then, a full description of the proposed framework is given in Sect. 3.3. Section 3.4 explains the workflow of the proposed framework, and the case study is discussed in Sect. 3.5. Finally, the conclusion is presented in Sect. 3.6.

2 Crowd Information Acquisition and Management

Crowd information acquisition and management can be classified into three main technologies: vision, wireless/radio-frequency (RF), and web/social media data mining [3]. Figure 3.2 summarizes the most recent work for crowd information acquisition and management.

Fig. 3.2
figure 2

Most recent work for crowd information acquisition and management

Full size image

A vision-based technology can estimate the crowd density by analyzing the aggregated real-time images, video, thermal videos, or satellite images. The work in [4,5,6] introduce a video data analysis-based system for crowd management, which are capable of detecting and tracking visitors [4], estimating the possible route, actively guiding alarm and signal [5], and building crowd energy modeling [6]. The works in [7,8,9] are using thermal video analysis in order to avoid overcrowding and estimating crowd density at Hajj ritual places. An analysis model was developed by [7] based on an FLIR camera and temperature sensor to detect crowd density. The authors in [8] have added two components to the previous model as visual camera and light intensity sensor in order to increase the density estimation accuracy. Another framework [10] proposes tracking pedestrians and crowd density estimation depending on the analysis of high-resolution satellite images.

RF-based technologies, e.g., RFID (radio-frequency identification), wireless sensor network (WSN), near-field communication (NFC), and Bluetooth are proposed for data acquisition in crowd management system. The methods in [11,12,13] use RFID tag in order to identify, track, count, and locate the pilgrims within a certain zone. However, the main drawbacks of RFID technology are that it is less reliable, expensive, and has less memory power [13]. The work in [14] developed a system for estimating crowd density by using Bluetooth technology. The system is composed of three components: a Bluetooth Low Energy (BLE) tag for identifying the pilgrims, a smartphone for counting the BLE, and a web server for crowd data analysis and monitoring. This approach is simple and inexpensive; however, BLE and RFID tags are easily damaged, lost, or mixed with other pilgrims, which may cause inaccurate crowd estimation [15]. The work of [15] uses a mobile phone for identifying and saving important information about the pilgrim, then exchanges this information with doctors or police officers using NFS technologies. The main challenge of using mobile phones for crowd density estimation is that pilgrims may be not carrying a mobile (or wireless devices) and other pilgrims may be carrying multiple mobiles, which can lead to incorrect crowd estimation [3]. Another system tracks and identifies missing pilgrims by using GPS on mobile phones and WSN [16, 17].

Finally, social media and web data mining-based technologies help for crowd management [18, 19]. An application called VIStologys HADRian is developed in [18] for integrating various information sources as social media into a common operational picture (COP) for humanitarian assistance/disaster relief (HADR) operations. An application called “Hajj Mabrour” is developed in [19] to help for crowd management. Under the social media sharing platform, the application (1) tracks visitors via Google Maps, (2) notifies and alerts via SMS and email, and (3) enhances the situational awareness via sharing information. The main challenge of this technology is how to extract critical information from a large amount of data and how to keep the personal privacy during data acquisition.

From the previous work, we can conclude that vision-based approach is recommended for crowd density estimation and RF-based technologies are better for tracking and identifying visitors. Hence, in our framework, a combination of these technologies is proposed to achieve our target to well distribute visitors over crowded areas.

3 Crowd Management Framework

This section introduces the crowd management framework (CMF) that aims to redistribute visitors in a crowded area and exploits RFID tags to locate missing children or old people. Mainly, CMF consists of three layers: the sensor layer, the management (middleware) layer, and the interface layer as shown in Fig. 3.3.

Fig. 3.3
figure 3

Crowd management framework (CMF) layers

Full size image

The sensor layer includes all sensors to be used for crowd data acquisition and people tracking. The management layer is acting as a middleware between the sensor and the interface layer; it collects data from sensors, analyzes it, and then transfers the results to the interface layer. The interface layer is an application which aims to help administrators and visitors to avoid crowd disaster, as it informs pilgrims about (1) current opening roads or doors, (2) how to find noncrowded areas, and (3) how to locate their groups and friends, as we will explain later in detail.

3.1 Sensor Layer

Sensors are divided into two categories: crowd density estimation sensors and tracking sensors. Crowd density estimation sensors consist of four components, connected through a computer system, as proposed by [8]: a monochrome visual camera, a low-cost infrared camera, temperature, and light sensors.

Tracking sensors aim to identify and locate the visitors within a certain zone using RFID (in controlled places like El-Haram) and mobile GPS sensors (in uncontrolled places like Mumbai station). RFID will be used to track children and old people. Vulnerable visitors (e.g., children, sick, and old people) will be given wristbands that include RFID tags. These tags contain visitor’s information, such as name, age, and nationality. Then, the admin can use RFID readers to retrieve visitors’ information from the RFID tags. In addition, Mobile GPS will be used by visitors to share their location when required.

3.2 Management Layer

The management layer consists of five components: analyzer, manager, monitor, dispatcher, and tracker:

  • Analyzer analyzes the data coming from the crowd density sensor using the same analysis model of [8]. It then notifies the manager component about crowded areas and saves it on the database in order to be shared with visitors after administrator’s approval.

  • Monitor is responsible for monitoring crowd estimation and tracking sensors. In the case of malfunctioning, it will notify the manager component about them for maintenance. It is also responsible for monitoring dispatcher service. It will notify the manager component to select another replica in order to reply to visitors’ requests (in case of failure) or to balance the load (in case of overloading) [20].

  • Manager provides a link between other framework components and administrators. It (1) notifies administrators about overcrowded areas, (2) asks for permission to close (doors or roads) and to share this information to the public through the mobile application, (3) asks for permission to locate vulnerable people when required, (4) notifies administrators about sensors required to be maintained, and (5) selects another dispatcher service replica to respond to visitors requests adaptively, in the case of dispatcher service failure or overload.

  • Dispatcher provides a link between other framework components and visitors. It is mainly web services invoked by visitors’ mobile application. It is responsible to publish information about overcrowded areas, closed doors, and roads on the mobile application. It receives also requests from visitors to search for missing people and forwards them to the manager to accept/refuse it. Acceptance is based on the trusted people circle that will be registered in advance for each user (e.g., family members and the group guide).

  • Tracker is responsible for tracking RFID tag holders after receiving an order from the manager. It communicates with all RFID readers to search for the requested RFID. It then sends results to the dispatcher in order to forward the search results to the person who placed the request.

4 Framework Workflow

The workflow of the suggested framework can be divided into three phases: monitoring, crowd management, and tracking. Monitoring phase is for sensing the failure which may occur to crowd estimation, tracking sensors, and dispatcher services. Crowd management phase is for distributing visitors over a crowded area and avoiding crowd disasters (e.g., stampede and suffocation). Tracking phase is dedicated to locating missing people. The three phases are explained in detail in the following subsections.

4.1 Monitoring Phase

The admin adjusts the monitoring component to check in predetermined intervals the availability of crowd sensors, RFID readers, and dispatcher service. If one of the sensors is malfunctioning, the monitor notifies the manager about that sensor failure. Then, the manager notifies the administrators to maintain these sensors. Also, if dispatcher services have failure or are overloaded, the monitor will notify the manager component for selecting another replica to reply to a visitor request (in case of failure) or to balance the load (in case of overloading). Figure 3.4 shows the sequence diagram of monitoring phase.

Fig. 3.4
figure 4

Sequence diagram of monitoring phase

Full size image

4.2 Crowd Management Phase

The analyzer gathers periodically the crowd data from the crowd sensors. It analyzes the data using the model in [8]. After that, it notifies the manager about overcrowded areas and saves it on the database. The manager receives the information from the analyzer. Based on this information, it determines the roads and doors that should be closed. The manager then asks the administrators to confirm publishing this information. Upon acceptance, the manager updates the information privilege on the database to be available for sharing. Then the dispatcher shares this information with the visitor’s application. Figure 3.5 views the sequence diagram of crowd management phase.

Fig. 3.5
figure 5

Sequence diagram of crowd management phase

Full size image

4.3 Tracking Phase

Unlike previous phases, this phase is triggered on demand. The visitor sends a request to locate a missing person. The dispatcher forwards the request to the manager. Then, the manager asks the administrator to confirm this request. After confirmation, the manager sends RFID tag to the tracker to locate the missing person. The tracker asks the RFID readers in the sensor layer. The RFID readers provide response to the tracker on whether the RFID tag has been found or not. After that, the tracker forwards results to both manager and dispatcher in order to inform the administrator and visitor, respectively. Note that if the missing person has a mobile phone with GPS, he/she can share his/her position with friends. Figure 3.6 shows the sequence diagram of tracking phase.

Fig. 3.6
figure 6

Sequence diagram of tracking phase

Full size image

5 Practical Example: Pilgrim Guide App

In this section, the main features of the Pilgrim Guide application are explained. The application is considered as a practical example of the interface layer in the proposed CMF framework. The Pilgrim Guide application (1) collects the data from sensors and RFID readers, (2) analyzes the collected data, (3) visualizes the collected data in appropriate forms and reports, and (4) periodically checks the performance of the sensor layer.

In the Pilgrim Guide application scenario, RFID tags will be given to pilgrims upon arrival to the Kingdom of Saudi Arabia airport. Also, pilgrims are kindly asked to install the application to help them during the journey. Furthermore, a number of sensors and RFID readers are installed on different ritual places. If the tag is missed, the pilgrim should contact the tour guide or one of the Haram administrators to locate the missed tag. In case the tag can’t be found as it may be missed outside the ritual places, a new tag will be provided for the pilgrim. The tag information is secured, as it is only accessible through the readers of Haram administrators.

Figure 3.7 provides a domain model for the Pilgrim Guide application. The aim of the diagram is to introduce the application’s objects and relationships between these objects. This diagram could be helpful for business analysts and software developers. Each pilgrim is guided by exactly one tour guide. Pilgrims and tour guides are able to get the current state of the Haram area (crowed or uncrowded) and search for other pilgrims upon the acceptance of Haram administrators. Haram administrators are able to confirm the request to search for a missed pilgrim, publish the current state of the ritual places, and check the performance of the sensors and RFID readers. Each ritual place has many administrators for managing the crowd, sensors for determining the crowd percentage, and RFID readers for finding lost pilgrims.

Fig. 3.7
figure 7

A domain model for the Pilgrim Guide application

Full size image

Pilgrim Guide consists of two types of applications: a web-based application for Haram administrators and a mobile one for pilgrims. Both applications are available in different languages (Fig. 3.9a) to increase the application usability. And it requires pilgrims to install the application on their mobiles, register, and login before being able to use the different available features. The following subsections explain these applications interface in detail.

Fig. 3.8
figure 8

Example of the administration application (suggestion table)

Full size image
Fig. 3.9
figure 9

(a) Pilgrim Guide application (b) Haram map overview (c) notification messages

Full size image

5.1 Administration Application

Through the administration application, the admin can explore the Haram map, identify crowded areas, and then accept or reject to close road or door (Fig. 3.8). Also, the admin can send notifications to all pilgrims. Another important feature is the management of pilgrims’ requests to track items/people that have RFID tags.

5.2 Pilgrim Application

Through the Pilgrim application, the pilgrim can explore the Haram map, identify crowded areas and closed doors (Fig. 3.9b), receive different notifications from Haram administrators (Fig. 3.9c), and store and retrieve different contacts.

The application provides a contact list that contains three different sections as seen in Fig. 3.10; the first section holds a close group (e.g., wife, children, and group coordinator). The second section stores all items that have RFID tags attached to it. Last but not least, the third section provides information about important emergency places (e.g., closest police station or hospital). Through the contact list, the user can send a request to the Haram admin to exactly locate one of his contacts. This feature is very important in case of children or property loss.

Fig. 3.10
figure 10

Contact list (a) Friends tab (b) RFID tab (c) General tab

Full size image

Figure 3.11 presents another important feature, that is, the selection of a suitable place inside the Haram. Here, the user can set some conditions including the suitable crowd degrees (low, middle, don’t care), air condition (with/without) air-conditional area, and the open preferable road and door to El-Haram; then, the best location is shown on the map with detailed path information.

Fig. 3.11
figure 11

Select your position (a) selection menu (b) selection result

Full size image

6 Conclusions

In this chapter, a crowd management framework for distributing visitors over a crowded area is introduced. The framework consists of three layers: sensors, management, and interface. Sensors layer aims to track visitors and gather information about visitors’ crowds. Management layer aims to analyze the collected data and extract the required information about the visitors upon administrative acceptance. The interface layer provides an application that aims to help administrators and visitors to avoid crowd disasters as it informs them about (1) current opening roads and doors, (2) how to find noncrowded areas, and (3) how to locate their groups and friends. The proposed framework will effectively save time and efforts to help administrators controlling and distributing visitors via low-cost sensors manipulated by smartphone. Also, the proposed framework shall be highly usable, reliable, available, and performance.