An Appraisal on Human-Centered Internet of Things

Abstract

This chapter outlines the current state of the Internet of Things (IoT) from the people’s association point of view. IoT is an intelligent environment where objects become smart and autonomously communicate with one another and human beings, through networks supported by interfaces. The IoT technologies have not only been widely studied in simulation and investigational circumstances but also dealt with real world scenarios. The IoT systems are enhanced by surveying diverse interactions between humans and the IoT to mine the implanted intelligence about individual, environment, and society. The IoT is facilitated by the latest developments in RFID, smart sensors, communication technologies, and internet protocols. In the upcoming years, the IoT is expected to bridge various technologies to enable new applications by connecting physical objects together in support of intelligent decision making. The spotlight of this chapter is to present the support of intelligent human-computer interaction for the IoT and to deal with human-centered concerns in the IoT.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Nowadays, the internet has changed from a network of interconnected computers to a network of interconnected objects. The dream of the IoT is to create the potential for objects of all kinds including people to communicate with one another by means of the internet. Apart from the technological hurdles such as privacy and trust, currently there is a lack of strong human-centered perspective on the IoT. Modern machines profoundly rely on the cognitive skills of their users, who make decisions and act in real-time. The cognitive skills of human beings include perception, knowledge, working memory, auditory and visual processing, judgment and evaluation, production of language, reasoning, problem solving and decision making.

The IoT systems [1] usually consist of a set of sensors that collect information, which is then transmitted among different devices without human intervention. But integration of the IoT and human interaction will make the IoT work smarter when input from people is considered. A system that collects data automatically can be compromised by issues of data qualities, such as inconsistent, missing, or unrecognized data which then can result in incorrect analysis. For example, human interaction with the IoT systems through mobile apps can augment automated data collection and correct, complement, extend, or even override the data gathered by the system or the actions it would undertake in response to the analysis of poor quality data. Also the functionalities of mobile apps such as camera, GPS, compass, bluetooth etc. make it possible for a person to collect a wide range of information which can in turn be used to interact with the IoT system.

The IoT systems roll out across industries, such as health management, industrial production, logistics, retail etc. The key to maximizing the usefulness of the IoT systems is to improve the capabilities of the IoT environments by making it easier for real people to contribute to them. The IoT should not be just about alarm clocks that start our coffee maker, or about accessing regular things over the internet but be the ultimate platform for human interaction with the physical world that would turn the internet into a mere medium.

Human-Centered Systems (HCS) are designed to complement human skills [2] and are not the systems that merely replace or automate human activities. HCS tap into our creative abilities that are unnoticed by the conventional problem-solving practices. It depends on human ability to be perceptive, to recognize patterns and to construct ideas that are emotionally meaningful as well as functional. Human-Centered Systems are inspired by behaviors rather than demographics, take place in natural contexts rather than controlled settings, and rely on dynamic conversations rather than scripted interviews. They are empathetic in understanding the needs and motivations of people who make up a community, collaborative in benefitting from multiple perspectives, optimistic in behaving irrespective of the constraints and experimental in making new things possible.

The design of Human-Centered Systems [3] is concerned with integrating the user’s view into the software development process in order to achieve a practical system. The significant principles of the Human-Centered systems should include the active involvement of end-users who have knowledge of the system context and should be a mutual process with an active participation of diverse experts with excellent technical skills. In this viewpoint, Sect. 2 describes some of the case studies on human-centered IoT and Sect. 3 discusses the conclusion and future directions of the IoT.

2 Case Studies of Human Centered IoT

This section describes a few of the case studies on IoT. The purpose of this description is to understand how the IoT intelligently help humans survive in vital situations. The IoT can be exercised by humans irrespective of their age and class such as patients, elderly people, disabled adults and children, night drivers etc.

2.1 Drug Tracker for Patient Safety

Patient safety is a major issue for human survival. The principal goals to augment patient care are drug compliance, nominal adverse drug reactions (ADR) and the elimination of medical errors. The drug compliance portrays the degree to which a patient correctly follows medical advice. The ADR results from the combination of two or more drugs. The above are caused mainly due to polypharmacy [4] in hospitals where patients undergo treatment for multiple illnesses. The polypharmacy increases with age and exists in poor countries where some diseases like TB cause the ADR. Such problems can be prevented if the prescribed drugs are examined to find the impediments by electronic prescribing of drugs and maintaining a record of prescribed drugs. An interaction between users and drugs is essential to bring out such drug checking tasks. Antonio J. Jara et al. have proposed a drug checker [5] as shown in Fig. 1 based on the IoT to watch the treatment for drug compliance and to detect ADR. The IoT adopts technologies such as barcodes, radio frequency identification (RFID) and near field communications (NFC) for identification of drugs. Once the drugs are identified, drug fitness according to the patient’s profile and personal health record (PHR) has to be verified by means of pharmaceutical intelligent information system (PIIS).

Fig. 1

Drug checker [5]

The components of the proposed architecture in Fig. 1 are listed as below:

• 1. The PIIS is a knowledge-based system which contains a rule engine system to detect the possible interactions between prescribed drugs and ontology where the drug concepts and patients information are described. After the drugs are prescribed by the doctors in hospitals the patients can confirm them before consumption.

• 2. The database contains a detailed drug description about ingredients and side effects.

• 3. The drugs are kept up-to-date and indexed by barcode numbers letting for easy and scalable identification.

• 4. A patient profile is set up, which contains the record of allergies and drug history ontology describes the anatomical, therapeutic, and chemical classification (ATC), dose, drug name, drug interactions, side effects, allergy causing ingredients etc.

• 5. Rule-based system detects drug allergies, skin rashes and breathing problems, drug interactions where one drug changes the effect of another drug, and drug loop where the side effect of one drug is solved by another drug.

The proposed drug checker has adopted many IoT technologies as depicted below:

• 1. Personal device based on IoT: A personal device called Movital based on IoT offering RFID and internet connectivity is developed and offered based on 6LoWPAN. This permits direct patient access to the internet and to PIIS. The RFID lets patient identification and a medium to load the patients’ health record by nearing their personal health card to the reader. The drug is identified by integrating RFID tags with drug box that store the drug code as a drug identifier as shown in Fig. 2. The Movital RFID raises an alarm or light signals to remind patients to take their medicines in time. The drawback of this adapted device is its highcost.

• 2. Movital IrDA: Another technology called Movital IrDA as shown in Fig. 3 is attempted in order to reduce the cost of Movital RFID. This tag is designed with low cost IC which consumes less power by transmitting the drug identifier through LED.

• 3. RFID/NFC using USB reader: This technology provides an option to store the patient profiles in the PIIS, and to access their health cards based on RFID.

• 4. GS1 barcode using smart phones: Currently the smart-phones have cameras by which the barcodes of drugs can be scanned and read. The drug ID reaches PIIS with the patient profile using the internet. PIIS checks the drug ID with its rule-based system and the patient profile and respond with informative details to the phone. This information verifies drug suitability for the patient with a proposed solution. The drug suitability is indicated by the display in green color and unsuitability in red color as shown in Fig. 4.

• 5. NFC using smart-phones and pocket PCs: The Pocket PC or smart phone is placed near the NFC drug tag, which reads the tag and starts the communication with PIIS as shown in Fig. 5.

Fig. 2

Drug box with RFID tag [5]

Fig. 3

Drug box with IrDA-based tag [5]

Fig. 4

Barcode reading [5]

From the above discussion, it is understood that the proposed system has used three media such as the RFID, the IrDA and the barcode. The proposed solutions have been analyzed and evaluated by a multidisciplinary group of experts and by two pharmacies. Each solution has its own pros and cons. As a result, Movital with RFID is recommended for richer countries whereas Movital with IrDA for low-income countries. The proposed system has made the possibility of utilizing new technologies to improve quality assurance in drug delivery, to improve adherence to drug appropriate consumption, and to reduce clinical errors caused by dosage mistakes and drug interactions.

Fig. 5

NFC with smartphone [5]

2.2 IoT Embedded Systems for Disabled Persons

Human-Computer Interaction (HCI) is a challenging task for people with disabilities. Davide Mulfari et al. have examined the use of low-cost embedded systems [6] to support the interaction between disabled users and computers. The main intention is to connect such embedded systems to any computer to help the user work on it without any prior configuration setup so as to adapt to the user disability. These embedded devices are designed in such a way that they manage heterogeneous wireless sensor networks and process data over cloud for IoT purposes. In general, disabled users interact with computers by means of Assistive Technology (AT) equipments. Such AT equipments need special configuration which makes the users difficult to work with their personal AT solutions. This issue can be solved by providing the end user with a smart AT hardware to interface the user with any computer without any installation of drivers or specific applications. As a result, the disabled user is enabled to employ his AT equipment to interact with different computing devices, such as PCs or smart phones. This setup process should make any computer to interact with the HCI system.

The proposed HCI system comprises of Atmega32u4, a microcontroller that supports mouse/keyboard libraries and acts as a resident mouse or keyboard when attached to a generic computer with an USB host port. Another key hardware component of the HCI system is a processor that needs to manage a network connection to access the internet and an USB host port to interface external peripherals. These constraints are accomplished by Arduino Yun board, a microcontroller board based on the Atmega32u4 and the Atheros AR9331 processor which supports a Linux distribution. Yun board is designed with quite a few kinds of sensor node, each sensor node consisting of a dedicated microcontroller, and an interface which enables the communication with a computing device. These sensor nodes are placed on human head or arms to capture their movements. Since the head is the highest body segment, head controls can be considered as alternative for hand controls to control the computer cursor as shown in Fig. 6.

Fig. 6

Accelerometer-based sensor node attached on the head [6]

The up and down vertical movements of the head are used for up and down displacement of the cursor while left and right horizontal movements are used for left and right displacement. Hence the disabled users can fully control a computer through head movements given that a sensor node can detect measurement of inclination changes less than 1.0$^{\circ }$. AT is mainly used by persons with disabilities in order to control their computers through resources other than a standard keyboard or pointing device. The other assistive solutions used by some of the related works are as follows:

• 1. Keyboards with alternative configuration for use with one hand.

• 2. Electronic pointing devices using ultrasound, infrared beams, eye movements, nerve signals, or brain waves to control the cursor on the screen without use of hands.

• 3. Sticks worn on the head, held in the mouth or strapped to the chin that are used to press keys on the keyboard.

The aforementioned discussion proves that the IoT devices can be effectively used to achieve human-computer interfaces for users with disabilities.

2.3 Elderly People Care Using IoT

Taking care of elderly people plays an important role in the society. Many elderly people are living alone because they are isolated from the society and are deficient in support from their family. Junbo Wang et al. have proposed a new application [7] using IoT to watch elderly people, to grasp the situations around them and to send the information to their care- takers to support them. While watching them, any abnormal activity detection need to be carried out since the services should be immediately offered to them in such cases to avoid any danger.

The proposed technique detects the situations around the user and provides the corresponding services or automatically changes environment to adapt the user. In this proposed work, not only the general situations are detected, but the abnormal activities in the situations are also recognized to support the users. The basic architecture of the proposed means is shown in Fig. 7. Initially the location of the elderly people need to be detected. A special type of sensor network called u- tiles sensor network is built below the floor which captures the location and position relation between the elderly people and surrounding objects based on the zones and other detail information namely status of the home appliances, actions of the user, etc. Then the abnormal activities in the detected situations are recognized by Support Vector Data Description (SVDD) and ultimately, the system sends the corresponding information to care-takers such as remote family and staff in care center.

Fig. 7

Architecture of the proposed system [7]

Following are the abnormal activities need to be detected based on some features as discussed below:

• 1. Forgetting to take medicine or taking medicines in an incorrect manner can be detected by tracing the duration between taking medicines

• 2. Going to restroom many times than normal times can be detected by starting time

• 3. Sleeping for a long time is detected by number of times per day

• 4. Taking unbalanced and insufficient foods by types of food

• 5. Inappropriate body exercise by number of times per week

The proposed technique has used a hardware environment called u-tile sensor network to detect the abnormal activities and SVDD for recognizing them. The basic idea of u-tiles sensor network is to detect precise location and position relation between the elderly people and surrounding objects by embedding various sensors, including pressure sensors and RFID antennas under the floor. This network consists of u-tiles under which there are pressure sensors and RFID antennas which are connected with a reader through a PIC based switch. The IDs of users and objects are read by each piece of u-tiles, and the position relation between users and objects are captured. To realize a smart home environment, the u-tiles sensor network need to be set up in the zones where elderly people often have several activities around the bed, in front of sofa, restroom etc. Apart from the u-tiles sensor network, a smart plug has been developed by NTT Company to detect and control the status of home appliances, i.e. turning on or off of a lamp, fan etc.

In the proposed technique four aspects about elders are needed to represent a situation, i.e. zone, status of related home appliances, actions of the elder people and environment. A situation s can be represented by the four tuple, $s = <z, D, A, E>$, where z is used to represent the current zone of the elderly people, D is a set including the status of related devices, A is a set including current actions of the elderly people, and E is a set including environmental factors.

A situation can be detected by framing some detection rules. An example of such a detection rule is “if the person is in the bedroom and if the bedroom light is off and if there is no action” then the situation detected is “sleeping in the bedroom”. The next task after detecting situations is to recognize the abnormal activity in these situations. The proposed technique uses SVDD to recognize abnormal activity which constructs a boundary for a dataset to detect target data or outlier. On analyzing the performance of the system by experimenting with 5 subjects the system has totally detected 136 activities out of 146 and achieved an accuracy of 93.2%.

The aforesaid details show the importance of elderly care and prove that the proposed situation-aware system detects abnormal activities of the elderly people on par with the other systems.

2.4 IoT Support for Children Affected by ASD

Autism Spectrum Disorder (ASD) is a serious neurodevelopmental disorder that impairs a child’s ability to communicate and interact with others. It also includes restricted repetitive behaviors, interests and activities. Since currently there is no cure for ASD, Ardiana Sula et al. have proposed a novel framework [8] based on Internet of Things (IoT) and P2P technology for supporting children with ASD to help them to interact, respond, and tell parents what they need for their survival.

The IoT is motivated by the victory of RFID technology, which is now widely used for tracking objects, people, and animals. RFID is primarily used to identify objects from a distance of a few meters and to determine the approximate location of objects provided the position of the reader is known. RFID systems consist of three main components namely RFID tags, the RFID readers, and RFID software. The RFID tag is an electrical device that receives a specific signal and automatically transmits a specific reply. The tag also carries an unique ID data and stores information contents depending on the size of its memory. The RFID reader is a hardware device that is used to read the transmitted data from the tag. The RFID software is a middleware that runs on the RFID reader.

The assistive technologies for the diseased children can be realized through the IoT where the children with autism live in their homes with smart objects, communicating to the outside world in an intelligent manner. The proposed framework is shown in Fig. 8. The system sends information about the children state in real time to therapists using P2P technology and also allows children to interact with other children and parents. The architecture of P2P distributed platform is developed using JXTA technology [9].

Fig. 8

IoT based framework for supporting children with ASD [8]

The Smartbox device used in the proposed system is integrated as a useful tool for monitoring and controlling children activities. The Smartbox has different sensors such as body sensor, chair or bed vibrator control, light control, smell control and sound control, and remote control socket. They are used to detect body movement, vibrating chair or bed, adjusting room light, controlling room smell, and emitting relax sounds respectively. Such functions relax the children affected by ASD and divert them to concentrate on some tasks.

Assistive technologies, apart from improving the health of the children with autism spectrum disorders, help them in assistive learning. Tagging physical objects to find and analyze data about the object is one way the IoT can be used in education. Figure 9 shows a part of the proposed framework for assistive learning. A child can learn new words through touching the physical objects that are in their vocabulary list. Each physical object would have a RFID tag placed on the item. When this tag is read by a RFID reader or scanned by an application running on a computer or mobile device it would prompt the device to open up a page of information or send a command for an action to happen. The RFID tags can be created and attached by the parents for each of the physical items in the vocabulary list. When the child places the RFID card on the RFID reader, it pronounces the word for the item in their native language. Touching the item will give the child another sense to be engaged and may help them learn new words faster.

Fig. 9

Assistive learning [8]

According to the above picture about the assistive technologies in the IoT, it is understood that even children affected by ASD can learn and survive on par with the normal children.

2.5 Tracking of Night Drivers for Accident Prevention

Prevention of road accidents during night time is an essential service for night drivers. Most of the road accidents happen due to the drowsiness state of the drivers. Aishwarya S.R et al. have proposed a novel system called Eye Blink Monitoring System (EBM) [10] to alert the drivers when they are under the condition of drowsiness. An embedded system based on psychological state of the drivers is developed to monitor eye and head movements which are useful in warning drivers during their sleepy phase of drowsiness. The sleep state of the drivers is determined by capturing the eye-blink rate using an IR sensor and head movement using an accelerometer. An IOT enabled sensor is used to transmit the entire data collected by sensors over a smart grid network for a quick response to take recovery actions under emergency conditions.

The existing technologies to prevent accidents are not cost effective and not easy to be implemented. Also they are inferior to the proposed technique due to many factors for the reason being that they are not well defined, load elimination is impossible and they respond only after the occurrence of accidents. The main objectives of the proposed framework are as follows:

• 1. Creation of an eye blink & head movement monitoring sensor system for Drowsiness detection.

• 2. If drowsiness is detected,

  • Alerting the driver by means of a buzzer

  • Reducing the speed and stabilizing the vehicle.

• 3. Mediating the sensor information and tracing accident location using GPRS for help and rescue.

• 4. Displaying the activities of designed system on LCD display.

The block diagram of the proposed framework is shown in Fig. 10. The designed embedded system is interfaced with another mobile phone having an android platform through an IOT application. Such mobile phone offers notification to the host about the status of the embedded system in case of drowsiness and accident occurrence via alarms, text messages and voice notifications.

Fig. 10

Block diagram of the proposed tracking system [10]

The proposed work involves measurement of the eye blink using an IR sensor and head movement using an accelerometer. The IR transmitter is used to transmit the infrared rays in our eyes. The IR receiver is used to receive the reflected infrared rays of the eyes. If the eyes are closed then the output of IR receiver is high, otherwise the IR receiver output is low. To know whether the eye is in closing or opening position, the output is provided to a logic circuit for alarm indication and the status will be displayed on LCD. The accelerometer is placed on the driver’s fore-head which measures tilting angle of the drivers either towards forward or backward direction and towards left or right direction from the driver’s knee. If the tilting angle exceeds a certain threshold range then the output is given to logic circuit to indicate the alarm and the status is displayed on LCD. Figs. 11 and 12 show the sensors and message display on LCD.

Fig. 11

IR sensor and accelerometer [10]

Fig. 12

LCD response on drowsiness detection [10]

Apart from helping the night drivers, the proposed technique prevents the drunkards from drowsing and rash drivers from over speed in driving. The main pros of the proposed work are as below:

• 1. Effective and easy implementation.

• 2. User friendly interface.

• 3. Prevention of over speeding of vehicles.

• 4. Prevention of accidents before their occurrence.

• 5. Load reduction due to the usage of cloud computing in implementation.

The countable demerits are requirement of active internet connection and standard bodies to enable IoT. From the above narration, it is clear that the IoT can revolutionize the way the embedded systems interact and respond for variety of applications especially in case of night drivers by monitoring the state of their drowsiness for a quick, safe and effective response for a safer road travel.

2.6 IoT Based Healthcare System for Wheelchair Users

Healthcare monitoring of residents such as disabled or elder persons has become a focus of recent researches and developments. Lin Yang et al. have proposed a home mobile healthcare system [11] for wheelchair users, based on IoT technologies. The primary ambition of the proposed work is to build an intelligent system with real-time monitor and interaction to take care of wheelchair users at home. The shortcomings of other related technologies have been overcome by the efficient design of the architecture of the proposed work as shown in Fig. 13.

Fig. 13

Architecture of the healthcare system [11]

The architecture of the proposed work constitutes three components namely Wireless Body Area Networks (WBANs) with smart objects, smart phone and data center layer. WBANs is a people-centric network for wheelchair users, containing nodes of vital physiological parameters and living environment to detect wheelchair falling and controlling. The purpose of detecting the falling of wheelchair is to indirectly perceive the falling down of the user sitting on it. Heart rate and ECG are the primary physiological parameters for the wheelchair users’ healthcare. WBANS contain sink node, heart rate sensor node and ECG sensor node to measure the heart rate of the users.

In addition to these sensors, a couple of sensors called pressure cushion and accelerator sensor are used to detect a human falling from the wheelchair and detect wheelchair falling down respectively. The aim of perceiving the surroundings is to monitor the emergency of wheelchair users. Environment parameters such as temperature, humidity and carbon monoxide are computed when the wheelchair shifts from one place to another. The smart phone is the core device of the people-centric networks, and is not only the gateway for transforming data and instruction between the nodes and internet, but also the interface between human and physical world. Wheelchairs, families or clinicians can remotely visit the data or control the nodes of WBSNs.

The data center platform connects different smart objects, builds the interoperation of the virtual nodes for smart objects and provide sharing of data to different applications via various interface. In concluding the above discussion it is obvious that the proposed method excels the other related methods in the sense that the former raises its portability and flexibility by providing a remote monitoring system.

2.7 Smart Home for Elderly and Disabled Persons

Home automation encompasses household activities such as centralized control of lighting, heating, ventilation and air conditioning appliances, security locks of gates and doors which would provide improved quality of life for the elderly and disabled persons who might otherwise have to oblige their caretakers. Commercial home automation systems are categorized as locally controlled systems and remotely controlled systems. The locally controlled systems use their automation system from within their home via a stationary or wireless interface. The remotely controlled systems use The IoT to allow the users to control their system from their mobile device, personal computer, or telephone.

Vishwajeet H. Bhide [12] has designed a fully smarthome environment monitoring various sensors. The key issue in designing a home automation system is to offer a cost-effective user-friendly interface on the host side, with the intention that the devices can be easily monitored and organized. For this reason, cloud networking and data infrastructure are used to monitor, manage, and control their personal data points through the internet. The framework for the proposed work is shown in Fig. 14. The components include sensors, a home PC, a cloud server etc.

Fig. 14

Home automation [12]

Different sensors for light, temperature, humidity etc. are used to gather the data to understand the environmental conditions and also to detect any fault in devices. The home PC is used to monitor sensor values, control the devices accordingly and send information about faulty devices to the cloud server. The cloud server on noticing the faults in the devices will send details to the owner. The aforesaid discussion shows that this novel technique would assist the elderly and disabled persons in monitoring and controlling the household activities without depending on their caretakers.

3 Conclusion and Future Research

This chapter presents a detailed survey on the applications of IoT. The key objective of the IoT is the “things” having different types of communication methods and belonging to different kinds of networks should be connected so as to share the required information.

In the future, first and foremost, the researchers have to focus on integrated IoT systems [13]. Moreover, if the number of the “things” in the network is significantly increased, proper protocols for the communication is required. Owing to the introduction of the IPv6 protocol, the IoT communications have been improved due to the large address space in it. Thus, the IPv6 over Low power Wireless Personal Area Networks (6LoWPAN) standard is likely to support the IoT communication in future. In view of the fact that different “things” are connected to the IoT network, security requirements are also varied according to network conditions. Since, the existing framework of security is not sufficient to keep the IoT secured, a safe and secured network is crucial for the IoT systems.

With the IoT being a product of information and communication technology, there will be demand for professionals with IT skills such as data scientists, user interface experts and digital-mechanical engineers. Also there will be a marked increment in the technology awareness of workers. Thus, instead of being afraid of the IoT, future workers should prepare themselves to work alongside it. The IoT apart from having robust cloud computing facilities, the smart Things in IoT should be easily deployed in the form of plug-n-play in any IoT framework. In the IoT, the data comes in smaller pieces and all the data is not needed. Hence the Big Data is required to collect only relevant data to filter irrelevant data and to handle massive amounts of unstructured data.