Keywords

5.1 Introduction

Invigorating researches are taking place in many fields such as healthcare, networking, information technology, and so on. Amongst all applications, healthcare is the unit where the elitist researchers are concentrating on. Smart healthcare applications instigate the growth in medical sciences. Though adequate medical facilities are there, most of the facilities are not reached to the needy on time. Here the technology comes and paves a way to resolve it. Emerging technologies are nowadays make circumstances topsy-turvy with the help of numerous smart applications. The backbone for all these things is the so-called “Sensors”.

A sensor can be of any type, a gadget, a piece of equipment, or simply a device that senses any type of events related to the application for the sensor specifically designed for. Sensor is the backbone of every existing and upcoming technology. Based on sensor information, applications are being designed for any purpose. There are wide-ranging kinds of sensors used everywhere which include temperature and humidity sensor to the flagship of sensor types. The applications of sensors are vast in various fields from the ground to galaxies. It includes Agriculture, Human comfort, Food processing, Building, Medical, Robotics, Remote sensing, and so on.

In today’s world, the Emerging technologies include the internet of Things (IoT), Artificial intelligence, Machine Learning, Deep Learning, Augmented Reality, Virtual Reality, and so on. These technologies made human life as ease. Among all those applications of emerging technology, Healthcare domain is being the top most important and needy field of public. Hence, so many researchers are working on the Healthcare domain by integrating all the existing and emerging technologies. Internet of Medical Things (IoMT), one among those technologies, helps in a great manner with the help of medical sensors. IoMT defines the Medical sensors as the sensors used to monitor, diagnose, and treat illness in the medical domain. Another term called Smart Sensors, which can be defined as a sensor package that employs “on-chip signal processing”. Therefore, to make the virtual world to be a real one, sensor technology plays a vital role to invent newer applications with the help of emerging technologies in all walks of life.

The short remainder of this chapter falls here; Sect. 5.2 is about the sensors, classification, medical applications. Sensors in great technologies are described in the Sect. 5.3. IoT and sensors, AR, VR, and MR and their corresponding role in health care are given in the Sections 5.4, 5.5, 5.6, and 5.7, respectively. Section 5.8 lists out the existing medical application of emerging technology. A brief about data handling and the summary with future work are explained in Sections 5.8 and 5.9.

5.2 Sensors and Sensor Networks

In general, the term “transducer ” is defined as “a device which converts one form of energy into another form” means it provides as output in a readable format of human, with respect to a particular object being measured or a specific quantity that meant to be measured. The definition of the transducer is given here just because it is being a near-synonym of the so-called “Sensor”. A sensor can be of any type, a gadget, a piece of equipment, or simply a device or a module defined for a specific purpose or a function that senses the changes or events in an environment and conveys the detected information to further electronic devices connected to the sensor.

Today’s digital era moves towards the emerging technologies which include the internet of Things (IoT), Artificial intelligence, Machine Learning, Deep Learning, Augmented Reality, Virtual Reality, and so on. These technologies made human life as easy as possible. There is a well-known saying, “Smart work is better than hard work” which means being smart in work saves time and energy a lot. Because of these technologies exist, the meaning for “smart” becomes more Smarter with the help of the “sensor”. Sensor is the backbone of every existing and upcoming technology. Based on sensor information, applications are being designed for any purpose. The new epoch in sensor technology has steered in since there is a demand for new sensors, allowing new approaches for transmitting physical events into an output that would be processed by a processor readily. For specific functions, the sensors can be customized with integrated circuits (IC).

Sensors are commonly used in our day-to-day things that start from touch sensor in smart mobile to MARG sensors in smart automobiles. Also there are so many applications that are being existed still in our lives, but most of them are unaware to common people. The uses and sensor-based applications are expanded to an unimaginable extinct. With advancement in manufacturing technology, sensors are beyond the conventional fields of pressure, humidity, temperature or distance measurements, for instance into gyroscope, magnitude, and altitude sensors, also chemical properties measurement of a material. Besides, conventional sensors like potentiometer, distance measuring sensors are still being used widely in so many fields.

The rate of sensitivity of a sensor can be identified by calculating the changes in output value of a sensor when input value of quantity is being changed. Consider a level sensor is placed in a water tank. If the water level in the tank increased by 1 cm when the intake is increased by 1 L, here the sensitivity of the level sensor is 1 cm/L. Basically, most of the sensors work on linear transfer function model. The sensitivity is simply stated in the form of the slope of the linear characteristics. In case of analog type of measurand, the sensor’s analog output or electrical output to be converted to the unit of measurand requires a mathematical operation i.e., dividing the electrical output of the sensor by value of slope. For digital type, an ADC (analog-to-digital converter) is required.

Classification

There are a wide-ranging kinds of sensors used everywhere. Certain sensors are intended to measure very simple measurands like temperature and humidity. In other hand, cutting-edge sensors are envisioned to sense accurate information in applications like remote sensing, satellite information, and even to detect the global position of a robot.

Quite a lot of classifications are made for sensor, based on the form of output signal, type of measurand, and means of application and in many ways. First type of classification is done based on the signal as Analog sensor and Digital Sensor. Analog sensor gives output as continuous signal in response to the quantity to be measured whereas Digital sensor deals with the digital data or discrete input devices. Probably analog-to-digital converter would have been used with digital types of sensors.

Next, broad classification is based on the form of signal and the respective measurand. It includes the Thermal sensors for measurands like temperature, heat, and entropy. Similarly, Radiation sensors for gamma rays, ultraviolet, radio waves, etc., and Mechanical sensors for displacement, acceleration, force, flow, and so on. And it extends as Magnetic sensor, Chemical Sensor, Biological sensors for measurands like magnetic permeability, chemical properties, and bio-medical metrics as sugar, antigens in that order.

Another classification is based on the application. In Robotics, sensors are of proprioceptive and exteroceptive sensors. Sensors used to measure the speed of motor, robot joints, voltage of battery used, are called as proprioceptive sensors whereas exteroceptive sensors are used to measure distance, amplitude, and intensity of light, etc. In Remote Sensing, the sensors are grouped into active sensors and passive sensors. Sensors that create its own drive, no need of external power supply (interacts with surface/atmosphere) are known as active sensors. In other hand, passive sensors work with naturally existing energy (solar energy or microwave radiations).

Whatever category the sensor lies on, it should obey the following:

  • Precise to the measured stuffs.

  • Unaffected by any other metrics probably come across in an application.

  • Does not sway the measured things.

If the above-mentioned rules are followed by a sensor, then it is said to be a first-class sensor. And the characteristics of sensors are generally grouped into static and dynamic. Static characteristic includes accuracy, threshold, hysteresis, and nonlinearity while dynamic characteristics comprise instability, drift, noise, repeatability, and step response.

Smart Sensor

Since IoT brings out everything as “smart and intelligent”, the massive advancement in sensor technology focuses on smart sensor development. The simple principle of smart sensors is that the “sensor complications must be covered inside and must be translucent to the system”. In the way of presenting as simple aspects to a network by means of a digital interface, the smart sensors are devised. In simple terms, a smart sensor can be defined as a sensor package that employs “on-chip signal processing”. In addition, a smart sensor primarily encompasses any type of sensor module, interfacing unit, signal processing circuit, and a power source. Along with these primary portions, sensing element, analog filter, data conversion , digital information and communication processing, and amplification systems are included as subsystems. Nevertheless, the concept “smart-sensing” makes fresh chances for using new innovative materials and compounds for sensor manufacturing. It avoids the constraints of applications on various fields. Possible advantages of smart sensor includes lower down time, lesser maintenance, increased reliability, less weight, minimum cost, fault tolerance, a lesser amount of complexity, and adaptability, etc. At present, the usage of smart sensors appears to be limited to specific applications.

The applications of sensors are vast in various fields from the ground to galaxies. It includes Agriculture, Human comfort, Food processing, Building, Medical, Robotics, Remote sensing, and so on. Different sensors are used in different applications based on certain criteria like environmental condition, range of sensor, calibration, accuracy, and cost of sensor. And based on sensor property too, can be applied in some matters. The focus of this chapter is all about the sensor with emerging technologies in the medical field.

Sensors in Medical

Medical sensors are defined as the sensors used to monitor, diagnose, and treat illness in the medical domain. Based on the risk profiles, the medical devices are classified into different classes such as Class-I for lower potential risk, Class-IV for highly potential risk, and it goes on.

The sensors used in medicine have some salient features. Sensors should act in accordance with the safety standard “IEC 60601-1” and statutory conditions including additional enforcement and criteria for quality management, usability, functional safety, and risk management in order to make sure that the prescribed device works properly in response with the given inputs. A good sensor should provide the accurate measurement with high precision, highly stable, and with quick response time. In order to connect with processor and controllers, a sensor must deliver measurements in the form of digital output.

Figure 5.1 represents the different types of sensors used in health care and its functions are briefly given in the Table 5.1.

Fig. 5.1
figure 1

Sensors in Healthcare

Full size image
Table 5.1 Types of Medical Sensors and Functions
Full size table

Advanced Sensor technologies offer leading cutting-edge medical sensors specifically for patient care and critical care applications that provide accurate and reliable sensing information from the medical devices to respective destination to monitor critical patients with chronic diseases.

In patient monitoring and tracking applications, the medical sensors are applied in medical instrumentation, clinical thermometry, blood pressure, drug delivery, respiratory, sleep apnea, and fall detection. Whereas in a critical care environment, high-performance medical sensors meant for their reliability, accuracy, affordable, and compact size are used. Under critical care, the medical sensors are implemented primarily in esophageal stethoscopes, normal thermometers, skin sensors, heart catheters, respiration monitors, blood analyzers, hypodermic needle sensors, and incubators. Also used in drug delivery, surgical temperature assemblies, fluid management and medical instrumentation process.

Wearable Sensors

Wearable devices are generally in direct contact with user’s skin membrane or implanted in the user’s body, therefore the size and physical compatibility of a sensor with human tissues have to be analyzed carefully. This necessity provokes the amalgamation of novel provisions and technologies. Incorporating sensing elements with weaves is a method that a lot many researchers are agreeing and take on to the next level. Few design principles are to be considered before going for wearable devices [1]. They are Comfortable and noninvasive, long-lasting in terms of power, highly accurate, secure, user friendly, customizable, and interoperable that leads to successful user acceptance for wearable devices.

Though sensor by itself as a super good technology grows massively today, it results in very great heights of applications when collaborated with one another great emerging technologies. For instance, finest technologies of today like Internet of Things (IoT), AR/VR, Mixed Reality, Radar, 5G, and goes on. Some technologies where sensors are used copiously are articulated here and moreover how it is helpful in the field of medicine and to what extent it could be benefitted too explained.

5.3 Sensors in Internet of Medical Things (IoMT)

The Internet of Things (IoT), a pulsating word everywhere today. As a technology, it touches an unimaginable zenith by its usages and demand in applications for each and every field than any other technology does [2]. IoT can be related with human for better understanding. Humans having skin, ears, eyes, and nostril as sensing elements for touch and feel habituates. Without these sensing elements, humans could do nothing in day-to-day life. Similarly, deprive of sensors, IoT is nothing. As the brain and nervous system in humans, controllers and actuators are playing their role in IoT. Additional to all above-mentioned hardware, middleware plays an important role especially in communication between devices and applications. It is a type of software which acts between an OS (operating System) and the applications running on the network. Basically functioning as hidden translation layer, the middleware permits communication and data management, data handling for disseminated applications. Middlewares, usually used in conventional distributed systems, are essential tools for the design and implementation of smart objects and smart environment applications. And it provides specific and general abstractions. For example, sensor and actuator interface, knowledge management, interobject communication, etc [3]. This technology is not only about connecting all devices which are capable of getting connected with the help of the internet but also it is about refining the way in all walks of life and also a great impact on the environs. To say stuff simply, with this interconnected devices people can set up their life in the way much more safer, extra productive, super smarter, and well-versed than ever before.

Figure 5.2 represents few sensors that are used frequently in IoT applications and the brief definitions of those sensors are given below:

Fig. 5.2
figure 2

Major IoT Sensors

Full size image

5.3.1 Gas/Smoke Sensor

One of the commonly used sensors in most of the environment is Gas Sensor . It used to detect gas in an environment, which is more supportive in safety systems. Specifically, this can be implemented in an area to detect leakage of gases and prevents explosions. There are so many readymade sensors available to detect harmful, flammable, and poisonous gases. For instance, MQ7 detects carbon monoxide, most dangerous gas. There are quite a lot of gas sensors are there named as MQ2, MQ3, MQ135, etc.

5.3.2 Temperature and Humidity Sensor

This sensor plays a foremost role almost everywhere. Agriculture, manufacturing industries, and atmosphere are few areas of its application. For example, to measure the machineries temperature in Industries, to detect the value of moisture of soil in agriculture field, temperature and humidity sensors are used. An affordable, basic, ease to use sensor is DHT11, DHT22 available easily in market. No need of analog pins for this category. It is a type of capacitive and digital sensor.

5.3.3 Infrared Sensor

Infrared (IR) sensor works based on the simple principle of emitting and receiving radiation. Here the radiation is infrared. It has been deployed in many more applications. Though it has the capability to detect heat and motion of an object, commonly it is used to detect obstacles. IR detectors are being used for applications as thermal imaging and night vision. IR sensors are also available in market.

5.3.4 Ultrasonic Sensor

Ultrasonic Sensor is a kind of reliable sensor. Similar to infrared sensor, ultrasonic sensor also works based on the principle of transmitting and receiving signal (e.g., SONAR). It transmits sound signal and receives an echo signal of the same. Ultrasonic sensor has the ability to compute the distance covered by the signal from transmitting end to receiving end. Hence, it is used to detect any object or to measure the distance between objects by hitting the signal on an object and receives it back.

5.3.5 Proximity Sensor

Without any physical contact, the sensor has the capability to spot the presence of human or objects nearby. Here, the sensor uses an electromagnetic beam or simply a field of electromagnetic signal and waits for the returning radiation and then it calculates the difference in transmitted and received signal strength. Fall detection and people counting are few common examples. Alarm technique in reversing, a car holds a proximity sensor which detects the obstacles and objects in the pathway. Available in smartphone too, detects human face during a video call. Example for proximity sensor is Si114x, Si1102, etc.

One of the applications of IoT is IoMT. In medical industry, the technology makes things easy, convenient, and secure in a healthcare environment for both professionals and patients. Smart healthcare IoT devices provided with sensors used to monitor and track the patient’s condition. For instance, BP, Heart rate can be observed easily today by a smart band that is available with most of the people.

Primary part of IoMT is to sense a patient’s health conditions. To make the sensed information useful, the data have to be communicated over internet with people and make it accessible by computers. At the other end, the healthcare people analyses the patient’s data and gives medical advice to the corresponding person. This happens only because of the technology so called IoMT.

Nowadays, IoMT-based device manufactures are exponentially increasing and they are using various communication protocols as base for transmitting sensing data between doctor and patient over internet. Since each IoMT equipment has its own unique IP address, the data from applications like in-home glucose monitor, room temperature monitoring, fall detection devices in a room are transmitted first to the neighbor network may be a home wifi connection or hospital IT network. Finally, the data from IoMT devices are available in a database created for each patient by the hospital network, which can be accessed from anywhere by the authorized persons. This phenomenon leads to the development of smart hospitals.

Smart hospital is all about remote-tracking hospital asset, monitoring health condition of in and out patients, and smart medical equipment. Base for this smart hospital is Internet of Things and undoubtedly the soul for IoT is Sensors.

5.4 Sensors in Augmented Reality (AR)

Augmented reality, generally shortened as “AR,” is an emerging technology which combines real and virtual worlds. AR is a kind of display environment based on real-time existence and interactive in terms of computer effects includes sound, images, scenes, and text to enrich the user’s experience in visualizing the real-virtual worlds. AR enhances the objects in real world by computer-generated modalities and somatosensory information includes touching, sighting, hearing, and even sense of smell. Combination two different worlds, interactive, 3D visualization with accurate information are some features that should be satisfied by the system called Augmented Reality. This type of superimposed sensory modalities and information could be either constructive or destructive means that information may be an additive to the reality or masking of the real-time facts and figures.

The primary intention of AR is to provide a vivid audiovisual treat for users. It works by taking up computer-generated techniques and simulation models such as voice and image recognition, animated characteristics, head-mounted wearable, or handheld standby devices which add virtual presentation overlap with real- world surroundings. AR devices come in various forms as mentioned above as handheld, standby, glasses, or wearable entities. In mere future, advancements in AR technologies lead the user more interactive with the real-time environment and surroundings by integrating smartphone use cases with AR cameras and augmenting computer vision for object recognition like applications. Though many more applications are there, most common and well-known application of AR is video games (e.g., Pokeman go). People are going to stun in forthcoming days with greater advancements in AR applications.

The competence of AR system is getting speeding up day by day because of the immense advancement in IC (Integrated Circuit) manufacturing. Earlier, the technology was relentlessly limited for reasons like weight, size, cost, and power consumption of sensors by which lightweight headsets cannot be constructed. Thanks for vlsi (very large scale integration) and IC manufacturing techniques for optimizing the sensor size and improving the power factor of the same. Nowadays, lightweight headsets are available for AR applications. Since AR is based on computer-generated images as holographs, the system should place the predicted holographs in front of the user. This can be done with the help of computer which detects the motion and location of the user’s (headset wearer) head and it can be get completed by a unit of the system known as IMU (Inertial Measurement Unit).

Sensors play role in the Inertial Measurement Unit. IMU comprises of three types of sensors. They are Accelerometer, Magnetometer, and Gyroscope given in the Fig. 5.3. The sensed information from these sensors is denoted as 3-dimensional signals of axis x, y, z. Therefore, these three signals return an accurate position of the user’s head and help to find the head movement in a precise manner in order to place the holograph. Micro Electro-Mechanical Systems (MEMS) fabrication technology based on IC photolithography mechanism has been employed by most of the sensors. Let us see about these three sensors in brief.

Fig. 5.3
figure 3

Vital Sensors of Augmented Reality

Full size image

Accelerometer has numerous of functions and applications, but predominantly it is used to determine the direction and orientation where the device is facing. This tiny and sensible devise precisely gives the device positioning details with the pull of gravity in any line among the axis x, y, z, since all the three axes are fixed. For example, on smart phones, the display mode can be varied automatically either landscape or portrait mode by the way the user holds the mobile. This is possible only with the help of accelerometer sensor. The sensor also has the ability to measure the “acceleration” along a specific axis. Any handheld device or wearable holds this sensor, the user can track the speed of any automobile at which the user is journeying. Even Google maps are having this feature. For instance, in an IoT application, vehicle accident detection system or vehicle collision prevention system uses this accelerometer in order to detect the speed of a vehicle and its direction. If collision happens, the controller proceeds with further procedure that fed into the system. Airbag ejecting technology in cars also based on the same technology.

The accelerometer sensors are infinitesimal in size. It is a kind of comb structure, and the generated electrical current from the combs (needle like structures) transformed into acceleration data as soon as the gravity disturbs the combs of the accelerometer. Yet few limitations are there for accelerometer. It may not have the ability to produce information to denote the device rotation, when the motion data gathered from the combined readings of all the three axes x, y, z instantaneously. Sometimes, it happens in mobile when playing high-end games. Here the need for gyroscope arises.

The gyroscope is a type of sensor or electronic entity used to determine the orientation of any device. It can be instigated by creating multifingered structure and interdigitized assemblies which, when pulsated at a desired frequencies, are able to sense the fictitious force deeds on rotating objects and calculate the rate of rotation. As said before, IMU has the ability to detect accurate position and orientation is just because of the presence of gyroscope in the IMU unit. A gyroscope is naturally present in smart mobiles too, but typically it is very important in the field of aircraft, helps to find out the flying level of a plane when it is covered by dense cloud and during worst weather conditions.

Just like accelerometer, magnetometer is also a tiny device used to measure magnetic fields. For all intents and purposes the magnetometer acts as a compass and this is essential in the field to detect the magnetic pole north and tells the direction where it is facing. The magnetometer detects the direction of magnetism when it is comes into contact with the magnetic field once the sensor gets powered. It can be worked in various functions, it may use as permanent magnets or as electromagnets. The function of magnetometer can be decided according to the application where it is deployed. Google Cardboard is a best example where this sensor used effectively and cleverly.

Besides measuring the orientation and direction of a device by these above-mentioned sensors, the users are being aimed cameras for more precise identification of object in the space direction and orientation at the instant. Former methods of augmented reality entail complex sensors to place the computer-generated models to look as if real in the panoramas. Two dimensional pictures were used as reference by the cameras and headsets to render graphics on the scenes. Nowadays, advances in AR technology changes the AR headsets certainly about one or more distinctive types of cameras. Time-of-Flight (ToF) Camera, Binocular Depth Sensing sensor, Structured-light sensor are some extraordinary cameras used in AR technology. The operation ToF based on tempered InfraRed (IR) radiation and to measure the phase delay of reflected radiation to determine the distance between the object and the camera. Marketable examples of this type of high-defined cameras are MLX75023 QVGA Time of Flight sensor and DS325/DS525 from Softkinetic.

Structured Light Sensor technology intends a well-defined array pattern of any type of light or radiation which includes visible as well as IR (IR or visible) onto the computer-generated model. The differences between the pattern emitted and returned are calculated by analyzing the pixel information of a camera. Microsoft’s Kinect cameras, Intel F200, Occipital’s Structure Sensor are few real time examples of structured-light sensor.

5.5 Sensors in Virtual Reality (VR)

Experiencing and exploring things via a specially designed system that do not exist in real are named as Virtual Reality commonly known as “VR”, is a technology similar to Augmented Reality in terms of simulation and projection of two different worlds. AR just combines real and virtual worlds whereas the VR gives the new world which is entirely different from the existence of reality. Here, instead of looking into the screen or the hologram model in front of them, the user actually gets inside the experience which means the user feels they are there on the 3D virtual entity by mentally as well as physically and also users can interact with the artificial world in real.

VR is in essence, computer-generated, believable, interactive, explorable, and particularly more immersive. For instance, in a 3D movie, people can see things which come near to them from the screen, they try to reach out towards it and touch it but the illusion will just fade away. This is an example for submissive environment. In terms of immersive environment, people be able to touch it, sense it, and even reach out; this is possible only in the technology so called virtual reality. It is a two-way communication channel; people can respond to it and get back what they expected. The VR simulation gets adjusted according to the user’s head rotation and perspective if they wore a VR headset. The application of VR in various fields includes education, entertainment (games), field of medicine (surgeons), aerospace, and so on. For instance, pilot training in aerospace and military training. In few cases, VR technology takes in augmented reality and mixed reality. A person can interact with the computer-generated virtual 3D environment with the help of extraordinary electronic devices which support VR, as like elite goggles with display, hand gloves with sensors. At present, to generate realistic pictures, acoustics, and other sensations such as haptic, hearing, and viewing are simulated with the help of typical virtual reality-based multiprojected displays or special headsets comprise of head-mounted device with screen in front of eyes. The computer-generated simulation locates the user’s eyes position and according to that the simulated environment reacts in real by changing its graphics automatically. By the way, VR creates such an interactive and convincing world for users.

Humans come under chromatic living beings; displaying techniques are easily understandable by humans than any other mode. Hence, display technology makes the huge difference between the standard and artificial user interfaces. VR employs some sensors to detect the movement of user’s body and its position. HMD (head-mounted display) is the most commonly used product in VR technology, has two screens for each eyes separately, stereo speakers around in a room, and special gloves were powered by a supercomputer or a workstation. HMD comprises of built-in position sensors, accelerometers to detect the orientation, exact movement of a person, and the accurate position of the person’s head, and adjust the VR pictures accordingly. The two screens are used to display stereoscopic images and create a 3D virtual environment in real. Perfect example is the Google’s cardboard goggles. It has built-in lenses used to convert a normal smart phone into a type of HMD that supports VR technology. Google gives this product for low-cost and makes affordable for everyone.

As afore said, HMD plays a vital role in VR technology which comprises of various sensors namely for head-motion monitoring and tracking sensors, eye tracking sensors, and gaming controllers. IMU (Inertial Measurement Unit) included in the VR headsets as a chunk of electronic entity is responsible for projecting the virtual environment on screen in VR immersive headsets. Related to the eye movement and direction and the head position of the user, the computer controls the generated image and stereo information. There is no need for position sensor if and only if the user is static in position. In gaming application, there is need for monitoring and tracking the position and movement of user. Such application needs additional light sources or markers that should be positioned at permanent locations around the gaming environment. Special light sensor or camera ICs are used to sense those light source or markers placed in the gaming space are commonly termed as outside-in tracking.

Most essential sensors for the specialized VR devices in particular for the head-mounted display are accelerometer, gyroscope, and magnetometer. These sensors allow user to measure the orientation, movement, direction, and position in the space. A broad explanation for the above-mentioned three sensors was discussed already under the topic augmented reality sensors. At earlier days, hardware and resources for constructing VR technology were very expensive and unmanageable. Due to massive growth in mobile devices and IC manufacturing, the sensors become very tiny and affordable, highly cheap in terms of cost.

The mentioned sensors are responsible for detecting motion and space. The micro electro mechanical systems (MEMS) technology paves the path for building devices like magnetometer and gyroscopes at a small scale. Deprive of this technology, so called smartphones and other electronic entities including AR/VR headsets would become very complex and chunky. Instead it gives smart and slim models by that today, each and every one hold a smartphone as a mini computer in a pocket which comprises of tiny and sophisticated sensors.

A VR device can hold any number of accelerometers, magnetometers and gyroscopes to produce precise and highly accurate information that can be passed to controllers and software to place a rich experience to the people who are incredibly immersive with the technology is all about. Apart from these three sensors, some sensors are used for VR technology. They are, structured light systems, eye tracking sensors, gaming controllers, and MARG sensors are precisely described in the Fig. 5.4.

Fig. 5.4
figure 4

Sensors used in Virtual Reality

Full size image

Eye tracking sensors are used to detect the gaze point, which means it makes a computer system or the VR headset display to determine the position of the user’s eyes and where they are looking at actually. Gaming controller, simply called as a controller used as an input device in several kind of video games. Primarily it is used to control a character or an object in the video game particularly in VR-based gaming applications. Oculus Rift S and HTC Vive are commercial examples that are available in market.

MARG (magnetic, angular rate, and gravity) sensor is one of the most dominant sensors in mobile devices and also in AR/VR technologies. Total of nine sensors including orthogonally mounted trios of micromachined rate sensors, magnetometers, and accelerometers are enclosed in every single MARG sensor. This sensor is primarily designed for measuring degree-of-freedom orientation of any number. But the ultimate objective is to attain the 6DoF (six degrees-of-freedom) which gives the freedom for motion within the 3D environment. Six degrees-of-freedom (6DoF) is traditionally defined as the combination of three rotations (pitch, roll, and yaw) and three transformations (up/down, left/right, and forward/backward). MARG sensor is used to measure the movement of person accurately such as user’s head and hand gestures in the virtual environment within the 3D space else the illusion may shatter.

Even researches are going to give orientation on 9DoF for better experience. However six degrees-of-freedom is basically essential, if not the 3D environment would not be pleasant. Therefore, modern headsets of VR should offer tracking and monitoring across all 6DoF at the maximum for rich experience.

Here comes the medical application of augmented reality (AR). It is the knack of overlaying computer-generated content over real objects of the world. AR incorporates real objects with the digital data and makes a 3D environment and that is affordable, easily reachable, and accessible for medical training and imaging, nurse training, and dentistry. Examples of AR medical solutions are, AccuVein, ARnatomy, VA-ST. Some of the applications of AR are Dentistry, Training nurses, Medical imaging, Medical education, Pediatric MRI evaluation, Helping the visually impaired, Visualization of peripheral vasculature, Remote surgical expertise, etc.

Apart from the given applications, AR mobile apps are developed for healthcare. It has been illustrated in the Table 5.2.

Table 5.2 AR mobile application and its description
Full size table

Virtual Reality along with the surgical training assists and uses in drug design, “telemedicine” is also possible. Telemedicine is defined as monitoring, tracking, examining, and even surgeries on patients from remote location. The real time example is daVinci surgical robot, developed in 2009 for doing surgeries by the instruction of surgeons from another remote location even from another continent. This can be possible with VR, consider a surgeon with virtual reality control panel at one location and a robot using knife in another location. After the advancements in VR technology and the successful implementation of the robot daVinci, several robots have been installed in various hospitals across the world. Collaboration of various specialists can be possible now via virtual reality. World’s top surgeons are now able to be connected and work together on a particular complex surgery. Till date, VR technology has been treated for various types of psychiatric disorders which include phantom limb pain, agoraphobia, and schizophrenia, and also in stroke patients rehabilitation and for those who are suffering from degenerative illnesses like multiple sclerosis.

5.6 Medical Applications of Emerging Technology

Before all the emerging advancement in healthcare, the foremost technology that has been employed and still in practice is “Telemedicine”. Since there are a few notable issues and challenges are incurred in tele-medicine like real-time connectivity, low standardization, QoS (quality of service), cost, and so on [4]. Therefore the field of medicine is in need of a system that takes over all these hassles in practicing methodologies and makes the medicine field outreach.

The corpus of asymmetrical cells inside the brain leads to brain tumor, which can harm the brain and that will be a life-threatening one for all. Thus, detecting brain tumors in initial grade is quite critical for its verdict, prognosis, and cure. To detect accurate brain tumors, smart and secure automated system is required. In [5], the authors implemented a computational method called “PART” identifies correctly the brain tumor with respect to its grade which is nothing but the level of infection. By using advanced feature set, the disease has been identified accurately and facilitated the healthcare supporters in treating the brain tumor.

An application has been proposed by the author Qi An and et al. [6] with the help of Gyro sensor and Accelerometer named as “PocketIMU2”. It is a wireless wearable sensor precisely designed for elderly people to detect their motions. It is essential to measure the body state of the patients in particular the elder people who are in need of rehabilitation by any means. Hence the “PocketIMU2” device is helpful for monitoring the human motions.

A very few existing AR and VR applications in medicine are given here in brief. Let us see one after another. In a work “Augmented reality-based upper limb rehabilitation system”, the author Wang Ying et al. introduced three different augmented reality games. By playing that, the patient can control the virtual objects in the real-virtual world by any fixed markers in the piece of equipment used. The performance of the patient has been recorded. The therapist from remote location be able to monitor the patient’s movement by looking over the training video; hence, the therapist easily gets the data of a patient and guides the patient in a professional manner. The three games are, tea pouring game, simulated driving, and brick breaker [7].

In the tea pouring game, in the virtual environment, the patient be able to see a virtual tea cup and a kettle. As per AR, the patient be able to interact with the virtual objects. Here, the patient views a cup and a kettle. The patient has to hold the virtual kettle by the affected hand. Next, the patient should move the kettle towards the virtual cup and hold the kettle for few moments and then tilt the kettle just like pouring water. The system detects the designated position of the kettle and cup based on its coordinates. As the same process repeats for a while, the ability of the motor in the upper limbs and flexibility of the joints are improved. Similarly, in simulated driving game, patient has to drive virtually whereas in the brick breaker game, the patient moves the virtual reflector to bounce back the virtual balls. So these are some exercises in terms of AR games for rehabilitating the upper limbs.

“Mobile RehApp – AR-based mobile game for ankle sprain rehabilitation” authored by Jaime Andres Garcia et al., proposed a technique that uses Mobile Augmented Reality (MAR) to provide a kind of motion training exercises to patients suffering from an ankle sprain. The game-based system constructs based on AR aspects to permit endless real-time tracking and monitoring of patients movement. The games used in this work are easy to deploy in all type of smart phones. This enhances the interactions between the game controllers and patients. It is based on home-therapy which increases the level of motivation and observance to rehabilitation [8].

A research work related to smart healthcare proposed by Yin Zhang et al. gives an idea about a hybrid ECG feature extraction method that combines standard- and nonstandard-based features to pull out more wide-ranging ECG features and thus increase the endorsement stability. Pattern recognition framework has been used in this application to increase the efficiency in the place of multiple ECG usage [9].

“YouMove” is a novel training system for full-body movement based on AR technology. An AR mirror has been implemented in a real world. The mirror records the full-body movements of a patient and sends to the respective trainer or therapist. Trainer can teach movements via the same mirror to the trainee and the trainer can increase the level of difficulty gradually. This application is based on the video-based learning. “YouMove” approach was proposed by the author Fraser Anderson and et al. to improve the patients’ health by doing physical exercise includes yoga, dance, and games with the help of professional trainer from the remote location [10].

“ARTESH – Augmented Reality Tele-Rehabilitation System with Haptics” system facilitated 2D force feedback, enabling synchronous isolated patient’s physical assessment with video, audio, and haptic information conveyed alive with the help of internet. The system takes up a few engineering features like a force augmentation algorithm to develop force rendering and choking the existing limitation in it and improved force reparation patient’s recuperation [11].

Many more applications are there in medical field with the help of advanced technologies. Spinal Surgery is one of the risky operations. Though advancements are there in medicine, there is a trade-off between the cost of the surgery and in reaching all sorts of people. A study on these stuffs is given in an article [12].

In an another work [13], “Augmented Reality with Application in Physical Rehabilitation” done by the author Yu Jin et al., is an IoS application proposed for the patients who are suffering from strokes. Gait training is one of the insignificant parts of the physical rehabilitation. Augmented Reality (AR) provides a solution for the patients suffered by strokes by giving a proper training and assessment from the specialized trainer or physiotherapists from the remote location. During gait training, the velocity and position of the patient have been recorded by the smart carpet which comprises of capacitive sensors powered by a computation unit with WiFi-based communication strategies. Virtual objects can be added over the head-mounted 3D display to the real world, and the smart sensors in the carpet provide the information about the patients’ movement and positioning characteristics.

Limited researches are done on Rehabilitation for hands. One amongst is a system based on AR/VR technology framed by Jia Liu et al. [14] The author designed the framework with four rehabilitation programs that includes trainings for trajectory, shelf, batting, and spile. These trainings and the framework allow patients to interact in real time with virtual environment.

In an article [15], the author Niccolò Butti et al., proposed a system specifically for the patients with a disorder called congenital cerebellar. Rehabilitation for brain functions is essential for these sorts of patients. By the way, the proposed VR-based rehabilitation method has the ability to process on social stimuli of a human and implementing a designed training program over social cognition to improve their ability in predictive social circumstances. This phenomenon is named as “VR-Spirit”.

5.7 Sensors in Mixed Reality (MR)

Mixed reality “MR” is a technology that incorporates augmented reality as well as virtual reality with the help of immersive technology. To put it in simple words, MR is the combination of AR and VR. It merges the real and virtual worlds to place a computer-generated simulation in front of the user as like in AR mechanism and by using the immersive technology, MR resembles as VR technology. Therefore, MR produces a fresh environment and 3D visualization where the real (physical) and computer-generated (digital) objects are interacted with each other in real-time. MR will not take place either in physical world or virtual world, instead it is a hybrid of AR and VR which means objects in two different worlds coexist in the environment produced by the mixed reality. During 1992, the first MR project was developed at Armstrong Laboratories of US named as virtual Fixtures platform. It demonstrated that by overlapping virtual objects on top of a real environment of a user’s direct gaze point, human performance can be improved significantly. This was the first project used the immersive technology.

Mixed reality affords the capability of keeping one foot in real world and the other in the virtual world. Without removing the headset, people can immerse themselves in the artificial environment created by mixed reality which allows people to interact with the virtual world using their hands. By means of next-generation imaging and sensing technologies, the user can manipulate real world as well as the virtual world objects and the environment proposed by MR. It brings two different worlds together and offers a rich experience that could change the way of user’s daily activities.

Mixed reality is sometimes called as “enhanced AR” as it blends both augmented reality and virtual reality, but more similar to AR with higher physical interaction.

Mixed reality is the combination of augmented and virtual reality. Hence, the sensors used in MR are the mixture of sensors employed in AR and VR technologies. Sensors in MR head-mounted display are Spatial mapping sensors (stereo cameras, ToF sensor, structure-light sensor), Head trackers, Six degrees-of-freedom sensors, Spatial anchors, SLAM (simultaneous localization and mapping), Eye tracking sensors, and hand-gesture sensors. A brief description about most of the given sensors was explained in previous sections.

While coming to the MR in healthcare, Patient’s previously taken CT scan reports can be overlapped with 3D digital prototype by using the prodigious technology called as mixed reality which allows surgeons to place the 3D model onto the patient’s affected area during surgery. This is one of the extraordinary applications that helps surgeons to a great extent during surgeries and assists them to find the blood vessels accurately and move the respective blood vessels from one place to other part of the body in order to make the open wound to heal rapidly.

Significant benefits can be derived by implementing the AR and MR technologies in the areas of education and surgical practices. Major difference between the AR and MR is the interaction between the objects by the users other than that there is no difference to mention. The same becomes as the important advantage for MR technology mainly in the field of medicine particularly to the surgeons during surgeries. With the help of MR, surgeons be able to handle each and every parts of the body virtually as like in the real world, this leads to remote surgery.

In healthcare, MR can be used in medical schools to make students to understand about the anatomy of human being safely. Even a MR library could be created for case studies and practices. Patients also able to access their medical report to study about the diseases and the approaches, treatments available for getting cure. Many MNCs are currently doing research on these projects and developing products and applications that make patients and doctors to come together from their remote area itself. When an expert advice is needed in a surgery, it is possible to connect with an expert remotely and can get the guidance of them immediately for surgical decision-making situations. Endless applications will be developed in future in MR technology particularly for medicine.

While comparing medical applications of augmented reality, virtual reality, and mixed reality, all three technologies own their distinctive advantages and limitations. In terms of upcoming growth of 3D visualization environment and augmented reality for surgical procedure, mixed reality with its additional interactive abilities, has the potential to make the transformation in both the real and virtual environments of medicine.

MR vs VR

The virtual reality endows an artificial/virtual environment created with full of digital objects. A person be able to experience the 3D environment with the help of head-mounted display or a specialized VR headset that consists of video as well as audio and other sensing elements too. VR head-mounted display comes with handheld modules and special gloves used by the person in the virtual environment to make interaction with different objects there and the VR headset display remains black once the power turned off. Though, mixed reality mixes both real and virtual objects and produces an environment, where a person can interact with all the existing objects even without wearing any gloves or handheld sticks and another feature offered by mixed reality is, the MR headset which would become a see-through like a normal glass when the power for the headset is turned off.

MR vs AR

Augmented reality uses the real world objects and overlapping with virtual object on it. Yet, the person cannot be able to interact with the objects directly which are available on the 3D environment. For instance, consider a 3D object has been created using AR as well as using MR. A person be able to see the object with both AR and MR technology. But in AR, the person be able to only see the object whereas in MR, the person can see, sense, and touch the same 3D object. The AR technology requires hardware-like handheld sticks with cameras and screens while MR requires only a HMD (head-mounted display) that enables the person to interact with the objects in 3D environment directly by bare hands. HoloLens from Microsoft is a real time example for MR HDM.

Mixed Reality has quite a lot of applications, such as cybernetic (virtual) training for pilots, virtual education and training for surgeons, and remote guidance for engineers. The applications of augmented reality, virtual reality, and mixed reality are escalating from entertainment to medicine, to robotics. Some applications are,

  • Entertainment – gaming, movies, and shows to offer a rich experience.

  • Healthcare – Surgical simulations and training for fresher.

  • Aerospace – Pilot training.

  • Virtual tour – from home, travel to any place includes museum, colleges or another planet to explore.

5.8 Data Handling: AR, VR, and IoT

Massively a new trend is rising by encompassing three great pulsating and innovating technologies such as AR (augmented reality), VR (virtual reality), and IoT (internet of things).

In this era, the digitalization becomes the reason for this hottest trend in information technology and ICT (information and communication technology). Most of all the applications are transferring a very big volume of data over internet to one another. Due to this type of transactions, the network faces hassle in data handling and leads to security, privacy, and cyber threats. It consumes people and high processes to manage the data and it is hard to share the Big Data in a cost-effective, practical, and meaningful way. A framework has been developed by IBM to deal with the Data handling, based on four distinct ‘V’s [16]. They are Volume: convey a message that the excel sheet is not enough to hold the “big data”; Velocity: data visualization and handling takes place as soon as the data have been received; Variety: collect various types of data such as plain text, sensor data, rich document, etc., and analyze it accordingly; and Veracity: have to find out the reliability of the received data because trust is most important in data handling over the internet. Data Handling plays a vital role already in the field of data science, helps data scientists to realize the pattern and structure that the data holds, before causing serious issues over the network.

AR and big data have constantly gone together. In a real-time environment, to render the information virtually, a large data set is necessary for AR and the same is provided by Big data. Data visualization depends on type of information, techniques used for visualization and compatibility of data. Big Data needs to be transformed as human intellectual is inhibited [17]. AR meets Big data since AR and big data have a logical maturity that certainly makes both giant technique to be united in the visualization part. The pace of hitching AR and big data to raise new exciting applications is beginning to have a concrete existence [18]. In [19], the proposed is the increase of big data visualization in virtual reality environments to prevailing considerations of human insight and communication via a phenomenological lens and the concepts of personified interaction and perception. Hence, the fundamental perception of effective data visualization is to represent congested and complex data in a way that is more convenient for the user [20].

The graphical representation (Fig. 5.5) given below represents the approximate amount of data received and handled by AR/VR applications.

Fig. 5.5
figure 5

Data Analytics in various applications of AR/VR

Full size image

The arena of sensor technology is tremendously broad, and its future growth will encompass the interaction of every scientific and technical aspects. The basic definitions of sensors and its terminology have been briefly presented, still considerable uncertainty exists in the sensor technologies, classifications, and its applications. Most of the modern sensors are integrating few types of signal processing in transmitting information as signals. The backend processes of sensors are well-hidden from the user end in modern sensors today. Novel materials are used to create more precise sensors to use in high-end technologies.

From internet of things (IoT), to augmented reality (AR), to virtual reality (VR), to mixed reality (MR), and now reaches AI (artificial intelligence) applications for each and every sector. By these advancements in emerging technologies, the universe is overfilled with numerous projects to make humanitarian aid as more effective. From 3D gaming headset, movies, virtual tour, and interior design experience to never-ending applications in elevating the human life better and ease. One of the best examples of AR technology is the game “pokemon Go”, it reaches to every nook and corner in kids’ world. Even adults get addicted over the game since it gives unique and different experience. Apart from the entertainment and other applications, the primary use of these emerging technologies is in the field of medicine. It saves life of many people. It cures so many traumas and pains of young and old people to a greater extent. The next upcoming technology is the Robotics. Robotics in medicine will be the pinnacle of technology in healthcare. These technologies play a vital role in terms of advancement in the medical field and the same is illustrated in the Fig. 5.6.

Fig. 5.6
figure 6

Timeline of Technology in Health Care

Full size image

5.9 Conclusion and Future Scope

By the above discussed applications, the AR and VR make their presence into the hearts of most of the users’ globally. By collaborating these different technologies on a single network layer, IoT infrastructure, mobile devices, and applications be able to interact with each other and permit the advanced technology such as AR and VR along with IoT grow into globally accessible by all level of users and offer opportunities to companies to connect with various set of people around the world. Therefore, to make the virtual world to be a real one, sensor technology plays a vital role to invent newer applications with the help of emerging technologies in all walks of life.

Future work would be the optimization of sensors, precise data analytics to be done for improved accuracy. Security must be provided great for the confidential biomedical data exchanges between one another by using emerging technologies like blockchain.