Keywords

1 Introduction

Human communication is at its root, where it is completely based on the text and speech [1]. While this communication has been accepted as a daily part of our life that requires little effort, it is not as easy for some. According to World Health Organization [WHO] report on vision [2,3,4], the number of visually impaired people present across the globe is approximately 2.2 billion, of whom at least 1 billion have a visual impairment not yet been addressed or prevented. According to a study published as of 2010 in the Global Estimates of Visual Impairment [5], almost 20.5% of the world’s unsighted [6] are Indian citizens, along with 22.2% of the world’s low vision population, and 21.9% of those are with vision impairment [7]. Disability takes a toll on a person’s day-to-day life. With the ever-expanding and unparalleled technological growth, this part of the population will actually require a modern device that helps them to be more self-reliant and efficient without increasing the cost/expense [8]. Such modern devices are termed as ‘assistive devices’, and as the name suggests, they assist the user in partially or wholly for overcoming a problem/disability. Among all the other available assistive devices, wearable devices have been proven as the most efficient and helpful, out of which the head-mounted display [9] devices and clip-on devices remain as the most popular consideration. Recent studies and technologies in clip-on devices [10] have also shown promising results. A large fraction of the devices are available for assistance and are either focused on a singular aspect of the problem, or it is not efficient as expected [11].

This paper focuses on different approaches that are considered to aid the visually impaired through technology and compare the solutions with a new concept (focusing specifically on the user) to see how it is different, or in some cases, better than what is already available in the market.

1.1 Types of Computer Glasses

Different types of computer glasses exist, and this adds information alongside what the user can see, or in this case, it will not be able to view. Like other computers, smart glasses may collect information from external or internal sensors. Using the sensors, smart glasses can guide the wearer, inform the user about his/her surroundings, warn them of threats and generally assist them in carrying out simple tasks that would otherwise be difficult for them [12, 13]. Various types of computer glasses are mentioned below.

1.1.1 Monocular Smart Glasses [14]

Monocular smart glasses as the name suggests are glasses in the form of HMDs where an optical engine is placed on one of the lenses [15, 16]. Vuzix M300, Google Glass [17], Optivent’s Ora-2 and the Lumus Sleek are some examples of such devices.

1.1.2 Binocular Smart Glasses [18]

Binocular smart glasses are another form of a head-mounted display system which consists of two transparent displays, which deliver a stereoscopic view to the users. The advantage of such HMDs over monocular glasses is that a more diverse enhancement of the overall view perspective of the wearer is conceivable because of the optical engines mounted before each eye. Much like monocular glasses, these also display the information just beyond the user’s line of sight, but to both eyes. Verge IT [19], Epson Moverio [20], the ODG R-9 [21], the Sony SED-E1 [22, 23] and SED-100A [24] are some of the current binocular smart glasses in the market.

1.1.3 Audio Augmented Reality Smart Glasses [25]

ARSG are basically head-up transparent displays that add virtual information to what the user sees with an integrated wearable miniature computer [26]. As the name suggests, the glasses provide the users with augmented reality by overlaying virtual data on the real worldview of the wearer, and doing so makes it conceivable to enhance a human’s perception of reality [27]. ARSG are an ideal user interface for industry usage by workers since they are hands-free gadgets that show information at eye level, right where it is required.

1.1.4 Immersive or Mixed Reality Smart Glasses [28]

Mixed reality is the combination or merger of the real-world view with the virtual world to create new environment. A real-world environment is superimposed with virtual objects with the help of computer graphics. The objects visually act like a physical object in the real-world environment, change shape, and appearance according to the angle and position of where they are interacted from and also, based on the external environment or actions/movements of the user. Hence, a user can view the virtual object from different bearings and directions by moving around spatially in a room, as if it were a real object [29].

Some mixed reality glasses are Microsoft HoloLens, ODG-9 AR and VR glasses and the Meta Space glasses. These gadgets are fully immersive and inclusive stand-alone systems with outstanding displays, allowing you to render 3D objects on board [30].

A field that has not been utilised to the best of its capacity is that of audio augmented reality [31]. Audio augmented reality is when the real or physical world can be augmented with a virtual layer of auditory signals, which allows the user to acquiescently interact with it. In certain glasses, the sensors on the glasses gather appropriate information and the focused speaker directly informs the user. The technology in this field can be used to provide maximum utility to the user.

1.2 Motivation for Conducting the Survey

The motivation for this survey was to clearly analyse the existing technologies/devices that exist, which aid the visually impaired and the features incorporated in them. This analysis helps to identify the problems that were solved by these technologies and the ones that could not be. Along with that, it provides a detailed overview of how these technologies can be better utilised to serve the visually impaired and make it easier for them to participate in their day-to-day activities. An exhaustive survey was conducted, particularly motivated from these facts:

  • The need and demand for understanding the benefits and drawbacks of the impact of existing assistive devices for the visually impaired.

  • The need and demand for understanding the various technologies these devices use.

  • Can these technologies be utilised in a better way to suit the needs of the customer base?

  • Can newer technologies be incorporated in such devices to provide an enhanced product?

Thus, there is an immediate need to understand and explore technologies that can be used to assist the visually impaired and the status of their applications in the physical world. Not only does the technology assist the visually impaired, but its application also includes safety; thus, it is our duty to work towards the enhancement of this area, so as to make it a better place to live in for those with disabilities. However, to gain an understanding of technologies that can be used, it is highly required to analyse the use of existing technologies and their applications, and only then it can contribute towards making a change in this field.

1.3 Outline

The remaining article is designed as: Sect. 2 presents the background. Section 3 goes into a detailed analysis of the existing devices and technologies, highlighting their use in the device and their impact. Section 4 highlights the inferences made by conducting researches on the existing devices and what better technologies can be used in those devices. Section 5 presents the concept of a prototype device that could be used to effectively solve various problems faced by the visually impaired population, in a single device. The proposed research work is concluded in Sect. 6.

2 Background

Until recently, solely medical solutions and practices used for treating disabilities have become more popular. Any assistive technology will only be considered as a branch of that medical practice. It is only in the past few years that assistive devices have really taken the centre stage in not just aiding people with visual disabilities but also making them largely autonomous and independent [32].

Particularly, the problems like navigation, reading and providing visibility to people with partial visibility problems are considered as the different challenges associated with the technologies.

Society today has changed considerably to eradicate the discrimination against impaired population. The combination of new laws and awareness has asserted the idea of acceptance and independence of the impaired. But with that, there is a need to devise a highly effective way to help the visually impaired in order to move a step closer towards normalcy by asserting self-dependence. This can only be done by implementing various advanced technologies. Various products have been integrated in the past few years to help the visually impaired to deal with these problems. This article will extensively explore these products along with the technologies employed and how they evolved over the years.

2.1 Review Process

Figure 1 shows the growth of research in the field of assistive technologies for the visually impaired since 1990 until date. The graph shows a tremendous increase in research contributions in the last decade (Table 1).

Fig. 1
figure 1

Graphical representation of the growth of research in the field of assistive technology for the visually impaired

Full size image
Table 1 Classification of papers
Full size table

Figure 2 shows a graphical representation of the distribution of research articles in various sources such as journals, conferences and proceedings. The graph depicts the maximum contribution of research articles in journals followed by Web articles.

Fig. 2
figure 2

Graphical representation of the distribution of articles in various sources, used for the review process

Full size image

The proposed research work has framed 6 research questions as shown in Table 2. These research questions helped the authors to explore the literature and collect all the information related to assistive technologies for the visually impaired.

Table 2 Research questions
Full size table

3 Related Works

3.1 BuzzClip [36]

BuzzClip caters to that part of the society that lives with blindness or partial sight. It aims to reduce the dread and anxiety related to navigation by providing a dependable way to make the user aware of their immediate surroundings and obstacles.

Construction: BuzzClip is a small and discreet wearable device. It aids in navigation by detecting obstructions in the wearer’s path using ultrasound. It is an additional supportive device to the cane and can also be used on its own for those with partial sight looking for a subtle solution.

Methodology: It informs the user of the hindrances through instinctive vibrations and helps them safely navigate their way around these objects. It additionally offers fundamental obstruction detection at head level and can be attached to clothing, proving it exceptionally adaptable and helpful.

3.2 Aira AR Smart Glasses [37]

The Aira smart glasses, developed by La Jolla, are augmented reality smart glasses that are intended to aid users who are completely or partially visibly impaired, to read significant text (expiration date on grocery items, information on medicines, etc). It is the blend of cutting-edge technology, hardware and live assistance support provided by highly attentive aides.

Construction: The glasses are equipped with a camera and inbuilt earpiece.

Methodology: With these smart glasses, Aira has built a service that fundamentally places a human assistant into a visually impaired user’s ear and sends live video of the surroundings of the user through the camera on the glasses to the assistants, who can then give instructions to the end-users and help them with their basic tasks, be it directions or describing scenes.

3.3 Sunu Band [38]

Sunu manufactures the Boston-based wearable, one of which is Sunu Band: a sonar sensor device that helps the visually impaired not collide with people and objects by guiding them.

Construction: Uses sonar sensors for obstacle detection portable system [39].

Methodology: When the sensor detects an object or person 15 feet away from the user, the device vibrates to let the user know of a potential collision, and with increasing proximity, the device vibrates with increased frequency and intensity.

3.4 Maptic [40]

Maptic devices are wearable navigation systems.

Construction: A necklace that acts as a visual sensor and a device that clips around the wrist of the user acts as a series of feedback units. A voice-controlled iPhone app uses GPS to direct the user.

Methodology: The device vibrates to guide the user to desired destinations. To steer the user towards either direction, the device sets off a series of vibrations to either side of the body.

3.5 MyEye2 [41]

The aim of these low-vision electronic glasses is to make day-to-day activities like reading, writing, recognising faces and items easier for the visually impaired.

Construction: Using a magnetic mount, the device is attached to any pair of glasses. A light camera is attached to the sides of the glasses’ frame as well.

Methodology: The power button of the OrCam device is near the speaker, which is located on the flat underside. A touch bar (raised for easy location) is placed on the top side of the device which is rounded. It allows the user to read multiple languages, identify currency and also face recognition. OrCam also learns to identify objects, as it does with faces.

3.6 NuEyes Pro [42]

NuEyes Pro is a pair of head-mounted smart glasses that are lightweight and wireless. It is specially designed to aid the visually challenged to see better. NuEyes Pro can help with conditions like glaucoma, diabetic retinopathy and macular degeneration. It converts print to speech and allows users to carry out daily activities like watching TV, writing and reading.

Construction: An image is captured by the camera placed at the front of the glasses and then shown in a magnified form inside of the lenses. Magnification of up to 12 times can be obtained on images, which helps the user see the finer details more clearly. NuEyes also has a 30° field of view (FOV) and a 1080i stereo display.

Methodology: A remote-handheld controller and vocal orders are used to control the device.

3.7 OXSIGHT [43]

In 2016, OXSIGHT was established based on the effort of a team of Oxford University, who started researching and working closely with the low-vision community a decade ago. Their main aim was to bring out an innovative device to help this community. Its main features include expanding field of vision, enhancing light and shape detection. At Oxford University, Dr Stephen Hicks and his colleagues researched and came up with the idea of these smart assistive glasses that would be able to present essential information about the proximity of obstacles from the wearers. This was achieved with the help of a camera and display option. Along with that, the glasses are also capable of increasing the brightness and clarity of obstacles.

3.8 Intelligent Reader for the Visually Handicapped [1]

This Raspberry Pi-based reader captures an image of some textual material and enables visually challenged users to passively interpret the text present in that image.

Technologies used: Raspberry Pi Camera [44], to capture the image; ImageMagick software is used to get a scanned image from the picture obtained; Tesseract Optical Character Recognition software obtains text converted from the image; Text to Speech (TTS) engine is used for the transfiguration of text into speech.

Table 3 effectively summarises these latest technologies and works in the field of visual aid devices and provides an efficient comparison between the same.

Table 3 Existing technologies to aid the visually impaired
Full size table

4 Discussion

The existing technology has greatly helped aid the visually impaired in providing them independence in day-to-day activities. They have immensely impacted their social status and mental well-being. While the contribution of these devices in bringing technological advancement (as in present day) to help the visually impaired is significant, there is always scope for improvement. Most of the existing wearable devices are not effectively solving the root of the problem. They focus only on subsets rather than the main obstacle. The most popular devices for smart glasses are Braille readers [45] which can read and write using the arrangement of dots. Another device commonly used is an audio reader which reads out books and papers saved in the audio format. This would essentially point to the fact that for each task, there exists an individual product solving only one part of the problem. Both these devices being costly, it leaves a huge dent on the pocket of those who really need them, especially those who will need to buy different assistive devices for different tasks. This ultimately leads to sacrificing basic comfort in their day-to-day lives.

Figure 3 gives a summary of the problems current assistive technology is solving. With the help of this figure and further research, one can conclude that there are only a few devices (35–44) that solve a multi-purpose function.

Fig. 3
figure 3

Graphical representation of the number of assistive technology devices for the visually impaired in each type of field

Full size image

Moreover, most of the devices explored in this article focus on improving visibility for those who are only partially visible. The devices that claim to provide aid to all visually impaired, like Aira smart glasses, do not provide autonomy to the user and force them to lean onto a person that is aiding them through a live call feature. These techniques are not just outdated, but also unattractive to the visually impaired.

A few devices target text reading, others target object detection in the path of the user, and a minute number of devices help in recognising objects or faces using machine learning or artificial intelligence tools, but these are limited to partial visual impairment. And to date, no device exists that combines all the features into a single device. There has been no development of even a costly all-round device, let alone a cost-effective solution.

The technology is assistive devices started with object detection solutions using ultrasound and moved onto braille readers, and even after leaps of technological advancement, it seems that assistive devices have only progressed to improve or build upon the technology of existing object detection devices and text readers. Assistance in the form of recognition of faces or objects is largely absent. Artificial intelligence and machine learning have rarely been incorporated into these devices, and when they have been, the devices do not seem to include the most necessary features of navigation and obstruction detection.

Among existing features, there can be multitudes of advancement. Technologies like IoT can be used for wireless detection (similar to how it might be effective in vehicle collision detection [46]) instead of ultrasound. For live video assistance technologies, instead of limiting to verbal assistance by a human aid, motor control systems for wireless automation [47] can be explored to help navigate the user.

While object and face recognition is trivial in these devices, recognition models like R-CNN, fast R-CNN (and YOLO—you only look once) should be implemented for faster results, as real-time recognition requires speed.

Most importantly, there is still scope for technology that can combine various factors into a single device and provide immense autonomy to the visually impaired for most activities.

Research and studies have found out that as the number of assistive products and related technology increase, so does product suspension. When asked, visually impaired users of such products communicated disapproval of most assistive devices currently used because of the factors of tactile feeling and visual balance. Many complain of self-consciousness linked with the design of the device and its functionality. There has yet to be a breakthrough that can model a device according to the user’s feedback and advice.

5 Device Concept

The main aim is to investigate the dynamics of converging mobile devices and their accessibility features with wearable devices coming into the market. This shall make the lives of visually impaired more comfortable and closer to that of the people around them.

In addition to that, the main focus is given to the possibility of developing a new technology, which would be able to accommodate the most crucial aspects of the problems faced by these people as well as manage to stay efficient by keeping its cost low.

Figure 4 represents a state chart (flow chart) of the working of a close to ideal assistive device. Current devices only include the object detection flow path or a branch of the object recognition flow path (as text recognition).

Fig. 4
figure 4

A general state chart representation of the proposed ideal assisting device

Full size image

6 Conclusion

The proposed research work leverages a new concept of smart device that aims to study the existing assistive technologies that were developed, to analyse their drawbacks, and furthermore it creates a technology that overcomes the challenges that were previously faced. While performing a detailed study, many research gaps are observed in the existing designs of this field. Most of these technologies focus only on one aspect of the problem; for example, the majority of the devices work on assisting the visually impaired with reading, or to an extent, identifying limited objects. With this device, it will always provide a comprehensive product that not only assists in their day-to-day lives but gives a wholesome experience to the user which they can enjoy without being constantly reminded of their disabilities. Most of the devices discussed above are focusing on just a fraction of the problem, making it difficult for users to have a better experience. This research work can be concluded with some findings that are listed below:

  1. (1)

    Larger parts of the technologies already present in the market do not highlight all the problems and instead develop a product focusing on only one aspect of the issue.

  2. (2)

    After finishing our research, it has been realised that out of all the technologies that has been analysed, where it cannot find any device, which addresses all the problems in one device.

  3. (3)

    All the devices mentioned are very heavy on a middle-class person’s pocket in comparison with the features it provides, making it very difficult to afford. This makes it really difficult for the disabled to even participate in day-to-day activities without proper assistance, and as a result of this situation it makes them feel dependent and liable.

The emerging innovative perspectives of different existing technologies can be incorporated into the making of new wearable technology. This work served the purpose of gaining insight into designing a new technology for smart devices which strives for maximum potential associated with minimum cost [48].