Abstract
Assistive technology for the visually impaired and blind people is a research field that is gaining increasing prominence owing to an explosion of new interest in it from disparate disciplines. The field has a very relevant social impact on our ever-increasing aging and blind populations. While many excellent state-of-the-art accounts have been written till date, all of them are subjective in nature. We performed an objective statistical survey across the various sub-disciplines in the field and applied information analysis and network-theory techniques to answer several key questions relevant to the field. To analyze the field we compiled an extensive database of scientific research publications over the last two decades. We inferred interesting patterns and statistics concerning the main research areas and underlying themes, identified leading journals and conferences, captured growth patterns of the research field; identified active research communities and present our interpretation of trends in the field for the near future. Our results reveal that there has been a sustained growth in this field; from less than 50 publications per year in the mid 1990s to close to 400 scientific publications per year in 2014. Assistive Technology for persons with visually impairments is expected to grow at a swift pace and impact the lives of individuals and the elderly in ways not previously possible.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
The field of assistive technology (AT), commonly considered to be technology designed for individuals with some form of impairment (or the elderly people), is a vital field expanding at a swift pace since it derives from many disciplines and is mainly driven by technology. Assistive technology for the visually impaired (VI) and Blind people is concerned with “technologies, equipment, devices, apparatus, services, systems, processes and environmental modifications” [43] that enable them to overcome various physical, social, infrastructural and accessibility barriers to independence and live active, productive and independent lives as equal members of the society.
Vision being an extremely vital sensory modality in humans, the loss of it affects the performance of almost all activities of daily living (ADL) and instrumental activities of daily living (IADLs); thereby hampering an individuals’ quality of life (QoL), general lifestyle, personal relationships and career. Therefore, technology that facilitates accessibility, safety, and an improved quality of life has a very relevant social impact [77]. Moreover, with our ever-increasing ageing and blind populations, it has the potential to broadly impact our quality of life in the future. This has driven novel research across many disparate disciplines, from cognitive psychology and neuroprosthetics to computer vision and sensor processing to rehabilitation engineering. More recently, advances in computer vision, wearable technology, multisensory research, and medical interventions have facilitated the development of numerous assistive technology solutions of both kind, invasive and non-invasive (a varied selection of a few compelling recent works on Assistive Technology for Visually Impaired and Blind individuals is shown in detail in Fig. 1).
Research on assistive technology for the visually impaired and blind people has traditionally focused on—mobility, navigation, and object recognition; but more recently on printed information access and social interaction as well [109]. Over the last decade there has been an expansion of research interest in the field and significant developments have taken place in the form of novel (miniaturized) wearable electronic travel aids (ETAs), smart canes, (wearable) form factors, smartphone based devices and apps, tactile displays and interfaces, cortical and retinal implants (bionic eyes), etc. The paper presents an insight into the current state of assistive technology for the visually impaired and blind people, with an emphasis on what can be learnt from the last two decades of published research and what the future trends are. This work is a survey of the state-of-the-art in research in the field based on information analysis of a database of scientific research publications. The publications are relevant to assistive technology for the visually impaired and blind individuals across various fields of technology, medicine and related sub-disciplines.
While there are many excellent reviews: on mobile and wearable assistive devices for the blind by Tapu et al. [107], Velazquez [112] and Ye et al. [114]; reviews of electronic travel aids or locomotion aids by Dakoupoulos and Bourbakis [28], Freiberger [37] and Jacquet et al. [51]; on mobile assistive technologies by Hakobyan et al. [40]; on computer vision-based assistive technologies by Manduchi and Coughlan [77] and Terven et al. [109]; on the use of haptics for the design of aids by Levesque [68]; and on sensory substitution by Maidenbaum et al. [76] and Bach-y-Rita and Kercel [7]; and systematic reviews of electronic navigation aids by Roentgen et al. [90, 91]; all of these accounts are subjective in nature and emphasize subjects and themes that are part of the author’s expertise or research priorities. Table 1 compares and contrasts all prior subjective surveys with this current objective survey of literature. It shows how the scope and focus of our survey differs from prior surveys. At the same time, our purpose in this survey is to present a complementary review to the existing ones and an insight into the field from the viewpoint of a more objective statistical survey across the various sub-disciplines in the field. In particular we focus on the following five key questions by analyzing the corpus: What areas does assistive technology for the visually impaired and blind people encompass? Where is research for assistive technology for visually impaired and blind people published? How rapidly is Assistive technology for the visually impaired and blind people expanding as a research field? Are there research communities within the field of assistive technology for the visually impaired and blind people? Where is assistive technology for visually impaired and blind people headed in the future? Focusing on these questions will benefit the field, as we believe that the answers will clarify how the field evolved to the current state, how the field derives from novel research across other fields, what are the current research patterns and trends in the field, and where they appear to be heading in the future. Though design/interface, performance and usability would make a very interesting study, we felt we were limited from pursuing it in the current work as the application of information analysis and network theory techniques required us to objectively analyze the constructed publication database.
The methodology employed in this paper is inspired by the one employed by Lepora et al. [67] in a prior article for the field of Biomimetics. Though this methodology was found useful in guiding our approach, we differ from the Lepora et al. paper in the following ways: (i) To begin with, we consider four major international scientific databases, detailed in Appendix A (instead of three, in [67]), (ii) The publications we considered for analysis span over twenty years (1994–2014) (instead of fifteen, in [67]), (iii) We present growth patterns and publication trends of journals, conferences and book chapters, and also report briefly on growth of patents (instead of only journals and conferences, in [67]), (iv) Information analysis was carried out using Xpath (XML Path Language) and XSLT 2.0 (EXtensible Stylesheet Language Transformations), both of which are best suited for XML publication data (MATLAB was used in [67]); our collocation metric also differs (Appendix B). (v) We considered past and current research in the field and put forward our opinion of emerging trends, recommendations and discipline hierarchies for the near future (which we believe is one step further from the work by Lepora et al. [67]), and finally, (vi) Lepora et al. [67] focused on Biomimetics, while our work applies the information analysis methodology and network analysis to a totally different field, i.e. assistive technology for the visually impaired and blind people. We present an insight into the past, present and future of the field, which we believe makes the resulting objective statistical survey significant and different from all prior surveys in the field.
2 Background to assistive technology for the visually impaired and blind people
The field of assistive technology for the visually impaired and blind people is quite complex. There are different aspects of it and it can be approached from different points of view. The scope of it extends from the physiological factors associated with vision loss, to the psychological and human factors influencing orientation, mobility and information access for an individual with visual impairments, to the technological aspects in the development of rehabilitation devices (for mobility, wayfinding, object recognition, information access, entertainment, interaction, education), to medical interventions and prostheses including both current and cutting edge research. Hence, it is very hard to characterize or capture the essence of this field within a single snapshot.
Since the inception of the field, believed to be about five decades ago [16], i.e. when serious developments in electronic mobility aids occurred (1950s–1960s) and development continued throughout (1960s–1970s) [42], one of the key focus areas has been the design and development of electronic travel aids for the blind or elderly people. This has resulted in the continuous evolution of ETAs; beginning with the early sonar-based mobility aids such as the Russell Pathsounder (1965) and the Mowat Sensor (1977) [16] which functioned as obstacle detectors in the traveler’s immediate path. A string of prototypes based on similar operating principles followed thereafter; however, owing mostly to their disappointing performance the user acceptance was low. Soon computerized ETAs came into the picture—NavBelt [13, 100] was a wearable, computerized and sonar-equipped system that enabled blind users to safely walk through unknown, obstacle-cluttered environments. Numerous useful ETA prototypes have resulted from efforts to leverage technology for blind mobility. We focus on just the representative works done in the area. An ETA that has been a commercial success story is the award winning primary mobility aid Ultracane [47] which combined the long cane with ultrasonic sensors. Over the last decade many advanced hand-held, wearable, and embedded assistive devices have been developed, but a comprehensive device still remains an elusive goal.
Technological innovations led to minimizing the form factor and paved the way for miniaturized, wearable ETAs such as those with system on a chip (SoC) and sensors mounted on the shirt’s pocket, or attached to the garment or disguised as a brooch; a good example of which is the clear-path indicator for the blind people by Jameson and Manduchi [54]. With the advent of Microsoft Kinect and depth sensing, its use as a sensing modality for navigation in indoor environments by the blind has also been experimented in many systems including work by Bhowmick et al. [11] and Filipe et al. [33]. In recent years computer vision capabilities coupled with the freedom and flexibility of mobile computing has shown great promise and resulted in many interesting developments like the Crosswatch system [22, 23] for providing assistance to the blind at traffic intersections; the Roshni project [52, 53] for mobility in unknown indoor environments; ARIANNA for both indoor and outdoor navigation [24]; and the 3D smartphone-based aerial obstacle detection system that is discreet and also favours social integration [94]. These ETAs assist visually impaired and blind individuals in known or unknown, indoor or outdoor environments by providing rich environmental information, obstacle avoidance, object recognition, and navigation through the use of ultrasonic systems, GPS, cameras, infrared, laser, and mobile phone technology.
Another central theme of research over the decades has been the accessibility and inclusion of the visually impaired and blind individuals into mainstream society. Various projects have been developed over the decades to empower them towards accessibility, inclusion and participation. Exciting and emerging topics associated with this field include sensory substitution, visual prostheses, visual neuroprostheses, brain plasticity, brain-computer interfaces (BCI), artificial vision, tactile human-machine interfaces (HMI), accessible computing, and human factors research. Since Bach-y-Rita’s pioneering work on tactile vision substitution systems (TVSS) to translate visual information into tactile cues [8], the theoretical neuro-physiological basis of which is discussed in [6], major advancements over the decades have followed based on this formative work and has steered the field into many offshoot areas of research. SmartTouch [56] is a notable example of a tactile interface that uses electrodes to activate sensory nerves under the skin. Such systems (known as electrocutaneous display systems) have been developed to work on other skin receptors like the Optacon for finger tips [72] and The tongue display unit (TDU) [61] that creates real-time tactile images using the tongue as a HMI. Brainport V100 [3, 85] is a recent investigational, non-surgical visual prosthetic using tactile tongue stimulation to translate information. Critics however state the device is overrated and debate over its awkward camera-tongue combination feature and its limited value in decoding the clutter and chaos of everyday life [111]. HamsaTouch [58] is a novel smartphone-based TVSS evincing the advances of current technology. These are some noteworthy examples of latest versions of TVSSs.
Research on auditory vision substitution attempts to create a visual prosthesis through auditory vision substitution systems (AVSS). The auditory sense has great potential as an alternate medium for meeting the needs of people with visual disabilities. Sonification focuses on the use of non-speech audio to convey information or perceptualize data. Sonification techniques and Auditory Displays (AD) have evolved in relatively few years and is seen as a standard technique on par with visualization for presenting data in a variety of contexts, including development of interfaces for visually impaired people [41]. Keller and Stevens [60] categorized sonification into three categories: direct, indirect ecological, and indirect metaphorical. Direct sonification maps data to sounds associated with certain phenomena, e.g. sound of crackling flames represent fire. Indirect ecological sonification uses related ecological sound associations such as the sound of branches snapping represent a tornado. Finally, indirect metaphorical sonification uses analogical sound associations, e.g., the sound of a mosquito buzzing to represent a helicopter. Sonification and auditory displays have achieved significant progress in areas of computer access and mobility aids. Screen readers represent a major advance in computer access for people with visual disabilities. The efficacy of screen readers is still limited in complex tasks such as while tracking focus in a windowed graphical user interface (GUI), or reading spreadsheets or accessing dynamic website content [15, 42]. For the visually impaired people computer access is still a relatively smaller need than avoiding obstacle and navigating to places. The vOICe Footnote 1 vision system [4, 81] converts live camera feeds into ‘soundscapes’ or sound patterns representing specific environmental information. Participants equipped with the device were able to use auditory stimulation and obtain information necessary for locomotor guidance, localization, and recognition of objects in a 3-D environment. The vOICe for Android has been available since 2008. EyeMusic [1] shown in Fig. 1 is another AVSS that conveys visual information through pleasant musical notes via bone-conductance headphones. A comprehensive list of electronic travel aids including ETAs using sound for sensory substitution may be found in [28] and [112].
An emergent theme in sensory substitution research is cortical visual prosthesis, sometimes called brain implants, that provides visual sense directly to the brain via electrodes in contact with the primary visual cortex. Brindley and Lewin [18] are credited with producing the first functional cortical implant (consisting of 80 electrodes) on a blind subject causing her to experience sensations of light (‘phosphenes’), given the appropriate signals. This was the first major advancement in artificial vision. Dobelle experimented with surface-electrode implants (Dobelle artificial vision system) [30, 31] and reinforced the idea that a functional implanted visual prosthesis (IVP) could be developed. These early developments laid the foundation for the current status of “bionic vision” through brain implant technology. Modern cortical implants, such as the Monash Vision Group’s ‘Gennaris’Footnote 2 device designed to bypass the retina and optic nerve and wirelessly stimulate the visual cortex of the brain with electrical signals using arrays of micro-sized electrodes are still in the preclinical phase.
In recent years, retinal implant technology i.e. inducing visual perception by electrical stimulation of the retina with implantable micro-electrode arrays, has also emerged as a popular alternative. There are three retinal implant strategies: (i) epiretinal implants—placed on the surface of the retina, such as the EPIRET3 device [82, 92] and the Argus II Retinal Prosthesis System Footnote 3 [2, 32, 49] both of which have already been implanted into tens of blind subjects in clinical trials and found safe and beneficial. Argus II is currently the first and only bionic vision device licensed for use anywhere in the world after being approved by the U.S. Food and Drug Administration (FDA) in 2013. (ii) subretinal implants—placed between the retinal pigment epithelium (RPE) and the retina, such as the Alpha IMS subretinal implant developed by Retina Implant AG, Germany [103, 104], IMI Retinal Implant System [46], and the Boston Retinal Implant Project wireless device [89] which have ongoing implantation clinical studies, and (iii) suprachoroidal implants—placed in a more surgically-accessible location than the earlier two i.e. between the vascular choroid and the outer sclera. An example is Bionic Vision Australia’s first-in-human trial of retinal implants in the suprachoroidal space [5, 96] that has opened a new door of possibilities. IVP principles of operation, current state of research and development, and functional aspects are excellently covered in the extensive book edited by Dagnelie [26]. While a direct interface to the cortex or retina is a big advantage, this also makes it an invasive approach, requiring surgery. Audio and tactile vision substitution systems are an inexpensive and biocompatible alternative to surgical implants.
Of late, the integration of mobile devices, smartphones and computer vision has fostered exciting themes and applications in areas of independent living, accessibility, print access, social interaction, and web accessibility. With the evolution of smartphones, a lot of smartphone-based assistive technologies have been developed that offer a better quality of life in daily living activities. Some notable examples are the Carnegie Mellon University’s smartphone-based Trinetra project [65, 84] that scans product barcodes to aid in independent grocery-shopping experience for the blind people; StopInfo [20] that provides detailed information about bus stops tailored to the needs of blind riders; and the display reader that uses computer vision to help the blind user access household appliance displays [38]. Concerning access to print information, traditional Braille devices and low-vision aids (e.g. screen readers and magnifiers) are fairly well developed and have been the primary access routes. Audio descriptions (AD) are known to make the visual content of films, theater, opera and TV programmes [97] more enjoyable, interesting, and informative for the blind people. Web accessibility for the visually impaired and blind people is more complex due to the Web’s rich multimedia content [78]. The inevitable transition of the Web to rich internet applications (RIA) introduces additional accessibility challenges. Screen readers still struggle to deal with dynamic Web content (AJAX content updates, drop down menus), form filling process and automatic page refreshes [15]. Lately, smartphones and wearable devices are transforming the assistive technology and artificial vision landscape by boosting independence and helping the blind people overcome accessibility barriers. Some remarkable developments include Project Ray [86]Footnote 4, GeorgiePhone Footnote 5, MIT Fifth Sense Footnote 6, and the very promising portable device OrCam Footnote 7 which offer a full set of ground-breaking features to the visually impaired based on just voice and touch. Exciting features include reading printed text, outdoor signs; recognizing faces, products, currency notes, credit cards, colors; knowing what’s around, avoiding obstacles and hazards; identifying bus lines, navigating busy places like airports, etc., some of which are already available.
3 An information analysis methodology
In order to be as objective as possible in this task we chose a methodology in which the state-of-the-art and future trends were surveyed using techniques from information analysis. An important aspect of this type of analysis of datasets is how the results are visualized, evaluated and interpreted. Data visualization which is a modern branch of descriptive statistics is a critical component in scientific research; one of its primary goal is to communicate information clearly and efficiently to users. Accordingly, we employed a variety of conventional and modern visualization techniques, including bar charts, word clouds, and co-occurrence graphs in order to analyze and survey the research field from a broad range of perspective.
Our first strategy was to identify scientific publications relevant to the field of “Assistive Technology for the Visually Impaired and Blind people” and retrieve or download the source metadata of all these publications in order to construct a publications database of the chosen field. Four major international scientific databases namely Elsevier ScienceDirect, IEEE Xplore, ACM Digital Library, and Thomson Reuters Web of Science were used as the sources for scholarly publications in our field of interest. A wide range of keywords and phrases were used as search terms including—vision rehabilitation, visual prosthesis, sensory substitution, wearable assistive devices, electronic travel aids, navigation systems, assistive technology solutions, rehabilitation technology, vision substitution. The focus of this research study was mainly on the past, present and future of assistive technology for the visually impaired and blind people. We narrowed down the search to include publications mainly from engineering, rehabilitation engineering, assistive and rehabilitation technologies, computer vision, sensor processing and technologies, cognitive science, cognitive psychology, vision prosthesis and multisensory research. The resulting publications database was then analyzed to infer interesting patterns and statistics concerning the main research areas and underlying themes. Leading journals and conferences which publish and disseminate knowledge in the field are identified; the growth of the research field is captured; and the active research communities are identified. Finally we present our opinions and predictions of future trends that are expected to shape the near future of the field.
The implementation details of this methodology are described in the two appendices: Database Construction in Appendix A and Database Analysis in Appendix B. In particular, we described how the publication metadata was retrieved and downloaded from the four major international scientific databases so that they could later be analyzed on a standard personal desktop computer. We retrieved a total of 3010 scientific publications on research relevant to assistive technology for the visually impaired and blind people covering the period from 1994 to 2014. The bar chart in Fig. 3 represents the overall split of publications. Our analysis consists of a series of tests, beginning with a survey of the relevant publications by topic based on the frequency of occurrence of terms and collocations appearing in the titles and abstracts of the published research. This is followed by the analysis of the year, journal and conference in which they were published and an analysis of the publication data in order to detect the underlying community structures. Finally our opinions are presented where the research field is heading in the future.
The information analysis methodology we adopted has limitations which must be noted at the outset. Firstly, though we believe that the database construction is systematic, it is possible for some publications that may be relevant to the field of assistive technology for the visually impaired and blind people and yet not be captured in the database. This is possible in rare cases, if the range of keywords and phrases used as search terms in the experiment does not match with any of the metadata information of the publication; or, if the publication is incorrectly labelled; or, if the descriptive stored metadata for a published document misleads the scientific database’s search engine; all of which are unlikely, but possible. Secondly, it is possible for a publication, whose subject matter may not be relevant to the field of assistive technology for the visually impaired and blind people research, and still have its source metadata captured in the database. This is possible but again unlikely because the descriptive stored metadata for each published document includes the title, publication title, abstract and author-defined keywords along with other information; and a scientific database’s search engine matches on each key word individually as well as on the whole phrase with the metadata information. However, none of the scientific databases can be said to be fully consistent and none of their search engines can be said to be free from errors; some degree of bias is present due to different referencing styles, typos, incorrectly recorded collaborations, citations, etc, as is reported in the analysis by Subelj et al. [106]. Hence, these limitations are unavoidable. We believe that the source metadata captured on our publications database is a reliable data of research being undertaken in the field of assistive technology for the visually impaired and blind people during the last two decades.
4 Analyzing the corpus: what areas does assistive technology for the visually impaired and blind people encompass?
A survey of topics reveals the big picture and provides a general introduction to the research field. Hence, the first question for our analysis concerning assistive technology for the visually impaired and blind people was: What are the individual topics that constitute the field of research, and how do they reflect the focus and breadth of the field? In order to answer this question, we extracted topics based on their frequency of occurrence in the ‘titles’ and ‘abstracts’ in the publication database through a word frequency counter written in EXtensible Stylesheet Language Transformations (XSLT), and listed them. We present a static word cloud visualization of these topics in Fig. 2 accompanied by the statistics for the 100 most common topics (summarized in Table 2).
A word cloud is a visual representation for text data that gives greater prominence to words that are more frequent in the source text. The words are placed in the cloud randomly, and the overall layout is determined by readability. More details on our word cloud construction and generation is discussed in Appendix B. Stop words and other non-informative words were excluded from the analysis.
Results Given that the database was constructed from scientific publications concerned with assistive technology for the visually impaired and blind people, it is not surprising to find that many of the leading concepts in the word cloud (Fig. 2) relate directly to the overall subject/research area. Furthermore, within the Assistive Technology research process words such as ‘systems’, ‘design’, ‘technologies’, ‘method’, and ‘applications’ (Table 2) are popular. Early and contemporary researches which have been recognized and published in (prominent) engineering, neuroscience, computer science and medicine journals and proceedings have sought to address the issues of accessibility, independent travel, information access, indoor and outdoor navigation for the visually impaired and blind people. The occurrence of words like ‘information’, ‘navigation’, ‘environment’, ‘objects’, ‘detection’, ‘aid’, ‘accessibility’, ‘access’, ‘indoor’, ‘obstacles’, ‘travel’, and ‘reading’, in the list reflects these efforts. The fields of sensor technology and computer vision have contributed to remarkable development in assistive technology systems for the visually impaired and blind people [77]. Leading terms from these fields such as ‘vision’, ‘sensory’, ‘learning’, ‘model’, ‘algorithm’, and ‘camera’, also appear. The prominent appearance of words ‘haptic’, ‘tactile’, ‘audio’, ‘auditory’, and ‘speech’ are due to the fact that these are the most popular alternate feedback modalities; though secondary auditory pathways such as bone conduction [10] have also been employed for navigation and obstacle avoidance purposes. Good user-centered design matches technology to the user’s needs. Usability testing or evaluation plays a vital role in the design and development of assistive technology. It is well known that lack of suitable interfaces, poor performance, slow user feedback, longer training times contribute to their lack of adoption [77]. Hence these words—‘interface’, ‘performance’, ‘feedback’, ‘training’ are prominent and appear in the list.
Other common topics from the word cloud again point to inspiration by advances in computer vision, machine learning, sensor processing and technology, human computer interaction, augmented reality, etc. Some terms are adopted directly from cognitive pyschology, such as ‘cognitive’, ‘map’, ‘space’, ‘spatial’, and ‘orientation’. This emphasis on cognitive psychology is consistent with the journals Neuroscience & Biobehavioral Reviews and ACM Transactions on Accessible Computing which have leading publications with one aspect of the research focus as cognition; specifically the psychological processes which have relevance to rehabilitation of the visually impaired people. In addition, a rich set of concepts from human-computer interaction (HCI) are also represented, including ‘wearable’, ‘tasks’, ‘model’, ‘service’, ‘interactive’, ‘camera’, ‘algorithm’, ‘display’, and ‘features’. These concepts are consistent with research on “Assistive Technology for the Visually Impaired and Blind people” being published in Computer Science and Human-Computer Interaction journals such as Accessibility and Computing which is the Newsletter of ACM SIGACCESS, International Journal of Human Computer Studies, and conferences such as ASSETS, CHI, and ICCHP.
5 Analyzing the corpus: where is research for assistive technology for visually impaired and blind people published?
Secondly, we had a strategic question: Which journals and conferences publish research on Assistive Technology for the Visually Impaired and Blind people? Identifying the leading journals and conferences which publish or disseminate knowledge in the field is crucial. First-time authors need to know the leading journals and conferences in the field (so as to follow or attempt to publish). Experienced authors also look for better publication opportunities. A sensitized research community will lead to more contributions and progress. To answer the above question, we queried the XML publication database we constructed at the beginning (Appendix A) with XPath expressions. We then explored more factors such as their scholarly reputation as indicated by the impact factor (IF), the proportion of the journal’s total published content that focuses on assistive technology for the visually impaired research, and the journal’s field of study.
From a total of 3010 publications in our database, 1106 (36.74%) were published in journals, 1723 (57.24%) in conference proceedings, and 181 (6.01%) in book chapters. Altogether, 351 distinct journals, 673 distinct conferences, and 78 distinct book titles had atleast one publication relevant to Assistive technology for the visually impaired and blind people. The 57–36% overall split between conferences and journals is less in comparison to the approximate split of 86–14% in IEEE Xplore, 85–15% in ACM Digital Library; while the journal to book chapter approximate split is 95–5% in Thomson Reuters Web of Science, and 72–28% in Elsevier ScienceDirect. Based on these statistics, “Assistive Technology for the Visually Impaired and Blind people” has a higher proportion of research published in conferences than in journals and book chapters. However it must be noted that there are considerable differences between the scientific databases in terms of coverage. IEEE Xplore includes high quality technical literature in engineering and technology and thus contains a high number of relevant content published and disseminated via conference proceedings; Elsevier ScienceDirect includes medical research as well and disseminates via journal articles and book chapters. The proportion of relevant publications captured from all four scientific databases is shown visually in a bar chart (Fig. 3).
Results Based on the coverage of the journals in the field in our publication database, the twelve leading journals in “Assistive Technology for the Visually Impaired and Blind people” are listed in Table 3. They were selected based on the total number of relevant publications published by these journals in the same field till 2014. Additional details, including, the 2013–14 SCI impact factor (IF), number of relevant publications till 2013 as compared to the total journal output, and the journal’s field of study are also shown in the table. The content of these journals are split over general areas of human-computer interaction and vision rehabilitation. At present ACM SIGACCESS Accessibility and Computing (SIGACCESS ACCESS COMPUT) is publishing the maximum number of publications per year relevant to assistive technology for the visually impaired and blind people research, followed by ACM Transactions on Accessible Computing (TACCESS). ACM SIGACCESS Accessibility and Computing is the newsletter of ACM SIGACCESS indexed by ACM Digital Library as a periodical/journal. The journal International Congress Series is discontinued as of 2008; while Procedia Computer Science focuses entirely on publishing high quality conference proceedings.
Based on our publication database, the twelve leading conferences publishing on Assistive technology for the visually impaired and blind people research are listed in Table 4, along with the conference full names and the number of relevant publications till 2014. They were selected based on the total number of relevant publications published by these conferences in the same field. The top three conferences in the field of assistive technology for the visually impaired and blind people research are the International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) with a total of 263 publications, ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) with 126 publications, and the Web for All (W4A) Conference with 65 publications. Clearly, the ASSETS conference dominates the research output produced in this field contributing about 15% of all conference publications relevant to the field. The W4A Conference focuses on wearable and mobile assistive technologies as well as Web accessibility for the blind. Six of the twelve leading conferences are ACM-sponsored, while five of them are IEEE-sponsored, which also indicates the emphasis of the applications of Assistive Technology for the Visually Impaired people in fields of engineering, technology and computing. The conferences listed have topics of interest such as assistive and rehabilitative technology, technologies for wellbeing, technologies which improve living environments, accessible computing, sensors, wearable computing, etc and have highly competitive publication requirements with acceptance rates for ASSETS-2015 being 24%, W4A-2015 being 35%, ICCHP-2015 being 50%, MobileHCI-2014 being 28%, and ISWC-2016 being 19%.
6 How rapidly is assistive technology for the visually impaired and blind people expanding as a research field?
Our next question concerns: How rapidly is Assistive Technology for the Visually Impaired and Blind people expanding as a research field? From adding up the number of publications each year in our database we note that there has been a remarkable growth in the field in the last two decades. The number of research publications per year roughly doubled every 4 years (Fig. 4). From a fledgling field in the mid-1990s with less than 50 publications per year, Assistive Technology for Visually Impaired and Blind people has expanded to become a mature field today reaching about 400 scientific publications per year in 2014. Over the last two decades the growth in this field has outpaced the growth of modern science, which is growing at 8–9% (doubling every 9 years) [14]. Today, the field is evolving swiftly; reliable and sophisticated computer vision algorithms can execute in realtime on embedded computers [109]. Smartphones come to the aid of the visually impaired and blind people (reducing the usual stigma attached to assistive devices). The miniaturization of processors and sensors would enable new wearable devices that may be embedded in clothing [112]. Advances in bionic vision offers hope for partial restoration of visual function in individuals in the near future [69]. This leads us to believe that the remarkable growth of the field is far from saturation. It is expected that rapid advances in computer vision, (wearable) sensor and systems, multisensory research and bionic vision will continue to drive this field further [69, 109].
Results Based on this initial analysis, it is safe to say that there has been a remarkable growth of interest in research on assistive technology for the visually impaired and blind people. Leading developments in this field have contributed to a greater understanding of the challenges and needs of the target community and laid the foundations for broader areas of present and future research. The changing research landscape is also evident today as the field is driven by diverse experts, practitioners, researchers, research groups and departments in leading universities and institutes. The expansion of research interest and developments in this field is also reflected in the increased number of publications within individual journals and conferences (Fig. 5). A great extent of this could be attributed to some relatively new journals and conferences in this area that have flourished in a short span of years, e.g. ACM Transactions on Accessible Computing (2008), Procedia Computer Science (2009), ACM Transactions on Applied Perception (2004), and the conference proceedings for International Conference on Pervasive Technologies Related to Assistive Environments (PETRA) (2008), International Convention on Rehabilitation Engineering and Assistive Technology (i-CREATe) (2007), Web for All Conference (W4A) (2004), etc. In other cases, this growth can be related to the change of focus of well-established peer-reviewed journals and conferences; journals such as—International Journal of Human-Computer Studies (1994), Personal and Ubiquitous Computing (1997), and conferences—IEEE Engineering in Medicine and Biology Society (EMBC) (1988), IEEE International Conference on Systems, Man and Cybernetics (SMC) (1988), and IEEE International Conference on Multimedia and Expo (ICME) (2000). The steady growth of the field appears to have been stunted in 2011 and since then there is a visible dip in journal output although conference outputs are unaffected.
Figure 5(left) reveals that ACM SIGACCESS Accessibility and Computing (SIGACCESS ACCESS COMPUT), ACM Transactions on Accessible Computing (TACCESS) and Procedia Computer Science (PROCEDIA COMPUT SCI) have been consistently publishing over the last decade, even though there appears a dip in TACCESS output after 2010 followed by a recovery, SIGACCESS ACCESS COMPUT and International Journal of Human Computer Studies (INT J HUM-COMPUT ST) has been consistent, while PROCEDIA COMPUT SCI appears to have random growth. The steady expansion of International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS), ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) and the Web for All Conference (W4A) is also clearly evident in Fig. 5(right), compared with the modest but appreciable growth of other conferences.
As developments in wearable computers, sensor processing and visual prostheses are translated into technology and commercialized, they have the potential to make long-term social and economic impacts on the community. The WIPO Patent Landscape Report [102] identified a total of 35,251 relevant patent documents on hearing and vision impairment technologies. 30% of these patents were filed between 2007 and 2011. This remarkable growth in patents on assistive technology for the visually impaired and blind individuals reflects the confidence of an inventor that a product has a fair chance to bring profit, as patents can be expensive to obtain and maintain. Recent patenting and innovation trends show the annual growth rate are highest in areas including ‘voice and sound control associated with vision assistance’ (22%) and ‘sensor technology adapted for vision impaired persons’ (10%); indicating that rapid development of new technologies derived from these areas are taking place. We believe that additional analysis of corporate developments and patents would prove to be useful and an important study.
7 Are there research communities within the field of assistive technology for the visually impaired and blind people?
One of the major questions in this analysis relates to the inter-connectedness of assistive technology for the visually impaired and blind people as a research field. A question of interest to researchers is—Whether the field functions as a coherent discipline or unites many disparate fields together into a coherent whole.
We note that there are several research groups and academic networks in assistive technology research, many of which have an active interest in the visually impaired community. To determine if there are sub-disciplines and communities and what these are, we considered a network theory analysis of the common nomenclature found in the titles and abstracts of published research. The result of the analysis is shown in Fig. 6. Two nodes are connected in the network if they co-occur within the database; with the size of the nodes proportional to its degree and the thickness of the edges proportional to the Log-likelihood Ratio scores. Modularity analysis detects the underlying community structures within the word co-occurrence graph. The coloring and positioning of the nodes in the co-occurrence graph are part of further analysis that we describe next.
To address the question, we construct a co-occurrence graph which is very helpful for a visual inspection of the connectedness of fields in research. Our corpus at this stage consists of titles and abstracts extracted from published research, so the word co-occurrence graph is given by all significant word co-occurrences found within these two contextual units of information; this resulted in an undirected graph with 135 nodes and 303 edges. These nodes represent terms more than those in Table 2. The connectedness between word pairs is quantified by their co-occurrence within the contextual units (title and abstract), defined by the log-likelihood ratio (LLR). The Log-likelihood Ratio basically examines the difference between alternate explanations (\(H_0\) and \(H_1\)) for the occurrence frequency of a word pair \(w_1\) \(w_2\). Under the null hypothesis \(H_0\), the words occur independently; under the alternate hypothesis \(H_1\), the words do not (Refer to Appendix B). The frequent terms are interpreted as nodes of the graph; the LLR score defines the strength of the connections between the nodes.
In order to highlight the research areas well, the top 50 most common word pairs are presented in Table 5. Some of these word pairs e.g. “touch screen”, “stereo earphones” etc, do not really reveal the underlying research areas well but do serve in interconnecting with other terms and in the community structure. Much of the other word pairs reveal the underlying interesting themes and sub areas. For e.g., “sensory substitution” which is basically a technique for replacing one of the disabled sensory input by other senses that are still intact. Sensory substitution studies the different non-surgical visual prostheses and is known to specialize in “object recognition”, “obstacle avoidance”, “vision rehabilitation”, etc, themes which were captured well in Table 5. There are also separate issues for the visually impaired people like “web accessibility” for blind web users, ensuring the blind are able to sense and walk within “pedestrian crosswalks” and multidisciplinary research areas like “bionic eyeglass” or bionic eye development and “vision rehabilitation” efforts.
These related terms are positioned together in a word co-occurrence graph for network analysis using the Gephi visualization tool [9]. First, the Force Atlas algorithm was used to produce a layout with the strongly connected nodes pulled together and the weakly connected nodes drawn apart. Consequently, the terms that tend to co-occur frequently within the contextual units are positioned closely on the graph while terms that co-occur infrequently are positioned apart. The layout result is seen in Fig. 6. The next step was to perform a modularity-based analysis of the network in order to discover and study community structures—i.e. natural divisions of nodes within which connections are dense but between which the connections are sparser. Gephi implements a popular modularity optimization method called the Louvain method, for community detection. In practice, modularity score (Q) values >0.3 are known to indicate significant community structures in the graph [35].
Results The modularity analysis applied to our graph resulted in a modularity score (Q) of 0.625; which implies that there are distinct communities. Four communities (C = 4) are evident from the modularity analysis, and Fig. 6 depicts each community node with a unique color designated by the community detection algorithm. Considering the labels, connections, and color codes as cues, we have the following interpretation of the four communities:
-
1.
Multisensory Research (Green)—A strong community with emphasis on several key areas such as sensory substitution and brain processes, computer vision, image processing and sensor signal processing, visual prosthesis, opthalmology and neuroscience, etc. Strong connections are seen in the community such as Computer-Vision, Low-Vision, Visual Impairment, Visual-Acuity apart from regular connections among other nodes. Methods include computer vision, ultrasonic sensors, tactile substitution via skin receptors, auditory substitution via the ears, vibro-tactile substitution, electrical stimulations (e.g. of the visual cortex [31], of the forehead [57], of the tongue [3]), medical or surgical approaches, etc. Applications include wearable assistive devices, mobility aids and electronic travel aids, visual prosthetic devices, brain implants, bionic eyes; for e.g. computer vision-based cortical implants for bionic vision [70, 71] and retinal implants, etc.
-
2.
Accessible Content Processing (Purple)—A community which attempts to cover urgent areas such as human computer interaction (HCI), Braille-related technologies, printed information access (such as textbooks, street signs, product information, bar codes), optical character recognition (OCR) technology, speech synthesis technology, sign detection and recognition, web accessibility, etc; Methods include tactile access, sonification of data (e.g. use of nonspeech sound for accessing geo-referenced data [115]), audio transcriptions of printed information, audio browsers to access web sites and web content [93]; Applications include reading devices, smartphone apps (e.g. the Trinetra project [65], or the bar code reading app for the blind from the Smith-Kettlewell Eye Research Institute [108]), low vision aids, screen readers, tactile touchscreens and tactile maps.
-
3.
Accessible User Interface Design (Blue)—We interpret this small community in the co-occurrence graph as an active community involved in areas such as human-computer interaction (HCI), interface design, user interface modelling, Braille technology, ubiquitous computing, human factors and ergonomics research, etc; Methods include user-centered design, auditory interactions and feedbacks, multiple accessibility features, design and usability evaluations; Applications include accessible user interface designs, frameworks for dual interfaces, non-visual interfaces to ubiquitous services (such as ATM machines, kiosks, home appliances), accessible games for the visually impaired [39], etc. An interesting application of an accessible user interface on mainstream technology is the EasySnap application for blind photography [55].
-
4.
Mobility and Accessible Environments Research (Red)—A wide community that is actively involved in research areas covering human-computer interaction (HCI), electronic travel aids, rehabilitation robotics, obstacle detection and avoidance, indoor and outdoor navigation, wayfinding, cognitive psychology, augmented reality (AR), etc; Methods include pattern recognition and machine learning, computer vision methods, ultrasonic audio feedbacks or echolocations, sensor-based techniques, global positioning system (GPS)-based navigation, spatial cognitive mapping and orientation; Applications include (wearable) electronic travel aids, mobility aids, smartphone apps, accessible public transport [12], navigation technologies, accessible airports and accessible offices (as proposed by the ARIANNA navigation system when GPS information is unavailable [24]), augmented reality systems, etc.
Overall, assistive technology for the visually impaired and blind people as such remains a very coherent discipline with growth in many sub-disciplines. Among these sub-disciplines, the field does form distinct communities as is confirmed by the observation that each of these communities have identifiable themes.
8 Where is assistive technology for visually impaired and blind people headed in the future?
Finally, we aim to identify—what are the emerging trends and discipline hierarchies that may influence the future direction of research in this field? This section covers recent trends, together with our interpretation of developments and recommendations, opportunities and challenges that will shape the near future of this field (Table 6).
Although the advancement of technology is evident, only a limited number of assistive technology solutions have emerged to make a social or economic impact and improve quality of life. Fundamental challenges e.g. enabling the visually impaired individuals and blind people to—localize themselves in an unknown indoor environment, retrieve lost/fallen objects, locate a person of interest, read notices, enjoy inclusion in society, or evacuate independently from a building in an emergency, are still to be thoroughly addressed in a practical and cost-effective manner. Researchers and technologists undoubtedly play a critical role in meeting the target community’s need but it is important to first understand the nature, scope, complexity, and diversity of the challenges faced by the visually impaired people in order to come up with an acceptable solution. The challenge further complicates because the target community is quite diverse in terms of degree of visual impairment, age, gender, ethnicity, and abilities. The causes of visual impairment are also diverse including uncorrected refractive errors, cataract, glaucoma, corneal opacities, age-related macular degeneration (AMD), and diabetic retinopathy, childhood blindness, trachoma, etc [29, 36, 113].
Sensory substitution has an enormous potential for visual rehabilitation of visually impaired and blind individuals [75]. Sensory substitution devices have been undergoing developments for many years in controlled settings in laboratories but are yet to be adopted for widespread use. Their adoption depends on deciding factors such as—performance of visual processing algorithms, interface, apparatus size, cost, amount of training, portability, etc [28, 77]. In recent years retinal implant technology or retinal prosthesis, or the development of “bionic eyes” is seen as a ground-breaking strategy for restoring some functional vision to visually impaired patients and thereby improve their independence and quality of life. Retinal prosthesis in the suprachoroidal space is expected to drive research in the near future because it is less invasive and therefore a viable clinical option. While cortical visual prostheses (Sect. 2) appears to remain mostly experimental and uncertain in the risk versus benefit balance, retinal prostheses appears to show more promise in accelerating the development of bionic eyes. To name a few examples—Bio-Retina Footnote 8 is an ultra small implantable chip that attaches to the retina and is powered by its own nanoelectrodes and photosensors; PRIMA [73, 80] is another promising miniaturized retinal prosthesis. A small number of private companies and research groups in laboratories are reported to be at an advanced phase of clinical trials [27, 75]. It remains to be seen whether retinal prostheses overcome the clinical, surgical, and engineering issues and finally transition from laboratories to industry. The most notable claim so far is of a patient who was able to discern the direction of white lines on a computer screen with Argus IIFootnote 9 [74]. Despite the accomplishments made since Humayun et al. [48] pioneered retinal implant technology and Sakaguchi et al. [95] and Zhou et al. [116] independently pioneered early engineering on suprachoroidal devices, the field of bionic vision is still considered a fledgling and promising field. The field is wide open and currently garners a lot of interest, effort and funding [27]. Research trends reveal a strong interest in the field in the near future. There are also sobering challenges: the surgical techniques for retinal prosthetic implantation are technically challenging and take long hours, signal and image processing challenges are significant, hardware technology is new and experimental, adverse biocompatibility issues such as risks of complications or infection still remain, conclusive outcomes from clinical trials are still awaited. Recent advances also point to several research groups actively pursuing prostheses based on stimulation of optic nerve, thalamus, and lateral geniculate nucleus (LGN) as they provide alternative stimulation targets [69, 88].
Computer vision integrated with assistive technology has tasted much success recently in the development of appropriate devices and interfaces for the visually impaired and blind individuals. Image description has been a long-standing challenge in computer vision; it is considered the hallmark of image understanding. A good image description is often said to “paint a picture in our mind’s eye” [21]. A blind guide describes the scene verbally (in natural language), for e.g. “The escalator is directly in front of you about 10 feet away. You’ll hear it as you approach”. A meaningful description provides the spatial awareness that is crucial for the formation or reinforcement of cognitive maps in the visually impaired or blind person. Mental imagery or the use of diagrams and pictorial representations held in the human memory is known to play a critical role in problem solving [66]. Physiologists have demonstrated that visually impaired and blind people have the capability to form cognitive maps of spaces, and this is also present to some extent in the congenitally blind people [50, 83]. Although non-visual senses are ‘inferior’ at encoding spatial information, yet studies [83, 110] confirm that the visually impaired and blind people have the potential to acquire concepts and develop maplike representations of space. This form of encoding requires more cognitive effort. Whether the representation is equivalent to those of the sighted people is logically answered by three broad theoretical positions—(i) the ‘deficiency’ theory (ii) the ‘inefficiency’ theory (iii) the ‘difference’ theory [34].
Kesavan and Giudice [62] were one of the earliest to note that automatic and meaningful natural language (NL) image description provided through a smartphone or other portable camera-equipped mobile device could assist blind people in knowing the contents of an indoor scene (e.g., room structure, furniture, landmarks, etc.) and support independent and efficient navigation of the space based on these descriptions. Computer vision and image understanding have taken remarkable steps forward; recent Convolutional Neural Networks based software such as [59, 101] can accurately describe scenes shown in photos in natural language. This opens up a host of possibilities; implementing deep learning in mobile contexts can be a game changing combination and could lead to a variety of mobile sensing tasks—such as scene, activity, emotion, and speaker recognition powered by deep learning. Lane and Georgiev’s exploratory study [64] is an early step in this direction. The authors applied deep neural network (DNN) to inference tasks such as activity, emotion recognition without over burdening the mobile hardware. There are alternate non-neural network approaches; such as Kernel canonical correlation analysis (KCCA)-based [45] and text-mined knowledge-based [63] novel algorithms for mapping images and videos to meaningful natural language descriptions respectively. Hands-free devices such as the Xowi Footnote 10 which engages the user in pure voice interactions have the potential to simplify many basic activities for the visually impaired and blind people. Image/scene descriptions in natural language (i.e. sentence-based image description) is the meeting point between state-of-the-art algorithms in mobile computer vision and off-the-shelf hardware (consumer smartphones or mobile devices) and promises to be a technology solution for the visually impaired people to help ambulate independently whether it is navigation online or in the real world.
Navigation and wayfinding particularly in unknown indoor scenarios is a long-standing challenge. There is a lack of established technologies capable to fill this need. It is dangerous for the visually impaired or blind individuals when they are in an unfamiliar building, e.g. a hotel or an airport, and they have no awareness of where the exits are, where the elevators are or if there is a wet floor. The NPR Labs Nipper One Footnote 11 is a telecommunications device designed to translate emergency radio broadcasts (e.g. in the event of a fire) and display onscreen text and flashing alert lights for the hearing-impaired. It is a more complicated case for individuals suffering from visually impairments or blindness. Wearable and unobtrusive indoor wayfinding system such as the smartphone-based system by Jain et al. [52, 53] (Roshni project) which downloads the floor plan of the building, localizes and tracks the user and provides step-by-step direction to the destination from any location inside the building using voice messages and vibration is a small step forward in this direction. Such a system could safely assist the visually impaired and blind individuals in scenarios requiring emergency evacuation from the building. ARIANNA [24] is another commendable smartphone based attempt to tackle the problem but the downside is that it will require retrofitting the whole floor layout with colored paths. Smartphones are expected to play a crucial role in the future in the daily lives of individuals with impairments, because many accessibility and social interaction barriers still remain. ETAs must be unobtrusive, light, discreet and effective if they are to be adopted; it should require minimum training time and give fast user feedback. These challenges are sure to push the boundaries of sensory substitution research even further. Aerial obstacles or head-height obstacles (e.g. overhanging signs, branches, open awnings) poses as another long-standing challenge for they are known to cause head injuries to the visually impaired. Saez et al. [94] has tackled the problem with 3D smartphones in the context of assisting the visually impaired but 3D smartphones are very expensive and still to reach the masses. Perceiving depth through monocular vision provided by consumer smartphones and camera-equipped mobile devices, for the purpose of detecting and avoiding aerial or frontal obstacles (as in miniature Unmanned Aerial Vehicles) is another direction of research that can empower the Visually Impaired and Blind people users to be aware of surrounding objects and avoid collisions.
9 Summary of main results
By applying an information analysis methodology and a set of transformations to our database (in XML) of publications on assistive technology for the visually impaired and blind people over the last two decades we could answer a set of strategic questions concerning the past, present and future of the research field.
First, what areas does Assistive Technology for Visually Impaired and Blind people encompass? From a total of 3010 scholarly publications (relevant to the field of Assistive Technology for the VI and Blind people) retrieved and downloaded from four major international scientific databases, we have extracted the 100 most frequent and informative topics in order to provide a bird’s eye view on the subject. The topics reveal leading concepts, terminologies and impact and contribution of one field on the other. The concepts are consistent with current research and also reveal the underlying thrust in the field.
Second, in which journals and conferences is research for assistive technology for the visually impaired and blind people published? We identified the leading (top 12) journals and conferences which publish or disseminate knowledge in the field. We provide additional information, such as the proportion of published content by journals and conferences and year besides their reputations and the coverage of research in scientific databases. A higher proportion of research is published in conferences than in journals and book chapters in this field.
Third, how rapidly is assistive technology for the visually impaired and blind people expanding as a research field? We have been able to capture interesting patterns of growth of the field over the last two decades (1994–2014). The growth of this field has outpaced the growth of modern science during the same period. We speculate on the causes of expansion of research interest and growth in this field. The publication trends of each of the leading individual journals and conferences over the last two decades is presented and it reveals the field is far from saturation. The remarkable growth in patents in the field is reported.
Fourth, what are the research communities within the field of Assistive Technology for VI and Blind people? We considered a network theory analysis of the common nomenclature to identify the top 50 most common word pairs (collocations). Word pairs reveal community structure besides the underlying themes and subdisciplines. We quantified the connectedness between word pairs by the log-likelihood ratio. Four distinct communities—multisensory research, accessible content processing, accessible user interface design, and mobility and accessible environments research are depicted with a unique colour each. The detection of communities and interpretation of the results are presented. We note that though there is growth in many sub-disciplines, the field remains a very coherent discipline.
Finally, where is assistive technology for VI and blind people headed in the future? Based on the analysis of past developments and the current research characterized by the sub-disciplines and communities obtained, we identified the major emerging trends and discipline hierarchies together with our interpretation of opportunities and challenges that will influence the near future of this field.
10 Conclusion
The needs of the visually impaired and blind people are greater than ever before. Assistive technology as a mature field will continue to gain prominence and impact the lives of the visually impaired and blind individuals (and the elderly people) in ways not previously possible. The increase in functionality of mainstream mobile technologies, advances in computer vision processing algorithms, miniaturization of electronic devices, and cutting-edge new medical interventions are expected to drive this field further towards the challenges and reality of creating successful assistive technology.
Notes
References
Abboud S, Hanassy S, Levy-Tzedek S, Maidenbaum S, Amedi S (2014) EyeMusic: introducing a visual colorful experience for the blind using auditory sensory substitution. Restor Neurol Neurosci 32(2):247–257
Ahuja AK, Dorn JD, Caspi A, McMahon MJ, Dagnelie G, daCruz L, Stanga P, Humayun MS, Greenberg RJ (2011) Blind subjects implanted with the Argus II retinal prosthesis are able to improve performance in a spatial-motor task. Br J Ophthalmol 95(4):539–543
Arnoldussen A, Fletcher DC (2012) Visual perception for the blind: the brainport vision device. Retin Physician 9(January 2012):32–34
Auvray M, Hanneton S, O’ Regan JK (2007) Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with the vOICe. Perception 36(3):416–430
Ayton LN, Blamey PJ, Guymer RH, Luu CD, Nayagam DAX, Sinclair NC, Shivdasani MN, Yeoh J, McCombe MF, Briggs RJ, Opie NL, Villalobos J, Dimitrov PN, Varsamidis M, Petoe MA, McCarthy CD, Walker JG, Barnes N, Burkitt AN, Williams CE, Shepherd RK, Allen PJ, Consortium ftBVAR (2014) First-in-human trial of a novel suprachoroidal retinal prosthesis. PLoS One 9(12):e115,239
Bach-y Rita P (1967) Sensory plasticity. Acta Neurol Scand 43(4):417–426
Bach-y Rita PW, Kercel S (2003) Sensory substitution and the human-machine interface. Trends Cogn Sci 7(12):541–546
Bach-y Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Nature 221(5184):963–964
Bastian M, Heymann S, Jacomy M (2009) Gephi: an open source software for exploring and manipulating networks. In: Third international AAAI conference on weblogs and social media
Belinky M, Jeremijenko N (2001) Sound through bone conduction in public interfaces. In: CHI ’01 extended abstracts on human factors in computing systems, CHI EA ’01. ACM, New York, pp 181–182
Bhowmick A, Prakash S, Bhagat R, Prasad V, Hazarika SM (2014) IntelliNavi: navigation for blind based on kinect and machine learning. In: Murty MN, He X, Chillarige RR, Weng P (eds) Multi-disciplinary trends in artificial intelligence, Lecture notes in computer science, vol 8875. Springer International Publishing, UK, pp 172–183
Bischof W, Krajnc E, Dornhofer M, Ulm M (2012) NAVCOM WLAN communication between public transport vehicles and smart phones to support visually impaired and blind people. In: Proceedings of the 13th international conference on computers helping people with special needs—Volume Part II, ICCHP’12. Springer, Berlin, pp 91-98
Borenstein J (1990) The NavBelt—a computerized multi-sensor travel aid for active guidance of the blind. In: CSUN’s fifth annual conference on technology and persons with disabilities, Los Angeles, pp 107–116
Bornmann L, Mutz R (2014) Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J Assoc Inf Sci Technol 66(11):22152222
Borodin Y, Bigham JP, Dausch G, Ramakrishnan IV (2010) More than meets the eye: a survey of screen-reader browsing strategies. In: Proceedings of the 2010 international cross disciplinary conference on web accessibility (W4A), W4A ’10. ACM, New York, pp 13:1–13:10
Brabyn JA (1982) New developments in mobility and orientation aids for the blind. IEEE Trans Biomed Eng BME 29(4):285–289
Brassai ST, Bako L, Losonczi L (2011) Assistive technologies for visually impaired people. Acta Univ Sapientiae Electr Mech Eng 3:39–50
Brindley GS, Lewin WS (1968) The sensations produced by electrical stimulation of the visual cortex. J Physiol 196(2):479–493
Calder D (2009) Assistive technology interfaces for the blind. In: 3rd IEEE international conference on digital ecosystems and technologies, 2009. DEST ’09. IEEE, Istanbul
Campbell M, Bennett C, Bonnar C, Borning A (2014) Where’s my bus stop?: supporting independence of blind transit riders with StopInfo. In: Proceedings of the 16th international ACM SIGACCESS conference on computers and accessibility, ASSETS ’14. ACM, New York, pp 11–18
Chen X, Zitnick CL (2015) Mind’s eye: a recurrent visual representation for image caption generation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 2422–2431
Coughlan JM, Shen H (2012) The crosswatch traffic intersection analyzer: a roadmap for the future. In: Miesenberger K, Karshmer A, Penaz P, Zagler W (eds) Computers helping people with special needs, Lecture Notes in Computer Science, vol 7383. Springer, Berlin, pp 25–28
Coughlan JM, Shen H (2013) Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. J Assist Technol 7(2)
Croce D, Gallo P, Garlisi D, Giarre L, Mangione S, Tinnirello I (2014) ARIANNA: a smartphone-based navigation system with human in the loop. In: 2014 22nd mediterranean conference of control and automation (MED), pp 8–13
Csapo A, Wersnyi G, Nagy H, Stockman T (2015) A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research. J Multimodal User Interfaces 9:275–286
Dagnelie G (ed) (2011) Visual prosthetics. Springer US, Boston
Dagnelie G (2012) Retinal implants: emergence of a multidisciplinary field. Curr Opin Neurol 25(1):67–75
Dakopoulos D, Bourbakis NG (2010) Wearable obstacle avoidance electronic travel aids for blind: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 40(1):25–35
Dandona L, Dandona R (2006) Revision of visual impairment definitions in the International Statistical Classification of Diseases. BMC Med 4:7
Dobelle WH (2000) Artificial vision for the blind by connecting a television camera to the visual cortex. ASAIO J (American Society for Artificial Internal Organs: 1992) 46(1):3–9
Dobelle WH, Mladejovsky MG, Girvin JP (1974) Artifical vision for the blind: electrical stimulation of visual cortex offers hope for a functional prosthesis. Science (New York, NY) 183(4123):440–444
Dorn JD, Ahuja AK, Caspi A, daCruz L, Dagnelie G, Sahel JA, Greenberg RJ, McMahon MJ, Group gIS (2013) The detection of motion by blind subjects with the Epiretinal 60-Electrode (Argus II) retinal prosthesis. JAMA Ophthalmol 131(2):183–189
Filipe V, Fernandes F, Fernandes H, Sousa A, Paredes H, Barroso J (2012) Blind navigation support system based on microsoft Kinect. Proc Comput Sci 14:94–101
Fletcher JF (1980) Spatial representation in blind children. 1: development compared to sighted children. J Vis Impair Blind 74(10):318–385
Fortunato S, Barthlemy M (2007) Resolution limit in community detection. Proc Natl Acad Sci 104(1):36–41
Foster A, Resnikoff S (2005) The impact of vision 2020 on global blindness. Eye 19(10):1133–1135
Freiberger H (1974) Mobility aids for the blind. Bull Prosthet Res, pp 73–78
Fusco G, Shen H, Murali V, Coughlan JM (2014) Determining a Blind Pedestrian’s Location and Orientation at Traffic Intersections. In: International conference, ICCHP : proceedings international conference on computers helping people with special needs, vol 8547, pp 427–432
Grammenos D, Savidis A, Stephanidis C (2009) Designing universally accessible games. Comput Entertain 7(1):8:1–8:29
Hakobyan L, Lumsden J, OSullivan D, Bartlett H (2013) Mobile assistive technologies for the visually impaired. Surv Ophthalmol 58(6):513–528
Hermann T, Hunt A, Neuhoff JG (2011) The sonification handbook. Logos, Berlin
Hersh MA, Johnson MA (eds) Assistive technology for visually impaired and blind people. Springer, London
Hersh MA, Johnson MA (2008) On modelling assistive technology systems part i: modelling framework. Technol Disabil 20(3):193–215
Hicks SL, Wilson I, Muhammed L, Worsfold J, Downes SM, Kennard C (2013) A depth-based head-mounted visual display to aid navigation in partially sighted individuals. PLoS One 8(7):e67,695
Hodosh M, Young P, Hockenmaier J (2013) Framing image description as a ranking task: data, models and evaluation metrics. J Artif Int Res 47(1):853–899
Hornig R, Zehnder T, Velikay-Parel M, Laube T, Feucht M, Richard G (2007) The IMI retinal implant system. In: Humayun MS, Weiland JD, Chader G, Greenbaum E (eds) Artificial sight biological and medical physics, biomedical engineering. Springer, New York, pp 111–128
Hoyle B, Waters D (2008) Mobility AT: the Batcane (UltraCane). In: Hersh MA, Johnson MA (eds) Assistive technology for visually impaired and blind people. Springer, London, pp 209–229
Humayun MS, deJuan E, Dagnelie G, Greenberg RJ, Propst RH, Phillips DH (1996) Visual perception elicited by electrical stimulation of retina in blind humans. Arch Ophthalmol (Chicago, Ill: 1960) 114(1):40–46
Humayun MS, Dorn JD, daCruz L, Dagnelie G, Sahel JA, Stanga PE, Cideciyan AV, Duncan JL, Eliott D, Filley E, Ho AC, Santos A, Safran AB, Arditi A, Del Priore LV, Greenberg RJ, Group AIS (2012) Interim results from the international trial of Second Sight’s visual prosthesis. Ophthalmology 119(4):779–788
Jacobson RD (1998) Cognitive mapping without sight: four preliminary studies of spatial learning. J Environ Psychol 18(3):289–305
Jacquet C, Bellik Y, Bourda Y (2006) Electronic locomotion aids for the blind: towards more assistive systems. In: Ichalkaranje N, Ichalkaranje A, Jain LC (eds) Intelligent paradigms for assistive and preventive healthcare, Studies in computational intelligence, vol 19. Springer, Berlin, pp 133–163
Jain D (2014) Path-guided indoor navigation for the visually impaired using minimal building retrofitting. In: Proceedings of the 16th international ACM SIGACCESS conference on computers and accessibility, ASSETS ’14. ACM, New York, pp 225–232
Jain D, Jain A, Paul R, Komarika A, Balakrishnan M (2013) A path-guided audio based indoor navigation system for persons with visual impairment. In: Proceedings of the 15th international ACM SIGACCESS conference on computers and accessibility, ASSETS ’13. ACM, New York, pp 33:1–33:2
Jameson B, Manduchi R (2010) Watch your head: a wearable collision warning system for the blind. In: 2010 IEEE Sensors, pp 1922–1927
Jayant C, Ji H, White S, Bigham JP (2011) Supporting blind photography. In: The proceedings of the 13th international ACM SIGACCESS conference on computers and accessibility, ASSETS ’11. ACM, New York, pp 203–210
Kajimoto H, Kawakami N, Tachi S (2002) Optimal design method for selective nerve stimulation and its application to electrocutaneous display. In: 10th symposium on haptic interfaces for virtual environment and teleoperator systems, 2002. HAPTICS 2002. Proceedings, pp 303–310
Kajimoto H, Kanno Y, Tachi S (2006) A vision substitution system using forehead electrical stimulation. In: ACM SIGGRAPH 2006 sketches, SIGGRAPH ’06. ACM, New York
Kajimoto H, Suzuki M, Kanno Y (2014) HamsaTouch: tactile vision substitution with smartphone and electro-tactile display. In: CHI ’14 extended abstracts on human factors in computing systems, CHI EA ’14. ACM, New York, pp 1273–1278
Karpathy A, Fei-Fei L (2015) Deep visual-semantic alignments for generating image descriptions. In: The IEEE conference on computer vision and pattern recognition (CVPR), Boston
Keller P, Stevens C (2004) Meaning from environmental sounds: types of signal-referent relations and their effect on recognizing auditory icons. J Exp Psychol Appl 10(1):3–12
Kercel S, Bach-y Rita P (2006) Human nervous system, noninvasive coupling of electronically generated data into. In: Wiley encyclopedia of biomedical engineering. Wiley, New York
Kesavan S, Giudice NA (2012) Indoor scene knowledge acquisition using a natural language interface. In: Workshop on spatial knowledge acquisition with limited information displays 2012, Kloster Seeon, Germany, vol 888, pp 1–6
Krishnamoorthy N, Malkarnenkar G, Mooney RJ, Saenko K, Guadarrama S (2013) Generating natural-language video descriptions using text-mined knowledge. In: Proceedings of the 27th AAAI conference on artificial intelligence (AAAI-2013), pp 541–547
Lane ND, Georgiev P (2015) Can deep learning revolutionize mobile sensing? In: Proceedings of the 16th international workshop on mobile computing systems and applications, HotMobile ’15. ACM, New York, pp 117–122
Lanigan PE, Paulos AM, Williams AW, Rossi D, Narasimhan P (2006) Trinetra: assistive technologies for grocery shopping for the blind. In: 2006 10th IEEE international symposium on wearable computers, pp 147–148
Larkin JH, Simon HA (1987) Why a diagram is (sometimes) worth ten thousand words. Cogn Sci 11(1):65–100
Lepora NF, Verschure P, Prescott TJ (2013) The state of the art in biomimetics. Bioinspir Biomim 8(1):013,001
Levesque V (2005) Blindness, technology and haptics. Center for intelligent machines, Technical Report, McGill University, UK pp 19–21
Lewis PM, Ackland HM, Lowery AJ, Rosenfeld JV (2015) Restoration of vision in blind individuals using bionic devices: a review with a focus on cortical visual prostheses. Brain Res 1595:51–73
Li WH (2013) Wearable computer vision systems for a cortical visual prosthesis. In: 2013 IEEE international conference on computer vision workshops (ICCVW), pp 428–435
Li WH (2014) A fast and flexible computer vision system for implanted visual prostheses. In: Agapito L, Bronstein MM, Rother C (eds) Computer vision—ECCV 2014 Workshops, Lecture notes in computer science, vol 8927. Springer International Publishing, pp 686–701
Linvill JG, Bliss JC (1966) A direct translation reading aid for the blind. Proc IEEE 54(1):40–51
Lorach H, Goetz G, Smith R, Lei X, Mandel Y, Kamins T, Mathieson K, Huie P, Harris J, Sher A, Palanker D (2015) Photovoltaic restoration of sight with high visual acuity. Nat Med 21(5):476–482
Luo YHL, Cruz Ld (2014) A review and update on the current status of retinal prostheses (bionic eye). Br Med Bull 109(1):31–44
Maidenbaum S, Abboud S, Amedi A (2014a) Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation. Neurosci Biobehav Rev 41:3–15
Maidenbaum S, Hanassy S, Abboud S, Buchs G, Chebat DR, Levy-Tzedek S, Amedi A (2014b) The EyeCane, a new electronic travel aid for the blind: technology, behavior and swift learning. Restor Neurol Neurosci 32(6):813–824
Manduchi R, Coughlan J (2012) (Computer) vision without sight. Commun ACM 55(1):96–104
Mankoff J, Fait H, Tran T (2005) Is your web page accessible?: A comparative study of methods for assessing web page accessibility for the blind. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’05. ACM, New York, pp 41–50
Manning CD, Schtze H (1999) Foundations of statistical natural language processing. MIT Press, Cambridge
Mathieson K, Loudin J, Goetz G, Huie P, Wang L, Kamins TI, Galambos L, Smith R, Harris JS, Sher A, Palanker D (2012) Photovoltaic retinal prosthesis with high pixel density. Nat Photonics 6(6):391–397
Meijer PBL (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39(2):112–121
Menzel-Severing J, Laube T, Brockmann C, Bornfeld N, Mokwa W, Mazinani B, Walter P, Roessler G (2012) Implantation and explantation of an active epiretinal visual prosthesis: 2-year follow-up data from the EPIRET-3 prospective clinical trial. Eye 26(4):501–509
Millar S (1988) Models of sensory deprivation: the nature/nurture dichotomy and spatial representation in the blind. Int J Behav Dev 11(1):69–87
Narasimhan P (2006) Assistive embedded technologies. Computer 39(7):85–87
Nau A, Hertle RW, Yang D (2012) Effect of tongue stimulation on nystagmus eye movements in blind patients. Brain Struct Funct 217(3):761–765
Preece A (2013) An evaluation of the RAY G300, an android-based smartphone designed for the blind and visually impaired. AccessWorld 14(July 2013)
Rabia J, Ali SA, Arabnia HR, Fatima S (2014) Computer vision-based object recognition for the visually impaired in an indoors environment: a survey. Vis Comput 30(11):1197–1222
Ren Q (2014) Visual prosthesis, optic nerve approaches. In: Jaeger D, Jung R (eds) Encyclopedia of computational neuroscience. Springer, New York, pp 1–3
Rizzo JF, Wyatt J, Loewenstein J, Kelly S, Shire D (2003) Methods and perceptual thresholds for short-term electrical stimulation of human retina with microelectrode arrays. Investig Ophthalmol Vis Sci 44(12):5355–5361
Roentgen UR, Gelderblom GJ, Soede M, de Witte LP (2008) Inventory of electronic mobility aids for persons with visual impairments: a literature review. J Visu Impair Blind 102(11):702–724
Roentgen UR, Gelderblom GJ, Soede M, de Witte LP (2009) The impact of electronic mobility devices for persons who are visually impaired: a systematic review of effects and effectiveness. J Visu Impair Blind 103(11):743
Roessler G, Laube T, Brockmann C, Kirschkamp T, Mazinani B, Goertz M, Koch C, Krisch I, Sellhaus B, Trieu HK, Weis J, Bornfeld N, Rothgen H, Messner A, Mokwa W, Walter P (2009) Implantation and explantation of a wireless epiretinal retina implant device: observations during the EPIRET3 prospective clinical trial. Investig Opthalmol Vis Sci 50(6):3003
Roth P, Petrucci L, Pun T, Assimacopoulos A (1999) Auditory browser for blind and visually impaired users. In: CHI ’99 extended abstracts on human factors in computing systems, CHI EA ’99. ACM, New York, pp 218–219
Saez JM, Escolano F, Lozano MA (2015) Aerial obstacle detection with 3-D mobile devices. IEEE J Biomed Health Inf 19(1):74–80
Sakaguchi H, Fujikado T, Fang X, Kanda H, Osanai M, Nakauchi K, Ikuno Y, Kamei M, Yagi T, Nishimura S, Ohji M, Yagi T, Tano Y (2004) Transretinal electrical stimulation with a suprachoroidal multichannel electrode in rabbit eyes. Jpn J Ophthalmol 48(3):256–261
Saunders AL, Williams CE, Heriot W, Briggs R, Yeoh J, Nayagam DAX, McCombe M, Villalobos J, Burns O, Luu CD, Ayton LN, McPhedran M, Opie NL, McGowan C, Shepherd RK, Guymer R, Allen PJ (2014) Development of a surgical procedure for implantation of a prototype suprachoroidal retinal prosthesis. Clin Exp Ophthalmol 42(7):665–674
Schmeidler E, Kirchner C (2001) Adding audio description: does it make a difference? J Vis Impair Blind 95(4):197–212
Shilkrot R, Huber J, Liu C, Maes P, Nanayakkara SC (2014) A wearable text-reading device for the visually-impaired. In: CHI ’14 extended abstracts on human factors in computing systems, CHI EA ’14. ACM, New York, pp 193–194
Shilkrot R, Huber J, Meng Ee W, Maes P, Nanayakkara SC (2015) FingerReader: a wearable device to explore printed text on the go. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, CHI ’15. ACM, New York, pp 2363–2372
Shoval S, Borenstein J, Koren Y (1998) Auditory guidance with the Navbelt-a computerized travel aid for the blind. IEEE Trans Syst Man Cybern Part C Appl Rev 28(3):459–467
Simonite T (2014) Google creates software that tells you what it sees in images
Solomon N, Bhandari P (2015) Patent landscape report on assistive devices and technologies for visually and hearing impaired persons. Tech. rep, World Intellectual Property Organization (WIPO)
Stingl K, Bartz-Schmidt KU, Besch D, Braun A, Bruckmann A, Gekeler F, Greppmaier U, Hipp S, Hrtdrfer G, Kernstock C, Koitschev A, Kusnyerik A, Sachs H, Schatz A, Stingl KT, Peters T, Wilhelm B, Zrenner E (2013) Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS. Proc Biol Sci R Soc 280(1757):20130,077
Stingl K, Bartz-Schmidt KU, Besch D, Chee CK, Cottriall CL, Gekeler F, Groppe M, Jackson TL, MacLaren RE, Koitschev A, Kusnyerik A, Neffendorf J, Nemeth J, Naeem MAN, Peters T, Ramsden JD, Sachs H, Simpson A, Singh MS, Wilhelm B, Wong D, Zrenner E (2015) Subretinal visual implant alpha IMS clinical trial interim report. Vis Res 111(Part B):149–160
Strumillo P (2010) Electronic interfaces aiding the visually impaired in environmental access, mobility and navigation. In: 3rd Conference on human system interactions (HSI), 2010, Rzeszow, pp 17–24
Subelj L, Bajec M, Mileva Boshkoska B, Kastrin A, Levnaji Z (2015) Quantifying the consistency of scientific databases. PLoS One 10:e0127390. doi:10.1371/journal.pone.0127390
Tapu R, Mocanu B, Tapu E (2014) A survey on wearable devices used to assist the visual impaired user navigation in outdoor environments. In: 2014 11th international symposium on electronics and telecommunications (ISETC), pp 1–4
Tekin E, Coughlan JM (2010) A mobile phone application enabling visually impaired users to find and read product barcodes. In: Proceedings of the 12th international conference on computers helping people with special needs, ICCHP’10. Springer, Berlin, pp 290–295
Terven JR, Salas J, Raducanu B (2014) New opportunities for computer vision-based assistive technology systems for the visually impaired. Computer 47(4):52–58
Thinus-Blanc C, Gaunet F (1997) Representation of space in blind persons: vision as a spatial sense? Psychol Bull 121(1):20–42
Upson S (2007) Loser: tongue vision—IEEE spectrum. http://spectrum.ieee.org/consumer-electronics/portable-devices/loser-tongue-vision
Velzquez R (2010) Wearable assistive devices for the blind. In: Lay-Ekuakille A, Mukhopadhyay SC (eds) Wearable and autonomous biomedical devices and systems for smart environment, Lecture notes in electrical engineering, vol 75. Springer, Berlin, pp 331–349
WHO (2015) WHO—causes of blindness and visual impairment. http://www.who.int/mediacentre/factsheets/fs282/en/
Ye H, Malu M, Oh U, Findlater L (2014) Current and future mobile and wearable device use by people with visual impairments. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’14. ACM, New York, pp 3123–3132
Zhao H, Plaisant C, Shneiderman B, Lazar J (2008) Data sonification for users with visual impairment: a case study with georeferenced data. ACM Trans Comput Hum Interact 15(1):4:1–4:28
Zhou JA, Woo SJ, Park SI, Kim ET, Seo JM, Chung H, Kim SJ (2008) A suprachoroidal electrical retinal stimulator design for long-term animal experiments and in vivo assessment of its feasibility and biocompatibility in rabbits. J Biomed Biotechnol 2008:547428
Zollner M, Huber S, Jetter HC, Reiterer H (2011) NAVI a proof-of-concept of a mobile navigational aid for visually impaired based on the microsoft Kinect. In: Campos P, Graham N, Jorge J, Nunes N, Palanque P, Winckler M (eds) Human-computer interaction INTERACT 2011, Lecture notes in computer science, vol 6949. Springer, Berlin, pp 584–587
Acknowledgements
The authors are grateful for insightful communications from—(Dr Nathan Lepora, University of Bristol and Bristol Robotics Laborotary) at the early stage of this work.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A: Database construction
Four major scientific databases—IEEE Xplore database Footnote 12, Thomson Reuters Web of Science Footnote 13, ACM Digital Library Footnote 14, and Elsevier ScienceDirect Footnote 15 were used as sources of quality multidisciplinary publications related to research in assistive technology for the visually impaired and blind people published in journals, conferences and book chapters. The bar chart in Fig. 3 illustrates the proportions of publications captured from the different scientific database sources. While the search within Web of Science, IEEE Xplore, and ACM Digital Library was restricted to a topical search from multiple fields containing relevant key words or phrases, the search within ScienceDirect was restricted to computer science, engineering, medicine, and design. A range of keywords and phrases were used as search terms including—wearable assistive devices, electronic travel aids, portable assistive device, navigation systems, assistive technology solutions, rehabilitation technology, vision substitution systems, etc all in the context of the blind and visually impaired. The search engines in the scientific databases matches on each key word individually as well as on the whole phrase with the descriptive stored metadata for each published document, which includes the title, publication title, abstract and author-defined keywords. Many articles on just pure medicine, neuroscience, psychology, general disability and impairments did appear in later search listings and were not considered as there was no any clear relation to engineering or technology. These terms restricted the type of research publications that we consider to that published in engineering, rehabilitation engineering, assistive and rehabilitation technologies, computer vision, augmented reality, virtual reality, sensor processing and technologies, cognitive science, vision rehabilitation and sensory substitution research.
The information analysis was carried out using Xpath (XML Path Language) and XSLT 2.0 (EXtensible Stylesheet Language Transformations), both of which are best suited for navigating, manipulating, querying, or transforming of XML data. It was first necessary to convert the results of publication search to a readable document in a non-proprietary format. This was achieved in two steps. First, the search results were exported to a personal library in Zotero (http://www.zotero.org), a powerful research tool to organize and analyze sources. Zotero comes with several styles for creating bibliographies. One can automatically export all of the references in a certain library in Zotero to specific bibliography formats like BibTeX or Endnote XML. XML is a markup language for representing structured information like documents, books, transactions, etc in a format that is both human-readable and machine-readable. XML markups are a form of document metadata.
A separate XML database was maintained for the search listings from each scientific database. Within XML, the metadata content concerning each publication is enclosed within standardized tags of elements. Hence, these were then accessible through XPath queries and expressions meant to match specific patterns and retrieve results. An XML editor provides the platform to perform XPath queries and regular expression searches and XSLT transformations meant for information analysis. Before the information analysis, it is essential to have the publication databases of metadata cleaned up of duplicate entries which arise from overlapping search terms. The database also needed to be preprocessed as it was found that unexpected characters, strings of symbols did creep into XML elements during the process of capturing the metadata from the source. The end goal is to have a well formed XML document for analysis.
Appendix B: Database analysis
The complete database with metadata content of publications was then analysed using a variety of methods. A survey of topics provides a broad introduction to the research area. Hence, the first step of analysis was to survey the publications by topic. As a useful starting point, we retrieved the specific fields i.e. titles and abstracts from the publication database through a pattern matching program in XSLT. Next we extracted common topics (based on their frequency of occurrence in the ‘titles’ and ‘abstracts’ of the publications) through a word frequency counter in XSLT and listed them. Unfortunately the most frequent words and collocations are found to be stop words. Stop words are extremely common words that are of little semantic value and so they must be excluded from analysis. Stop words like ‘the’, ‘and’, ‘of’, ‘have’ and non-informative words like ‘proposed’, ‘developed’, ‘approach’, ‘towards’, etc were compiled into a stop word list and filtered out. This left us with a refined list of most frequent words (Table 2 lists 100 most frequent words), that identifies the most interesting and informative words or topics in the field of assistive technology for the visually impaired and blind people today. The same corpus of title and abstract information from the publications database was used to generate a word cloud from WordleFootnote 16, a web-based tool. The static word cloud (Fig. 2) visualizes our analysis giving greater prominence to the more frequent words. Except for a mismatch of two words, the word list and rankings from our XSLT word frequency program matched that of Wordle’s program.
Our second step of analysis considered identifying the leading journals and conferences which publish or disseminate knowledge in the field of assistive technology for the visually impaired and blind people. These steps were performed with XPath queries (using path expressions) on the XML publication database. We gathered additional information, such as the proportion of published content by journals and conferences and year besides the reputation of these journals and conferences. The total number of journal articles, conference publications, book section publications are mentioned in Sect. 5. The analysis by journals and conferences are published in Tables 3 and 4 respectively; while the analysis by year is plotted in Fig. 4 and are discussed in detail in the main text of this article. This was followed by the adding up the number of publications each year in our database in order to determine the growth of journals and conferences in the field in the last two decades. The expansion of research interest and developments in this field is reflected in the increased number of publications within individual journals and conferences (Fig. 5).
Our next step was to consider the semantic environment of the most frequent words by analyzing their collocations. In corpus analysis, examination of word collocations is known to reveal the main underlying themes in the corpus. In this step, we employed a list (of most frequent words) longer than the one presented in Table 2. The goal was to obtain a fairly good overview of the main themes and sub-disciplines in the field of assistive technology for the visually impaired and blind people. Using absolute frequency as a collocational measure is the simplest method, but it is not recommended, as it does not lead to interesting results always. A more appropriate and widely accepted measure for statistical significance is the collocation metric based on Log-Likelihood Ratio. In collocation discovery the likelihood ratio examines the likelihoods \(L(H_0)\) and \(L(H_1)\) of two hypotheses \(H_0\) and \(H_1\) about words \( w_1 \) and \( w_2 \) [79]. If N is the total number of words in the corpus, and \(c_1\), \(c_2\), and \(c_{12}\) denote the occurrences of \(w_1\), \(w_2\), and bigram \(w_{1}w_{2}\); then maximum likelihood estimates for \(p = c_2/N \), \(p_1 = c_{12}/c_1 \) and \(p_2 = (c_2-c_{12})/(N-c_1)\). Assuming binomial distribution, the log of the likelihood ratio \( \lambda \) is calculated as follows:
where L(k, n, x) = \( x^{k}(1-x)^{n-k} \). On applying the log-likelihood test to our database consisting of preprocessed titles and abstracts from 3010 publications on the concerned field of research, we obtained the most interesting two-word collocations and ranked them based on their corresponding LLR scores (Table 5 lists 50 of them). The LLR score is simply a statistic; the higher the number the closer the candidate is to being a collocation. A trigram analysis could yield terms such as “electronic travel aid”, “object detection system”, or “low vision rehabilitation” which could be interesting for analysis. However we feel that a second-order analysis of terms may not necessarily lead to more insights or novel terms.
Community detection is a popular way for uncovering important structures and functions of complex networks. We performed a modularity-based analysis of the word co-occurrence graph (publication data) in order to detect underlying community structures. The network analysis tool Gephi Footnote 17 was used to visualize and understand the graph. Our word co-occurrence graph regards words as vertices and the LLR score of word pairs as edge weights. We constructed a temporal graph in the GEXF file format retaining these information. The Force Atlas algorithm was used to produce a layout with the strongly connected nodes pulled together (Fig. 6). Gephi’s Modularity statistic was used to detect communities. Force Atlas algorithm and the Modularity statistic are described more fully in the Gephi software’s documentation [9].
Rights and permissions
About this article
Cite this article
Bhowmick, A., Hazarika, S.M. An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends. J Multimodal User Interfaces 11, 149–172 (2017). https://doi.org/10.1007/s12193-016-0235-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12193-016-0235-6