Smart University: Software Systems for Students with Disabilities

Abstract

Smart universities, smart classrooms and smart education are the wave of the future in a highly technological society. One of the distinctive features of a smart university is its ability of adaptation to and smooth accommodation of various types of students/learners such as regular students and life-long learners, in-classroom/local and remote/online students/learners, regular students and special students, i.e. students with various types of disabilities including physical, visual, hearing, speech, cognitive and other types of impairments. This chapter presents the outcomes of an ongoing research project aimed at systematic identification, analysis, and testing of available open source and commercial text-to-voice, voice-to-text and gesture recognition software systems—those that could significantly benefit students with disabilities. Based on obtained outcomes of completed research and analysis of designated systems we identified and recommended top text-to-voice, voice-to-text and gesture recognition software systems for implementation in smart universities.

1 Introduction

Smart universities (SmU) and smart classrooms (SmC) can create multiple innovative opportunities for students to learn material and communicate to classmates in a variety of highly technological ways. In addition, they can give students who would normally not have access to these learning materials opportunities to interact with digital learning content as well as the instructors and other in-classroom and/or remote/online students. Although not designed or even conceptualized to benefit students with disabilities, this concept would definitely have an impact on the learning process and access to learning content for students with different types of disabilities.

1.1 Literature Review

1.1.1 Smart Classrooms: Literature Review

Pishva and Nishantha in [1] define a SmU as an intelligent classroom for teachers involved in distant education that enables teachers to use a real classroom type teaching approach to teach distant students. “Smart classrooms integrate voice-recognition, computer-vision, and other technologies, collectively referred to as intelligent agents, to provide a tele-education experience similar to a traditional classroom experience” [1].

Glogoric, Uzelac and Krco [2] addressed the potential of using Internet-of-Things (IoT) technology to build a SmU. “Combining the IoT technology with social and behavioral analysis, an ordinary classroom can be transformed into a smart classroom that actively listens and analyzes voices, conversations, movements, behavior, etc., in order to reach a conclusion about the lecturers’ presentation and listeners’ satisfaction” [2].

Slotta, Tissenbaum and Lui [3] described an infrastructure for SmUs called the Scalable Architecture for Interactive Learning (SAIL) that “employs learning analytic techniques to allow students’ physical interactions and spatial positioning within the room to play a strong role in scripting and orchestration”.

Koutraki, Efthymiou, and Grigoris [4] developed a real-time, context-aware system, applied in a SmU domain, which aims to assist its users after recognizing any occurring activity. The developed system “…assists instructors and students in a smart classroom, in order to avoid spending time in such minor issues and stay focused on the teaching process” [4].

Given all the research publications that focus on SmUs, no literature was located on “SmC’s software systems and students with disabilities” topic. This is the reason that this topic is in the center of our research activities.

1.1.2 Smart Universities: Literature Review

Coccoli in [5] argue that “…primary focus of SmU is in the education area, but they also drive the change in other aspects such as management, safety, and environmental protection. The availability of newer and newer technology reflects on how the relevant processes should be performed in the current fast changing digital era. This leads to the adoption of a variety of smart solutions in university environments to enhance the quality of life and to improve the performances of both teachers and students. Nevertheless, we argue that being smart is not enough for a modern university. In fact, all universities should become smarter in order to optimize learning. By “smarter university” we mean a place where knowledge is shared between employees, teachers, students, and all stakeholders in a seamless way” [5].

Aqeel-ur-Rehman et al. in [6] present the outcomes of their research on one feature of future SmU—sensing with RFID (Radio frequency identification) technology; it should benefit students and faculty with identification, tracking, smart lecture room, smart lab, room security, smart attendance taking, etc.

Lane and Finsel in [7] emphasize the importance of the big data movement and how it could help to build smarter universities. “Now is the time to examine how the Big Data movement could help build smarter universities—in situations that can use the huge amounts of data they generate to improve the student learning experience, enhance the research enterprise, support effective community outreach, and advance the campus’s infrastructure. While much of the cutting-edge research being done with Big Data is happening at colleges and universities, higher education has yet to turn the digital mirror on itself to innovate the academic enterprise” [7]. Big data analytics systems will strongly support inferring characteristic of a SmU.

Al Shimmary et al. in [8] analyzed advantages of using RFID and WSN technology in development of SmU. “The developed prototype shows how evolving technologies of RFID and WSN can add in improving student’s attendance method and power conservation”. RFID, WSN as well as Internet-of-Things technology are expected to significant parts of a SmU and strongly support sending characteristics of SmU.

Doulai in [9] presents a developed system for a smart campus. This system “… that offers an integrated series of educational tools that facilitate students’ communication and collaboration along with a number of facilities for students’ study aids and classroom management. The application of two widely used technologies, namely dynamic web-based instruction and real-time streaming, in providing support for “smart and flexible campus” education is demonstrated. It is shown that the usage of technology enabled methods in university campuses results in a model that works equally well for distance students and learners in virtual campuses”.

Yu et al. in [10] argue that “… with the development of wireless communication and pervasive computing technology, smart campuses are built to benefit the faculty and students, manage the available resources and enhance user experience with proactive services. A smart campus ranges from a smart classroom, which benefits the teaching process within a classroom, to an intelligent campus that provides lots of proactive services in a campus-wide environment”. The authors described 3 particular systems—Wher2Study, I-Sensing, and BlueShare—that provide sensing, adaptation, and inferring smart features of a SmU.

One area that so far has had a limited attention is “students with disabilities and SmU”. Although features, components, and systems of SmU taxonomy have been discussed in [11], only one publication could be located that discussed SmU, SmC, and students with disabilities [12]. Given that 10% of all school/college/university students have some kind of disabilities, this is definitely an area that needs a more thorough investigation.

2 Students with Disabilities and Software Systems

Categories of students with disabilities. Students of schools/colleges/universities may experience a variety of different categories of disabilities; they include but are not limited to:

• (1) Deaf/hearing impairments

• (2) Learning disabilities

• (3) Physical disabilities

• (4) Psychological/neurological disorders

• (5) Speech or language impairments

• (6) Visual impairments

• (7) Cognitive impairments

Software systems for students with disabilities. Software systems allow students with disabilities equal access in the classroom and learning environments. Often these systems also help them learn more efficiently and effectively and in many cases allow them to interact better with their professor and classmates. Where traditional classrooms do not specifically address software systems and how students with disabilities could be impacted, the implementation of specific advanced software systems in SmU and learning environments would definitely approach learning barriers from the perspective of universal accessibility: providing greater learning opportunities for all students in the SmU classroom—including students with disabilities.

A list of possible software systems that may benefit students with various types of disabilities are listed in Table 4.1.

Table 4.1 Types of students with disabilities and software systems that may be beneficial

Bradley University and students with disabilities. Bradley University (Peoria, IL USA) is a top-ranked private university that offers 5,400 undergraduate and graduate students opportunities and resources of a larger university and the personal attention and exceptional learning experience of a smaller university. Bradley offers more than 185 undergraduate and 43 graduate academic programs in business, communications, education, engineering, fine arts, health sciences, liberal arts and sciences, and technology.

The Center for Learning and Access (CLA) at Bradley University is the University’s primary academic support service responsible for helping students acquire skills essential to achieve academic and personal success (https://www.bradley.edu/offices/student/cla/). Under the CLA umbrella, the Office of Access Services currently serves approximately 310 students (or, about 6% of the total student number) that have provided appropriate documentation and registered for services.

In accordance with information from Bradley’s Center for Learning and Access (CLA) the current distribution of students with disabilities at Bradley by various designated categories is follows: (1) with a health impairment—19 students, (2) with a hearing impairment—6 students, (3) with learning disabilities—84 students, (4) with a physical disability—5 students, (5) with psych/neuro impairments—186 students, including 11 students with ASD (Autism Spectrum Disorder) and 61 students with ADHD (Attention Deficit Hyperactivity Disorder), (6) with a speech impairment—2 students, and (7) with a visual impairment—8 students.

The software systems currently in use at CLA by various categories of students with disabilities at Bradley University are summarized in Table 4.2.

Table 4.2 Software systems used by students with disabilities at Bradley University

The CLA specialists identified a list requirements for software systems to be used by students with various categories of disabilities—those features and functions should provide users with significant benefits; a list of CLA of most important requirements is presented in Table 4.3.

Table 4.3 CLA’s list of most important requirements to software systems for students with disabilities

The outcomes of our research as well as CLA requirements clearly shows that (1) text-to-voice (or, text recognition), (2) voice-to-text (or, speech recognition; also including captioning of all lectures and video materials), and (3) gesture (and, face) recognition systems are among most actively systems that may be used by students with disabilities in SmU. This is the main reason that during initial part of our project we focused research activities primarily on these types of software systems.

3 Project Goal and Objectives

The performed analysis of above-mentioned and multiple additional publications and reports relevant to (1) SmU, (2) university-wide smart software and hardware systems and technologies, (3) SmC, (4) smart learning environments, (5) smart educational systems, and (6) students with disabilities undoubtedly shows that SmU-related topics will be in the focus of multiple research, design and development projects in the upcoming 5–10 years. It is expected that in the near future SmU concepts and hardware/software/technological solutions will start to play a significant role and be actively deployed and used by leading academic institutions in the world.

Project Goal. The overall goal of this ongoing multi-aspect research project is a) to research and analyze various open source and commercial software systems in the areas of text-to-voice, voice-to-text, and gesture recognition, and b) identify top systems that could be recommended for implementation and active use in SmU and/or SmC to aid students with disabilities (and possibly students without disabilities).

The premise is that these software systems will make the curriculum more accessible for students with and without disabilities and will help traditional universities to understand the impact this software could have on the learning of students with disabilities and how this software could aid universities to a possible transformation from a traditional university into a smart one.

Project Objectives. The objectives of this project are

• (1) close collaboration with subject matter experts and identification of most desired features and functions for software systems to be used by students with disabilities in SmU and SmC;

• (2) extensive research and identification of available software systems in text-to-voice, voice-to-text, and gesture recognition areas;

• (3) identification and thorough analysis of available software systems in each designated area, including at least 10 commercial and 10 open-source systems,

• (4) identification of a list of most important (i.e. most useful for students with disabilities) features (functions) of existing software systems in each designated area;

• (5) perform analysis of most powerful (in terms of functionality) existing software systems in each area;

• (6) ranking of analyzed systems, i.e. identification of top 3 commercial and top 3 open-source systems among analyzed systems in each area, and

• (7) develop lists of open-source and commercial software systems in each designated area that are recommended for in-depth testing by actual students with various categories of disabilities and subject matter experts in smart classrooms and smart universities (and, probably, traditional universities).

The obtained research and analysis outcomes are presented below.

4 Research Outcomes: Analysis of Text-to-Voice Software Systems

There are many available text-to-voice software systems that could be implemented in a smart classroom within a SmU. This software will allow the user to convert text to voice so they can hear what information the text is trying to convey if they have issues with reading and comprehending text. Instead of students focusing on reading the text they can focus on comprehending it. For example, the act of reading for some students is a cognitive process. These students see words and have to figure out what letters are in the words, what the letters sound like, and what the actual word is so all there energy is spent on the task of reading, not comprehending the material. Using this software will make the material more accessible to the student with these difficulties. For other students, the actual act of reading is automatic and they can focus on comprehending what they are reading.

After investigating the desired features of text-to-voice software systems (Table 4.4) that, in our mind, should be available for students with disabilities in SmU, the next steps in our research and analysis project were:

• (1) Identification and thorough analysis of about 10 commercial and 10 open-source text-to-voice available software systems,

• (2) identification of a list of most important (i.e. most useful for students with disabilities) features (functions) of existing text-to-voice software systems,

• (3) examples of obtained analysis outcomes of powerful (in terms of functionality) text-to-voice existing software systems, and our ranking of those systems,

• (4) our recommendations, i.e. top 3 commercial and top 3 open-source text-to-voice software systems to be implemented and actively used in SmU.

The obtained research and analysis outcomes are summarized and presented in Tables 4.5, 4.6, 4.7, 4.8, 4.9, 4.10, 4.11, 4.12 and 4.13.

Table 4.4 A list of desired features of text-to-voice software systems for SmU

Table 4.5 Analyzed 10 commercial and 10 open source text-to-voice software systems

Table 4.6 A list of most important for SmC/SmU features in existing text-to-voice software systems

Table 4.7 Natural Reader [13] commercial text-to-voice system: the analysis outcomes

Table 4.8 Read the Words [15] commercial text-to-voice system: the analysis outcomes

Table 4.9 TextHelp Read&Write [22] commercial text-to-voice system: the analysis outcomes

Table 4.10 Balabolka [23] open-source text-to-voice system: the analysis outcomes

Table 4.11 Text-to-Speech Reader [24] open-source text-to-voice system: the analysis outcomes

Table 4.12 Text-to-Speech TTS [25] open-source text-to-voice system: the analysis outcomes

Table 4.13 Our recommendations: existing top 3 commercial and top 3 open-source text-to-voice software systems to be implemented and actively used in SmC/SmU

5 Research Outcomes: Analysis of Voice-to-Text Software Systems

There are many available voice-to-text software systems that could be implemented in a smart classroom within a SmU. This software will allow the user to convert their voice to text if they have issues with written expression. Instead of students focusing on the actual writing process they can focus their attention on producing a high quality product. For example, the act of writing for some students is a cognitive process. These students think of a word, have to think of the letters that make up this word, and then have to think of how the letter looks so they can retrieve it from memory and write it down. This process is very time consuming and by the time they have written a few words they have lost their thoughts on what they initially had planned to write. Using voice-to-test software systems will allow the student with a disability more access and the ability to produce higher quality written products. For other students, the actual act of writing is automatic (i.e., letter formation, word spellings, punctuation, etc.) and they can focus on the content of the message or assignment they are involved in writing.

Based on our current and past research project and obtained research outcomes, a generalized list of desired features of voice-to-text software systems for SmU is presented in Table 4.14.

Table 4.14 A list of desired features of voice-to-text software systems for SmU

After investigating the desired features of voice-to-text software systems (Table 4.14) that, in our mind, should be available for students with disabilities in SmU, the next steps in our research and analysis project were:

• (1) identification and thorough analysis of about 10 commercial and 10 open-source voice-to-text available software systems,

• (2) identification of a list of most important (i.e. most useful for students with disabilities) features (functions) of existing voice-to-text software systems,

• (3) examples of obtained analysis outcomes of powerful (in terms of functionality) voice-to-text existing software systems, and our ranking of those systems,

• (4) our recommendations, i.e. top 3 commercial and top 3 open-source voice-to-text software systems to be implemented and actively used in SmU.

The obtained research and analysis outcomes are summarized and presented in Tables 4.15, 4.16, 4.17, 4.18, 4.19,4.20, 4.21,4.22 and 4.23.

Table 4.15 Analyzed 10 commercial and 10 open source voice-to-text software systems

Table 4.16 A list of most important for SmC/SmU features in existing voice-to-text software systems

Table 4.17 Braina Pro [35] commercial voice-to-text system: the analysis outcomes

Table 4.18 Dragon Naturally Speaking Premium (Home Edition) [33] commercial voice-to-text system: the analysis outcomes

Table 4.19 Dragon Professional Individual [34] commercial voice-to-text system: the analysis outcomes

Table 4.20 Google Doc–Speech Recognition [52] open-source voice-to-text system: the analysis outcomes

Table 4.21 Windows Speech Recognition [43] open-source voice-to-text system: the analysis outcomes

Table 4.22 TalkTyper [45] open-source voice-to-text system: the analysis outcomes

Table 4.23 Our recommendations: existing top 3 commercial and top 3 open-source voice-to-text software systems to be implemented and actively used in SmC/SmU

6 Research Outcomes: Analysis of Gesture Recognition Systems

Gesture recognition software systems, in general, will allow the user to communicate with the machine naturally, using human-machine interface (HMI) and without any mechanical devices. For example, using the gesture recognition technology, it is possible to point a finger at the computer screen so that the computer cursor on a screen will move accordingly. Potentially, this technology could make conventional input devices such as mouse, keyboards and even touch-screens redundant. For the individual with any type of motor difficulties this could make a huge contribution to them having access to content in the Smart Classroom.

Currently, there are several gesture recognition software systems available that potentially in the future could be implemented in a smart classroom within a SmU. Unfortunately, most of them are not mature enough to be recommended at this moment for an implementation and active use in SmU.

Based on the outcomes of extensive literature review and creative analysis of existing gesture recognition systems, we arrived with a generalized list of desired features of gesture recognition systems suitable for SmU; those features are presented in Table 4.25.

After investigating the desired features of gesture recognition software systems (Table 4.24) that, in our mind, should be available for students with disabilities in SmU, the next steps in our research and analysis project were:

Table 4.24 A list of desired features of gesture recognition software systems for SmU

• (1) identification and thorough analysis of about 10 commercial and 10 open-source available gesture recognition software systems,

• (2) identification of a list of most important (i.e. most useful for students with disabilities) features (functions) of existing gesture recognition software systems,

• (3) examples of obtained analysis outcomes of powerful (in terms of functionality) gesture recognition existing software systems, and our ranking of those systems,

• (4) our recommendations, i.e. top 3 commercial and top 3 open-source gesture recognition software systems to be implemented and actively used in SmU.

The obtained research and analysis outcomes are summarized and presented in Tables 4.25, 4.26, 4.27, 4.28, 4.29, 4.30, 4.31, 4.32, 4.33.

Table 4.25 Analyzed 10 commercial and 10 open source gesture recognition software systems

Table 4.26 A list of most important for SmC/SmU features in existing gesture recognition systems

Table 4.27 Microsoft Kinect [53] commercial gesture recognition system: the analysis outcomes

Table 4.28 Intel Real Sense [54] commercial gesture recognition system: the analysis outcomes

Table 4.29 IISU [55] commercial gesture recognition system: the analysis outcomes

Table 4.30 Hand Vu [63] open-source gesture recognition system: the analysis outcomes

Table 4.31 FUBI (Full Body Interaction Framework) [64] open-source gesture recognition system: the analysis outcomes

Table 4.32 Wiigee [65] open-source gesture recognition system: the analysis outcomes

Table 4.33 Existing top 3 commercial and top 3 open-source gesture recognition software systems among analyzed systems

7 Research Outcomes: Strengths and Weaknesses of Tested Text-to-Voice and Voice-to-Text Systems

The next step of research and analysis was to

• (1) download the actual trial or demo versions of selected ranked software systems,

• (2) test and evaluate those systems against CLA requirements (Table 4.3), and

• (3) summarize lists of strengths and weaknesses of analyzed systems.

The outcomes of systems’ testing and lists of identified strengths and weaknesses of each system (using evaluation criteria from Table 4.3) are presented in Tables 4.34, 4.35, 4.36, 4.37.

Table 4.34 Strengths and weaknesses of ranked commercial text-to-voice software systems

Table 4.35 Strengths and weaknesses of ranked open source text-to-voice software systems

Table 4.36 Strengths and weaknesses of ranked commercial voice-to-text software systems

Table 4.37 Strengths and weaknesses of ranked open source voice-to-text software systems

Based on outcomes of the performed SWOT (Strengths-Weaknesses-Opportunities-Threats) analysis and obtained testing outcomes of designated systems, we recommend the following systems to be considered for an implementation, testing by actual students with disabilities of various categories, and active use in SmU, and, probably, traditional universities:

• (1) text-to-voice systems: Natural Reader [13] (about $ 70/copy) and Read The Words [15] (about $ 40/year) commercial systems, and Balabolka [23] and Text to Speech Reader [24] open source systems;

• (2) voice-to-text systems: voice-to-text systems: Braina [35] (about $ 30/year) and Dragon Naturally Speaking Home Edition [33] (about $ 100/copy) commercial systems, and Google Docs–Speech Recognition [52] and Windows Speech Recognition [43] open source systems.

8 Conclusions. Future Steps

To be successful in a college/university environment, students with disabilities need more support than students without disabilities. We believe the implementation of specific software systems in SmU and SmC is a key for this to happen. Software systems that address speech-to-text, text-to-speech, and gesture recognition will help students with disabilities to be more successful in the educational setting. In addition, students without any disabilities may benefit as well.

We are suggesting that SmC be equipped with various software systems so that all students (a) will have better access to the content being delivered, (b) be able to adequately interact with the professor and classmates, and (c) feel they are an integral part of the innovative learning environment—SmC in SmU.

Although not all university professors have knowledge or experience with students with disabilities, all of them should try to include them in the learning environment.

Conclusions. The performed research helped us to identify new ways of thinking about “students with disabilities in smart classroom and smart university environment” concept. The obtained research findings and outcomes enabled us to make the following conclusions:

• 1. Smart universities and smart classrooms can significantly benefit students with disabilities even though they are not the focus.

• 2. Many technologies that are geared towards students without disabilities will actually impact the learning of students with disabilities.

• 3. Some students with disabilities may need specialized technology to have equal access in the classroom.

• 4. Some technologies and software systems focusing on the success of students with disabilities may help students without disabilities to be successful.

• 5. There are a variety of commercial and open-source software systems in the areas of text-to-voice, voice-to-text and gesture recognition to aid students with disabilities.

• 6. Given the variety of commercial and open-source software systems an in-depth hands-on assessment of these systems by actual students with disabilities and subject matter experts should be conducted.

• 7. Each of the commercial and open-source software systems in the areas of text-to-voice, voice-to-text and gesture recognition have different features and capabilities.

• 8. More research and testing in real-world scenarios needs to be completed addressing commercial and open-source software systems in the areas of text-to-voice, voice-to-text and gesture recognition to decide which of them would have the most benefits for students with disabilities.

• 9. More research needs to be completed where actual students with disabilities experience and evaluate commercial and open-source software systems in various learning environments and scenarios.

• 10. More research needs to be completed that directly focuses on students with disabilities in smart classroom and smart university environment.

Next steps. The next steps of this multi-aspect research, design and development project deal with

• 1. Implementation, analysis, testing and quality assessment of numerous components of text-to-speech, speech-to-text, and gesture recognition software systems in Bradley Hall (the home of majority of departments of the College of Liberal Arts and Sciences) and in some areas of the Bradley University campus.

• 2. Implementation, analysis, testing and quality assessment of text-to-speech, speech-to-text, and gesture recognition software systems (a) in everyday teaching of classes in smart classrooms and (b) with actual students with disabilities.

• 3. Organization and implementation of summative and formative evaluations of local and remote students and learners with and without disabilities, faculty and professional staff, subject matter experts, administrators, and university visitors with a focus to collect sufficient data on quality of implemented text-to-speech, speech-to-text, and gesture recognition software systems.

• 4. Creation of a clear set of recommendations (technological, structural, financial, curricula, etc.) regarding a transition of a traditional university into a smart university pertaining to software and students with and without disabilities.