Abstract
Massive open online courses (MOOCs) have a prominent role in achieving universal e-education, i.e., education offered via the Internet to diverse learners around the world independently of their motivations, backgrounds, capacities, and limitations. Regrettably, current MOOCs platforms and contents are not accessible enough for all learners. This study presents the results of a systematic literature review on the combined field of accessible MOOCs that covers from the years 2008 to 2016. We followed a four-staged method than included a within-study and between-study literature analysis, and a descriptive synthesis. A total of 40 relevant studies was identified and mapped to eight research dimensions that form a lifecycle: problem characterization; needs identification; use of industry guidelines, specifications and standards; accessibility requirements specification; architectures; design strategies; verification of accessibility requirements compliance; and validation of user needs satisfaction. The results presented in this study give a head start to researchers interested in pursuing the combined field of accessible MOOCs, providers of MOOCs platforms and contents, as well as decision-makers of educational institutions that offer e-education can also benefit.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
This study presents the results of a systematic literature review performed to collect, comprehend, analyze, synthetize, and evaluate relevant literature to provide a foundation to the combined research field of accessible massive open online courses (MOOCs). The acronym MOOC was coined in 2008, to refer to the course “Connectivism and Connective Knowledge” offered by Stephen Downes and George Siemens from University of Manitoba, which attracted 2,200 online students. Since then, MOOCs have steadily increased their presence in digital learning becoming an important trend. Simply explained, MOOCs are online courses with unlimited number of participants and no entry requirements.
Due to their massive and open nature, MOOCs have a great potential to offer access to education to millions of people worldwide. Unfortunately, there is a contradictory situation between MOOCs pretending to democratize education while at the same time ignoring the need of making them accessible for all learners, including those with disabilities, elderly students, and foreign students.
Around 15% of the world’s population lives with some form of disability [1]. This makes this community the largest minority in the world. People with disabilities should be able to live as independently as possible and participate in all aspects of life, including education. In this context, this study is important because it presents the challenges that need to be addressed to make MOOCs truly accessible. The audience that may find useful the results presented in this systematic literature review includes researches, providers, and decision-makers from educational institutions. Researchers need to be aware of the existing work to advance this combined field. Providers can take into account the accessibility dimensions proposed in this study to improve the accessibility of their platforms and content. Decision-makers can set up institutional policies regarding the use of accessible MOOCs platforms and contents.
The rest of this paper is organized as follows: Section 2 presents the method used to perform the systematic literature review; Sect. 3 presents the results including the final documentary corpus and the mapping of studies to research dimensions; Sect. 4 presents the discussion of empirical results reported in the relevant studies and research challenges; Sect. 5 presents conclusions and future work.
2 Method
The method used in this study was a systematic literature review. The goal was to collect a set of relevant primary studies in the combined field of accessible MOOCs and provide a review of the empirical results reported in these studies. A primary study is a published research that presents original findings or data collected by the authors.
A systematic literature review addresses specific research questions and reduces bias in the review by being systematic and explicit about how the review is conducted. This involves using a stepwise approach and defining a research protocol [2–5]. The four stages followed in this study are shown in Fig. 1 and explained below.
2.1 Stage 1: Pretest literature reviews
This stage involved:
-
1.
Checking if previous reviews on the general field of MOOCs exist, using the pretest search string shown in Table 1;
-
2.
Performing general keyword searches in bibliographic and research databases to get an initial approximation of the size of the documentary corpus;
-
3.
Identifying specific keywords to include in the search string.
2.2 Stage 2: Initial search
This stage involved:
-
1.
Developing a research protocol by determining the set of clearly defined research questions and their motivations (as shown in Table 2), developing the search string (as shown in Table 3), identifying the bibliographic databases to be used in the searches: Scopus, Web of Science (WOS), Directory of Open Access Journals (DOAJ), and Education Resources Information Center (ERIC); establishing the period of time the review will cover: from the year 2008 to the year 2016 (the review starts this year because 2008 was the year the term MOOC was coined), and limiting the language: English;
-
2.
Conducting searches in the four bibliographic databases to get studies for the initial documentary corpus.
2.3 Stage 3: Literature selection
This stage involved:
-
1.
Analyzing the titles, abstracts, and keywords of the studies found in terms of the inclusion criteria: accepting only peer-reviewed journal articles and conference papers; and the exclusion criteria: rejecting studies with unrelated research topics or keywords out of topic;
-
2.
Minimizing the bias by having two researchers working independently in the literature selection, comparing their individual selections, and reach common agreements to refine the initial documentary corpus.
2.4 Stage 4: Analysis and synthesis
This stage involved:
-
1.
Obtaining the full content of the selected studies from the following research databases: IEEE Xplore Digital Library, ScienceDirect, Springer Digital Library, and ACM Digital Library;
-
2.
Analyzing the full content of the studies found in terms of the inclusion criteria: accepting as relevant studies those semantically related to accessibility of MOOCs for learners with disabilities;
-
3.
Applying a snowballing technique to expand the final documentary corpus: using Google Scholar (GS) to identify studies that have cited the studies already found and review their references, as explained by [6];
-
4.
Performing a within-study and between-study literature analysis in the final documentary corpus;
-
5.
Performing a descriptive synthesis to identify categories of analysis, e.g., year of publication, authors, institutional affiliations, countries, research dimensions;
-
6.
Answering the research questions.
3 Results
The systematic literature review was performed from April 2016 to December 2016. In this section, we present the findings of the pretest of existing literature reviews on MOOCs in general, the partial results of the search process, the final documentary corpus, the most researched dimensions, and the least researched dimensions.
3.1 Pretest literature reviews
In Stage 1, we applied the pretest search string defined in Table 1 and fetched a total of 15 literature reviews on the general field of MOOCs, corresponding to the time period 2013–2016: one review from the year 2013, five reviews from the year 2014, five reviews from the year 2015, and four reviews from the year 2016, as detailed in Table 4.
We reviewed these 15 literature reviews on the general field of MOOCs to check if they had explicitly considered as a dimension of study the accessibility for learners with disabilities; only six of them [13, 16–19, 21] had. This preliminary review also served as a starting point for the subsequent search of primary studies.
Balula covered the time period 2014–2015. This author selected 82 studies and identified three dimensions: MOOC design, accessibility, and employability. In accessibility, this review defined six subcategories: participants; financial issues; technical issues; interaction skills; digital skills; and language and cultural translation. In the subcategory participants, they briefly mention visual, hearing, and motor disabilities. This review concluded “Research as to MOOCs potentialities in terms of digital (and consequently social) inclusion is still sparse and somehow atomized” [13, p. 1].
Rolfe covered the time period 2009–2014. This author selected 68 articles and identified two categories: learning analytics and socio-ethical aspects. In socio-ethical aspects, this author included the subcategory learner diversity and equality. This review concluded “If MOOCs are to deliver their promises of inclusivity and equality, and if education providers do not wish to deviate from the academic and ethical values that are the mainstay of campus-based provision, the requirements of diverse learner groups is another area that should be more fully explored” [16, p. 63].
Sangrà et al. [17] covered the time period 2013–2014. They selected 228 studies and identified 11 categories, including a category about cultural and accessibility issues. They found out that this category was one of the least researched: only 7% of the reviewed studies considered it.
Liyanagunawardena and Williams covered the time period from the year 2008 to the year 2012. They identified only three studies regarding the accessibility needs of elderly learners in MOOCs. This review concluded “We have shown the lack of research into the use of MOOCs by elderly learners while at the same time establishing their presence in MOOCs” [18, p. 9].
Ossiannilsson et al. [19] covered the time period 2013–2015. They selected 22 studies and identified two categories: learning experiences and quality. Within quality, these authors included the subcategory inclusiveness in the sense of diversity of language, culture, setting, pedagogy, and technology.
Finally, Zancanaro and Domingues [21] covered the time period 2008–2014. They selected 294 articles and identified 9 categories, including one category about target public, in which only one study about accessibility of MOOCs for seniors was identified.
3.2 Initial search
In Stage 2, we performed searches in the bibliographic databases defined in the research protocol by applying the search string shown in Table 3. We fetched 388 studies. Table 5 shows the total number of studies identified, distributed per year, and source from the year 2009 to the year 2016. At this stage, we did not eliminate duplicate studies between sources. It is interesting to note that:
-
No studies were found in the year 2008;
-
A total of 10 studies were identified from the year 2009 to the year 2012;
-
The number of studies increased significantly from 2013 onwards;
-
The source with the highest number of studies found was Scopus with 235 studies, followed by DOAJ with 77 studies.
3.3 Literature selection
In Stage 3, we filtered 114 out of the 388 studies. Table 6 shows the total number of selected studies distributed per year and source. Again, at this stage we still did not eliminate duplicate studies between sources. It is interesting to note that:
-
No studies were selected from the year 2008 to the year 2010;
-
A total of four studies from the year 2011 to the year 2012 were selected;
-
The number of selected studies increased significantly from 2013 onwards;
-
The source with most selected studies was Scopus with 48 studies, followed by WOS with 39 studies.
The unrelated research topics or keywords out of topic of discarded studies were multiple: accessibility in other senses, e.g., open access to educational content for general users, access from developing countries, access for underserved populations; MOOCs that teach about accessibility; pedagogical issues, e.g., dropout rates, student’ s motivation and engagement, self-regulated learning, personal learning paths, student’s authentication, student’s behavior, student’s lived experiences, assessment at scale, instructors’ role.
3.4 Analysis and synthesis
In Stage 4, we applied the inclusion criteria and accepted 52 out of the 114 selected studies. Among these 52 studies, 20 duplications between Scopus and WOS were found and one quadruplication among Scopus, WOS, DOAJ, and ERIC (as shown in Table 8, column Sources). Hence, we eliminated a total of 23 repetitions, keeping 29 unique studies. Then, we applied the snowballing technique described by [6] using GS and found 11 additional relevant studies, given a total of 40 relevant studies. We used GS for snowballing because this database usually includes new research studies faster than other databases. These 40 relevant studies form the final documentary corpus.
Table 7 shows the total number of relevant studies distributed per year and source. It is interesting to note that:
-
No relevant studies were indicated in 2011;
-
A total of four relevant studies from the year 2012 to the year 2013 were found;
-
We found 18 relevant studies from the year 2014 to the year 2015, nine studies per each year;
-
The number of studies increased significantly in the year 2016;
-
The sources with most relevant studies were Scopus and WOS, with 25 studies each.
The next step was to perform a within-study and between-study literature analysis using the final documentary corpus and a descriptive synthesis to identify categories of analysis and answer the research questions defined in the research protocol (as shown in Table 2).
RQ1. How many relevant studies are there? What are their titles, authors, institutional affiliations, and countries? Which are the journals and conferences that have published them?
As already stated, the final documentary corpus has 40 relevant studies. Table 8 shows detailed information of each of them in chronological order including: sequential ID and reference number, short reference, first author’s affiliation ID (see Table 9), publisher ID (see Table 10 for journals and Table 11 for conferences), sources (Scopus or SCO, WOS, DOAJ, ERIC, GS), number of academic citations in SCO, WOS, and GS, and date of publication (Year/Month). Citations have been included to provide an indication of reach and impact.
Table 9 shows the institutional affiliations of the first authors with their correspondent relevant studies. The most active institutions until 2016 have been: Escuela Politécnica Nacional of Ecuador with 11 studies (27.5%), Universidad Nacional de Educación a Distancia UNED of Spain with seven studies (17.5%), the Open University of UK with six studies (15%), and the Oslo and Akershus University College of Applied Sciences of Norway with three studies (7.5%). The other institutions have contributed with one study each (2.5%).
As for countries, taking into account the institutional affiliations of all co-authors, Spain has participated in 25 studies (62.5%), Ecuador in 11 studies (27.5%), UK in eight studies (20%), USA and Germany in four studies each (10%), Norway in three studies (7.5%), Portugal in two studies (5%), and Saudi Arabia, Egypt, Austria, Italy, Guatemala, and South Africa in one study each (2.5%).
The fact that Spain has been leading the combined research field of accessible MOOCs is not surprising for two reasons.
First, according to the Open Education Consortium [62], as of December 2016, Spain has 19 members in this Consortium, followed by UK and France with 6 members each. Similarly, according to Open Education Europa [63], as of February 2016, Spain was the European country with the biggest MOOC offer with 481 courses, followed by UK with 427 courses, France with 269 courses, and Germany with 199 courses. Therefore, Spain is a very active country in the open education and MOOCs landscape.
Second, on the accessibility field, Spain became in 1988 one of the first countries worldwide to have a national standard for web accessibility: the norm UNE 139803 “Accessibility requirements for web content” [64]. In the same line, in 2002, Spain approved a mandatory regulation requiring public institutions, including educational institutions, to have accessible websites.
Table 10 shows the international peer-reviewed journals with their correspondent country, Journal Citation Report Impact Factor (JCR IF), SCImago Journal Rank (SJR), relevant studies, total number of studies, and frequency. The most active journal, until 2016, has been the Journal of Information Computer Science (J.UCS) with two relevant studies, one in the year 2016 and one in the year 2013.
Table 11 shows the international peer-reviewed conferences with their corresponding CORE ranking [65]. CORE provides assessments of computing conferences, which are assigned to one of the following categories: A*, A, B, or C. At the moment, the conferences that have published relevant studies are either ranked C or unranked. The four most active conferences have been: the International Conference on Computers Helping People with Special Needs (ICCHP) with four studies, two in the year 2016 and two in the year 2015; the Open Education Global Conference (OE Global) with three studies, two in the year 2016 and one in the year 2015; the International Symposium on Computers in Education (SIIE) with three studies, one in the year 2016 and two in the year 2014; the International Conference on Applied Human Factors and Ergonomics (AHFE) with two studies, one in the year 2016 and one in the year 2015; the IEEE International Conference on Information Technology Based Higher Education and Training (ITHET) with two studies, one in the year 2015 and one in the year 2013; and the International Conference on Quality and Accessibility of Virtual Learning (CAFVIR) with two studies in 2014. In these conferences, the main topics related to accessible MOOCs were usability and accessibility in MOOCs, universal learning design, inclusive virtual education, open education as strategy, design for all, universal usability, e-accessibility, e-ageing, and digital inclusion.
RQ2. What research dimensions have been tackled in the relevant studies? What are the most researched dimensions? What are the least researched dimensions?
Based on the technical processes proposed in the standard ISO/IEC/IEEE 15288:2015 “Systems and Software Engineering System Life Cycle Processes” [66], we defined a set of research dimensions using an iterative process that consisted in reviewing inductively the topics on each relevant study and mapping them to the proposed research dimensions. If a topic did not map to any currently existing dimension, a new dimension was created. At the end, we came up with eight dimensions that were organized in the logical order of a lifecycle as follows: problem characterization, needs identification, use of industry guidelines, specifications and standards, accessibility requirements specification, architectures for accessible MOOC platforms and courses, design strategies for accessible MOOC platforms and courses, verification of accessibility requirements compliance, and validation of user needs satisfaction.
Table 12 shows the mapping of the relevant studies and the dimensions. During the period 2012–2016, the most researched dimensions were: use of industry guidelines, specifications and standards with 26 studies, problem characterization with 25 studies, and needs identification with 23 studies. During the same period, the least researched dimensions were: architectures for accessible MOOC platforms and contents with 7 studies, design strategies for accessible MOOC platforms and contents with 11 studies, and accessibility requirements specification with 14 studies.
Figure 2 shows the distribution of relevant studies per year and research dimension. In the year 2012, the main research focus was in problem characterization with one study and needs identification also with one study. In the year 2013, the main research focus still was in problem characterization with three studies followed by needs identification and use of industry guidelines, specifications, and standards with two studies each. In the year 2014, the main research focus shifted to needs identification with seven studies followed by validation of user needs satisfaction and problem characterization with six studies each. In the year 2015, the main research focus was in the use of industry guidelines, specifications and standards with six studies, followed by accessibility requirements specification with five studies. In the year 2016, the main research focus was in the use of industry guidelines, specifications and standards, and the verification of accessibility requirements compliance with 12 studies each, followed by problem characterization with 11 studies.
4 Discussion
The number of relevant studies in the combined field of accessible MOOCs is still scarce, although it is continuously growing. Figure 3 shows data from years 2012–2016, showing the aggregated total number of relevant studies per semester. The dotted line depicts the approximation trend with a coefficient of determination which denotes a high level of predictability, i.e., it is expected that the number of relevant studies will continue to grow in the future.
To identify the research challenges in the combined field of accessible MOOCs, the eight dimensions were broken down into several subdimensions using content analysis. As a result, we identified a total of 58 subdimensions distributed among the dimensions, which are presented in detail in the accompanying digital datasetFootnote 1. Below, for each dimension, we discuss its importance, summarize the empirical results reported in the relevant studies, and present some associated research challenges.
MOOCs and the characterization of the accessibility problem
The importance of this dimension lies in the necessity that researchers have a correct understanding of the problem of accessibility in all its aspects.
The existing relevant studies have sufficiently explained the general concepts of accessibility, disability, diversity, universal access as well as the demographics [22–27, 29, 30, 33, 35, 38, 42–44, 46, 54, 56, 57, 60, 61]. In the same way, the Convention of the Rights of Persons with Disabilities [29, 44, 59] and related legislation had also been addressed, both in general [29, 44, 61] and in country-specific level, e.g., the US Rehabilitation Act Section 504 and Section 508 [34, 45], and the US Department of Education Universal Design for Learning Provisions in the Higher Education Opportunity Act [45].
Nevertheless, one challenge is the need of elaboration and implementation of legal regulations in more countries, especially developing ones, to establish as legal requirement the accessibility of e-Education for learners with disabilities. Another challenge is the establishment of government incentives to those who comply with these regulations. A final challenge is to demonstrate the social and institutional benefits of the implementation of accessible MOOCs platforms and contents in comparison with the associated costs [22, 25, 46, 53, 57, 61].
MOOCs and the identification of accessibility needs
This dimension is important because a complete identification of accessibility needs for different types of disabilities, alone and in combination, would make it possible to overcome the subsequent specification, design, and implementation of solutions in existing and future MOOC platforms and contents.
The existing relevant studies have made important efforts to identify the needs related mainly to blindness and low vision [27–31, 35, 45, 48, 53, 57, 59], deafness and reduced hearing [29, 45, 49, 57], and combined disabilities due to natural aging [23, 33, 51, 54].
However, more research is necessary to identify needs not only for those disabilities but also for other types of disabilities, e.g., speech [29], motor [29, 33, 57], cognitive [29, 33], psychosocial [29]. If some disabilities are left apart, there is no real accessibility. Another challenge is to understand accessibility needs that appear in the context of MOOCs’ use, e.g., cognitive issues experienced by non-native speakers [24, 26, 29, 43], cross-cultural issues [22, 24, 29], and needs related to the limitations of the technology available [22, 26, 29].
MOOCs and the use of industry accessibility guidelines, specifications, and standards
This dimension is important because even though there are well-known guidelines, specifications and standards related to web content accessibility, educational content accessibility, and mobile accessibility, these are largely ignored when it comes to developing MOOC platforms and contents. Hence, their general use must be promoted.
In addition to incorporate the use of the World Wide Web Consortium (W3C) Web Content Accessibility Guidelines (WCAG) in the context of MOOC [23, 24, 27–30, 33–36, 38, 39, 43, 44, 46–48, 50, 51, 55–57, 59, 60], other guidelines, standards, and specifications should be more deeply studied, e.g., the W3C Authoring Tool Accessibility Guidelines (ATAG) [35, 36, 44, 46, 50, 55, 59], W3C User Agent Accessibility Guidelines (UUAG) [35, 36, 44, 46, 55], W3C Website Accessibility Conformance Evaluation Methodology (WCAG-EM) [44, 46, 56], W3C Web Accessibility Initiative Ageing Education and Harmonization (WAI-AGE) [23], Guidance on Applying WCAG to Non-Web Information and Communications Technologies (WCAG2ICT) [44], W3C Mobile Accessibility Task Force [44], IMS Access for All (AFA) Personal Needs and Preferences (PNP) [41, 55, 57, 58], IMS AFA Digital Resource Description (DRD) [36, 41, 55, 58], IMS Learner Information Profile (LIP) [58], IMS AFA Learner Information Package Accessibility for LIP (ACCLIP) [36, 41, 55, 57, 58], IMS AFA Meta-data Information Model (ACCMD) [36, 41, 55, 58], IMS Accessible Portable Item Protocol (APIP) [36, 55], Learning Resource Metadata Initiative (LRMI) [41, 58], ISO 24751-2 Information technology—Individualized adaptability and accessibility in e-learning, education and training—Part 2 “Access for all” personal needs and references for digital delivery [36, 41, 43, 57, 58], ISO 24751-3 Information technology—Individualized adaptability and accessibility in e-learning, education and training—Part 3 “Access for all” digital resource description [41, 58], and ISO 9241-110 Ergonomics of human–system interaction—Part 110: Dialogue principles [37].
Among the challenges in this regard, it is necessary to use the guidelines, specifications, and standards beyond verification and validation of existing MOOC platforms and contents, and adopt them during the development of new MOOC platforms and contents.
MOOCs and the specification of accessibility requirements
This dimension is important because it focuses on the process of transforming the needs of diverse learners in engineered specifications of accessibility requirements, so further development of MOOCs platforms and contents takes them in account.
The existing relevant studies have mainly explored the specification of accessibility requirements for learning contents [28, 35, 36, 42, 46, 53, 55, 57, 59, 60], MOOC user interfaces [28, 35, 36, 48, 53, 55, 57, 59, 60], and MOOC platforms [28, 35, 36, 40, 46, 53, 55, 57]. Nevertheless, more research is needed for the specification of accessibility requirements at these levels and also for learning and assessment activities, online and offline, e.g., a chemistry course may require the learner to make an experiment at home and report the results on a discussion forum [40, 53]. Another challenge is the use of scenarios and persona descriptions as a mechanism for specifying requirements [39, 52].
Definition of architectures for accessible MOOC platforms and courses
This dimension is important because with proper architectures, existing and future MOOC platforms would be able to leverage accessible contents.
Existing relevant studies propose architectures that incorporate accessibility adaptation rules based on algorithms [35, 43, 44] to deliver both adaptive interfaces [35, 36, 44, 55, 58] and adaptive contents [35, 43, 44, 55, 58].
This is the least researched dimension, and there are several challenges to work on, e.g., the definition of comprehensive lifecycles or methods for developing accessible MOOCs [32], the implementation of adaptive learning paths [34], and the implementation of adaptive engines.
Design strategies for accessible MOOC platforms and courses
The importance of this dimension lies in the necessity of providing MOOC developers with proven design strategies. Existing relevant studies have explored strategies for profiling students [25, 35, 36, 41, 43, 44, 54, 55, 57, 58, 61] and educational content [41, 43, 44, 55, 58].
Nevertheless, there is a need for more research about profiling courses [43] and instructors [25] so recommender services can guide learners with disabilities through selecting MOOCs according to their specific needs. Other strategies that pose challenges include providing educational content in several alternative formats for different types of disabilities [43, 44, 55, 61], the implementation of on-the-fly video captioning based in speech recognition, user interface navigation with alternative mechanisms such as audio markers or screen overlays. Also, it is important to define design patterns to develop accessible MOOCs.
MOOCs and the verification of accessibility requirements compliance
This dimension is important because MOOC platforms and contents need to be tested for accessibility during the development process.
Existing relevant studies report results on testing accessibility at content level [33, 45–47] using heuristic evaluations [23, 30, 38, 46, 50, 53, 56], automated testing tools [27, 28, 33, 34, 38, 46, 51, 53, 56], disabilities simulators [28, 33, 46, 47], and manual testing [45, 48, 50, 53].
Nevertheless, more research is needed about new methods, tools and techniques for verification of accessibility requirements compliance at platform level [34, 47, 48, 50, 51, 53, 56, 58, 61] and content level [47, 53]. Also, it is important to research about accessibility testing in mobile contexts [44, 46, 54].
MOOCs and the validation of user satisfaction of accessibility needs
Finally, it is important to validate with users the level of accessibility of MOOC platforms and contents. Existing relevant studies report accessibility assessments from the perspective of learners, e.g., user testing [30, 31, 37, 38, 45–47, 51, 53, 54], study cases [23, 27, 28, 30–33, 37–39, 45].
It is important to also consider the instructors’ and providers’ perspectives regarding the level of accessibility when creating contents, as well as to have a holistic validation of the learning experience as perceived by the users. Another challenge is the definition of crowd-sourced mechanisms of validation, e.g., collecting and sharing participants accessibility reviews [47, 60].
5 Conclusion
This systematic literature review identified 40 relevant studies in the combined field of accessible MOOC that have been published between the year 2012 and the year 2016. Moreover, eight research dimensions and 58 subdimensions were also identified, and the relevant studies were mapped to them.
The main highlights of this review are summarized in the following:
-
The number of citations of the relevant studies hints to the fact that there is not enough impact within the research community. The top study as of 2016, as far as number of citations is concerned, is Santos, Boticario, and Pérez-Marín [32] with a total of 36 citations (9 in Scopus, 8 in WOS, and 19 GS), followed by Baker, Bujak, and DeMillo [22] with a total of 22 citations (8 in Scopus, 3 in WOS, and 11 in GS). From April 2015 onwards, there is neither Scopus nor WOS citations. From the year 2016, there is no GS citations either, with the exception of Iniesto and Rodrigo [47]. This might be due to the inherent citation latency, which is threefold: first, the time it takes from the publication in a journal or the proceedings of a conference to the inclusion in databases; second, the time needed to actually being cited in a new study; and third, the time needed from the publication of this new study to its inclusion in databases.
-
A total of 10 studies from the year 2009 to the year 2012 have been identified;
-
Most of the conferences that published the relevant studies are unranked. The novelty of the combined field of accessible MOOCs might be the reason for the current nonexistence of specialized conferences. Even in the general topic of MOOCs, there are few specialized conferences held annually, e.g., the European MOOCs Stakeholders Summit EMOOCsFootnote 2, MOOCs, Innovation and Technology in Education MITEFootnote 3, Learning with MOOCs ConferenceFootnote 4, Web Learning International MOOC ConferenceFootnote 5;
-
The summary of empirical results from the relevant studies gives a head start to researches interested in pursuing the combined field of accessible MOOCs, making it easier for them to define an initial roadmap and introduce themselves to the research community, e.g., know which researches to follow;
-
The research challenges identified in this study might also provide potential directions for future research in the combined field of accessible MOOCs.
To the authors’ knowledge, this is the first systematic literature review specific to the combined field of accessible MOOCs. The main strengths of this study are the time period coverage (from the inception of MOOCs in 2008 to December 2016) and the ample coverage of bibliographic and research databases.
As future work, we plan to keep the literature review dataset up to date and to explore additional categories of analysis to understand the evolution of this combined research field, e.g., the research teams composition: computer science teams, pedagogical teams, or mixed teams; the research methods followed in the studies: quantitative, qualitative, or mixed methodology; and the research techniques used, e.g., experimental, document analysis, content analysis.
References
World Health Organization. World report on disability (2011). http://www.who.int/disabilities/world_report/2011/en/. (2017/01/06)
Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences. Blackwell Publishing, Oxford (2006)
Kitchenham, B.: Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report (2007). https://www.cs.auckland.ac.nz/~norsaremah/2007GuidelinesforperformingSLRinSEv2.3.pdf (2017/01/06)
Staples, M., Niazi, M.: Experiences using systematic review guidelines. J. Syst. Softw. 80(9), 1425–1437 (2007)
Onwuegbuzie, A., Leech, N., Collins, K.: Qualitative analysis techniques for the review of the literature. Qual. Rep. 17(56), 1–28 (2012)
Jalali, S., Wohlin, C.: Systematic literature studies: database searches versus backward snowballing. In: Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 29–38. ACM (2012)
Liyanagunawardena, T.R., Adams, A.A., Williams, S.A.: MOOCs: a systematic study of the published literature 2008–2012. Int. Rev. Res. Open Distrib. Learn. 14(3), 202–227 (2013)
Ebben, M., Murphy, J.S.: Unpacking MOOC scholarly discourse: a review of nascent MOOC scholarship. Learn. Med. Technol. 39(3), 328–345 (2014)
Hew, K.F., Cheung, W.S.: Students’ and instructors’ use of massive open online courses (MOOCs): motivations and challenges. Educ. Res. Rev. 12, 45–58 (2014)
Jacoby, J.: The disruptive potential of the massive open online course: a literature review. J. Open Flex. Distance Learn. 18(1), 73–85 (2014)
Kennedy, J.: Characteristics of massive open online courses (MOOCs): a research review, 2009–2012. J. Interact. Online Learn. 13(1), 1–16 (2014)
Yousef A.M.F., Chatti, M.A., Schroeder, U., Wosnitza, M., Jakobs, H.: The state of MOOCs from 2008 to 2014: a critical analysis and future visions. In: Proceedings of the International Conference on Computer Supported Education, pp. 305–327. Springer (2014)
Balula, A.: The promotion of digital inclusion through MOOC design and use: a literature review. Indag. Didact. 7(1), 145–164 (2015)
Chiappe-Laverde, A., Hine, N., Martínez-Silva, J.A.: Literature and practice: a critical review of MOOCs. Comunicar 22(44), 9–17 (2015)
López-Meneses, E., Vázquez-Cano, E., Román, P.: Analysis and implications of the impact of MOOC movement in the scientific community: JCR and Scopus (2010–13). Comunicar 22(44), 73–80 (2015)
Rolfe, V.: A systematic review of the socio-ethical aspects of Massive Online Open Courses. Eur. J. Open Distance E-learn. 18(1), 52–71 (2015)
Sangrà, A., González-Sanmamed, M., Anderson, T.: Meta-analysis of the research about MOOC during 2013–2014. Educación XX1 18(2), 1–27 (2015)
Liyanagunawardena, T.R., Williams, S.A.: Elderly learners and massive open online courses: a review. Interact. J. Med. Res. 5(1), 1–11 (2016)
Ossiannilsson, E., Altinay, F., Altinay, Z.: Analysis of MOOCs practices from the perspective of learner experiences and quality culture. Educ. Med. Int. 52(4), 272–283 (2015)
Veletsianos, G., Shepherdson, P.: A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. Int. Rev. Res. Open Distrib. Learn. 17(2), 198–221 (2016)
Zancanaro, A., Domingues, M.: Analysis of the scientific literature on massive open online courses (MOOCs). RIED Revista Iberoamericana de Educación a Distancia 20(1), 59–80 (2017)
Baker, P.M., Bujak, K.R., DeMillo, R.: The evolving university: disruptive change and institutional innovation. In: Proceedings of the 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI), pp. 330–335 (2012)
Sanchez-Gordon, S., Luján-Mora, S.: Web accessibility of MOOCs for elderly students. In: Proceedings of the 11th International Conference on Information Technology Based Higher Education and Training (ITHET), pp. 1–6. IEEE (2013)
Sanchez-Gordon, S., Luján-Mora, S.: Accessibility considerations of massive online open courses as creditable courses in engineering programs. In: Proceedings of the 6th International Conference on Education, Research and Innovation (ICERI), pp. 5853–5862. International Association of Technology, Education and Development (IATED) (2013)
Dias, S.B., Diniz, J.A.: From blended to inclusive learning: accessibility, profiles, openness, and higher education. J. Univ. Comput. Sci. 19(18), 2722–2742 (2013)
Sanchez-Gordon, S., Luján-Mora, S.: MOOCs gone wild. In: Proceedings of the 8th International Technology, Education and Development Conference (INTED), pp. 1459–1468. International Association of Technology, Education and Development (IATED) (2014)
Calle-Jimenez, T., Sanchez-Gordon, S., Luján-Mora, S.: Web accessibility evaluation of massive open online courses on geographical information systems. In: Proceedings of the 2014 IEEE Global Engineering Education Conference (EDUCON), pp. 680–686. IEEE (2014)
Iniesto, F., Rodrigo, C., Teixeira, A.M.: Accessibility analysis in MOOC platforms. A case study: UNED COMA and UAbiMOOC. In: Proceedings of the V International Conference on Quality and Accessibility of Virtual Learning (CAFVIR), pp. 545–550 (2014)
Sanchez-Gordon, S., Luján-Mora, S.: Web accessibility requirements for massive open online courses. In: Proceedings of the V International Conference on Quality and Accessibility of Virtual Learning (CAFVIR), pp. 530–535 (2014)
Al-Mouh, N.A., Al-Khalifa, A.S., Al-Khalifa, H.S.: A first look into MOOCs accessibility: The case of Coursera. In: Proceedings of the 14th International Conference on Computers Helping People with Special Needs (ICCHP), pp. 145–152. Springer (2014)
Bohnsack, M., Puhl S.: Accessibility of MOOCs. In: Proceedings of the 14th International Conference on Computers Helping People with Special Needs (ICCHP), pp. 141–144. Springer (2014)
Santos, O.C., Boticario, J.G., Pérez-Marín, D.: Extending web-based educational systems with personalised support through User Centred Designed recommendations along the e-learning life cycle. Sci. Comput. Program. 88(1), 92–109 (2014)
Iniesto, F., Rodrigo, C.: Accessibility assessment of MOOC platforms in Spanish: UNED COMA, COLMENIA and Miriada X. In: Proceedings of the 2014 IEEE International Symposium on Computers in Education (SIIE), pp. 169–172. IEEE (2014)
Pascual, J., Castillo, C., García-Díaz, V., González, R.: Method for analysing the user experience in MOOC platforms. In: Proceedings of the 2014 IEEE International Symposium on Computers in Education (SIIE), pp. 157–162. IEEE (2014)
Sanchez-Gordon, S., Luján-Mora, S.: Adaptive content presentation extension for open edX. Enhancing MOOCs accessibility for users with disabilities. In: Proceedings of the 8th International Conference on Advances in Computer-Human Interactions (ACHI), pp. 181–183. International Academy, Research, and Industry Association (IARIA) (2015)
Rodrigo, C., Iniesto, F.: Holistic vision for creating accessible services based on MOOCs. In: Proceedings of the 2015 Open Education Global Conference (OE Global), pp. 1–5 (2015)
Yousef, A.M.F., Chatti, M.A., Schroeder, U., Wosnitza, M.: A usability evaluation of a blended MOOC environment: an experimental case study. Int. Rev. Res. Open Distrib. Learn. IRRODL 16(2), 69–93 (2015)
Sanchez-Gordon, S., Calle-Jimenez, T., Lujan-Mora, S.: Relevance of MOOCs for training of public sector employees: Enrollment, completion and web accessibility challenges. In: Proceedings of the 13th IEEE International Conference on Information Technology Based Higher Education and Training (ITHET), pp. 1–5. IEEE (2015)
Kelle, S., Henka, A., Zimmermann, G.: A persona-based extension for massive open online courses in accessible design. In: Proceedings of the 6th International Conference on Applied Human Factors and Ergonomics (AHFE), pp. 3663–3668. Elsevier (2015)
Draffan, E.A., Wald, M., Dickens, K., Zimmermann, G., Kelle, S., Miesenberger, K., Petz, A.: Stepwise approach to accessible MOOC development. In: Proceedings of the 13th European Conference on the Advancement of Assistive Technology (AAATE), pp. 227–234 (2015)
Iniesto, F., Rodrigo, C.: Accessible user profile modeling for academic services based on MOOCs. In: Proceedings of the XVI International Conference on Human Computer Interaction (Interacción), pp. 1–2. ACM (2015)
Sanchez-Gordon, S., Luján-Mora, S.: An ecosystem for corporate training with accessible MOOCs and OERs. In: Proceedings of the 2015 IEEE International Conference on MOOCs, Innovation and Technology in Education (MITE), pp. 123–128. IEEE (2015)
Sanchez-Gordon, S., Luján-Mora, S.: Accessible blended learning for non-native speakers using MOOCs. In: Proceedings of the 2015 IEEE International Conference on Interactive Collaborative and Blended Learning (ICBL), pp. 19–24. IEEE (2015)
Sanchez-Gordon, S., Luján-Mora, S.: How could MOOCs become accessible? The case of edX and the future of inclusive online learning. J. Univ. Comput. Sci. 22(1), 55–81 (2016)
Van Rooij, S.W., Zirkle, K.: Balancing pedagogy, student readiness and accessibility: a case study in collaborative online course development. Internet High. Educ. 28(580), 1–7 (2016)
Iniesto, F., McAndrew, P., Minocha, S., Coughlan, T.: The current state of accessibility of MOOCs: What are the next steps? In: Proceedings of the 2016 Open Education Global Conference (OE Global), pp. 8–14 (2016)
Iniesto, F., Rodrigo, C.: Can user recommendations be useful for improving MOOCs accessibility? A project for inclusive design and profitable feedback. In: Proceedings of the 2016 Open Education Global Conference (OE Global), pp. 1–7 (2016)
Sanchez-Gordon, S., Estevez J., Luján-Mora, S.: Editor for accessible images in e-Learning platforms. In: Proceedings of the 13th Web for All Conference (W4A’16), pp. 1–2. ACM (2016)
Gupta, P., Fatima, S.: Massive Online Course for Deaf and Dumb People. In: Proceedings of the 21st Western Canadian Conference on Computing Education (WCCCE), pp. 21–24. ACM (2016)
Sanderson, N.C., Chen, W., Bong, W.K., Kessel S.: The accessibility of MOOC Platforms from Instructors’ Perspective. In: Proceedings of the 10th International Conference on Universal Access in Human-Computer Interaction (UAHCI), pp. 124–134. Springer (2016)
Bong, W.K., Chen, W.: How Accessible Are MOOCs to the elderly? In: Proceedings of the 15th International Conference on Computers Helping People with Special Needs (ICCHP), pp. 437–444. Springer (2016)
Coughlan, T., Rodriguez-Ascaso, A., Iniesto, F., Jelfs, A.: OLA! A scenario-based approach to enhance open learning through accessibility. In: Proceedings of the 15th International Conference on Computers Helping People with Special Needs (ICCHP), pp. 445–452. Springer (2016)
Ferati, M., Mripa, N., Bunjaku, R.: Accessibility of MOOCs for Blind People in Developing Non-English Speaking Countries. In: Proceedings of the 8th International Conference on Applied Human Factors and Ergonomics (AHFE), pp. 519–528. Springer (2016)
Fernández, C., Esteban, G., Conde, M.Á., Rodríguez-Lera, F.J.: ICT for older people to learn about ICT: application and evaluation. In: Proceedings of the 2016 International Conference on Learning and Collaboration Technologies (LCT), pp. 292–302. Springer (2016)
Iniesto, F., Rodrigo, C.: Strategies for improving the level of accessibility in the design of MOOC-based learning services. In: Proceedings of the 2016 IEEE International Symposium on Computers in Education (SIIE), pp. 1–6. IEEE (2016)
Martín, J.L., Amado-Salvatierra, H.R., Hilera, J.R.: MOOCs for all: evaluating the accessibility of top MOOC platforms. Int. J. Eng. Educ. 32(5–B), 2374–2383 (2016)
Rodriguez-Ascaso, A., Boticario, J.G., Finat, C., Petrie, H.: Setting accessibility preferences about learning objects within adaptive elearning systems: user experience and organizational aspects. Expert Systems. pp. 1–12 (2016) (in Press)
Iniesto, F., Rodrigo, C.: A preliminary study for developing accessible MOOC services. J. Access. Des. All 6(2), 126–150 (2016)
Ngubane-Mokiwa, S. A.: Accessibility strategies for making MOOCs for people with visual impairments: a universal design for learning (UDL) perspective. In: Proceedings of 8th Pan-Commonwealth Forum on Open Learning (PCF8), pp. 1–12. Commonwealth of Learning (COL) and Open University Malaysia (OUM) (2016)
Osuna, S., Tejera, S.: ECO European project: inclusive education through accessible MOOCs. In: Proceedings of the 4th ACM International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’16), pp. 881–886. ACM (2016)
Iniesto, F., McAndrew, P., Minocha, S., Coughlan, T.: Accessibility of MOOCs: understanding the provider perspective. J. Interact. Med. Educ. 2016(1), 20 (2016)
Open Education Consortium. The global network for open education (2016). http://www.oeconsortium.org/about-oec/. (2017/01/06)
Open Education Europa. Open education scoreboard: Distribution of MOOCs by country (2016). https://www.openeducationeuropa.eu/. (2017/01/06)
Valderrama, J.A.: The new norm UNE 139803:2012. Accessibility assurance in web pages. Spain Association for Quality (2012). http://www.aec.es/c/document_library/get_file?uuid=dfa2b505-0605-49da-8ef4-fc091ef9114f&groupId=10128. (2017/01/06)
CORE. CORE rankings (2014). http://www.core.edu.au/conference-portal. (2017/01/06)
ISO. ISO/IEC/IEEE 15288:2015 Systems and Software Engineering-System Life Cycle Processes (2015)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sanchez-Gordon, S., Luján-Mora, S. Research challenges in accessible MOOCs: a systematic literature review 2008–2016. Univ Access Inf Soc 17, 775–789 (2018). https://doi.org/10.1007/s10209-017-0531-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10209-017-0531-2