1 Introduction

During the COVID-19 pandemic, organisations worldwide turned to the Internet to provide services to citizens [1, 2]. In some countries, essential procedures such as interacting with the administration, health services, or banks, were only avaiable online [3, 4]. Companies and individuals had to adapt to this situation during and after the pandemic through digitalisation [1, 5].

The percentage of the population with access to the Internet is increasing, although 33.7 % of the world’s population still does not have access to the Internet [6]. Therefore, a significant proportion of the population experienced serious barriers in coping with the exceptional situation imposed by the COVID-19 pandemic lockdown [7, 8]. The closure of COVID-19 presented an unprecedented challenge, particularly for higher education institutions, which had to turn to online platforms to continue their activities. This shift to online learning further highlighted the existing barriers that persons with disabilities face in accessing online education [9, 10].

Accessing the Web is not equal to everyone. About 16 % of the world’s population, estimated at 1.3 billion people, live with some form of disability and this value is increasing, in part due to the ageing of the population and the prevalence of non-communicable diseases [11]. Hence, web accessibility needs to be monitored to raise awareness for continuous improvement.

There are more reasons to monitor web accessibility. First, the performance of the Digital Accessibility Rights Evaluation Index (DARE) [7] has not improved significantly in recent years, although it is rising slightly [12]. Second, non-compliance with the Web Content Accessibility Guidelines (WCAG) has led to lawsuits, such as those against the University of California Berkeley [13] and Harvard University [14]. Third, the 2023 Digital Accessibility Index shows that only 3 % of the Web is accessible to persons with disabilities [15]. The key findings of this report are that around 56 % of images on corporate websites are not accessible to persons with visual impairment, 64 % of pages have links that are not clear to persons with some visual and cognitive impairments, and 25 % of forms lack clear labels. Fourth, the 2024 report of The WebAIM Million report shows that by February 2024, 95.9 % of the pages analyzed have some WCAG failure. [16].

Monitoring the evolution of web accessibility performance is of special importance for higher education. Quality education is one of the United Nations Sustainable Development Goals (UN-SDGs) [17]; because education enables upward socio-economic mobility and is a key to escaping poverty. In the information society, equal access to the Web is essential for inclusive and quality education. Improving web accessibility compliance can improve the web experience for all users, whether they have a disability or not [18, 19]. Furthermore, achieving the UN-SDGs requires good web accessibility policies and compliance [20].

An open question is whether the efforts made by universities to cope with the COVID-19 pandemic have continued after the pandemic has ended. This is the motivation for this research: its aim is to verify, from the point of view of web accessibility, how a group of universities responded to the constriction caused by the pandemic, whether they improved the accessibility of their web pages, and whether this improvement has been maintained over time. The review is based on the WCAG 2.1 [21] of the World Wide Web Consortium (W3C). Although WCAG version 2.2 was released in October 2023, no automated web accessibility evaluation tools (AWAETs) have implemented it yet [22].

As this research aims to assess web accessibility over a period of time, it needs to make use of a digital preservation project called the Internet Archive,Footnote 1 which is dedicated to preserving the websites of organisations around the world. The archived home pages of selected universities are retrieved from the Internet Archive’s Wayback Machine for all the years from 2018 to 2024.

Laws and international treaties such as the Convention on the Rights of Persons with Disabilities (CRPD) [18] enforce the implementation of web accessibility. The CRPD has conceptualised the accessibility of information and communication technologies (ICTs) as a prerequisite for the enjoyment of human rights and has led to a number of reforms in national legal frameworks worldwide. Many studies have evaluated the level of compliance of websites of different organisations. Most of these evaluations are very punctual in time, but this study evaluates the performance of web accessibility over a period of seven years (2018 to 2024): the two years before the COVID-19 pandemic, the two years during the pandemic and the two years after its end. This general objective is broken down into the following specific objectives:

  • To determine the web accessibility scores of selected web pages before, during and after the COVID-19 pandemic;

  • To evaluate the evolution of web accessibility errors of selected web pages during the period from 2018 to 2024;

  • To measure the level of implementation of Accessible Rich Internet Applications (ARIA) [23], on selected web pages during the period from 2018 to 2024.

Web accessibility can be evaluated in three main ways: by experts in the field, by AWAETs, and by user testing. The most widely used method, which is less affected by subjective opinions, is the use of AWAETs. However, a combination of automated and manual evaluation is recommended, as these AWAETs do not detect all possible web accessibility issues [24]. Nevertheless, to ensure that this study can be reproduced by other researchers, only AWAETs are used.

The remainder of this article is structured as follows. Section 2 describes the background information needed to understand web accessibility, its regulation around the world, and its relationship to the COVID-19 pandemic. Section 3 presents the methodology used to collect web accessibility information for the period before, during and after COVID-19. Section 4 presents the web accessibility results obtained from selected universities around the world in a way that is understandable to the audience, according to the objectives. Section 5 discusses the results and section 6 presents the conclusions, highlighting possible future work that could be derived from this study.

2 Background

2.1 Web accessibility regulation and legislation worldwide

The W3C periodically publishes WCAG, which remains the standard for evaluating web accessibility [25]. In 2008, WCAG version 2.0 was released and in 2012 it became the ISO/IEC 40500:2012 standard [26]. WCAG 2.0 states that a web page is accessible if it is perceivable, operable, understandable and robust. In June 2018, WCAG 2.1 was released [21]. To achieve these four principles, WCAG 2.1 provides 13 guidelines, 78 testable criteria and 3 levels of conformance (A, AA and AAA) [27]. WCAG 2.2 was also released in October 2023; however, no AWAET has implemented it yet.

The most important current web accessibility laws [28] are the Americans with Disabilities Act (ADA) and Section 508 from the United States of America, the Accessibility for Ontarians with Disabilities Act (AODA) from Canada, and the Web Accessibility Directive (WAD) and the European Accessibility Act (EEA) from the European Union. Some countries around the world have also made web accessibility mandatory by law and have established web accessibility certification programmes [29].

Web accessibility is already regulated in many countries around the world. Each country follows a normative hierarchy structure similar to Kelsen’s pyramid [30]. In the European Union, for example, this hierarchy is organised from top to bottom in the national constitution, European Union directives, international treaties, national laws and regulations. Accessibility regulation around the world could be placed at any level of this hierarchy [31]. For example, in Ecuador it is in the form of a regulation, “Reglamento Técnico Ecuatoriano RTE INEN 288" [32], in the European Union it is in the category of a directive [33], “Directive (EU) 2016/2102 on the accessibility of the websites and mobile applications of public sector bodies", in countries such as the United States, Australia, Argentina, Israel, Canada, Republic of Korea, Switzerland and others, it is in the form of a law. This law could be about persons with disabilities in general, like the “Section 508 Amendment to the Rehabilitation Act of 1973 from United States of America" [34] or the “Disability Discrimination Act 1992 from Australia" [35], or strictly related to web accessibility in particular, like “Ley de Accesibilidad de la Información en las Páginas Web de Argentina" [36]. However, some countries, such as Taiwan, Hong Kong, India and others, still regulate web accessibility in the form of policies, which may be translated into legislation in the future. Regrettably, numerous countries still lack any governmental policy regarding web accessibility [28].

2.2 COVID-19 and web accessibility

The COVID-19 pandemic crisis highlighted the importance of the Web as a means of communication between governments and their citizens. The imposition of quarantine during the pandemic forced human interaction to take place through ICT. Several universities and educational centres continued to offer their courses online, through the Web. However, some institutions were unable to do it due to economic reasons or the digital divide.

The post-COVID-19 era has normalised the Web as the main way for citizens to interact with e-government, e-health, e-education, e-commerce and e-entertainment services; it has also highlighted the importance of web accessibility in achieving the UN-SDGs and emphasised that the development must be inclusive. Of the 17 UN-SDGs that make up this agenda, there are several that contain specific references to persons with disabilities. These are: UN-SDG 4 on inclusive and quality education; UN-SDG 8 on economic growth, full employment and decent work; UN-SDG 10 on the reduction of inequalities; UN-SDG 11 on inclusive and sustainable cities and communities; and finally UN-SDG 17 on the implementation of the agenda and the necessary alliances [37].

2.3 Related works

Web accessibility of universities is a trendy topic and many researchers have conducted studies on it. The scope of this brief literature review is limited to the period from 2018 to 2024. In a previous work [38], we summarized 42 scientific papers about accessibility evaluation of university websites. Some of these studies have focused on evaluating general web accessibility issues of universities around the world [38,39,40,41,42,43,44]. However, the focus of this literature review will be on the core issue of this research, which is to determine to what extent web accessibility and its relationship to COVID-19 has been researched and what aspects are still missing and need to be addressed.

Acosta et al. [45] carried out an evaluation of the web accessibility of the universities of the European Union in times of COVID-19. The aim of this work was to evaluate the accessibility of the websites of the 1,584 best universities in the European Union according to the Webometrics ranking [46], using Pa11y, an AWAET based on WCAG 2.1. The results of the statistical analysis showed that the position in the ranking of European Union universities was not an indicator of the level of accessibility of their websites. It was also shown that the percentage of WCAG 2.1 compliance failures was very similar to the percentage of HTML coding standard failures. This study highlights the urgent need to include accessibility features in to the websites of European Union universities; therefore, web designers, developers and authorities of educational institutions must reflect and take decisions to eliminate accessibility barriers and deficiencies in their websites, since the right of access to education of persons with disabilities is being violated.

Doush et al. [47] carried out an evaluation of the web accessibility of Palestinian universities during the COVID-19 pandemic against WCAG 2.0. They found that the most violated guideline was empty link, which is related to success criterion 2.4.4 Link purpose. The second most violated error was linked image missing alternative text, which is related to success criterion 1.1.1 Non-text content. The obtained results show that the websites of selected university do not comply with WCAG 2.0 Level A.

Ennam [48] conducted and assessed the COVID-19 pandemic forced transition to distance e-learning in Moroccan universities. About 274 students were interviewed to collect and analyse their perceptions, reactions and attitudes towards the COVID-19. Much of the data collected testified to the inadequacy of online teaching/learning, mainly due to low web accessibility/affordability.

Russ et al. [49] investigated the problems faced by students and teachers during the abrupt transition to online education during the COVID-19 lockdown. They reviewed and analysed 14 papers on e-learning accessibility published over the past 11 years, in order to translate their findings into actionable recommendations for improving the accessibility of platforms at the time of the COVID-19 crisis, as well as for future pandemics. Their findings highlighted the need to build organisational cultures of accessibility, with support for educators as accessible content creators, and to raise awareness of the many types of disabilities that can affect students and how accessible content can prevent widening opportunity gaps.

Laamanen et al. [44] conducted an empirical study on the accessibility of Finnish higher education web pages. The aim was to examine the accessibility of all Finnish higher education landing pages based on WCAG 2.1 using a combination of two AWAETs. The results showed that the web pages of higher education institutions were not accessible and that there were enormous differences between the institutions.

Finally, an interesting study that deserves to be mentioned, despite the fact that it has nothing to do with universities and COVID-19, is a study carried out by Lazar et al. [50]. They evaluated the potential implementation of new metrics in AWAET for managers. Its significance in relation to this study is that they also used digital preservation, via the Wayback Machine, to automatically crawl and evaluate websites from the past in order to provide feedback to managers about the accessibility of their organisations’ websites.

In summary, this literature review on web accessibility and COVID-19 shows that most of these articles are categorized as cross-sectional studies, as there were focused on assessing the status of web accessibility in universities at a particular point in time. Those articles that analyzed a time period were basically a literature review of articles published on the subject during that same time period. Other articles presented the analysis of data collected by surveys. Therefore, there is still a gap of research for performance indicators of web accessibility of universities from past periods in general and during the COVID-19 pandemic in particular. Therefore, our study tries to fill this gap by performing a longitudinal study that shows the web accessibility performance of selected universities from 2018 to 2024, in order to observe its evolution before, during and after the COVID-19 pandemic.

3 Materials and methods

The method used to determine the web accessibility performance of universities from five continents around the world for the period from 2018 to 2024 is divided into six phases, as shown in Fig. 1.

Fig. 1
figure 1

Phases of the applied research methodology

Full size image

3.1 Phase 1: review bibliography on web accessibility and COVID-19

We conducted a comprehensive bibliographic review of research related to web accessibility of universities’s websites and COVID-19 in Scopus, Web of Science and Google Scholar. The aim of this review was to find out what has been done and what aspects still need to be researched. The search results were filtered by years between 2018 and 2024.

3.2 Phase 2: select universities to evaluate their web accessibility from 2018 to 2024

We chose a set of 25 universities from around the world. Six universities were pre-selected as this work is part of a project with them as partners. The rest of the universities were randomly selected. The source of the universities was Webometrics [46], although ShangaiRanking was also an option, but Webometrics was preferred because it is easy to search by continent. The procedure for random selection consisted of creating a random calculation in Microsoft Excel that ranged between number one and five, being five the maximum number of universities on each continent. Once the universities were selected, a web archive repository was needed to be able to retrieve websites from the past. The Wayback Machine holds historical website information for many organisations around the world and was chosen as the source of information for this research.Footnote 2

Table 1 Short names of universities that participates in the evaluation
Full size table

3.3 Phase 3: choose the automated web accessibility evaluation tool to be used

Many tools can be used to evaluate web accessibility [51]. In this research, Mauve++Footnote 3 and WAVEFootnote 4 were selected. Mauve++ was selected among them because of its ability to perform server-side rendering evaluations. In this research, server-side rendering was used because it provides more accurate evaluations. Using the Server Side Rendering Validation, the validation tool does not parse the static web page code, but it uses Selenium to load the HTML and CSS code in a headless version of the Chrome browser. In this way, it simulates the loading phase as if the page were open in the user’s browser and then performs the validation on the DOM of the resulting page obtained by also performing the scripts included in it. Mauve++ supports WCAG 2.1 (conformance levels A, AA and AAA). Mauve++ provides two evaluation results, by technology and by success criteria. This research considered only the results by success criteria.

The WAVE tool was chosen for the evaluation of web accessibility errors presented in the web page and the level of ARIA implementation. Having good ARIA implementation is an indication that the organisation and its web developers have considered the need for robustness of the website so that it can communicate the roles, states and properties of user interface elements to assistive technologies.

There was a need to set a threshold score for a university website to be considered accessible; in this research this score was set to 80 for WCAG 2.1 and conformance level AA. Here we considered that 20 % of non WCAG conformance cause 80 % of web accessibility barriers encountered by persons with disabilities. This is an arbitrary score set for this research based on Pareto’s principle [52, 53]. In order to categorise performance over the years, this research sets three levels of performance (poor, partial and good). Scores below 50 were considered poor performance, scores between 51 and 80 were considered partial performance, and scores above 80 were considered good performance.

3.4 Phase 4: perform web accessibility evaluation from 2018 to 2024

The evaluations were carried out on the archived home pages of the selected universities using the Wayback Machine of the Internet Archive’s Wayback Machine,Footnote 5 which is a digital preservation repository that holds copies of websites from previous years for many websites on the Internet. The home pages of the selected universities were evaluated for the period from 2018 to 2024. In this research, 2018 to 2019 represents the pre-COVID-19 period, 2020 to 2021 represents the COVID-19 period, and 2022 to 2024 represents the post-COVID-19 period. This research used the last backup of the year for the university’s website as the primary source for evaluation, i.e. the last backup of December; however, if this backup was not available, the source was the next backup going backward through the months of the year.

3.5 Phase 5: present and discuss results aligned to the objectives

We present the results and discussion in relation to the objectives of this research. To improve the readability of the results presented, the names of the universities were shortened as shown in the Table 1, composed of the short name of a university followed by the country code. The results are grouped and presented by continent to facilitate understanding and to avoid overwhelming the reader with a lot of information in one place. Finally, these results are discussed, including a final comparison of web accessibility performance by continents. The content of this phase is presented in Section 4 and Section 5.

3.6 Phase 6: establish conclusions from obtained results and recommend future work

We draw conclusions from the results presented in phase 4 and the discussion made in phase 5. Finally we suggest future work that could be derived. The content of this phase is presented in Section 6.

4 Results

What follows are the results of the web accessibility scores, errors and ARIA implementations of selected universities from around the world for the period from 2018 to 2024. These results are derived from phase 4 of the methodology shown in Fig. 1 and are presented grouped by the continent that the universities belong, in alphabetic order.

The values from 2018 to 2019 represent the pre-COVID-19 period; 2020 to 2021 represent the COVID-19 pandemic period; and 2022 to 2024 represent the post-COVID-19 pandemic period. A total of 175 evaluations (25 universities by 7 years) were conducted using two AWAETs.

The Mauve++ tool was used to determine the web accessibility score, and WAVE was used to collect web accessibility errors and the number of ARIA implementations. These results are grouped by continent and presented in alphabetical order. The web accessibility score is represented by the acronym AC, the errors by ER and the ARIA by AR. The names of the universities are also replaced by the corresponding short names presented earlier in Table 1.

4.1 Africa

Table 2 presents the results of web accessibility scores, errors and ARIA of universities from Africa, for the period from 2018 to 2024. This table comprises five columns; the first column is the short name of the university being evaluated, the second column is the archived URL of the its home page, the third column is the web accessibility score of that home page, the fourth column is the number of web accessibility errors encountered, the fifth column is the number of ARIA implementations found and the sixth column is the number of contrast errors. The archived URL is the one that appears in the “timestamp” of the page file in the Internet Archive. For example, the first URL has 20181228231929, which means it was archived in December 28, 2018 at 23:19:29. This table structure is the same for all continents and will be used in Tables 3,  4,  5 and  6.

Most of the web accessibility scores of African universities are lower than 80, which is the established threshold, which means that either they are partially or poorly accessible. From the five universities evaluated, only one can be considered as having good level of web accessibility as achieved scores higher than, and for three consecutive years. The values of ARIA implementation are low, although web accessibility errors prevail throughout the period.

Table 2 Web accessibility data from selected African universities for the period from 2018 to 2024
Full size table

Figure 2 shows the performance of African universities in the form of a line chart for the period from 2018 to 2024. An increase in web accessibility score or ARIA represents a good performance, while an increase in errors represent a decrease in performance.

Most of these universities show low performance over the years, and only the University Cheikh Anta Diop de Dakar (UCAD-SN), Senegal, maintained a good web accessibility performance from 2018 to 2021; however, its performance decreased after the end of COVID-19. The performance of the University of Johannesburg (UJ-ZA), South Africa, was good before the start of the COVID-19 pandemic; although it has made good efforts to reduce the number of errors, ARIA and web accessibility have not improved.

Fig. 2
figure 2

Web accessibility performance of African universities for the period from 2018 to 2024

Full size image

4.2 America

Table 3 shows the results of web accessibility scores, errors and ARIA of universities from America for the period from 2018 to 2024. Most of the web accessibility scores of American universities are lower than 80, which is the established threshold. The values of ARIA implementation are also low and only one university, out of five, the Universidad de Veracruz (UV-MX), Mexico, has improved in ARIA implementation. However, American universities present a number of web accessibility errors.

Table 3 Web accessibility data for selected American universities for the period from 2018 to 2024
Full size table

Figure 3 shows the performance of American universities in the form of a line chart for the period from 2018 to 2024. In general, the performance of web accessibility remained low during this period. An increase in performance was observed during the COVID-19 pandemic, but it decreased after the end of the pandemic. The performance of the Universidad Politécnica Salesiana de Ecuador (UPS-EC), was good before the start of the COVID-19 pandemic, but it decreased immediately afterwards and has remained partial since then.

A decrease in web accessibility errors is observed at the beginning of COVID-19, but increases immediately after its end. ARIA implementation also increases during this period. The Instituto Tecnológico de Aguas Calientes (TECNM-MX), Mexico, did not have archived data from the Wayback Machine for 2018.

Fig. 3
figure 3

Web accessibility performance of American universities for the period from 2018 to 2024

Full size image

4.3 Asia

Table 4 shows the results of web accessibility scores, errors and ARIA of universities from Asia for the period from 2018 to 2024. Most of the web accessibility scores of Asian universities are lower than 80, which is the established threshold. The values of ARIA implementation are also low and only one university, out of five, the Tel Aviv University (TAU-IL), Israel, has improved in both ARIA implementation and web accessibility scores. Asian universities present significant number of web accessibility errors. We did have issue to open the archived URL of University of Calcutta (CALUNIV-IN), India, for 2024, therefore we performed the web accessibility evaluation directly from the online URL at 27 of March 2024.

Table 4 Web accessibility data from selected Asian universities for the period from 2018 to 2024
Full size table

Figure 4 shows the performance of Asian universities in the form of a line chart for the period from 2018 to 2024. The Tel Aviv University (TAU-IL), Israel, stands out, although in terms of web accessibility performance before and during the COVID-19 pandemic, although it has decreased after the end of the pandemic. For the rest of the Asian universities evaluated, some maintained their web accessibility performance during the COVID-19 pandemic, while others experienced a decline. After the end of the COVID-19 pandemic, all these Asian universities experienced a decline in web accessibility performance, although their performance remained partial throughout the period. The implementation of ARIA did not improve in these universities during this period, but a decrease in the number of errors was observed.

Fig. 4
figure 4

Web accessibility performance of Asian universities for the period from 2018 to 2024

Full size image

4.4 Europe

Table 5 shows the results of web accessibility scores, errors and ARIA of universities from Europe for the period from 2018 to 2024. Most of the web accessibility scores of European universities are lower than 80, which is the established threshold. Two universites out of five, Universidad de Alicante (UA-ES), Spain, and National Technical University of Ukraine (KPI-UA), have good web accessibility scores. The values of ARIA implementation are also low and only one university, out of five, Universidad de Alicante (UA-ES), has improved on ARIA implementation. European universities present very low web accessibility errors. We did have issue to open archived URL of Universidade Aberta (UAB-PT), Portugal, for 2024, therefore we performed the web accessibility evaluation directly from the online URL at 27 of March 2024.

Table 5 Web accessibility data from selected European universities for the period from 2018 to 2024
Full size table

Figure 5 shows the performance of European universities in the form of a line chart, for the period from 2018 to 2024. At the beginning of COVID-19, the Universidad de Alicante (UA-ES), Spain, experienced a decrease of the good performance it had before the beginning of the pandemic and it has remained partial since then. The National Technical University of Ukraine (KPI-UA) stands out with a good performance before and during the COVID-19 pandemic, but it also dropped to partial performance when the pandemic ended, and an increase in web accessibility errors is observed since then. Most of these universities have reduced the number of errors in their web pages, but have not improved their ARIA implementations.

Fig. 5
figure 5

Web accessibility performance of European universities for the period from 2018 to 2024

Full size image

4.5 Oceania

Table 6 presents the results of web accessibility scores, errors and ARIA of universities from Oceania, for the period from 2018 to 2024. Most of the web accessibility scores of Oceanian universities are higher than 80, which is the established threshold. However, the values of ARIA implementation are is low. Oceanian universities present a number of web accessibility errors.

Table 6 Web accessibility data from selected Oceanian universities for the period from 2018 to 2024
Full size table

Figure 6 shows the performance of Oceanian universities in the form of a line chart for the period from 2018 to 2024. Australian National University (ANU-AU), Australia, University of Waikato (WAIKATO-NZ), New Zealand, Fiji National University (FNU-FJ), Fiji and National University of Samoa (NUS-WS) had good performance before and during the COVID-19 pandemic, but it dropped to partial performance when the pandemic ended. The University of Papua New Guinea (UPNG-PG) had partial performance before, during and after the COVID-19 pandemic. The number of errors on these sites did not improve during this period, but there was a slight increase in the use of ARIA features.

Fig. 6
figure 6

Web accessibility performance of Oceanian universities for the period from 2018 to 2024

Full size image

Figure 7 shows the web accessibility performance of selected universities grouped by continent in the form of a line chart for the period from 2018 to 2024. First, it can be seen that the web accessibility score in 2024 is lower than in 2018 on all continents. Second, web accessibility errors fluctuate up and down throughout the period for all continents except Asia, which shows a steady decrease in errors. Third, the performance of ARIA in general improves steadily throughout the period for all continents, especially for the Americas and Europe. Fourthly, contrast errors are higher in Africa than in any other continent, and in Asia they increase steadily throughout the period.

Fig. 7
figure 7

Web accessibility performance by continents for the period from 2018 to 2024

Full size image

Figure 8 shows the six most frequent web accessibility issues, out of a total of 12, recorded for selected universities from all continents, in the form of a bar chart for the period 2018 to 2024. The values are the sum of each type of web accessibility problem registered from 2018 to 2024. On the one hand, low contrast, lack of alternative text on images and empty links are the top three recurring issues across all continents. On the other hand, empty headings, empty buttons and empty form labels are recurring issues that vary between continents. Other common web accessibility problems recorded are: broken ARIA reference, missing language, marquee, missing form label and missing page title. However, these problems did not occur on all continents at the same time, so they were excluded.

Fig. 8
figure 8

Cumulative web accessibility errors of selected universities by continents for the period from 2018 to 2024

Full size image

Figure 9 shows the six most frequent web accessibility problems at selected universities on all continents in the form of a bar chart for the period 2018 to 2024. Low contrast ranks first, followed by missing alternative text on images, empty links, empty headings, empty form labels and empty buttons. The values are the sum of each type of web accessibility problem registered from 2018 to 2024.

Fig. 9
figure 9

Total cumulative errors of selected universities for all continents for the period from 2018 to 2024

Full size image

5 Discussion

The websites of African universities show partial performance in web accessibility scores, low performance in errors and ARIA. Low scores in web accessibility and errors mean that these universities do not comply with the WCAG guidelines and present barriers for persons with disabilities to access their websites. Moreover, low performance in ARIA means that African universities may not be compatible with assistive technology for persons with disabilities.

The websites of American universities show partial performance in web accessibility scores, low performance in errors and ARIA. Low scores for web accessibility and errors mean that these universities do not comply with the WCAG guidelines and present barriers for persons with disabilities to access their websites. Low performance in ARIA means that American universities may not be compatible with assistive technology for persons with disabilities.

The websites of Asian universities show partial performance in web accessibility scores, and low performance in errors and ARIA. Low scores for web accessibility and errors mean that these universities do not comply with the WCAG guidelines and present barriers for persons with disabilities to access their websites. Low performance in ARIA means that Asian universities are not compatible with assistive technology for persons with disabilities.

The websites of European universities show partial performance in web accessibility scores, low performance in ARIA and good performance in errors. Low performance in web accessibility means that these universities do not comply with the WCAG guidelines and present barriers for persons with disabilities to access their websites. Good performance in web accessibility errors means that these universities are making efforts to improve web accessibility. Low performance in ARIA means that European universities are not compatible with assistive technologies for persons with disabilities.

The websites of Oceanian universities score well on web accessibility and poorly on ARIA and errors. High scores for web accessibility mean that these universities comply with the WCAG guidelines and do not present many barriers for persons with disabilities to access their websites. Low performance in ARIA means that African universities are not compatible with assistive technology for persons with disabilities.

Overall, there is a progressive decrease in web accessibility performance (AC) from 2018 to 2024 for all continents, as well as the prevalence of web accessibility errors (ER). However, the implementation of ARIA (AR) has, on the contrary, increased since 2018, except for the Asian continent, which shows a sharp decrease in performance from 2023. This can be better appreciated in Fig. 7, which shows the web accessibility performance of universities from all continents in the form of a line chart for the period from 2018 to 2024.

This longitudinal study also shows that digital preservation of web pages provides researchers with a good source of information to understand the evolution of web accessibility in universities around the world years before the start of the COVID-19 pandemic, during its duration, and years after its end. The ability to analyse web accessibility scores, errors and ARIA on web pages from the past provides indicators that organisations can use in their scorecards when conducting performance evaluations.

The differences in the web accessibility performance of different universities from different continents are due to differences in the commitment, implementation capacity and results in web accessibility of universities and the associated countries and supranational organisations to which they belong. Nevertheless, the results of this longitudinal study can provide managers with a better understanding of what needs to be done to improve web accessibility by relating these web accessibility indicators to other performance variables and finding their correlations.

The six most common web accessibility issues illustrated in Figs. 8 and  9 mean that persons with disabilities have difficulty perceiving, navigating and understanding the content of the websites of selected universities from five continents. As such, this highlights non-compliance with the WCAG principles [21]. persons with visual impairment are most affected as low contrast error is the number one problem, which is consistent with the findings of the 2023 Digital Accessibility Index [15].

This study alone cannot clarify the reason for the decline in the universities’ web accessibility compliance performance after the end of the COVID-19 pandemic. Further studies are needed as other factors may be involved, such as legislation, regulations, policies, programs, organisation, processes, resources, etc. These factors are examined nationally by the Digital Accessibility Rights Evaluation Index (DARE Index); therefore, a correlation study between the web accessibility of universities and the DARE Index of the countries to which they belong could provide more information on the factors behind the decline in web accessibility performance of universities after COVID-19.

6 Conclusions and future works

The COVID-19 outbreak increased the web accessibility performance of many of the universities evaluated in this study. Regrettably, after the COVID-19 pandemic ended, this performance was found to have declined, in some cases to levels lower than before the outbreak. This behaviour suggests that the closure during the COVID-19 pandemic may have led stakeholders to consider web accessibility, but further studies should be conducted to verify that this was not due to other factors. In any case, web accessibility remains a challenge after the COVID-19 pandemic.

The lack of positive web accessibility performance by universities around the world is significant. Persons with visual impairment are most affected, as low contrast errors and lack of alternative text on images remain the main barriers to equal web access for persons with disabilities. Universities provide knowledge to society and should be committed to facilitating universal access in the information society. As institutions of advanced learning, universities should be concerned with the equal access and active participation of potentially all citizens in the Information Society.

The fact that the web accessibility of the universities selected in this study changed immediately after the end of the COVID-19 pandemic confirms that web accessibility is an ongoing issue. Therefore, this study emphasises the need to approach web accessibility in a proactive rather than reactive mode in order to ensure continuous improvement. In this way, the web of higher education institutions will always be perceivable, usable, understandable and robust, in line with the principles of WCAG, Article 9 of the CRPD and Goal 4 of the UN-SDGs.

The reasons for this drop in web accessibility scores after the end of the COVID-19 pandemic need to be investigated. However, on the one hand, it may be due to a decline in the following three variables of the DARE index [12], which are country commitments (laws, regulations, policies and programs), country implementation capacity (organisation, processes, resources) and actual digital accessibility outcomes for persons with disabilities in 10 areas of products and services [12]. However, there are no data available for the DARE index from 2020, and its evolution from 2018 to 2020 is positive. On the other hand, the decline in web accessibility performance in the post-COVID-19 era may also be due to a lack of funding; the World Bank’s index of government expenditure on education has declined globally since 2018 [54]. However, the available data go up to 2021, so this hypothesis needs to be analysed and confirmed by further studies in the future.

This study has a number of limitations. Firstly, it does not evaluate the accessibility of all the universities of the five continents of the world, but focuses on a selection of universities. Therefore, the results should be taken as indicators of web accessibility performance in a given period. Secondly, the variables analysed in the study, web accessibility scores, errors, contrast errors and ARIA implementation, are not separated because the main objective of the study is to evaluate the trend of web accessibility performance over a period of time. Therefore, the study is not intended to highlight which specific WCAG principle or technique is being violated the most, as there are other studies that do this. This study is more focused on providing web accessibility performance indicators to stakeholders to help them understand the importance of continuous improvement of web accessibility in higher education institutions. Finally, this study is not intended to explain the reason for the decline in web accessibility compliance performance of higher education institutions after the end of the COVID-19 pandemic. There may be other factors at play that require further correlational studies.

This study can be extended in the future by conducting a hybrid evaluation, combining AWAETs and manual validation by web accessibility experts to improve the accuracy of the results. Furthermore, this research could also be extended by including more universities and variables in the study, such as the Human Development Index (HDI) and the DARE, government spending on education, and analysing their correlations with web accessibility scores.