Keywords

1 Introduction

The Higher Education System of Ecuador, to date there are 59 accredited universities, of which three only offer postgraduate degrees, and 83 Technical and Technological Institutes, being 34 public and 49 privateFootnote 1. Each institution has a public web domain, from which academic and management resources are administered. After having carried out a Systematic Literature Review, no studies on the usability of all institutional websites have been found. In [1] n analysis is made of 24 websites of universities belonging to CEDIA, using the Prometheus tool from SIRIUS and Kmeans, obtaining as relevant data that 50% are at an adequate level, but fail to highlight in which aspects they present problems. In [2] the websites based on the ISO 9241-151 standard in 59 universities are analyzed, classifying them by the category obtained in the evaluation by CACES (A, B, C), also detecting problems in content design, general design and especially in the search criteria. In [3] the study at the University of Riobamba is singled out, determining that the main problems are the use of Search Engine and Interacting with Links and Download Information. This research complements the previous ones, completing the entire Ecuadorian higher education system. The proposed objectives are of interest to higher education authorities, knowing the usability of their web spaces, which are the point of contact with the community.

To assess usability on the web, there are many methods and techniques, which, Perurena [4] very accurately details. Granollers [5] proposes the use of heuristics, as a technique to evaluate usability in user interfaces; Storm [6] uses time Series Analysis of Selected Episode Graphs; In [7] they make use of the ISO/IEC 9126 quality model standard, applying techniques such as observation guide, satisfaction questionnaire and attractive interface questionnaire to evaluate each metric. In [8] card sorting method is applied to construct the shape of the site focused on user shared cognition. In [9] the use of the eye tracking tool is exposed, as an input for the analysis of user behavior when interacting with the web. Artificial intelligence has also been applied to enhance data collection and perform sustained analysis in machine learning algorithms; Thus, Zainab [10] applies clustering techniques as an input to categorize data obtained from the web, which are then analyzed from a standard system usability scale. To improve the automation of software evaluation (including usability), Stouky [11] proposes the use of machine learning techniques and big data. Chamba [1] applies data mining techniques to collect data from 24 Universities Web portals, as input for the subsequent application of usability techniques. In this work, Chamba’s proposal [1] is followed to collect data from 83 institutional websites.

In this research, it is proposed to work with the User-Oriented Web Usability Assessment System based on the determination of Cryptic Tasks (SIRIUS by its Spanish acronym), developed by Torrente [12] as part of his work on PhD research; It is composed of 10 aspects, divided into 83 variables (criteria) that cover the various areas of functionality of a web-based solution. Torrente exposes it in [13] like as a heuristic-based framework for measuring web usability adapted to the type of website. Sai Aparna [14] adapts it as SIRIUS-DWUEP (Web Usability Evaluation Process), operationalizing this empirical validation technique in the WebML method. Muñoz-Egido [15] applies it to analyze eighteen academic library web portals, obtaining as a result that the average usability of the websites of the analyzed university libraries is 72.30 out of 100 with a standard deviation of 5.57. In [1] his technique is used in 24 Universities Web portals, obtaining three clusters that relate the 10 aspects of SIRIUS. In this research, SIRIUS is applied, and techniques based on clustering that have allowed us to categorize the web portals of the 63 institutes of higher education since the fulfillment of each of the aspects of this technique.

2 Materials y Methods

As a general method of automatic learning, the CRISP-DM method is proposed [16], adapted to the singular requirements of the study. This method is composed of six iterative stages: business understanding, data understanding, data preparation, modeling, evaluation and deployment.

2.1 Business Understanding

The usability evaluation of 83 public (34 eq = 40%) and private (49 EQ = 60%) institutes websites was performed following the Sirius framework: A heuristic-based framework for measuring web usability adapted to the type of website [13]. Sirius has 83 evaluation criteria grouped into ten aspects, which are detailed in Table 1. The data for the analysis was obtained through a project carried out at the “Instituto Tecnológico Loja”, and is available on the webFootnote 2, in open access format.

2.2 Data Understanding

Table 1. Sirius aspects and criteria’s
Full size table

2.3 Data Modeling

Two clustering techniques are used for data modeling:

  1. a)

    Hierarchical, with the Ward.2 method and euclidean distance between two points p and q, which is defined as the length of the segment that joins both points. Hierarchical combined with descriptive stadia [17], o determine the suitable number of centers for Kmeans. Once the dendrogram is created, the extent to which its structure reflects the original distances between observations is evaluated, using the correlation coefficient between the cophenetic distances of the dendrogram (node height) and the original distance matrix.

  2. b)

    K-means clustering [18] to group the observations in K different clusters (centers), based on the evaluations previously carried out, it was determined that the value of K = 3, with 100 iterations to guarantee the process.

The computational analysis was carried out with the R StudioFootnote 3 software, using the packages: clValid to validate the ideal clustering algorithm; factoextra for Multivariate Analysis); and, cluster and stats for Clustering Analysis; In addition to the official sites, these packages can be downloaded from the project websiteFootnote 4. To find the similarity, the Euclidean distance is used, making a hierarchical classification where the groups are merged (or subdivided) successively, following a priority or hierarchy, decreasing the homogeneity as they become wider. Therefore, for the evaluation of clustering, the coefficient of correlation is considered, which allows finding the distance between the clusters.

3 Results

In Fig. 1 we observe the grouping of the institutes applying the Hierarchical method and obtaining 3 groups with a correlation coefficient of 0.9526396.

Fig. 1.
figure 1

Hierarchical cluster

Full size image

The clustering analysis allowed to identify 3 groups of institutes (Fig. 2) with 18, 32 and 33 institutions respectively. Regarding the classification by financing of the institutions, we find that Cluster 1 is made up of 17 private and one public institute, Cluster 2 of 17 private and 15 public, and cluster 3 of 25 private and 8 public.

Fig. 2.
figure 2

Kmeans cluster

Full size image

In Fig. 3 the values of the cluster with respect to the 10 aspects of SIRIUS are shown. Cluster1 predominates in 2 aspects (search and help); Cluster 2 predominates in 4 aspects (General aspects, layout, control and multimedia); and cluster3 predominates in 4 aspects (identity and information, structure and navigation, labeling and comprehensibility).

Fig. 3.
figure 3

Classification aspects

Full size image

Table 2 details the distribution of the 3 general clusters and the clustering of the 83 criteria among the 10 aspects evaluated by SIRIUS, classifying the institutes between public and private. These significance values allow us to observe that 73.49% of the institutes evaluated obtained a high evaluation in Labeling, besides that only 25.50% obtained a high evaluation in Control, we also highlight that 88% of institutes do not have a search incorporated in the website, finally we observed that only 19% of institutes show user help on site.

Table 2. Cluster Sirius Aspects
Full size table

Finally, to better understand the usability margins of the three clusters, a graph with measures of central tendency and a usability center line are detailed in Fig. 4. It is observed that the percentage of usability of the institutes belonging to the cluster1 is above the average; however, the institutes belonging to cluster2 and cluster3 are below the average, with better results for the last group.

Fig. 4.
figure 4

BoxPlot cluster – usability percent

Full size image

4 Discussion

Being our objective to evaluate the usability characteristics of the institutes’ websites, we have identified 3 types of institutes and the most significant aspects are search and help, the search aspect presents similarity with problems previously found in university websites [1, 3]. As indicated by research [19, 20] the user must be able to retrieve information easily, therefore, it is important to provide adequate search systems, because the websites of educational institutions allow the presentation of academic offerings, news and achievements in teaching, research and networking. In this area, the importance of the understanding of a website is highlighted [21, 22], as shown in this research, most Ecuadorian institute sites present information, but care must be taken to ensure that this is done through appropriate web interfaces that speed up understanding.

The analysis based on heuristics used in this research, and adaptable to the type of website, can be adapted to the information and usability needs of educational institutions; this presents a similarity to the work [2], However, research such as [3] use the ISO 9241-151 standard, which evaluates all types of sites with the same metrics; furthermore, these studies are based on the categorization of type A, B, C universities etc. and the present study, classifies according to the analysis performed by the artificial intelligence technique used, based on the 81 parameters and 10 aspects evaluated with SIRIUS.

Of the aspects found with SIRIUS, the one corresponding to Help is shown as a weakness in most institutes, so it presents an opportunity to decongest the traditional communication channels and provide the user with a better web experience in educational sites [23], being this an aspect to improve, implementing sections of frequent questions, tutorials, chatbots etc.

5 Conclusions

Group 1 is characterized by high values in Search and Help and low values in Understandability and Multimedia elements, Group 2 is characterized by high values in Page Layout, Control and Feedback, low values in Tagging and null values in Search and Help; on the other hand Group 3 presents high values in Site Identity, Structure and Navigation, Tagging, Understandability, Multimedia elements, low values in Help and null values in Search.

Evaluating the aspects and criteria of usability of 83 technological institutes of Ecuador we can conclude that Cluster 1 composed by 18 institutes is a group that to improve its usability must improve its understandability and multimedia elements, Cluster 2 composed by 32 institutes is a group whose scores must improve in most aspects except in Page Layout and Control and Feedback, on the other hand Cluster 3 composed by 33 institutes is a group whose values are high in 6 of the 10 but must improve in help and implement search in the web site.