Keywords

1 Introduction

Website portals are currently the simplest and most widespread way people and companies have to publicize information about a product or service. In relation to the aforementioned, it is necessary for these sites to comply and agree with what users expect to find in them, in such a way that it manages to capture their attention without ignoring the main idea for which the website was created [1]. Due to this, the concept of usability has gained strength with the objective of users being able to fulfill their expectations of use adequately in applications in different application contexts. According to the ISO 9241-11 standard, usability is understood as the degree in which a product can be used by specific users to achieve specific objectives with effectiveness, efficiency, and satisfaction within a given context of use [2].

Various methods of web usability inspection exist among which we can highlight the heuristic test by Sirius [3] and the Torres-Burriel test [4], which, from the definition of heuristics and usability criteria, permit identifying common usability problems in different types of websites. The heuristics defined for websites in both evaluation tests mentioned, start from the general-purpose heuristics by Nielsen and are specified to the context of the websites [5]. These two types of tests define a different evaluation scale for each of the evaluation criteria considered, permitting to obtain statistics by using documents or spreadsheets, making it necessary for the evaluator to draw graphics manually on the compliance of heuristics in websites. The importance of executing these evaluation methods lies in obtaining recommendations on the website portal evaluated, permitting to provide feedback to the quality of the product software [6]. Although both website usability tests are similar regarding the heuristics and criteria proposed, the website usability test by Torres-Burriel can be considered more complete by bearing in mind a set of accessibility criteria, a currently fundamental element, to guarantee that a greater number of people can access a website independent of the context of use [7]. Thus, in this paper the heuristics of Torres-Burriel are considered, taking into account that they address the main elements of a website, including aspects related to accessibility.

This article proposes a software system for usability inspection for web applications, considering the usability test by Torres-Burriel [4]. This system permits evaluating the compliance of a set of criteria associated with the eleven heuristics of the test (general aspects, identification and information, language and drafting, labeling, navigation structure, page layout, search, multimedia elements, help, accessibility, and control and feedback), obtaining a graphic report on the percentage of compliance and the evaluation average obtained for each heuristic. The software system seeks to become a support to conduct usability inspections on website portals, to identify different elements to improve of usability of a website portal. The rest of the article is organized in the following manner: Sect. 2 presents the different phases of the methodology used for this research; Sect. 3 presents a set of concepts considered to develop this work; Sect. 4 describes the functional structure of the usability inspection software system; Sect. 5 presents a case study conducted through the software system on the website portal of the Government of Cauca; finally, Sect. 6 presents the conclusions and future work derived from this research.

2 Methodology

To develop and validate the software system for the usability inspection of websites, the following phases were considered, namely, exploration of the usability test by Torres-Burriel, design of the inspection software system, implementation of the inspection software system, and case study (see Fig. 1).

Fig. 1.
figure 1

Proposed methodology

Full size image

Phase 1 - Exploration of the Usability Test by Torres-Burriel:

this phase conducted a detailed study of each of the guidelines and criteria present in the usability test by Torres-Burriel. This was done to identify the different heuristics and criteria, as well as the format of the template used to conduct the test.

Phases 2 and 3 - Design and Development of the Software System:

starting from the criteria obtained in the Torres-Burriel test and revising its structure in the document in which it is proposed, a software system was designed and developed using a system of tabs, each of which is associated with a Torres-Burriel heuristic.

Phase 4: Case study:

once the software system was generated to evaluate the usability of websites, the portal of the Government of Cauca was chosen, given that by being a government-type portal, it must guarantee usability and accessibility conditions necessary considering the amount of visitors.

3 Conceptual Framework

This section includes a set of relevant concepts considered to carry out this work. These are: usability, usability test, usability test by Torres-Burriel.

3.1 Usability

Usability is understood according to ISO 9241-11 as the measure in which a system, product, or service can be used by specific users to achieve specific objectives with effectiveness, efficiency, and satisfaction within a specific context of use [1]. Likewise, it may be defined as the discipline that studies the way of designing websites for users to interact with them more easily, comfortably, and intuitively [8]. A usability test comprises a series of practices and tests performed on an application or a website to test the comfort, ease or complexity with which it is managed. Expert users or conventional users who explore in detailed manner the functionalities of the interactive system can carry out these tests. The results of the interaction of the users are consigned in a test that accounts for the usability of the software system evaluated [9]. In this work an application for the execution of usability evaluations based on the inspection of Torres-Burriel heuristics is proposed.

3.2 Heuristic Test of Torres-Burriel

Torres-Burriel propose a set of heuristics to evaluate websites, which contain the most relevant aspects to keep in mind at usability level, where for each heuristic presented a set of criteria to evaluate is provided within a range from 1 to 5 [4]. These are posed in terms of questions. Said heuristics retake some of the elements proposed in the principles by Nielsen, so that they are distributed in 11 heuristics, as shown in Table 1 [10].

Table 1. Torres-Burriel heuristics
Full size table

In keeping with the aforementioned, Table 2 introduces the amount of criteria associated to each of the heuristics presented in Table 1. As mentioned, said criteria are presented to the evaluator in terms of questions.

Table 2. Criteria per heuristic of the test by Torres-Burriel
Full size table

4 Proposed Software System

The software system introduced in this work stemmed from a set of usability criteria associated with each of the 11 heuristics proposed by Torres-Burriel (Table 2). These criteria are presented in terms of questions and can be evaluated in a range from 0 to 5, with 0 being the absence of the criterion in the website portal and 5 being maximum compliance of the criterion within the portal. At design level, the software system is based on tabs, each of which comprises a heuristic and its associated criteria.

According to the aforementioned, the software system proposed is constituted by 15 tabs organized in the following manner: the first tab consigns the general data of the usability evaluation; tabs 2 to 12 include each of the usability heuristics and their associated criteria; tab 13 presents a table with the average obtained in each of the heuristics evaluated and the general average of the heuristics; tab 14 includes a graphic with the averages of each principle; finally, tab 15 presents a graphic with the percentage of compliance of each of the heuristics by Torres-Burriel.

A flow diagram illustrating the functioning of the software system is presented in Fig. 2. As observed in Fig. 2, the software system allows reviewers to evaluate one-to-one the different criteria associated with the standard’s accessibility principles to, subsequently, generate statistics of the mean value per accessibility principle and of the percentage of compliance of each principle of the application to inspect. In summary, according to Fig. 2, the functionality of the tabs is grouped into three: the first instance is related with filling out the test data (first tab); the second instance includes the evaluation of the criteria of the eleven heuristics (tabs 2 to 12); and the third instance contains that related with the analysis of the statistical results and graphics of the evaluation conducted. The results obtained seek to guide evaluators on those heuristics that the website portal evaluated is not complying fully.

Fig. 2.
figure 2

Functional diagram of the software system

Full size image

To comply with the idea presented in Fig. 2, a software system was constructed in Java language through tabs and with different tables per tab, using JTabbedPane (Swing class) and JTable components (see Fig. 3). Regarding the generation of the analysis graphics, the software system uses the JFreeChart library [11], which in this case permits obtaining bar graphs on the average of the evaluations per principle and their percentage of compliance.

Fig. 3.
figure 3

Heuristics evaluation tabs

Full size image

As shown in Fig. 3, each of the first four tabs has a table associated with three columns: Id, Criterion, and Value. The “Id” column permits distinguishing the different criteria of each heuristic principle, using the first letter of the heuristic and a number. In that regard, the first criterion of the “General” heuristic has the code of G1. The “Criterion” column includes a question that permits evaluators to verify compliance of the criterion of a given heuristic. Finally, the “Value” column comprises a set of values evaluators can assign to each criterion, depending its level of compliance in the website portal. The scale of values varies between 0 and 5, including the option of N/A, which can be used in case the criterion does not apply for the website portal in question. The 0 value in the scale corresponds to no application of the criterion in the website portal, while 5 corresponds to full application of the criterion in the website portal.

Upon filling out basic evaluation data (name of evaluator, URL of site to evaluate, and evaluation description) and ending the inspection by the different heuristics and usability criteria proposed by Torres-Burriel, it is possible to see in the “Results” tab the average obtained in each of the heuristics and general average obtained in the usability inspection (see Fig. 4). The average obtained within each of the heuristics permits evaluators to identify aspects of the web application that do not comply adequately with the usability criteria.

Fig. 4.
figure 4

Heuristics results tab

Full size image

Upon generating the heuristic averages and the general average of the evaluation, in the “Graphic – Average” tab it is possible to graphically visualize the average obtained by each of the usability heuristics for websites proposed by Torres-Burriel (see Fig. 5).

Fig. 5.
figure 5

“Graphic – Average” tab

Full size image

Likewise, in the “Graphic – Percentage” tab, it is possible to visualize the graphic with the percentage of compliance of each of the usability heuristics for websites proposed by Torres-Burriel (see Fig. 6).

Fig. 6.
figure 6

“Graphic - Percentage” tab

Full size image

5 Case Study

This work implements the inspection software system to facilitate application and analysis of the results of the heuristic test by Torres-Burriel and validate the heuristic test by Torres-Burriel, which proposes in its different tabs a set of criteria associated with the eleven heuristics website portals and applications must comply according to Torres-Burriel. The inspection software system was applied on the website portal of the Government of Cauca (http://www.cauca.gov.co/) in order to evaluate the usability of it (see Fig. 7). The Government of Cauca aims to plan social and economic development, promote the welfare of the community, promote the integral development of its municipalities and other territorial entities of its jurisdiction, through the exercise of its administrative functions of coordination, complementarity, concurrence, subsidiarity and intermediation, within the framework of the Constitution and laws. In this sense, considering the diversity of the target public, it is necessary to guarantee inasmuch as possible compliance of certain basic usability guidelines.

Fig. 7.
figure 7

Website portal of the Government of Cauca

Full size image

5.1 Results of the Inspection

The evaluation method used in this article was the inspection method, which took into consideration each of the usability criteria included in the usability evaluation software system, which – in turn – are associated with the different guidelines of the eleven heuristics of the usability test by Torres-Burriel. The evaluation was conducted by a group of three experts in the area of human computer interaction, who inspected the compliance of Torres-Burriel’s heuristics on the portal of the Government of Cauca. Thus, Fig. 8 presents the average of the evaluations assigned by the evaluators to the criteria of the “ Structures and Navigation” principle.

Fig. 8.
figure 8

Evaluation by experts

Full size image

Upon finishing the inspection of the Government of Cauca web page and considering the different criteria by Torres-Burriel, Fig. 9 presents one of the graphic results obtained by the usability inspection software system for web pages, which shows the average of the evaluations obtained for each of the eleven heuristics evaluated.

Fig. 9.
figure 9

Average graphic of the heuristics

Full size image

It may be noted that the portal is poorly usable, considering the inspection results, given that only one of its principles (General) reaches an average slightly above 4 (4.11) in a scale from 1 to 5, remaining <4 in the rest of the heuristics evaluated, which in mathematical terms means that only one of the heuristics achieves more than 80 percent compliance. Likewise, it is possible to analyze that six of the eleven heuristics evaluated have a compliance percentage greater than or equal to 70% and less than 80%. In the same way two of the eleven heuristics evaluated have a compliance percentage greater than or equal to 60% and less than 70%. Finally, two of the 11 heuristics evaluated have a compliance rate of less than 60%. The heuristics with the lowest percentage of compliance are “Help” and “Multimedia Elements” with less than 56%. Similarly, the percentage of the “Accessibility” heuristic is 70%, considered acceptable and in need of improvement, bearing in mind the variety of the public accessing the government portal evaluated (see Fig. 10).

Fig. 10.
figure 10

Compliance of heuristics of web usability

Full size image

Finally, the general usability average for the website portal is 3.466, corresponding to 69.32% compliance (see Table 3). It is convenient to consider different usability criteria for the portal to have a value >80%, which can be deemed acceptable. Thus, the inspection of the portal of the Government of Cauca helps to generate a set of recommendations keeping in mind the averages per heuristics (see Table 3), which seek to be of help to generate possible improvements, considering the number and diversity of users accessing these types of government portals.

Table 3. Average per heuristic
Full size table

5.2 Recommendations

Pursuant to the inspection conducted of the portal of the Government of Cauca, this section presents a set of recommendations (obtained by the test coordinator) for the eleven heuristics evaluated and considering the criteria obtaining the lowest value through the assessment of the evaluators (see Table 4).

Table 4. Recommendations of usability generated
Full size table

6 Conclusions and Future Work

The principal contribution of this research is to propose a software system for the usability inspection of web applications, keeping in mind the usability test by Torres-Burriel. This software system seeks to support developers, designers, and programmers in evaluating websites to construct more inclusive and usable websites.

The principal contribution of the software system proposed is the possibility of evaluating each of the criteria associated with the heuristics by Torres-Burriel, as well as automatically generating the graphic of averages and the graphic of percentage of compliance of the heuristics by Torres-Burriel. The above indicators are intended to be helpful in terms of improving the overall design of the evaluated portal and the increase in its number of potential users.

It is worth highlighting that portals, like that of the Government of Cauca, when aimed at a broad and diverse public, must pay great attention to heuristic criteria such as accessibility and help to enable inclusion and access to different users, independent of the context of use.

The Java language proved adequate to implement an inspection software system, bearing in mind the tab management component (JTabbedPane), the table management component (JTable), and the API to generate JFreeChart statistical graphics.

In reflection, it is important to highlight that given that many governmental website portals have been created by using content managers, the websites constructed through these bring along diverse usability problems that do not necessarily depend on the webpage designer, given that these managers were mostly conceived without considering usability criteria.

For future work derived from this research, the software system should include the possibility of generating automatic recommendations arising from the statistic results and graphics obtained in the test.