Keywords

1 Introduction

Usability applied to websites is commonly defined as a quality attribute that evaluates the ease of use of its interfaces and is classified as a survival condition [19]. Due to the importance of this attribute, specialists have developed different guidelines and methods that allow them to determine if a website reaches an adequate level of usability. An example of the guidelines developed are the ten heuristics proposed by Nielsen, which are general rules for the design of user interfaces [13].

For the design and evaluation of usability, it is possible to incorporate the cultural characteristics of societies. The problem is that many sites may not take into account the characteristics of each culture because, among other reasons, including cultural issues would involve greater effort in the design of the interfaces of the websites, because the requirements of Users are influenced by their local cultural perspective [8].

To meet these needs, there are studies that include cultural issues in their heuristic proposals, which are then applied and validated on different types of websites. The purpose of this study is to identify the cultural-oriented heuristics proposals applied to websites and the procedures used to establish them.

This paper is structured as follows: Sect. 2 presents main concepts related to cultural-oriented usability. Section 3 describes the process of systematic review. Section 4 presents the results of the research. Finally, conclusions and future works are presented in Sect. 5.

2 Theoretical Background

2.1 Usability

According to Jakob Nielsen, usability can be defined as “a quality attribute that evaluates how easy the interfaces are to use” [19]. In addition, it presents 10 heuristics [13] that are currently used to evaluate usability and to establish new heuristics. Those 10 heuristics proposed by Nielsen are listed are listed below:

  • Visibility of system status.

  • Match between system and the real world.

  • User control and freedom.

  • Consistency and standards.

  • Error prevention.

  • Recognition rather than recall.

  • Flexibility and efficiency of use.

  • Aesthetic and minimalist design.

  • Help users recognize, diagnose, and recover from errors.

  • Help and documentation.

2.2 Cultural Dimensions

According to Hofstede, a Dimension is “an aspect of a culture that can be measured in relation to other cultures” [11]. Accordingly, Hofstede initially proposed a four-dimensional model [11] that is composed of the concepts which are described below:

  • Power Distance: The degree to which less powerful members of institutions and organizations in a country expect and accept that power is distributed unevenly.

  • Individualism and Collectivism: Individualism belongs to societies in which everyone is expected to take care of themselves and their immediate family. Collectivism belongs to societies in which people are expected to protect each other in exchange for loyalty.

  • Masculinity and Femininity: Gender roles and behaviors such as competitiveness, assertiveness and quality of life.

  • Uncertainty Avoidance: The way in which members of a society feel threatened by ambiguous or unknown situations.

3 Systematic Literature Review

This systematic review was carried out using the methodology proposed by B. Kitchenham, which proposes the approach of a research question and the use of the PICOC method, which refers to Population, Intervention, Comparison, Outcome and Context [17].

3.1 Formulation of the Research Questions

The objective of the systematic review is to know the existence and application of usability metrics related to cultural aspects in different web domains. Therefore, the following questions are defined:

  • RQ1: What procedures are used to establish heuristics of evaluation in cultural aspects in the design of websites?

  • RQ2: What usability heuristics, focused on assessing cultural aspects in website design, are reported in the literature?

3.2 Database Selection

The following databases were selected for their relevance in the area of Computer Engineering:

  • SCOPUS.

  • Web of Science.

  • IEEE.

3.3 Development of the Search String

The PICOC table will be used to determine the search string. The objective of this systematic review is not to make a comparison between existing usability metrics, therefore, the “comparison” criterion is not taken into account. The concepts are detailed in Table 1.

Table 1. General concepts defined using the PICOC criteria
Full size table

Using the concepts established in PICOC, a search string was developed for each database. The search string for SCOPUS was as follows:

(TITLE-ABS-KEY (“metric” OR “metrics” OR “heuristic” OR “heuristics” OR “guideline” OR “guidelines” OR “evaluation”) AND TITLE-ABS-KEY (“UX” OR “Usability” OR “HCI” OR “interface design” OR “design” OR “web design”) AND TITLE-ABS-KEY (“web site” OR “web sites” OR “web page” OR “web pages” OR “web system” OR “web system” OR “web application” OR “web applications” OR “web apps”) AND TITLE-ABS-KEY(“cultural dimension” OR “cultural dimensions” OR “culture” OR “cultural-oriented”))

The search string for Web of Science was as follows:

TOPIC: (“metric” OR “metrics” OR “heuristic” OR “heuristics” OR “guideline” OR “guidelines” OR “evaluation”) AND TOPIC: (“UX” OR “Usability” OR “HCI” OR “interface design” OR “design” OR “web design”) AND TOPIC: (“web site” OR “web sites” OR “web page” OR “web pages” OR “web system” OR “web system” OR “web application” OR “web applications” OR “web apps”) AND TOPIC: (“cultural dimension” OR “cultural dimensions” OR “culture” OR “cultural-oriented”)

The search string for IEEExplore was as follows:

((“All Metadata”:“metric” OR “All Metadata”:“metrics” OR “All Metadata”:“heuristic” OR “All Metadata”:“heuristics” OR “All Metadata”:“guideline” OR “All Metadata”:“guidelines” OR “All Metadata”:“evaluation”) AND (“All Metadata”:“UX” OR “All Metadata”:“Usability” OR “All Metadata”:“HCI” OR “All Metadata”:“interface design” OR “All Metadata”:“design” OR “All Metadata”:“web design”) AND (“All Metadata”:“web site” OR “All Metadata”:“web sites” OR “All Metadata”:“web page” OR “All Metadata”:“web pages” OR “All Metadata”:“web system” OR “All Metadata”:“web systems” OR “All Metadata”:“web application” OR “All Metadata”:“web applications” OR “All Metadata”:“web apps”) AND (“All Metadata”:“cultural dimension” OR “All Metadata”:“cultural dimensions” OR “All Metadata”:“culture” OR “All Metadata”:“cultural-oriented”))

3.4 Selection Strategy

The inclusion and exclusion criteria were the following:

Inclusion Criteria

  • English language publications.

  • Publications that have at least been cited in a publication.

  • Publications since 2000.

Exclusion Criteria

  • Publications that are not applied to websites.

  • Publications that use usability but not related to the cultural approach.

4 Analysis of the Results

When performing the search, in the different databases, a total of 112 results were obtained. After applying the inclusion and exclusion criteria, 10 documents relevant to the Scopus source, 5 relevant to the Web of Science source and 2 relevant to the IEEE source were obtained, adding a total of 17 results relevant to this research. Table 2 shows the summary of what was found and Table 3 shows the list of selected articles.

Table 2. Summary of Search Results
Full size table
Table 3. Relevant documents for Systematic Review
Full size table

The 17 articles were classified according to their scope. Table 4 shows the classification.

4.1 Methodologies to Define Heuristics

In this section, the results obtained after the review are presented. The purpose was to obtain an answer for the following research question “What procedures are carried out to establish heuristics aimed at assessing cultural aspects in the design of websites?”

The following procedures reported in the literature have been identified to determine whether a website represents what it should, taking into consideration the cultural aspects:

  • In the paper entitled “A cultural-oriented usability heuristics proposal” (A13), a procedure to develop usability heuristics for multicultural websites is described. This procedure was proposed by Cristian Rusu, Silvana Roncagliolo, Virginica Rusu and Cesar Collazos in the paper “A Methodology to Establish Usability Heuristics”, published in 2011 [22]. In that study, it is indicated that, to establish usability heuristics, they are necessary six stages: (1) exploration, (2) descriptive, (3) correlation, (4) explanation, (5) validation, and (6) refinement.

    • Stage 1: Exploration. It focuses on obtaining the bibliography related to the main research topics. It includes related usability heuristics.

    • Stage 2: Descriptive. It focuses on highlighting the most important characteristics of previously obtained information, to establish the main concepts associated with research.

    • Stage 3: Correlation. It focuses on identifying the characteristics that usability heuristics have for a specific application, based on traditional heuristics and case studies analyzed.

    • Stage 4: Explanation. It focuses on specifying the set of proposed heuristics.

    • Stage 5: Validation. It focuses on verifying new heuristics against traditional heuristics through experiments, through heuristic evaluations.

    • Stage 6: Refinement. It focuses on the feedback of the validation stage.

  • On the other hand, in the article “A cross-cultural comparison of Kuwaiti and British citizens’ views of e-government interface quality” (A10), the relationship between Hofstede’s cultural dimensions and aspects of the interface of user and web design, a relationship that was proposed by Aaron Marcus and Emilie Gould in the paper “Crosscurrents. Cultural Dimensions and Global WebUser-Interface Design”, in the year 2000 [18]. This article expresses the relationship between each cultural dimension and the influence it could have on the interfaces.

Table 4. Articles according to scope
Full size table

4.2 Usability Heuristics to Evaluate Cultural Aspects

In the literature there are discussions about the advantages and disadvantages of different heuristic evaluation methods instead of providing guidelines for developing new usability methods and heuristics [22]. It is observed in the result of the systematic search resulted in a total of twelve documents (A01, A03, A16, A06, A12, A10, A04, A12, A17, A05, A09, A11) classified in the evaluation or comparison of sites Web. That is to say, in these documents defined methodologies and heuristics are used to perform the corresponding evaluations, instead of adapting said heuristics to make the evaluation that will be made to the website more specialized.

Only one article that evaluates aspects of culture in usability heuristics could be found. Case A13 entitled “A cultural-oriented usability heuristics proposal”, results in the application of a procedure where they propose thirteen heuristics, which are associated with the dimensions raised by [11].

5 Conclusions and Future Works

Culture-oriented heuristics are used in usability evaluations of different types of websites. These proposals were made through procedures, based on traditional heuristics to which the cultural factor has been added according to cultural models.

This work allowed us to identify the existing procedures in the literature that are used to establish culture-oriented heuristics, the characteristics of web interfaces that can be related to cultural aspects and the type of website to which the established heuristics is usually applied: e-commerce.

However, heuristics cannot necessarily be applied to all websites equally. Therefore, more analysis is needed to determine which aspect should be prioritized according to the type of website of interest and the cultural characteristics of its target audience, in order to focus on those aspects when cultural-oriented heuristics are established.

In this way, it is also necessary to develop procedures that take this aspect into account to establish culture-oriented heuristics that will be used to evaluate the usability of the website.