Keywords

1 Introduction

Nowadays, there is no doubt that the World Wide Web or WWW plays an important role in different sectors of society such as business, government, education, science, leisure, among others. Private and public organizations have benefited from this technology by offering information about their products or services through websites.

In the field of education, university websites offer the information of interest about degree programs they offer to prospective students. Although these websites should ideally reflect the needs of their users, the web design is often dictated by the technology used to develop and design them or by business objectives, not taking into account the needs of users [8]. As a result, searching for information of interest can be difficult for those who access these websites for the first time [3].

This situation has resulted in the owners of these websites, as well as the personnel involved in their design starting to study and incorporate aspects of usability in their university websites [17]. Although at present we have a body of knowledge that can help us to improve aspects of usability in these types of websites, we observe that there are still university websites that can be improved by taking usability aspects into account.

An example of this is observed in the national arena (Ecuador). The search for information about bachelor’s degree programs in different universities can be a complex task; this because each university offers a different structure on their website. In some cases, the required information is easy to find, but in other cases, it is difficult to access.

Due to prospective students are an important type of user, the owners and web designers must offer the necessary usability mechanisms that allow for this type of user to browse and find information of study programs in an agile and simple way.

In order to face this situation, in this work we present a merged interaction pattern focused on improving the navigability of university websites. This pattern is developed with the aim of facilitating the search for information of study programs for prospective students who wish to enroll in a given undergraduate university program. The proposed interaction pattern was implemented as part of a prototype for an Ecuadorian university website.

The proposed pattern was evaluated using a sample of 266 prospective students. As a research method, we conducted a pretest-posttest intervention study. Participants were assessed in two different time periods (sessions) using the website of an Ecuadorian university. One session without the interaction pattern developed (pretest or control) and the other session with the website prototype (posttest or the intervention) that includes the proposed interaction pattern (intervention).

The rest of the document is organized as follows: Sect. 2 presents the related work; meanwhile, Sect. 3 deals with concepts and basic terminology. In Sect. 4 the merged interaction pattern is presented. The research method used is described in Sect. 5. In Sect. 6, the results obtained are presented and finally, in Sect. 7, we present the discussion and conclusions.

2 Related Work

The study of usability in university websites has gradually begun to be of interest. For example, in [18], authors conducted a study in order to have more knowledge of how prospective students use university websites. A total of 55 students from four high schools participated in the study. Authors used a survey which addressed different aspects of a university website. Content, site architecture (organization), ease of navigation, speed of connection (download), focus on target audience, distinctiveness of site, and importance of graphics were among the aspects studied. Authors found that content, organization and ease of navigation were the most important aspects of a university website according to the participants.

In the work of [3], the author proposes a series of practices in order to help the developers and designers of university websites to improve the navigability and organization of their content.

In other work of [16], authors focused on evaluating nine different Jordanian universities. The evaluation was performed through a survey and the use of automated tools. Authors came up with a list of 23 website usability criteria.

In [8], authors carried out a study in order to identify factors affecting the usability of a sample of Lebanese universities websites. Authors found that management and leadership in the ICT department, user testing and ease of use were key factors that may affect the usability of university websites.

In the work of work [15], authors created a usability evaluation model, this model was used for evaluating ten university websites in Asia. The websites were evaluated in two phases. First authors conducted a survey using thirty participants (students) enrolled in three different universities. In this first stage, authors identified problems faced by the participants while interacting with the analyzed websites. The outcomes of the survey were used to create the usability evaluation model. This model consists of 17 measures divided into five feature categories such as website content, webpage design, navigation, page design layout and accessibility.

In the work of [6], authors examined the usability and user acceptance of various websites of African universities with the aim of identifying some criteria that best evaluate these aspects. Authors observed that interaction, appeal, application robustness and invisibility measures have a relevant impact on usability and user acceptance.

In the work of [19], authors evaluated the usability and accessibility of three university websites. Authors used two types of usability evaluation techniques: a questionnaire-based evaluation and a performance-based evaluation. Usability assessment was done by analyzing the observed task success rates, task completion times, post-task satisfaction ratings and the feedback of 68 participants.

Recently, in [7], authors studied the usability of 24 Ecuadorian university websites using data mining techniques. Authors found that 50% of the websites assessed meet certain usability standards. As we observe, the study of the usability in university websites has been performed either using participants or automated tools.

3 Basic Terminology

In this section, we present some basic terminology related to the topic addressed in this paper.

3.1 Usability

Different definitions of usability can be found in the literature; the usability definition is mainly based on the different attributes or factors by which it can be evaluated [9]. One of the first usability terms was coined in 1979 [4]. It was used to describe the effectiveness of human performance [4], In [20], it is defined as the ability of something to be used by humans in an easy and effective way. The ISO/IEC 25010 standard [11] defines usability as the degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

3.2 Navigability

The navigability is the ability of the user to be intuitively guided through a website. Usually, the navigability is integrated on the homepage of a website; it is closely related to meeting the needs of the different types of users that visit the site. Consequently, it can be said that good navigability exists if a type of user finds the links that lead you to what you are looking for in the pages of the site (especially in the beginning) [12], thus having a good mechanism of navigation will positively influence the usability experience of the user.

3.3 Design Patterns

According to [2], a design pattern describes a problem that occurs continuously in our environment, and it describes the base solution to a particular problem. In general, patterns describe solutions to problems that occur repeatedly in our environment [1].

In the software engineering arena, design patterns follow a series of steps that describe the problem, the solution and its application, as well as its consequences [10].

We can find different types of patterns; on the one hand, we have the interaction design patterns that have the objective of solving usability problems. While the software design patterns are aimed to solve functional problems, both of these two pattern-based design models complement each other, and can be used together with successful results in the design of interactive applications [21].

Although both types of patterns are clearly intended to solve different problems, usability and functionality cannot be treated as independent aspects of an interactive application. The usability not only depends on the design of the interface but also on the design at the architecture level of the application; this means that a design decision at the functionality level will affect the usability, and vice versa.

4 Proposed Interaction Design Pattern

In this section, we present the merged pattern focused on prospective university students. Taking the pattern structure used in [10] as reference, we describe the proposed pattern.

  • Pattern name: Prospective Student

  • Intention: To improve the navigability and usability of universities websites.

  • Problem: Based on a focused problem, higher education institutions may have websites with different designs and content organization, which makes the search for information by prospective students a difficult and demanding activity.

  • Usability principles:

    • Visibility of the system status.

    • Recognize before remembering.

    • Flexibility and efficiency of use.

    • Aesthetic and minimalist design.

  • Context: The present pattern can be implemented in university websites with the aim of improving the browsing experience of prospective students who are looking for information regarding study programs.

  • Solution: With regards to the website’s main menu, the use of a vertical dropdown menu is proposed [22], since it is necessary that the options of each section of the menu are directly visible.

We also suggest the use of breadcrumbs [22] as a solution to the problem of location that users may have when browsing the website; this also allows users to return to higher levels quickly because a hierarchical route is displayed at any given moment.

In this proposed pattern, the use of split navigation is also considered [22]. Having split navigation allows prospective students to change quickly between pages on different levels. It consists of having a home page (first level) and secondary thematic pages, the home page will contain the links of interest that the website has. By clicking on a link, the student will navigate to a second level or thematic pages.

The thematic web pages should contain an index to further levels (for example third and four webpage levels). All textual information is displayed on this thematic page. Users can switch between pages in the third and four levels by clicking on the index links. The links of the homepage of the split navigation will be grouped by means of a menu without header (headerless menu) [22]. This is a type of vertical menu that combines different menus using different visual clues instead of headers.

Finally, we propose that thematic pages be guided by doormat navigation [22], specifically speaking of the offered programs degrees section. It is important that prospective students be directed to the proper degree program. It will give them a quick overview of the programs offered without having to click on them.

  • Consequences:

    • It improves the navigability of prospective students, since links and contents are better organized.

    • It reduces redundant links and the resulting website structure is minimalist and clean.

  • Benefits:

    • It provides a user-friendly interface.

    • The contents are presented in an orderly manner.

    • The prospective student finds information of programs degrees more easily.

  • Example: An example of the proposed merged pattern is shown in Fig. 1. The pattern is implemented in a prototype of a university website.

    Fig. 1.
    figure 1

    Example of the proposed pattern implemented in a university website prototype.

    Full size image

  • Related patterns: vertical dropdown menu, breadcrumbs, split navigation, headerless menu, doormat navigation.

5 Method

In this section, we describe the steps involved in the creation and assessment of the proposed pattern. Figure 2 shows the steps we follow.

Fig. 2.
figure 2

Steps involved in the reported pretest-posttest intervention study.

Full size image

5.1 Usability Questionnaires Selection

For the first step, we selected two widely known validated instruments specializing in the assessment of usability [13, 14]. The first instrument also known as “USE Questionnaire” (Usefulness, Satisfaction, and Ease of use.) proposed in [14], is a questionnaire composed of four sections that assess usefulness, ease of use, ease of learning and satisfaction. Each of these sections includes items to be answered using a seven point Likert scale (ranging from strongly disagree that represents a minimum value of 1 to strongly agree that represents a maximum value of 7).

The second instrument we chose is the one known as After-Scenario Questionnaire and is described in [13]. We chose this questionnaire in order to triangulate results. It evaluates aspects such as the ease of completing tasks, the time to complete a task and the adequacy of the support or help information (not considered for this study). The After-Scenario Questionnaire (ASQ) is a three-item questionnaire (also uses a seven point Likert scale, from strongly disagree to strongly agree) that addresses three aspects of user satisfaction with regards system usability: ease of task completion, time to complete a task, and adequacy of support information (on-line help, messages, and documentation; this item was not considered in our study because university websites do not usually make online help available).

Both questionnaires are intended to measure the perceptions of the users with regards to the usability of a determined software product, in our case, a university website. Once the usability questionnaires to be used were selected, we adapted them to Spanish.

5.2 Pretest

An Ecuadorian university website was selected as a control group for the pretest. We contacted to the management staff from a high school located in Riobamba, Ecuador and we explained the study to them. They were interested in our study and agreed to participate.

In this step, an initial sample of 274 prospective students (students in the last year of high school) participated in the pretest. These students belong to different specialization areas from this school. Table 1 shows the distribution of the participants.

Table 1. Summary of prospective students that participated in our study.
Full size table

The day of the pretest, we gave to participants directions of how to complete the usability questionnaires; we also gave the participants a couple of tasks they had to complete, such as looking for specific information in a defined university website.

Participants were asked to search specific information of three bachelor’s program degrees. Participants only had ten minutes to complete this task. After the time assigned for this task was up, the participants were asked to respond to the questionnaires.

We also asked to participants to give comments and feedback of the tasks they performed. For example, we asked them to look out for and record any possible problems encountered while navigating through the university website.

For this study, we had access to a computer lab with 15 computers connected to the Internet. In conjunction with the teachers, we divided the participants into groups of 15.

5.3 Merged Interaction Pattern Development

Taking into account the comments and feedback of participants, we designed the merged interaction pattern described in Sect. 4. Once the pattern was designed we built a prototype of the university website used for the pretest. This prototype contains the designed merged pattern.

5.4 Posttest

Five weeks after applying the pretest and once the prototype containing the merged pattern was complete, we conducted the post-test. The same students that participated in the pretest also participated in the post-test. The same questionnaires with same directions were given to the participants. In this session, participants worked with the prototype developed, which included the merged interaction pattern.

It is worth noting that we had to remove eight measurements due to some participants not attended either the pretest or post-test sessions, thus resulting in a sample size of 266 participants. According to the study design used, it is necessary that the same participants in the two different sessions be exposed to the pretest and the intervention (posttest).

6 Results

In this section, we present the findings of the study with regards to each of the aspects assessed in the questionnaires used. Findings were examined by means of descriptive and inferential analysis.

6.1 Usefulness (First Questionnaire)

Usefulness is defined as the extent to which participants considered the information provided by the university website (actual [pretest] and the prototype [posttest]) to be useful. Table 2 shows a summary of the descriptive statistics observed in the pretest and the posttest (website prototype which implements the merged interaction pattern). For each participant, all of the items belonging to this aspect were averaged, so the mean and the median were calculated from the averages of the sample used. These two values (mean and median) are in the range of the Likert scale used (from strong disagree [the minimum value is 1] to strong agree [the maximum value is 7]).

Table 2. Descriptive statistics related to usefulness.
Full size table

As shown in Table 2, on average, participants yielded a higher score in the posttest that in the pretest, i.e. participants perceive the prototype to be more useful than the actual university website assessed. In order to visually assess the difference between the pretest and the posttest, we used a Bland-Altman plot [5] as shown in Fig. 3. A Bland-Altman plot or difference plot is used to analyze the agreement between two different essays or samples.

Fig. 3.
figure 3

Bland-Altman plot related to usefulness.

Full size image

As shown in Fig. 3, the differences between the pretest and the posttest are mostly distributed below zero suggesting a difference of these two samples in favor of the posttest.

6.2 Ease of Use

This aspect refers to the ease with which participants were able to use or navigate through the university website. Table 3 shows the descriptive statistics with regards this aspect.

Table 3. Descriptive statistics for the ease of use aspect.
Full size table

On average, participants in the posttest session showed a better performance than the pretest (as shown in Table 3). Figure 4 shows the difference plot related to this aspect.

Fig. 4.
figure 4

Bland-Altman plot related to ease of use.

Full size image

As we see, most of the differences shown in the Fig. 4 are distributed below zero, suggesting a difference in favor of the intervention (posttest session).

6.3 Ease of Learning

In this case, ease of learning is related to the amount of effort that participants expend learning to use the university website. Table 4 shows the descriptive statistics related to this aspect.

Table 4. Descriptive statistics related to ease of learning.
Full size table

Also for this aspect, participants showed a better score using the university website prototype than the actual university website. The difference plot (not shown due to the limitation of page space) also yielded a similar distribution to the observed in Figs. 3 and 4.

6.4 Satisfaction

It refers to the extent to which participants perceive the university website to meet their information requirements. Table 5 shows the descriptive statistics of the sample used.

Table 5. Descriptive statistics of the satisfaction aspect.
Full size table

As observed in Table 5, on average participants in the posttest session perceived a better degree of satisfaction than in the pretest session.

6.5 After-Scenario Questionnaire (Second Questionnaire)

In order to triangulate the results, we also used a second usability questionnaire. As mentioned in Sect. 5, this questionnaire is composed of three items. Table 6 shows the resulting descriptive statistics.

Table 6. Descriptive statistics related to the second questionnaire used.
Full size table

Results shown in Table 6 also suggest a better performance in the posttest session than in the pretest. The results yielded with this second questionnaire (After-Scenario Questionnaire [13]) are consistent with those of the first questionnaire (USE Questionnaire [14]). Figure 5 shows the resulting difference plot.

Fig. 5.
figure 5

Bland-Altman plot related to data of the After-Scenario Questionnaire [13].

Full size image

As observed in Fig. 5, most of the differences are distributed below zero, suggesting a difference in favor of the posttest session.

6.6 Inferential Analysis

Once we have an exploratory analysis of the data sample, we proceed with an inferential analysis in order to examine possible significant differences of our pretest-posttest study.

Because the items of the two questionnaires used are measured with an ordinal scale (Likert scale), it is probable that the sampled data is not normally distributed. In this situation, a non-parametric inferential analysis is more useful. In our case, after conducting the Shapiro-Wilk normality test for each of the aspects of the both questionnaires, for each aspect, we rejected the null hypothesis (at an alpha level of 0.01) of this test in favor of a lack of normality of the data.

In order to compare the pretest and posttest for each aspect of the questionnaires, we used the Wilcoxon signed-rank test. Table 7 shows the results of this test.

Table 7. Inferential analysis using the Wilcoxon signed-rank test.
Full size table

As noted in Table 7, all the tests performed show significant differences at an alpha level of 0.001, suggesting that the sampled data in the pretest and the posttest belong to different populations. Together with the descriptive analysis, we observe that the intervention studied (the prototype with the interaction pattern) caused significantly different results.

7 Discussion and Conclusions

According to our results, the use of a merged interaction pattern has improved the perception of usability among the prospective students who participated in this study. In the case of the first questionnaire used (USE Questionnaire [14]) the perception of usefulness among the participants increased by 77% (taking as reference the average of the pretest); the perception of the ease of use increased by 96%; the perception of the ease of learning increased by 97%; the perception of satisfaction increased by 100%. Combining these four aspects, on average, the perception of the usability has increased by 93% in the university website prototype. A similar percentage is observed in the second questionnaire (After-Scenario Questionnaire [13]), participants perceived an increase of the usability by 95% with regards to the current university website (pretest). Both instruments yielded consistent results in favor of the merged interaction pattern proposed.

Although our results are promissory, it is necessary to conduct further research in order to make improvements or adaptations to the proposed pattern. Also, this work can serve as reference for conducting replications with other prospective students in other schools. The main contribution of the present work is the presentation and assessment of a merged interaction design pattern focused on university prospective students. As a concluding remark, the use of interaction design patterns in university websites help to increase the usability experience of prospective students.