Keywords

1 Introduction

For the last twenty years or so, computing has provided the world with countless general and specific purpose computer systems to perform almost any task that a human must carry out in his or her social, work, and even leisure context [1, 2]. The use of mobile devices has considerably increased the use of software tools in different environments of everyday life. In particular, the continuous advance of mobile technologies is leading to a new generation of software tools, called mobile applications, which are those applications developed to be used on any mobile device. However, some of the success factors of software applications have to do with: the ease with which users can interact with them, the ease with which users can interact with them, and the ease with which they can use them [1,2,3], the accessibility [4, 5], and the effectiveness in the performance of tasks, among others. The existence or absence of these factors determines the success or failure of each of the developed systems, which is why, in the design of any computer system, it is necessary to pay special attention to the elements that must contemplate the interface [6]. But most mobile applications are difficult to use, inflexible, and not robust [7]. In addition, it is necessary to have methodologies and tools that consider specific usability elements for the development of applications used on mobile devices [7]. Currently, there is a lack of knowledge, lack of what and how to apply usability in specific contexts, therefore, considering its importance, this paper shows the process of building a usability guide for the design of interactive mobile applications in the context of learning mathematics for children aged 6 and 7 years, for which the methodology of multicycle action research with bifurcation was used, with which the conceptualization, construction, and validation of the usability guide was performed.

2 Related Works

In [8] guidelines for incorporating usability in Interactive Television (ITV) and evaluation methods are defined. In [9] the usability recommendations to be considered in order to improve student interaction with educational Web sites are mentioned. On the other hand, in [10] guidelines were defined for the development of interactive educational tools for psychomotor development in children. In [11] determines the elements, components, and factors that are key when designing interactive interfaces for augmented reality environments. In [12] best practices for usability in the design of the virtual learning environment interface are identified, similarly, in [13] systematizes the design and development of M-Learning applications by defining design and development guidelines for mobile educational applications. In [14] defines principles and guidelines on the design of 3D user interface applications. On the other hand, in [15] describes a study of the principles and requirements of user interface design using mobile devices as a tool for mobile learning purposes. At [16] a game for children was designed based on the data obtained in surveys on the mathematics axis and color preferences. Similarly, in [17] a mobile application was designed to support learning in the subject of Mexican history, whose purpose was to be a form of didactic support for teachers who teach this subject at the elementary school level.

3 Methodology

This work was developed following the multi-cycle action research methodology with bifurcation [18], which is detailed below:

3.1 Conceptual Cycle

Considering that for this work children are the end users of the applications to be designed with the usability guide, it was necessary to define their main characteristics and needs in the context. For this purpose, the user-centered design (UCD) methodology was used as a basis [19]. The UCD involves the user from the first steps of the development process and the type of user determines the design, where the scope of dissemination, the age of the user, and the user's needs must be determined. Thus, in order to identify the characteristics and needs of children, it was necessary to conduct a survey with teachers who teach mathematics, since they are the ones who directly know the needs that must be solved through the applications that are developed using the usability guide. The chosen age range of 6–7 years was due to the fact that the Colombian Ministry of Education establishes that children of this age must be in the first grade of primary school and is in the pre-operational stage, it is at this time where the child begins to develop its logical and cognitive stage and understand the basic elements of the different areas of education, this combined with its rapid adaptation to the technological world today makes children have other options to learn faster and more agile the nature of the subjects [20] The survey was conducted with a total of 23 first-grade teachers (grade in which the principles of mathematics are taught and the children are on average between 6–7 years old).

Some of the most relevant results obtained from the survey were as follows:

  • 70% of the teachers assure that the highest rate of the problem at the moment of learning is covered by addition and subtraction, given that children do not know at what moment to add and at what moment to subtract, in the same way, many times they do not recognize numbers or find it difficult to count numbers of 2 digits or more, so it can be determined that for applications the subject to be treated should be one of these two topics.

  • The most used and accepted method to make a better study environment is the use of tangible media (60%), which contributes to knowing how to adapt this to the final mobile prototype.

  • 65% agreed that the final prototype to validate the usability guide should be oriented to the development of a game, given that as they are children the best way to make this type of operations arrive.

  • 95% of the teachers felt that the use of mobile applications helps to learn and that it can be a powerful tool to reach children.

  • 60% of the teachers suggest that the application should have an activity that lasts 15 min on average and that it should include some process that helps the instruction of the class.

  • 60% of the teachers suggest that they should be logical thinking applications.

3.2 Methodological Cycle

The following steps were followed to define and adapt the guidelines:

Literature Review.

First, an analysis of the literature on usability guidelines applied to any context was made. A large number were found, mainly those defined by Nielsen [21], where most of them are oriented to the design of web applications, which are recommendations to take into account in applications without a specific context [22]. Despite the boom of mobile applications and their massive use, one of the pending issues is the definition and classification of guidelines and appropriate metrics to ensure usability in these applications, since they do not follow metrics, are not considered or there is simply no way to guide their application [23]. From the literature analysis, a list of general usability guidelines was obtained, the problem is that these are not focused on the mobile part, in addition, they were disorganized and in different sources, so it was necessary to structure them and adapt them to the mobile part.

First Version of the Guide.

In order to define the first version of the guide, the guidelines analyzed from the literature and the study of the needs of the end user were taken into account. This made it possible to define that mathematics topics and ages between 6-7 years old are characteristics that need to be supported in the definition of the guidelines. As a mechanism to standardize the description of the guidelines, a brochure format was designed, as shown in Table 1.

Table 1. Structure of the guidelines
Full size table

Table 2 shows a summary of the list of guidelines with the identifier and name of each guideline and its source.

Table 2. List of guidelines
Full size table

Classification According to Standard 9241-11 Usability

Once the guidelines were defined in their first version, the efficiency, effectiveness, and satisfaction factors of the ISO 9241-11 standard were used as a classification tool, thus obtaining a list of the guidelines that contribute to each of these factors. In order to classify the guidelines, it was necessary to ground the above factors in the context in which the guidelines are used, which is why they were redefined: Effectiveness: Degree of precision with which the interactive mobile application allows supporting the learning of mathematics (The sum) in children between 6–7 years old, in addition to generating in children the need for the use of the application. Efficiency: Degree to which the interactive mobile application supports the learning of mathematics (The sum) in children between 6 and 7 years old, taking advantage of the resources provided by mobile devices (interactivity, sound, interaction capacity, simplicity, multi-functions, motivation, gamification, processing, among others) to achieve its final objective. Satisfaction: The degree to which a user of the interactive mobile application (child between 6–7 years old) has positive attitudes, acceptability, or appreciation towards its use, in addition to the degree of ease of use presented by users at the time of use.

Once the usability factors were defined and grounded in the context, a classification was made according to the three factors as shown in the following table, in this work only the classification for the effectiveness factor is shown, see Table 3.

Table 3. Classification of guidelines considering effectiveness
Full size table

Inclusion and Exclusion Criteria

Once the classification process was done and considering what was analyzed in the literature, the usability standard, and the experience, inclusion and exclusion criteria were created to filter and discard the guidelines that do not provide usability or do not apply directly to the mobile context as for learning mathematics, the criteria are shown below:

Criteria for inclusion of guidelines.

  1. 1.

    Does the guideline comply with the user-centered development?

  2. 2.

    Does the guideline provide functionality to the application?

  3. 3.

    Is the guideline adaptable to mobile design?

Exclusion criteria for guidelines.

  1. 1.

    Does the guideline does not comply with user-centered development?

  2. 2.

    Does the guideline does not bring functionality to the application?

  3. 3.

    Is the guideline not adaptable to mobile design?

Applying the above criteria, the guidelines shown in Table 4 were discarded, which shows the justification for their elimination.

Table 4. Guidelines that do not meet the criteria
Full size table

Once the entire process was completed, a second version of the guidelines was obtained, with which a series of validations and refinements were made, which are shown as part of the validation cycle.

3.3 Validation Cycle

To validate the second version of the guidelines obtained in the previous chapter, a series of tests were generated, including a proof of concept, an evaluation with experts and a case study.

Proof of Concept (PoC)

Objective of the proof of concept: to know the problems that may arise when using the guidelines and to verify if the guidelines really contribute and guide the user when designing an interactive mobile interface for learning mathematics in children between 6–7 years old. Context: it was decided to generate a series of mockups considering the usability guidelines. This validation was carried out by the members of the research team, who developed two prototypes. Execution: In the execution two prototypes were developed, in the development of prototype A was invested between 2 to 3 h and for the development of prototype B, 1 h. Some shortcomings were found in this process. In order to analyze and solve these shortcomings, a meeting was held with the team involved, which resulted in the following considerations:

  • The material lacks ease of use since there are no guidelines to indicate the order of the guidelines or where to start using them.

  • Some guidelines could complement each other, since there are some that were similar and therefore there is a repetition of information

Results of the Concept Test:

From this test, the first measure obtained was the need to create a support guide for the management of the guidelines, which provides a series of steps and recommendations on how to apply the guidelines, for which it was considered that the guide should have a flow chart that would also support the management of the guidelines, in order to facilitate their use and provide a step-by-step textual and visual aid. In addition, the concept-to test identified the need to factor and refine the guidelines in order to solve the problem of repetition of information.

Expert Evaluation

The expert evaluation was carried out in order to obtain input, opinions and different points of view from experts in the area of usability to serve as a refinement mechanism and thus have a stable and reliable improved version. Objective: To validate with usability experts the completeness, suitability, ease of use and ease of learning of the guidelines and the guide. Context: The validation was carried out by means of questions addressed to six usability experts, in order to listen to suggestions that could be considered in subsequent versions of the guidelines. 66.7% of the experts have more than 5 years of experience in usability and 33.3% have between 3–5 years of experience. The survey was based on the Likert scale [41], and the questions were categorized according to the following variables: completeness, suitability, ease of use, ease of learning. Each of the experts was given the user's guide, the flow of use and the compendium of guidelines, as well as the link to the survey for validation. The results obtained from the expert evaluation are summarized in the following conclusions.

4 Conclusions

  • According to the perception of the experts, it can be stated that more than 80% of the respondents agree that the guidelines have the necessary elements to be easy to learn.

  • According to the perception of the experts, it can be concluded that the guidelines do not contain the necessary elements to affirm that they are easy to use, given that more than 50% neither agree nor disagree.

  • According to the perception of the experts, it can be concluded that the guidelines contain the necessary elements to affirm that they are complete, since more than 80% of the respondents agree.

  • According to the perception of the experts, it can be concluded that the guidelines do not contain the necessary elements to affirm that they are suitable, given that only 50% of the respondents agree with this statement.

  • According to the perception of the experts it can be concluded that the guidelines contain the necessary elements to affirm that they are easy to learn, given that more than 80% of the respondents agree with this statement.

  • According to the perception of the experts, it can be concluded that the guide contains the necessary elements to affirm that it is complete, given that more than 80% of the respondents agree.

  • The guide contributes optimally and positively to the ease of use of the guidelines, but there are also points to improve, such as the organization to provide a better experience.

  • The evaluation and interaction with experts in usability helped to positively improve the design of these, since they provided points of view that had not been considered, this experience provides greater feedback and improvement because the experts were willing to collaborate in different aspects.

According to the results and opinions of the experts, the following changes were made:

  • Guideline No. 42 is merged with guideline No. 25, since they complement each other, thus making No. 25 complete.

  • Guideline N°34 is eliminated, since it was repetitive with guideline N°38, which is more complete.

  • Guideline N°41 thematic scope is reformed, since its structure was poorly focused and was more a definition of the context to be addressed than a guideline on how to define the thematic scope.

  • A correction was made in the way the guidelines were written, most of the descriptions were changed since they were not very complete and clear.

  • The application steps of the guidelines were completely reformed since it was one of the major shortcomings of the list of guidelines, they were organized by numbers to facilitate their application according to how they are listed.

  • The steps that were mandatory and those that could be optional were placed, and their wording was improved since observations were made on the content of the description of the step.

  • The guidelines were ordered with their id according to the proposed guide for their use in this way they are ordered and easier to use.

  • The guidelines were classified into mandatory and alternative guidelines in order to provide flexibility in their use.

  • The examples were adjusted to the specific context proposed, described in more detail and thus provide better guidance to the developer.

  • The guide was supplemented so that each heading of the guideline explained in a general way about its content. In addition, the structure of the guide was organized by steps for a better description.

According to the results and changes considered in the expert evaluation, version three of the guidelines and its respective guide was generated. In addition, it was decided that in order to facilitate the use and understanding of the guidelines, a light version will be created, which only has the id, name and description of the guideline, so that there is a simple, more reduced, easy to use and understand version.

Case Study

The usability guidelines and their application guide were tested in the development of a case study, which allowed us to inquire their applicability.

Case Study Question:

Based on the research question of the project, it was necessary to evaluate the applicability of the guidelines to ensure usability in the design of interactive mobile applications for learning mathematics in children between 6 and 7 years old. Therefore, the question for the defined case study was: is the support for the design of interactive mobile applications for learning mathematics in children between 6 and 7 years old, through the application of the usability guidelines and its respective guide, useful, easy to use and complete? Objective of the case study: The objective of the case study was to verify that both the usability guidelines and the guide for their use were useful, easy to use and complete for the context they were developed. Selection of the case study: The practice of process definition, observation and execution of the case study is developed by the researchers, which consisted of evaluating the usefulness, ease of use and completeness of the proposal. The primary source of information is those responsible for applying the guidelines with the interest of ensuring usability in the final product, the case study is holistic type considering a unit of analysis with a research subject, the activity of using the guidelines and its guide was selected to emulate a real case in the development of mobile applications. Context of the case study: The development of the case study was conducted with a group of 15 students in their last semester of the systems engineering program at the Corporación Universitaria COMFACAUCA - Unicomfacauca. It is important to emphasize that the group that participated in the case study were students taking the elective in human computer interaction. Planning of activities: Table 5 shows a summary of the activities designed for the development of the case study and specifies the planned duration of the activities and the support instruments that were used for their development.

Table 5. Case study planning
Full size table

Evaluation Instrument:

For the evaluation of the case study different instruments were used, which are presented below:

  • Survey: Establishes a communication between the researchers and the study subjects in order to obtain data in writing and thus know the states of opinion, characteristics or specific facts, which are related to the case study.

  • Prototypes: Specification in mockups of the functionality of the outcome of the case study.

Observation Protocol:

Through this method, an attempt was made to establish a concrete and intensive relationship between the researchers and the case study participants. This protocol allowed the collection of data and information through observation presented in the case study. Indicators and metrics: in order to objectively evaluate the case study and answer the question formulated, it was necessary to define a set of metrics and indicators, which are described in Table 6 below.

Table 6. Indicators and metrics
Full size table

The detailed description of the indicators and how they are calculated through the identified metrics is as follows:

Usefulness:

This is defined as the degree of usefulness that a person perceives of the guidelines and the defined guide. This variable represents a useful perceptual judgment of the proposal. The guidelines that have been established to determine usefulness are:

  • The average degree of usefulness of the guidelines and the guide obtained from the students’ perception that is between 1 and 5 (5 being the highest degree of usefulness) must be equal to or higher than 80%

Ease of Use:

IT is defined as the degree of ease with which the person can understand the guidelines with their respective guide, this represents a judgment of perception of the effort required to understand what is proposed. The guidelines that have been established to determine the ease of use are:

  • The average degree of Ease of use of the guidelines and the guide obtained from the students’ perception that is between 1 and 5 (5 being the highest degree of ease of use) must be equal to or higher than 80%.

Completeness:

This is defined as the degree to which a person perceives that the guidelines and their respective guide contain the necessary elements to be described. The guidelines that have been established to determine ease of use are:

  • The average degree of completeness of the guidelines and the guide obtained from the students’ perception that is between 1 and 5 (5 being the highest degree of completeness) must be equal to or higher than 80%.

Execution of the Case Study:

For the execution of the case study, it was decided to apply the usability guidelines and their respective guide individually, while a group of student’s design mobile interfaces for learning mathematics, in order to analyze the pros and cons found by the end users in the guidelines. The students were asked to make a prototype with the necessary interfaces to meet a specific requirement, which was the following: “A school located in the city of Popayán, is looking for a group of developers in order to make a mobile application that provides support or help for math class, since it is one of the classes in which there is some difficulty in learning in students, teachers say that the difficulties occur in addition and subtraction so the application will be focused on these issues, the application seeks to motivate and encourage students for their progress in the app, teaching in a fun, didactic and creative way, the interface should be friendly and simple since this will be directed to children of 6 and 7 years old which have basic knowledge in reading and identification of numbers, the school wants to know the idea of the developers before selecting a final development so it is requested to make the Mockups corresponding to the app that will be offered to the institution”.

The development of the case study was carried out according to plan, but it is important to highlight that the time of some of the activities was exceeded.

Conclusions of the case study:

  • According to the perception of the students it can be concluded that 86% agreed on the usefulness of the guidelines in the activity.

  • According to the perception of the students 86% agreed on the completeness of the guidelines, therefore, it is affirmed that the guidelines are complete.

  • According to the students’ perception, 92% approve of the ease of use of the guidelines, largely because of the examples stipulated in the format, since these give a clearer vision of what is intended to be understood and expressed with the guidelines.

  • According to the perception of the students, it can be concluded that 96% agreed with the usefulness of the guide, since it complements the list of guidelines.

  • According to the perception of the students, 86% agreed with the completeness of the guide since it facilitates the understanding and gives a brief description of the guidelines, therefore the application and justification of the order of the guidelines is understood.

  • According to the perception of the students, more than 96% agreed with the ease of use of the guide since it provided the guidelines, steps to follow and the terms used in the guide were adequate.

  • It can be concluded that the purpose of the guidelines is fulfilled given that despite the fact that each of the attendees had a different idea, the usability guidelines were applied in the same way in their Mockups.

  • From the development of the case study a positive response was obtained from the students, they were satisfied with the use of the guidelines in the development of their Mockups, there were no problems and they felt correctly instructed by the guide, finally, they stated that the guidelines really provide a support to develop interactive mobile applications with usability elements, in the learning of children between 6 and 7 years old.

  • It was noted that the students did not necessarily read all the information to understand and apply the guidelines; in some cases, they only relied on the examples to understand them.

  • It was noted that the graphic that contains the guide to explain how to use the guidelines was omitted in the development of the activity.

  • Some students were confused at the beginning of the activity because they thought that the guidelines provided the idea to be developed, when in fact the objective of the guidelines is to guide an idea with concepts that provide usability.

From the results and opinions of the students, the following changes were made:

  • Guideline N°10 lacks an example to better understand its application.

  • Guideline N°4 has an example, but it does not have a guiding image alluding to what is being said.

  • It was suggested to change the name of step 2 of the guide (refine) to a synonym or a more technical term.

Conclusions

This paper presents the process of construction of usability guidelines for the design of interfaces in interactive mobile applications in the context of mathematics learning in children between 6–7 years old, which were built from a constant validation that was performed in several phases, the first was to check if the content of the guidelines were framed in the concept of usability proposed by the categories of the ISO 9241–11 standard, In the next phase, a proof of concept was evaluated, which allowed to analyze missing elements when applying the guidelines and difficulties that arose in this process, which generated a new version of the guidelines and the creation of a guide to help with the use of these guidelines to generate an order of application, With these elements in a third phase, their completeness, suitability, ease of use and ease of learning were evaluated by usability experts, who gave their points of view and it was obtained that the guidelines were easy to learn, complete, but they were difficult to use and suitable in the context, on the other hand, the guide contains what is necessary to be easy to use, complete and easy to learn, With these results the guide and the guidelines were refined, which were finally validated through a case study, which sought to analyze their usefulness, ease of use and completeness through the development of a prototype of an interactive mobile application, developed by a group of students using this proposal, where it was obtained that the guidelines are complete, useful and easy to use on the other hand the guide is also easy to use, useful and complete.