Keywords

1 Introduction

1.1 Dyscalculia

A child with dyscalculia can exhibit several symptoms, such as difficulty in recognizing numbers, in correlating numeric symbols (“5”) with their corresponding word (“five”), or even in recognizing patterns and putting things in order. Furthermore, dyscalculia not only affects children in a school environment, but also in day-to-day activities, like struggling to remember distinct numbers (e.g. postal codes or telephone numbers), issues concerning money, distinguishing left from right, watching the clock and telling the time [1].

Developmental Dyscalculia (DD) in children is relatively common, with a prevalence of 3 to 6% in the school population and with high rates of comorbidities such as ADHD and dyslexia [1, 8]. In point of fact, there is a relevant correlation between mathematical challenges and the inability to read: it is estimated that 40% of dyslexics also have a mathematical disability [5].

Furthermore, children with dyscalculia exhibit general deficits in number processing, including access to verbal and semantic numerical information, counting points, reciting number sequences and writing numbers [6]. Despite this, children with dyscalculia but no reading disabilities presented average, and even above average, results in tasks involving phonological working memory, access to non-numerical verbal information, non-verbal intelligence, language skills and psychomotor skills [10].

In this paper, we explore the impact of an application created with the purpose of teaching mathematic with students from different school years. After briefly presenting the related work and discussing the importance of such an application, we describe the methodology used for this case study and its results. By reporting our observations, we throw some light on how children of different school years may take advantage of an application purposefully created to help teach mathematics and, as such, how these types of applications could help children with developmental dyscalculia.

1.2 Related Work

In this section, we briefly outline scientific background related to developmental dyscalculia and the importance of creating interfaces that can help children with these disorders.

Research showed that children with arithmetic difficulties performed poorly on a task that required the memorization of dynamic visual information [11]. These results are consistent with other findings [4], that report lower performance in children with arithmetic disabilities regarding the memory game Corsi Block Tapping test. Thus, deficits in working memory systems have contributed substantially to specific deficiencies in the construction of a number’s cognitive representations, in the formation of concepts and procedures, and in the retrieval of arithmetic facts in children with DD [2, 3].

In another study [7], the aim was to assess the differences between strategy selection and information processing in children with and without mathematical disabilities, of the first and second school year. The chosen strategies and the time needed for the problem solving were recorded on an experimental basis and each was classified according to the association's strategy choice model. Based on performance, indexed by achievement test scores, the sample group with the learning disability (LD) was then subdivided and reclassified into two groups: an LD-enhanced group and an LD-no-change group (a group that did not show a great ability to work around problems). Nevertheless, there appeared to be no substantial differences between the two groups. The performance characteristics of the LD-no-change group, compared to the remaining group, included frequent errors in memory counting and retrieval, frequent use of an immature computational strategy, poor strategy choices, and a variable rate of information processing. These performance aspects were taken into account in terms of the strategy choice model and in terms of potential long-term memory deficits and working memory capacity.

During the research carried out, multiple applications were taken into account and analysed (Table 1) [12,13,14]. We concluded that none of the applications mentioned met all the necessary requirements for an improved learning by our target audience.

Table 1. Comparison of several applications already in the market.
Full size table

Hence, we want to create an easy-to-use application that tackles two challenges: (1) does not require constant adult supervision, but rather expects only assistance if needed; (2) has a simple and easy-to-understand vocabulary in which a child can understand any and all mathematical problems with no difficulty.

Also, as already mentioned, there are some situations where dyscalculia is associated with dyslexia, so we also intend to implement a reading function in order to simplify the problem’s interpretation. Indeed, our research shows that the only application with this feature is NeuroNation. However, it is not suitable for our target audience.

For all the described reasons, and considering the ultimate reality of the applications currently available on the market, we intend to develop an application that meets all of the following requirements for an improved learning experience: be ease of use; support the Portuguese language; provide mathematical problems, logical reasoning, memorization exercises and daily activities.

2 Case Study

We divided the case study into two different phases: (1) usability evaluation of the interface developed; (2) accessibility assessment of the application. The adopted methodology consists of a quasi-experimental design with a qualitative focus. A non-probabilistic convenience sampling technique was used.

2.1 Participants

In order to carry out the usability testing, we contacted the Augusto Moreno School Group, located in Bragança, Portugal. As such, the functional prototype was tested at the Santa Comba de Rossas primary school. Fourteen students participated: two from the first grade, six from the second, one from the third and five from the fourth grade. We assessed if the participants were used to using a computer and a computer mouse for study purposes through a questionnaire with closed-ended questions, at the beginning of each test. Indeed, only one first grader was not used to using the computer to study. However, he was used to working with the computer on a general basis and, as such, that fact would not skew our results in terms of performance. Out of all of the participants, ten of them had not yet used a computer to specifically study mathematics.

2.2 Apparatus

The activities were performed individually, in a controlled environment, with a participant observer (which oriented the user in the task’s fulfillment). All tasks were displayed on the computer screen. The following material resources were used: a computer monitor displaying the application, a computer mouse for interaction.

2.3 Application

The application was developed with precise requirements, which we describe next. Furthermore, a series of graphical assets were created for the branding of the application, such as a logo, icons and its graphical user interface.

Functional Requirements. In the application’s homepage (see Fig. 1), the user has access to the main menu, where he/she can select the intended option: either “Challenges” or “Teaching Games”.

On one hand, if the user selects the menu “Challenges”, the following steps will take place: (1) a new screen will appear, in which the user can choose their school year (see Fig. 2), and thus corresponding level of difficulty (“1st grade”, “2nd grade”, “3rd grade”, “4th grade”); (2) after selecting the intended option, the challenge begins, consisting of 5 multiple choice math problems related to the subject taught in the corresponding year (see Fig. 3); (3) at the end of each challenge, the application displays how many questions the user has answered correctly. Here, the user will have the option to repeat the challenge or exit to the homepage.

Fig. 1.
figure 1

Homepage featuring the main menu.

Full size image
Fig. 2.
figure 2

Screen of the second level of the application, when selecting the option “Challenges”.

Full size image
Fig. 3.
figure 3

Screen of the first question of the “Challenges” section, featuring multiple choice questions according to the level of difficulty selected.

Full size image

On the other hand, if the user selects the menu “Teaching Games”, the following steps will take place: (1) a new screen will appear in which the user can choose the educational game they want to play, either “Memory game”, “Quick Calculation”, “Association” (see Fig. 4); (2) after selecting one of the three games proposed, the game begins. The rules will depend on each educational game.

Fig. 4.
figure 4

Screen of the second level of the application, when selecting the option “Teaching Games” in the main menu.

Full size image

When the user selects “Memory game”, he/she can then choose the game’s difficulty level (see Fig. 5), between easy (numbers between 1 and 8), medium (numbers between 1 and 16), and difficult (numbers between 1 and 20). Each difficulty level works by the same principle: the user has to find the corresponding pairs of all the numbers displayed, using memory as the main element and exercising it. Similarly, if the user selects “Quick Calculation”, a new screen will display a sum and four possible answers (see Fig. 6). There is a time limit of 30 s for the game, but if the user correctly hits five questions in a row, this time will be extended. The game’s difficulty also progressively increases each time the user hits a correct answer.

Fig. 5.
figure 5

Screen displaying the memory game’s difficulty level.

Full size image
Fig. 6.
figure 6

Example of the quick calculation game layout.

Full size image

Finally, if the user selects “Association”, he /she will be asked to answer 10 problems following the same principle: associate cardinal numbers written in numerical form to their textual form and vice versa (see Fig. 7). Each question grants four answers, some of which are very similar to the right answer and others are quite different. At the end, the user submits the answers and the results obtained are displayed, as well as a motivating qualitative rating (see Fig. 8).

Fig. 7.
figure 7

Example of the association game layout.

Full size image
Fig. 8.
figure 8

Final screen with the results obtained by the user.

Full size image

Furthermore, throughout the application there must be a text-to-speech feature, where the user is provided with the possibility to hover the mouse over the words and hear them out loud.

Non-functional Requirements. The application was developed the specific non-functional requirements that dictated its correct use: (1) it must be in Portuguese; (2) it must have a responsive design; (3) it must only contain mathematical problems appropriate to the target audience’s age group; and (4) it must have a simple and straightforward language.

2.4 Experimental Design

The experiment made use of a within-participant design. The methods of data collection used are directly related to the research methods adopted.

At the beginning and end of each test, a specific questionnaire suitable for each age group was completed. During the test, direct observation and text annotations were very important to register all observation data and oral comments made by the users. Additionally, we performed interviews after the tests to better understand the users’ different attitudes and behaviors throughout interaction.

2.5 Procedures

We were given the opportunity to carry out the experiment for two hours during the school time. As such, due to the time constraints of having to perform the experiment with fourteen participants during only two hours, we decided to prioritize the application’s challenges section, instead of the teaching games. Each task was explained before the participant initiated it. The users were asked to: (1) select the intended option on the main menu of the application; (2) complete the corresponding tasks.

2.6 Results and Discussion

As previously explained, a final questionnaire was delivered to the participants, in order to assess their opinion on the application. Indeed, most of the participants acknowledged their positive feedback on the matter, saying they really enjoyed using the application. Out of all of the participants, only two had a negative feedback on the application, as they did not like operating it. Similarly, when asked if the interface was easy to use, one participant reported it was not easy, two said that it was roughly easy, and the rest stated it was easy to use.

Furthermore, when inquired about the understanding of the questions and the text in general, only three participants had issues with the text comprehension. However, all of the participants acknowledged that the use of the application could, indeed, help them with the study of mathematics if used on a regular basis. Also, only three participants were not sure if they would use the application again in the future. Contrarily, the rest of the students would definitely use the application again if the opportunity was provided. Regarding completion times, students completed their challenge within 4 to 8 min total.

Post-test Analysis.

After reviewing the results of the questionnaire, we can conclude that most of the students asserted that the application would be of great help for the study of mathematics, both inside and outside of the classroom context, regardless of the difficulties they may have experienced in their first use of the application.

Observations During the Test.

During the experiment, there were some observations worth mentioning: most students had difficulties in realizing that it was mandatory to click the “Next Question” button in order to move on to the next question; most students had difficulty identifying the “submit answers” final button as mandatory for the challenge submission; a significant number of students needed an explanation about the exercises to be able to answer correctly. Even though the application’s text was straightforward, the participants were not yet comfortable with the subjects.

Suggestions Provided by Students.

At the end of each test, we felt it was imperative to ask students for their suggestions, in order to improve the application. Overall, 1st, 2nd and 4th graders said that the challenges’ questions were too difficult and thus it would be good to have some easier queries, as the school subjects they had been taught so far were not sufficiently comprehensive. A 3rd grader felt that the questions for his grade should be more diversified and a 4th grader suggested that the application should allow some calculations to be executed, in order to facilitate the reasoning for the answer. As for the calculation game, a student suggested increasing the game time, as the 30-second limit was too short.

Accessibility Testing.

Regarding accessibility, during the development of the application we resorted to the “Wave platform” (Web Accessibility Evaluation Tool) for testing. It has a set of assessment tools that identify errors and warnings, in order to perfect the accessibility for individuals with disabilities. Thus, several tests were carried out and changes were made at the programming level, in order to rectify any errors. The last tests performed in Wave for our application did not show any errors. “Warnings” were presented; however, they did not alter the application's functioning. Also, as the final prototype had already been evaluated in the Usability Test whilst still manifesting these warnings, we decided not to make any changes in terms of programming. They should be, nevertheless, resolved later when continuing with our research.

Final Remarks.

Taking into account all the data collected from testing the prototype, we consider that there are some usability errors worth correcting, namely the buttons “Next Question” and “Submit Answers”. Indeed, only one of the fourteen students automatically inferred their use without any help. This problem could be solved by changing the design of the buttons (from text to arrows), thus rendering the buttons more intuitive, or by making sure the application automatically heads over to the following questions after the user clicks on the correct answer. As for the remaining pages (homepage, challenge choice, etc.), all students understood their content and there were no issues using the buttons. Therefore, we can consider that there is no usability error in them. With regard to colours and text, there were no negative observations reported by the students. In fact, the students who had some sort of difficulty with the text had, in general, difficulty in reading. This problem was easily overcome with the application’s text-to-speech feature.

3 Conclusions and Future Work

This exploratory investigation was intended to understand if an application developed for teaching mathematics could bring advantages to the students’ daily study. Although it is not possible to assume the benefits of the application for children with dyscalculia, as it was not possible to carry out a usability test in this context, we can state that the application would be an asset in both individual or supervised study for basic education students. Indeed, the results obtained suggest that these types of applications have a lot of potential as a driving force in the struggle against developmental dyscalculia.

Regarding usability, although there were some problems when testing the functional prototype, it fulfills our initial expectations. Furthermore, with minor adjustments we could improve its usability and overcome specific issues on this subject. On the other hand, regarding accessibility, tested using Wave, we believe that in time it would also be possible to correct the found “warnings”, thus achieving an even more accessible platform.

It is also important to report the suggestions and feedback provided by the school teacher. In this case, the teacher showed great interest in the application and even considered using it during classes or for home study. She pointed out that many of the problems encountered with regard to the questions’ difficulty stemmed from the COVID-19 obligation for confinement, causing a delay in teaching the corresponding subjects during the first 4 months of the calendar year. She explained that, perhaps, if the experiment was performed at the end of the school year, the aforementioned problems would not be so significant.

For future work, we consider correcting the interface’s design problems and perform a new, and complete, usability test, more focused on quantitative results. Also, the COVID-19 pandemic may become more controlled and allow for a school year completely on-site, minimizing the problems found due to the lack of these conditions.

Indeed, this pilot case study presents preliminary perceptions that may be vital to understanding the problems with teaching mathematics and the issues students face in this regard. Overall, we may infer that with some changes and a future testing of the prototype, the application would achieve a better outcome. Undeniably, the application created could be favorable to basic education students after the suggested changes.