Keywords

1 Concept Definition and Research Question

In educational practice and research, it is of crucial importance that educational practitioners and researchers to be concerned with the students with poor academic performance and to assist them to improve their academic performance. If one student in primary and secondary schools has poor academic performance, but cannot get timely help from the teachers and parents, he or she often has to receive remedial education before entering the ordinary higher education. How to identify a student as a student with poor academic performance remains to be discussed. The different cutoff scores used to identify poor academic performance often range from student examination performance at the 10th to the 45th percentiles. We define the student as a student with poor academic performance in one examination, if he or she has the score below the quartile in this exam (Geary and Hoard 2005). From the viewpoint of formative assessment, the student who just fails in one exam cannot be defined as a student with poor academic performance. We define one student as a student with poor academic performance during one learning period such as three school years, if he or she has poor academic performance in three quarters of all exams during this whole period.

In recent years, besides one-to-one tutoring and other traditional approaches, many intelligent or interactive online tutoring systems have been used to assist the students’ learning. The automatically recorded behaviors of the students using online learning systems can serve as one important data resource to analyze the learning features and to explore the relation between online learning behavior and normal exam performance. In this paper the following two questions will be addressed regarding the students with poor academic performance in mathematics study:

What are the online learning features of those students?

What is the relation between their online learning feature and their performance in normal school exams?

2 Related Work

One student with poor academic performance has learning difficulties, and cannot improve his or her competence by self-regulated learning. A great number of researchers have studied the students’ learning difficulties generally (Sweller 1994; Chappell 2000; Gustafsson et al. 2013) and in different school subjects including mathematics (Shalev 2007; Lewis 2014; Roschelle et al. 2016), language (Ellis 2006), science (Bahar et al. 1999; Sirhan 2007), and so on. Because mathematics is the foundation to learn other school subjects and very important to logical thinking, the learning difficulties in mathematics have been investigated for a long time. Many notions are used to describe the students with poor mathematic performance, such as mathematical learning disability (MLD), mathematical low achievement (MLA), mathematical learning difficulty (MLD), and mathematical struggling learner (MSL). The previous experience and basis, motivation, interest, habit, the attitude from parents, teachers and other students, and other factors all may contribute to learning difficulties in mathematics.

Educational technology has been used to help the students with poor academic performance, and the positive effect has been evidenced in many relevant studies (Burns et al. 2012; Ok and Bryant 2016; Roschelle et al. 2016; Satsangi and Bouck 2015; Zhang and Zhou 2014). However, the detailed analysis of the online behavior of the students with poor academic performance by using the learning systems and its relation with the normal exam performance has not been found in our literature review.

This study attempts to fill in this gap. We scrutinize the online learning behavior of one experiment class in China that regularly used one intelligent web-based mathematics instruction system, and explore the relation between their online learning behavior and the scores in regular exams.

3 The Students with Poor Academic Performance in the Experiment Class

The experiment class is one normal class in a junior middle school located in Chengdu City, Sichuan Province, one province in south-western China. This class existed between September 2015 and June 2018. In the first school year this class included 49 students. In the second and third school year, three students were absent in all exams, and are excluded in this analysis.

We collected the scores of 48 exams held between November second, 2015 and January 24th, 2018, including mid-term and final term tests. Not all students took part in every exam, as some were absent due to different reasons.

By the first screening, we set the exam score to zero for the absent students to examinations and calculate the quartiles for all examinations, just like the simple measuring regulation in school practice. The following result are found: none of the students had always been below the quartile in all 48 examinations; 11 students had never been below the quartile in any examination and belonged to 22.9% or near one quarter of all students; the students whose exam scores had at least once been below the quartile counted 35 and belonged to 77.1% or three-quarters of all students; the mean and median counts of all students with their exam score below the quartiles for all exams was 11.3 and 4, respectively. According to the previous definition, four students were classified as the students with poor academic performance during the study time between November second, 2015 and January 24th, 2018. The students are anonymously identified as Pu, Tang, Wen and Zhong, and had the poor academic performance in 47, 46, 41 and 43 examinations, respectively.

By the second screening, we exclude all the students who had not taken part in at least one exam for the sake of comparableness and then 28 students were included. After calculating the quartiles for all examinations, the following results are found: none of the students had always been below the quartile in all 48 examinations; seven students had never been below the quartile in any examination and belonged to 25% or just one quarter of all the students; the students whose exam scores had at least once been below the quartile counted 21 and belonged to 75% or three-quarters of all the students; the mean and median of all students’ count with their exam score below the quartiles for all exams was 11.85 and 2.5, respectively. According to the previous definition, four students were classified as the students with poor academic performance during the study time between November second, 2015 and January 24th, 2018. The students are identified as Liang, Wang, Wen and Zhong, and had the poor academic performance in 39, 39, 44 and 46 examinations, respectively. Because in this screening all the students who did not write the exam were excluded, the score quartiles of all examinations were more than that in the first screening. The improved quartiles explain why the two students Wen and Zhong had poor academic performance in more exams than in the first screening. The other two students Liang and Wang had poor academic performance in 32 and 33 exams respectively in the first screening.

Through the two screening, we classify the six students into students with poor academic performance in the research period: Liang, Wang, Pu, Tang, Wen and Zhong.

The time series distribution of the regular exam scores of the six students with poor academic performance is shown in Fig. 1. It cannot be estimated from the figure when the students had a better performance or worse performance. The only common feature of the six students is that they all had worse performance in the first examination.

Fig. 1.
figure 1

The time series diagram of the score rank in normal exams of the six students with poor academic performance, with the top circle representing that the student is below the quartile and the down circle representing that the student is above the quartile.

Full size image

Figure 1 shows that two students, namely Liang and Wen, performed better in the last exam of the whole period. From the point view of summative assessment, both should not be classified as students with poor academic performance in the last exam. Because mathematics learning is a cumulative process, the score of one student in the first exam is the starting base point, and the score in the last exam is the end point, the six students’ scores in the first and last exam are listed in Table 1. In the first exam, Pu performed the worst, and Wang performed the best. In the last exam, Laing and Wen performed the best, while Tang did not write the quiz.

Table 1. The scores of the six students in the first exam and the last exam
Full size table

In the next section we will analyze the students’ online learning behavior based on the OLAI model.

4 The OLAI Analysis of the Students Using a Web-Based Interactive Learning System

“Lexue 100” (with the Chinese meaning Happy Learning for 100 Percent, http://www.lexue100.com) is a web-based intelligent instruction system for school mathematics, developed by Beijing Lexue 100 Online Education Co., Ltd. More than 6.8 Million quizzes have been designed for the different versions of mathematics textbooks that are used in different provinces and metropolis in China. Writing quizzes is the main learning activity in this system. Each quiz is composed of a series of gap-filling or single-choice questions with predefined standard answers. As soon as one student submits the trial answer to the system, the trial answer can be compared with the standard answer, and the corresponding quiz score and feedback are instantly provided to the student. Users are allowed to pass the quiz only if every answer gets right, meaning that if the first try of one student is wrong, the student will have to try again until the answer hits the point.

The OLAI (Online Learning Activity Index) model was proposed by Jia and Yu (2017) to describe the students’ online learning activities. OLAI can be calculated by the summarization of standardized and dimensionless speed, quality and quantity to complete one activity, i.e. OLAI = speed + quality + quantity. This formula means that OLAI is also a dimensionless value and can comprehensively describe the learner’s online learning behavior. Furthermore, the OLAIMAA (OLAI Mean of All Activities) can be used to measure one learner’s online learning activities during a certain period on average and the OLAISAA (OLAI Sum of All Activities) can be used to measure one learner’s online learning activities from a summative perspective. Similarly, the online learning performance of a specific learner group, such as a class or a school, can also be described by similar indexes such as OLAIMAAAP (OLAI Mean of All Activities of All Participants) and OLAISAAAP (OLAI Sum of All Activities of All Participants).

In Jia and Yu (2017), the concrete formulas to calculate the three dimensions of the OLAI for the quiz activity in the “Lexue 100” system was introduced in details. Based on those formulas, we calculated all the OLAI values for the 46 students in the experiment class during the study period between November the second, 2015 and January the 24th, 2018. The quartile for the quizzes number completed by all students is 1061. Like the scores in all regular school examinations, the percentiles for the OLAI dimensions are also calculated so that the corresponding position of the six students with poor academic performance in the experiment class regarding their OLAI values can be shown as in Table 2. The cell value is set to Yes, if the student’s value is below the quartile among the 46 students in the class; otherwise the cell value is set to No. In addition, in order to compare the students’ performance, the values of the OLAI dimensions of the six students, together with the mean and median value of the whole class, are listed in Table 3.

Table 2. The positions of the 6 students with poor academic performance in the experiment class regarding the OLAI dimensions represented by whether the value is below the quartile in the class.
Full size table
Table 3. The values of OLAI dimensions of the 6 students with poor academic performance
Full size table

Tables 2 and 3 show that none of the six students performed poorly in all the eight OLAI dimensions. Among them, the student Pu and Zhong performed the worst in online activities with 6 dimensions below the quartiles. The student Tang and Wang performed worse with 4 dimensions below the quartiles. The student Wen performed better just with one dimension below the quartiles. The student Liang performed the best in online activities with all 8 dimensions above the quartiles. We depict the six students’ online behavior in the following details.

Student Pu performed better in speed mean and quantity mean, but the quizzes he wrote counted below the quartiles of all students. This behavior means he completed totally 199 quizzes, much less than others, but the questions in the quiz quantity counted as usually, and the average speed to complete the quizzes was also satisfying. However, the quality to complete the quizzed was bad and should be improved. This student completed only fewer required quizzes fast. In normal examinations, this student performed the worst in the whole class from the first through the last exam except twice, and thus should be paid special attention by the teachers and the parents.

Student Zhong performed better in both speed sum and mean, but the quizzes he wrote counted below the quartiles of all students. This behavior means he completed totally 569 quizzes, less than others, but the speed to complete the quizzes on average and in sum was satisfying. However, both the quality and quantity to complete the quizzed were bad. This student completed only fewer required quizzes very fast, but the completion quality should be improved. This means he did the online quizzes just perfunctorily. In normal examinations, this student performed worse in all exams except six, and thus should be paid special attention by the teachers and the parents.

Student Tang performed better in both speed sum and mean, as well as quantity mean and OLAIMAA, but the quizzes he wrote counted below the quartiles of all students. This behavior means he completed less quizzes (totally 764) than others, but the quality to complete the quizzes on average and in sum was not satisfying. This student completed only fewer required quizzes with more questions very fast, and the fast speed on average and in sum plus the average quantity lead to better OLAIMMA. This means he did the online quizzes just perfunctorily. In normal examinations, this student performed worse in all exams except three, and thus should be paid special attention by the teachers and the parents.

Student Wang performed better in both quantity sum and quantity mean, as well as quality sum and OLAISAA, and the quizzes he wrote counted above the quartiles of all students. This behavior means he completed totally 2019 quizzes, more than others, and the quizzed contained more questions than usual and are more difficult for him. But the speed and quality to complete the quizzes on average were not satisfying. This student very slowly completed more required quizzes with more questions, and the quality for some quizzes was very good. The better quantity on average and in sum plus the quality sum lead to better OLAISAA. In normal examinations, this student performed worse in 33 exams, and thus should take acre about the exams with bad records.

Student Wen performed online worse just in the dimension of quantity mean, but better just in 6 normal exams. Student Liang performed better in all dimensions, but better just in 10 normal exams. These two students completed the required quizzes with both better quality and speed on average and in sum, thus the OLAISAA and OLAIMAA were also satisfying. But they performed worse in most normal exams than the other average students in the class, though better than the other five students with poor academic performance. The reason may be that they did not complete the online quizzes by themselves, but with others’ help. This guess was just verified in half by checking the online questions-answering records. While writing the quiz online, the student can call for instant help from the distant teaching assistant. Through chatting the assistant can answer the student’s questions about the current quiz. The question-answering log data show that Student Wen had more times’ question-answering online than usual, but the Student Liang had usual question-answering. Through the online help, the student Wen could complete the quiz, but this help did not result in better performance in regular classroom exams.

The time series distribution of the OLAI and its three dimensions’ values including speed, quality and quantity for all online quiz activities of the six students is shown Fig. 2. Obviously, the student Pu had taken part in much less activities than others, and the student Zhong had fewer quizzes from July 2017 to January 2018. Except the winter vacations in January and February 2016 and 2017, as well as the summer vacations in July and August 2016 and 2017, the other four students continued to write quizzes. The OLAI and its three dimensions’ values of the six students varied not much. But exceptionally the student Wen had three very low and negative values in the dimension speed and corresponding OLAI values. A negative speed in one quiz means that this student completes this quiz slower than the average students.

Fig. 2.
figure 2

The time series distribution of the OLAI and its three dimensions’ values of the six students with poor academic performance.

Full size image

If the student Wen’s three exceptional values are excluded, the time series distribution is shown in Fig. 3 with greater clarity. The student Liang had the most stable online performance with less activities whose speed and OLAI value were below zero. This phenomenon confirms the quartile analysis that Liang has less activities whose OLAI values were below the quartiles of all students. This student’s better online performance corresponds to the better performance in normal exams among all the six students.

Fig. 3.
figure 3

The time series distribution of the OLAI and its three dimensions’ values of the six students with poor academic performance excluding three exceptional values.

Full size image

The student Wang and Wen constantly participated in most quizzes. Although they completed some quizzes very slowly so that the speed was negative enough to lead to the negative OLAI value, the speed of most quizzes was positive enough so that the speed mean and sum were above the quartile of all students. The Student Wen’s better online performance on average did not result in better performance in normal exams. The reason may be that the online quizzes were completed with the help of online teaching assistants. If this student could not get instant help, he or she might spend much longer time on completing the difficult quizzes.

The students Pu, Tang and Zhong could not completed the most quizzes, and so had also worse performance in normal exams.

5 Findings and Conclusion

After screening all students’ exam scores from Grade One to Grade Three of a class in a junior high school, we identified six students as students with poor academic performance during the study time. The quartile analysis of their exam scores show that their chance to get the worse scores, i.e. the scores below the quartiles was randomized. They performed worse in most examinations, but not bad in some examinations. It could not be predicted in which exam they could performed better.

The students’ online quiz activity analysis using the OLAI model shows that two students Liang and Wen performed better in the online quizzes, the student Tang and Wang performed at the intermediate-level, while the other two Pu and Zhong performed worse in the online quizzes. Among the six students, Liang was the best one in normal exams, Pu and Zhong performed the worst in normal exams. In the last exam of the research period, the scores of both Liang and Wen were above the quartile of all students, and were the best twos among the six ones. Considering that the performance of both Liang and Wen was not the best twos among the six ones on the first exam, but their online learning activities’ quality and quantity on average represented by OLAIMAA and in sum represented by OLAISAA were the best twos, we can infer that the online performance is positively correlated with the normal exam performance, at least from the point view of summative assessment.

The two students Liang and Wen had some difference. The student Wen had a better starting point (62) than Liang (54) in the first exam, and performed better online, but the performance in regular exams was worse than Liang. This paradox can be somewhat explained by the frequent online question-answering behavior of this student.

The student Wang had the best OLAISAA because of the best quantity sum, i.e. Wang completed more quizzes containing more questions, or this student liked to challenge more difficult quizzes. However, those difficult quizzes took this student more time, and the completion quality wasn’t good. This hard work might become cognitive load to the student (Sweller 1994), but could not lead to better mastery of the learned content and better performance in regular exams.

Based on the above findings, we can conclude that the online quiz activities with instant feedback is helpful for the students with poor academic performance in their normal exams, but too challenging quizzes may become cognitive load, and the frequent help from teaching assistants online must not lead to better self-thinking about learning content and corresponding performance in normal exams.

All the poor students had a bad performance in the starting point, i.e. the first exam. Their deficiency in previous study prevented them from understanding new knowledge and should be overcome at first. It is necessary for the school and parents to help them with reviewing the previous mathematic knowledge learned in the primary school before they began the learning in the junior high schools.

6 Limitation and Further Studies

This paper just studied six students with poor academic performance in one experiment class in a junior high school. In the future we will investigate more students in more classes to validate the findings from this study. With a bigger sample, statistical approaches like ANOVA, Pearson correlation analysis, and regression analysis could be adopted to determine how significant the difference between the students’ online behavior is, and what the correlation between the school exam and online behavior is.

The analysis method used in this paper, i.e. formative assessment based on the students’ learning behavior measured by the OLAI, may be also suitable for good and medium students. We will analyze the behavior of good and medium students and explore the behavior difference among them in the further studies.