Keywords

1 Introduction

A long-standing goal within educational technology research has been to use technological innovation to improve the learning experience and performance of students [1]. One means to support student learning is by having a student teach others, which has been shown to help students learn through the process of having to explain it to someone else. [2] The positive effects of learning by teaching persist even when a student is teaching a virtual agent or robot, rather than a peer [3]. There is increasing interest in examining what factors impact the development of positive perceptions of and rapport with a teachable agent or robot, and how that influences learning outcomes [4]. One factor that might influence how learners benefit from teachable agent interactions is whether they are teaching the agent collaboratively (i.e., with another peer), termed “learning by collaborative teaching.” [5]. By teaching the agent collaboratively, learners might have more of an opportunity to co-construct knowledge with each other through discussion in order to figure out how to explain it to the agent. In addition, learners may feel less frustrated with an agent that makes errors if they are having a positive experience with their peer, and in turn rate the agent more positively and feel more rapport with the agent.

The current study examined the extent to which teaching a social robot with a peer or alone influences students’ robot perceptions, rapport-building, and math performance. To explore this, we use a social robot named Emma, designed to engage students in spoken dialogue on ratio and proportions problems. Students explain the problem step by step, and Emma responds with questions about or self-explanations related to the current step. We predicted that teaching Emma with a peer may increase rapport between students and Emma, improve positive perceptions of Emma, and increase math learning and performance compared to teaching Emma alone. Our work builds on and extends prior work by focusing on the impact of collaborative teaching of a robot on both perceptions of engagement and learning compared to individual teaching.

2 Methods

We recruited 40 undergraduates (35 Female, 5 Male; 13 Asian, 5 Black, 1 Latin@, 17 White, 4 No Response; Mean age = 19.64 years, SD = 1.25) from a mid-Atlantic US university for an exploratory pilot study. Students were assigned to one of two conditions in which they would either engage in a 30-min collaborative activity in which students worked together in pairs (i.e., the dyad condition, n = 28) or an individual activity in which students worked alone (i.e., the solo condition, n = 12) to help a teachable robot (i.e., Emma) solve math problems on ratios and proportions. Students completed a battery of self-report questionnaires that included a variety of motivational and engagement constructs prior to and post teaching Emma.

We assessed student perceptions of Emma through a 35-item measure containing two ten-item subscales (e.g., Anthropomorphism, Intelligence) and two eight-items subscales (e.g., Animacy, Likeability) on a six-point Likert scale [6]. We combined the average score on each subscale into a single composite measure of robot perceptions (Cronbach’s alpha = .95). Higher scores indicated greater positive perceptions of the robot. We also assessed rapport with Emma with a 15-item measure containing three four-item subscales (e.g., positivity, attentive, and coordination) and one three-item subscale (e.g., general) on a six-point Likert scale (e.g., 1 = Strongly disagree, 6 = Strongly agree) [3]. Higher scores indicated greater rapport with Emma.

We assessed learning outcomes through a math test on ratio and proportion administered before and after students interacted with Emma. There were two counterbalanced versions of the test. The original test had 13 items. However, because of counterbalancing issues five were removed because different versions may have been easier or harder, leaving eight items for both pre- and post-tests. Each item was scored dichotomously as correct or incorrect and summed for a total score that could vary from 0–8 on pre and posttest.

3 Results

Initial analysis of pre-test math performance revealed an effect of condition with the solos (M = 4.58, SD = 2.19) performing worse than the dyads (M = 6.00, SD = 1.85), F(1,39) = 5.47, p = .03. This was unexpected as students were expected to perform similarly at pretest before interacting with each other and Emma, therefore we analyzed our three key questions both with math-pretest as a covariate.

A one-way analysis of covariance (ANCOVA) to test the effect of condition on participants’ average reported perceptions of Emma with the math pre-test as a covariate revealed a medium effect of the covariate, F(1,38) = 6.02, p = .01, showing that participants’ pre-test scores predicted their perceptions of Emma. There was also an effect of condition, F(1,38) = 6.95, p = .01, with participants in the dyad condition (M = 4.47, SD = .50) reporting more positive perceptions of Emma than participants in the solo condition (M = 4.12, SD = .69). If we take the pre-test covariate out of the model, the effect of condition is marginal in the same direction, F(1,38) = 3.17, p = .08.

A multivariate analysis of covariance (MANCOVA) to test the effect of condition on participants’ perceptions of rapport with Emma with the math pre-test as a covariate revealed that the overall model did not show an effect of condition, F(4,34) = 4, p = .15, nor an overall effect of covariate F(4,34) = 1.02, p = .41. Exploratory analyses for the rapport subcomponents revealed a trend for positivity F(1,37) = 5.93, p = .02, with dyads (M = 5.02, SD = .52) reporting more positivity than solos (M = 4.60, SD = .99). All other components of rapport were not significant, F’s < 1, p’s > .41.

A one-way ANCOVA to test the effect of condition on participants’ average math posttest performance with the math pre-test as a covariate revealed a large effect of the covariate, F(1,37) = 8.54, p < .01, showing that participants’ pretest scores predicted their posttest scores. There was no effect of condition, F(1,37) = .22, p = .65, with participants in the dyads (M = 6.86, SD = 1.43) and solo (M = 6.58, SD = 1.38) conditions performing similarly.

4 Discussion

Overall, we found evidence that working collaboratively to teach a social robot math is linked to increased positive perceptions with the robot, but not improved rapport or individual learning outcomes. One possible reason for the increased positive perceptions of Emma is that by working collaboratively participants may construct common ground [7] and then use that shared knowledge to work more productively and positively with the robot as it makes errors and mistakes. Future work should investigate whether learners who construct common ground are more tolerant of a robot’s errors in comprehension, and if there are potential benefits of these improved perceptions.

We did not find similar collaborative benefits for participants’ perceptions of rapport with Emma or their individual learning outcomes. Although there was a trend for an impact on rapport positivity, which aligns with the positive perceptions of Emma results, we did not see that trend reflected in any of the other sub-dimensions of rapport (general, attentive, or coordination). One possible reason for these null results is that dyads may not have been engaging in interactive ways with each other (e.g., via explanation or error correction), but may have instead focused on developing common ground to take turns to teach Emma. Future work should further examine the nature of the interactions between participants in the collaborative dyads. It could also be the case that the duration of our study was too short to see changes in rapport or learning, or that the material was generally too easy for the undergraduate students involved. Future research could examine whether longer collaborative engagement with the teachable robot shows benefits for rapport and learning over time.

Although our study is preliminary and uses a small sample size in an online rather than in-person setting, these initial results set up the foundation for future research to explore effective means and uses of a teachable social robot to facilitate student learning and student engagement with novel learning technology. Given the limited resources and time of teachers to address the individual needs of students, having students work collaboratively with a teachable robot may be a beneficial tool to improve the student learning experience through promoting student engagement with novel technology.