Keywords

1 Introduction

The way digital technologies have developed has changed the needs for the skills that future workers need to possess. A fundamental subset of these skills is called STEAM (science, technology, engineering, art, and mathematics) and it is often considered as one of the essential skill sets for anyone applying for any job in the twenty-first century (Matson et al. 2004). Also, the same future workers, due to the digital environment they have been living in since their birth, are functioning differently when compared to the people that were born either before or at the beginning of the era when digital technologies became mainstream. These developments create pressure in our existing educational systems for finding ways of teaching traditional and still relevant subjects like mathematics in a way that would be meaningful and engaging for the future workers who are being educated right now (OECD 2016; Prensky 2001; Gerretson et al. 2008). Special focus is often given to bring engagement to learn mathematics as this subject is necessary in all other STEAM subjects, being thus inevitably required by the majority of twenty-first century jobs (Ribeiro et al. 2011; Savard and Freiman 2016; OECD 2015).

Educational robotics in its current meaning was born during 1980s when rapid development made it possible to place electronics of a smaller computer into mechanized toys. The first commercial educational robotics sets, especially designed for schools, were announced in 1988 and were called LEGO TC Logo (Resnick 2016). Educational robots can be considered as simplified models of commercial robots but they can also be seen as robots without any other purpose than being easy to program and good to think with (Papert 1980).

One of the first uses of robotics as an educational tool was the Logo Turtle project, developed in the MIT Artificial Intelligence Academy by Seymour Papert and Marvin Minsky, in 1967. Logo programming activities included, among other fields, mathematics and science (Logo Foundation 2015). Since then, various efforts have been made to introduce robots into schools (and even kindergartens) as tools for teaching different subjects like physics, mathematics, engineering, and for personal development of cognitive, metacognitive, social skills like research skills, creative thinking, problem solving, and other twenty-first century skills (Alimisis 2013). The format of teaching these skills can be by using robots as an educational tool in an after-school robotics club, in a robot-based technology classroom or in a subject-based classroom, for example, in a mathematics classroom.

Implementing educational robotics is not free of problems. For example, it is easier to use robots in a technology class as the teachers there are more familiar with technology. Using robots as tools for teaching mathematics, however, can be challenging. Although programming and using robots can include a lot of mathematics (Papert 1980), the teachers tend to have no training in teaching technology (Rasinen et al. 2009) which results in mathematics teachers lacking the knowledge needed for implementing educational robotics in lessons.

In the spring of 2018, a pilot study was conducted in Estonia aimed to collect information necessary for preparing methods and instruments for a larger study to be conducted during the school year of 2018/2019. Pilot study involved 208 students from grades 3 and 6 as experimental group (EG) and 196 students from grades 3 and 6 as control group (CG).

The article is presenting a short overview of the existing literature concerning educational robotics as well as the preliminary results of the pilot study conducted in the spring of 2018 in Estonia.

2 Literature Review

2.1 The Search Process

For providing a summary of existing literature relevant to the subject of our study, a systematic literature review was conducted, following a process defined by Khan et al. (2003).

The EBSCO Discovery Service (EDS) was used for finding relevant papers. The EDS creates a unified, customized index of information resources by harvesting metadata from both internal and external sources, including journals, books, and conference proceedings (Pedaste et al. 2015), and then creating a pre-indexed service. Since 2010, the EDS also covers Thomson Reuters’ Web of Science results, provided that user’s institution is subscribed to both services (https://help.ebsco.com/interfaces/EBSCO_Discovery_Service/EDS_FAQs/can_I_view_Web_of_Science_results_when_searching_EDS; Europa Science 2010). The article search query used was the following: ((“educational robotics”) AND (mathematics)) . The main search was carried out on December 2017 and it resulted in 164 matches.

The results were further filtered. An article was only to be included in the literature review if (1) it included a study of the use of educational robots as an educational tool for teaching mathematics in a school environment, (2) the robots were used for teaching students from primary education to upper secondary education, according to the classification by ISCED (UNESCO Institute for Statistics 2011), (3) it studied physical robots, and (4) the full text was obtainable either through the EDS or directly from the authors.

At the end of the search procedure, there were 20 articles (Matson et al. 2004; Gerretson et al. 2008; Ribeiro et al. 2011; Savard and Freiman 2016; Highfield 2010; Highfield et al. 2008; Holgersson and Lindh 2007; Hussain et al. 2006; Iturrizaga 2000; Khanlari 2014; Kopcha et al. 2017; Ortiz 2015; Paula 2014; Samuels 2010; Samuels and Haapasalo 2012; Savard and Highfield 2015; Silk et al. 2009; Walker 2017; Williams et al. 2012; Greene et al. 1989) that fully satisfied the search criteria. The content of these articles was evaluated for finding their: (a) research objectives, (b) research type, (c) methods and instruments that were used for collecting and managing data, (d) sample size and duration, and (e) conclusions that were made. As an aid to the reader, the summary of the overviewed research papers is provided on a separate link (Link 1)Footnote 1.

2.2 The Results

The evaluation of the 20 articles that fully satisfied the search criteria brought out following findings.

Research Objectives of the Papers

The research objectives are listed in an online table, available on Link 1 (on the row “Goal of the article”). These objectives were analyzed and categorized into four generalized groups: (1) the goal was to explore teaching possibilities; (2) the goal was to design an artifact; (3) the goal was to evaluate learning outcomes; and (4) the goal was to evaluate students’ and/or teachers’ attitudes toward using robots for learning/teaching purposes.

Several studies had more than one goal. 65% of studies were exploring the ways robots could be used for teaching purposes. 60% of studies were evaluating the attitudes of students and/or teachers toward using robots as an educational tool. 40% studies were offering an artifact (typically a teaching method) with the purpose of helping novice implementers to start using robots as an educational tool. And lastly, a quarter of all studies were evaluating the influence on learning outcomes.

Research Types, Samples, and Durations of the Papers

Based on how the papers described the sample size, duration of the study and the strategies, and what methods and instruments the papers used for collecting and reporting data, we found that 75% of studies were using qualitative approach, 20% of studies were based on mixed methods research, and 5% were quantitative studies. The study length and the sample size were also evaluated. For easier manageability, the duration of the research was considered long term if the duration exceeded 1 month. The longest study lasted 2 school years; the shortest studies were a couple of hours long. The average sample size (calculated based on studies where the sample size was presented clearly) was 348, with the largest sample size being 1653 and the smallest 6.

Qualitative Studies

Of these, 20% were theoretical studies and 80% were case studies, using the following data collection instruments: observation (60%), questionnaires (20%), and interviews (53%).

Quantitative Study

The only quantitative study was designed as an experiment with the purpose of measuring learning outcomes in mathematics. The study was a short-term study (under 1 month) with a relatively small sample size.

Mixed Methods Studies

All of these four studies had a sample size over 100. 75% of these studies were relatively long-term studies (lasted more than 1 month). All mixed methods studies were experimental, having either control groups or testing students’ knowledge before and after intervention. For data collection, all of these works used learning outcome testing and questionnaires. 75% of mixed method studies additionally used observations, and 50% used interviews for gathering data.

Conclusions of the Papers

For each study, we recorded a one-sentence overall conclusion, shown in the row “Conclusion string” in an online table, available on Link 1. The conclusions were further analyzed and following findings were made: 55% of the papers analyzed recommended using robots as an educational tool for teaching mathematics. Only one study came to the conclusion that such usage does not improve mathematics skills. 30% of the papers found that using robots did improve learning outcomes. 50% of the papers concluded that students had a positive attitude toward using robots for teaching mathematics, and of these, 40% noted that also teachers had a positive attitude toward using robots as an educational tool. There were no negative conclusions toward using educational robots in mathematics teaching (with the one exception described above).

Conclusions

Of all of the analyzed works, only 25% stated one of their goals as measuring the learning outcomes. However, this is probably one of the areas that would need development, having the potential of persuading teachers to start using robotics in classrooms. Similarly, it is necessary to do more research about teachers’ attitudes toward using robots as an educational tool and about the real-life problems teachers face when implementing such tools. From reviewed works, only 20% made conclusions about the teachers’ attitudes.

Another area that needs additional attention is the sample size and the duration of studies. Although 55% of papers are considered as long-term studies in the context of this work, in reality, this is not enough for making conclusions about permanent effects of the use of robotics as an educational tool. Also, the sample size of present works is often small. Only 25% of studies had a sample size larger than a few hundred students, and more than 50% of studies had a sample size smaller than 50 students, representing thus a relatively small part of the learning community. There is an obvious need for studies with durations of at least 1 school year with students using robots on weekly basis.

3 Effects of Educational Robots in Math Lessons: A Pilot Study

3.1 Aim

The aim of the pilot study was to clarify how educational robotics can be implemented in mathematics lessons, what kind of results this implementation brings, what kind of learning theories will be needed to support this method, and what the teachers’ and students’ attitudes toward educational robotics as a learning tool are.

3.2 Research Questions

Based on the reviewed literature and the aims of the pilot study, we formulated the following research questions to answer:

  1. 1.

    What were the biggest problems and challenges the teachers faced while using robots in their math lessons?

  2. 2.

    What did the teachers consider as the biggest difference compared to their regular approach of teaching?

  3. 3.

    What is the attitude of students toward learning mathematics by using educational robots?

  4. 4.

    What are the differences between experimental and control groups in learning outcomes and student’s motivation?

3.3 Sample

Two groups were formed for the study: an experimental group (A) and a control group (B). In group A, the robots were used as an educational tool in embodying numbers, results, shapes, distances, calculations, and other mathematical constructs. In group B, the lessons were carried out in a traditional way.

The criteria for school participation were the following: in each participating school there had to be parallel classes in either grade 3 or grade 6 (ISCED level 1), or both. One of the parallel classes was to be included into EG, the second into CG. It was preferred that the mathematics teacher had no significant previous experience with educational robots or programming. Also, schools had to provide a technical support person to support the teacher in the lessons with robotics content.

Altogether eight schools volunteered with six fulfilling the criteria. One of the volunteering schools did not have a parallel class and another school wanted to participate by using robotics material that was to be taught by a robotics coach who was not a mathematics teacher. These two schools were allowed to participate unofficially for the purposes of gathering additional information for designing the main study. In all participating schools, the majority of students had no significant previous experience with educational robotics. Few of the students had either taken part in robotics clubs or had robots at home.

The pilot study was performed in 10 experimental group classes, having altogether 208 students, 106 students in the third grade, and 102 students in the sixth grade. There were also 10 control classes with a total of 197 students, 111 students in the third grade, and 86 students in the sixth grade. The average class size was 22 in the third grade and 19 in the sixth grade.

3.4 Procedure

Educational Robotics Platforms Used

In the pilot study, three different educational robotics platforms were used: the Edison robot, the LEGO Mindstorms EV3 robot, and the LEGO WeDo 2.0 robot. In the third grade lessons, all of these platforms were used; in the sixth grade lessons, only the LEGO Mindstorms EV3 robot was used. The more detailed description of the robots and their programming platforms is given on Link 2.Footnote 2

The Edison Robot

The Edison educational robot was launched in mid-2014. The base robot is small, self-contained, and relatively robust. It has two individually controlled motors as actuators, a speaker, two LED lights, two IR transmitters, three buttons, and following sensors: IR receiver, line tracking sensor, two light sensors, and a sound sensor. The main advantages of Edison are its low cost, relatively simple use, and the possibility of enhancing the robot by using LEGO bricks. The main disadvantage is that the robot’s motor rotation sensors cannot be controlled in programming languages that are suitable in the basic education level.

The LEGO Mindstorms EV3 Robot

The EV3 is a constructor robot, meaning that students need to build the robot before they can use it. In the educational set, there are three motors, one color sensor, two touch sensors, one ultrasonic sensor, one gyro sensor, robot’s brain, and necessary cables and bricks for building a robot, altogether 541 pieces. For the purposes of the study the EV3, “driving base” model was used. The building of models had to take place outside the mathematics lessons (being built either by support personnel, teachers, or robotics club students). The main advantages of the EV3 robot are its relatively high accuracy and availability in (Estonian) schools, its main disadvantage is its price that is up to seven times higher than that of the Edison robot.

The LEGO WeDo 2.0 Robot

WeDo is a constructor robot designed for children 7 or more years old. In the box, there is one motor, one tilt sensor, one motion sensor, robot’s brain, and necessary bricks and cables for building different models of the robot. For the purposes of the current study, the Milo science rover model was used. The main advantages of the WeDo robot are its child friendly appearance and easy usability, its disadvantages for the purposes of this study are its inaccuracy, lack of a second motor (making impossible for the robot to turn), and relatively high price (costing more than twice the price of the Edison robot).

Curriculum for the Pilot Study Lessons

For the pilot study, an extensive curriculum was designed that included 20 lesson plans for each selected robotic platform, for both grade 3 and grade 6. The main aim of these lesson plans was to provide a context that allows students to apply mathematics on a robotics exercise, encouraging students’ abstract thinking by studying and predicting individual robot movements. The robot functioned also as an external embodied agent representing student’s thinking about shapes, time, distance, etc., when solving the mathematical exercise.

The creation of lesson plans was carried out in cooperation with pilot study teachers. Half of the mathematical exercises were supplied by the participating teachers by either choosing a text book example or by creating their own mathematical exercise. Another half of the mathematical exercises was created by the authors of the study, following the examples of national standardized mathematics tests. The lesson plans were created with different levels of difficulty by including either only one or several algebraic expressions. The robotics exercises that illustrate the mathematical exercises, thus also have different levels of difficulty, illustrated by the number of programming blocks (from 1 to 14) needed for solving the exercise. Examples of the lesson plans are provided on Link 3.Footnote 3

Each lesson plan was designed for conducting one 45 min class period, and was composed of the following blocks:

  1. 1.

    A mathematical textual exercise that served as a topic of that lesson, and descriptions of three accompanying robotics exercises.

  2. 2.

    A reference to previous knowledge and guiding material necessary for conducting the robotics exercises.

  3. 3.

    Programming blocks with their descriptions that were needed for conducting the robotics exercises.

  4. 4.

    Each robotics exercise with explanations about the mathematical solution and about the coding solution, and textual description of the code.

  5. 5.

    A visual example of the program code.

  6. 6.

    A link to the video that showed the robot performing the robotics task.

All lesson plans were shared with participating teachers through Google Drive. The teachers were also given 1.5 h trainings demonstrating the use of lesson plans. The lesson plan design saw two students using one robot. The lessons were to be conducted during regular mathematics class periods, either on each week or on every second week. In two out of 10 participating classes, an additional class period per week was allocated for conducting the lesson plans of the pilot study. In the implementation phase, the teachers were encouraged to work out their own methodology for using the lesson plans. For example, some of the teachers shared the lesson plans with their students beforehand, some shared only parts of the lesson plan to students, and some teachers created their own slides based on the lesson plans.

3.5 Methods

In this study, both qualitative and quantitative research methods were used for gathering data and analyzing it with the purpose of complementarity (Greene et al. 1989).

National Standardized Math Test (NSMT) Scores

NSMT is a test that maps pupils’ knowledge at the end of the first and second stage of studies, i.e., in grades 3 and 6. This is a low-stakes test that is not evaluated and is aimed at providing help and tools for the teacher for further organizing the teaching. As the test results were not available for grade 3 at the time of writing this paper, then only the test scores of grade 6 for both EG and CG students were evaluated at the end of the pilot study.

Questionnaires

For gathering data about teachers’ and students’ attitudes, questionnaires with Likert-type scales were used. The design process of the surveys was partially based on the findings of the literature review. The teachers’ attitudes were surveyed at the beginning and at the middle of the pilot study; the students’ attitudes were surveyed after each school hour that had been conducted according to the curriculum of the pilot study. For teachers, both EG and CG attitudes were gathered. For students, only EG attitudes were gathered. Survey included also open-ended questions concerning specific aspects of using educational robots in school and in mathematics classes:

  1. 1.

    What did teachers find robots to be suitable in classrooms?

  2. 2.

    What did teachers find robots to be suitable in mathematics lessons?

  3. 3.

    What was teachers’ previous experience with robotics and coding?

  4. 4.

    What kind of influence teachers believed the robots would have on learning process and teacher’s work organization?

  5. 5.

    How attractive did students find the use of robots and how did they cope?

Additionally, each teacher kept a diary of their lessons, documenting the activities performed during the lessons, engagement of students, encountered problems and their solutions. The surveys were analyzed statistically, using methods of descriptive statistics.

Interviews

Semi-structured interviews were performed with six of the 10 participating mathematics teachers, with interviews lasting from 35 to 60 min. The interviews were guided by five questions that asked teachers to reflect on the use of the educational robots when conducting robotics–mathematics lessons:

  1. 1.

    What is her experience of conducting mathematics lessons with educational robots?

  2. 2.

    What is the opinion of students?

  3. 3.

    What is the opinion of the teacher herself?

  4. 4.

    Is there any influence, and what kind of influence, on learning results?

  5. 5.

    Were there any technical problems and how were the problems solved?

The interviews were recorded and transcribed. The transcriptions were subjected together with the teachers’ diaries to the thematic content analysis for finding relevant clusters of meaning.

3.6 Results

Biggest Problems and Challenges the Teachers Faced

The data sources for answering this question were surveys conducted before and in the middle of the experiment, interviews, and teachers’ lesson diaries. The data indicated that one of the biggest challenges for the teachers was their lack of robotics and programming skills. Half of the teachers had never coded before and 2/3rd of the teachers had either not used educational robots before or had used them only a couple of times. Quotation from an interview:

Before starting with mathematics lessons in school, the teacher should have the option to participate in a training that is focused on solving the robotics experiments. The questions will arise and the teacher is not able to answer (all of the) students’ questions by herself while solving technical problems with robots at the same time!

Another problem raised was the lack of time for conducting the robotics experiments. The mathematics curriculum is usually very tight and it is difficult to reserve time for alternative teaching methods. Another quotation:

Students had to work more at home so that in the classroom there would be more time for conducting the robotics enhanced lesson.

Usually, it was difficult for a teacher to conduct these lessons alone. They relied heavily on the support of the school’s educational technologist who had to support preparing and conducting the lesson in order to ensure solving of technical problems. If for some reason the technical support person was unable to participate, then the quality of the lesson suffered. For example, the duration of a regular lesson is 45 min and teachers working alone were often unable to correct occasional problems that took place during the lesson. Any potential technical failure put the teacher into potentially bad situation due to the time pressure. Quotation from a lesson diary:

Students were confused when the robot failed to perform the 2nd experiment as expected. Two of the students started playing with foam plastic cubes and there was a small hassle.

The teachers sometimes felt uncomfortable because conducting robotics lessons requires a different mindset compared to regular mathematics lessons. Students are expected to have more cooperation and less individual work, students move around the classroom, the results with robots cannot be as accurate as regular mathematics tasks, and teachers had to be prepared for conducting each lesson. For the teachers, it was sometimes difficult to accept these changes.

The lack of functional reading skills was a student related challenge for teachers as some students (with special educational needs) were unable to understand textual exercises or to understand the logical sequence of entering programming blocks. Teachers needed additional time for motivating and encouraging these students to participate as team members.

The challenges of students initially lacking necessary knowledge and having different paces of work were more common and required more time at the beginning phase of the experiment. With the lessons, students had constructed necessary coding knowledge and it was accepted that not everyone had to finish all robotics experiments within the lesson.

Differences Compared to Their Regular Approach of Teaching

We used interviews and teachers’ lesson diaries for answering this question. It turned out that conducting a mathematics lesson with educational robots has many differences compared to the regular lesson. For example, teachers admitted that it was more difficult to plan the lesson’s flow: some of the students were able to finish all three robotics experiments in time while some of the students were only finishing their first experiment. Also, to ensure that lesson would be completed during a school hour, teachers had to familiarize themselves with the experiments beforehand. Otherwise, the motivation could fall:

I liked it very, very much. But then we had this problem that we did not understand … As soon as I got stuck … we have a wrong line here, let’s correct it … This is when it starts going wrong, the whole lesson gets stuck, the students do not know what the teacher wants and so they just dive into their smartphones.

For some teachers, it was really challenging to understand that with robotics experiments, the answers are rarely exact, there is no individual work, and therefore traditional evaluation models could not be applied. Also, as the robotics experiments embraced embodied cognition principles, there was more physical movement in the classroom compared to a regular mathematics lesson, creating pressure to recognize the need for different classroom order:

So there was this confusion at the beginning that what’s going to happen and how to organize all this, how it all works, and … And the first lesson was like, for me like, not stressful, but like … there was a lot of action in the lesson.

In few cases, using robots in the lesson also needed a different place for conducting the lesson. Instead of the regular classroom, the lesson could be conducted in the computer class (where the robots were) or even in the hallway (to have more floor room for the experiments).

Attitudes of Students Toward Learning Mathematics by Using Educational Robots

To answer this question, we gathered data from students’ lesson feedback surveys, from teachers’ lesson observation diaries, and from teachers’ interviews conducted at the middle of the pilot study.

The students’ lesson feedback surveys (n = 686) contained a text input field for student’s free form opinion. In practically all occasions, the field contained a phrase of approval, such as “Cool,” “Cool lesson,” “I would have liked the lesson to last longer,” “I have never experienced such a lesson,” etc. A few entries were about technical problems and only one entry contained negative meaning. The overall positive attitude was confirmed by the answers to a quantitative Likert-scale question “Do you find the lesson interesting?”—86% of answers considered the lessons being as “very interesting” or “interesting.”

For the purposes of ensuring the match between mathematics tasks and robotics experiments, we explored the connection between the perceived difficulties of the mathematics task and the robotics experiment that was based on it. The results proved that the perceived difficulty of the robotics experiment was directly related to the perceived difficulty of the mathematics task. The connection was stronger with grade 3 students and slightly weaker with grade 6 students. For example, in the third grade, if the mathematics task was perceived as “easy,” then of these respondents, 31% considered the robotics task as “too easy,” and 55% as “easy.”

It is interesting to recognize that while girls and boys had in general very similar attitudes, we found one remarkable difference: boys were more likely to evaluate both the mathematics task and the robotics experiment as “too easy” compared to girls: of 342 boys 34% found the experiment “too easy,” and 37% as “easy.“ On the other hand, of 330 girls, only 19% found the experiment “too easy” while 47% considered it as “easy.”

The teachers’ interviews and lesson diaries revealed that although students were highly motivated and engaged, there were also some rare occasions when student’s interest declined, caused by technical problems with the robot, by failure in experiment, or simply by having too easy experiments in cases where students had studied robotics before. However, in these cases. teachers’ encouragement usually solved the temporary loss of motivation.

Differences Between Experimental and Control Groups

Based on the data from the National Standardized Mathematics Tests (NSMT), interviews, and teachers’ lesson diaries, we are able to present some of the findings.

At the moment of writing this article, the NSMT results were available only for grade 6. The average score of EG students was 27.3 and the average score of CG students was 19.3 while the national average was 26.6 (from 44.5). However, due to the small sample size, no statistical generalizations can be made. In interviews that were conducted during the pilot, most teachers did not find any direct connections. Nevertheless, in cases when the same teacher teaches both EG and CG, there exists some evidence supporting hypothesis that the use of educational robotics in mathematics lessons could improve learning outcomes:

And then there was this test, and I tried to … just tried to evaluate these two classes … What was their average level? And, in reality I could see that the A-class that I conduct “robotics in mathematics” lessons with, they had higher average. But of course there may be other factors also.

Some of the teachers pointed out the importance of using the robot as an embodied agent that helps students to learn “with their body”:

It is very important and necessary for many kids … To learn, using the cognition of the body, a lot of children need it.

Most teachers pointed out that the improvement of students’ motivation in itself could improve learning outcomes in a long term. They also found that students had improved the skills of functional learning, logical thinking, and understanding the cause and effect relationship. Following quotes are from the lesson diaries:

More and more there are signs of improvement of self-control skills. Children are actively searching for cause and effect relationships, and are analyzing the results! Good!

Students are finding independently cause and effect relationships. At the same time they are inattentive to written guidelines, meaning that they are unable to correctly and attentively read from the slides.

Many teachers also described the improvement of measuring, coding, and cooperation skills. One teacher noticed that students became better in recognizing connections:

Let’s bring forward another thing, besides the motivation: recognizing connections. She (the student) will start seeing connections and logic. How mathematics is connected to real life, how are the different topics of mathematics connected with each other.

4 Discussion

Using robots as teaching aids in the classroom is a relatively new subject of study, limited by the fact that educational robots in a modern sense became available only since 1980s. Although there are several studies exploring the use of educational robots as an educational tool for teaching mathematics, the number of such studies is still limited. Besides using robots for teaching mathematics, most of the evaluated studies also explore other possibilities, including teaching STEAM subjects and twenty-first century skills. The majority of analyzed studies find that educational robotics is a valuable tool for teaching mathematics, having a positive effect at least on students’ motivation but in best cases also improving their learning outcomes. However, there is no clear indication if educational robots have justified their use as an educational tool.

The preliminary results of the Estonian pilot study about using educational robotics in mathematics lessons mostly confirm the observations of the previous studies. The collected data show that robots do bring motivation and engagement into classroom, help students to develop many mathematics related skills, and have a positive impact on students’ learning motivation. The results of NSMT for grade 6 also showed significant improvement of learning outcomes for EG compared to CG but due to small sample size, this development would need additional research. The results also point out that using robots as an educational tool can be challenging for teachers. The teachers’ confidence can be hindered by the lack of previous experience with robots, they need to acquire different types of mindset for conducting these lessons and evaluating the students, and sometimes, it is just difficult to allocate time for robots in the existing curriculum.

The present paper has its own shortcomings. For example, the small sample size of literature review, being caused by the lack of prior research papers on the subject, is one of the factors seriously limiting the scope of the study and preventing finding possible trends and meaningful relationships. The results of the pilot study are mainly limited by its goal of gathering information for the preparation of instruments and methods for a larger study.

At the present moment, the question about positive effects of using educational robotics in mathematics lessons will remain without conclusive answer. We perceive the need for further systematic and experimental studies about that subject.

The authors of the present paper are using the findings of the paper to prepare a full-scale study in Estonia with the goal of bringing more clarity to understanding whether educational robotics, compared to traditional classroom teaching, would bring about significant growth of students’ mathematics skills. The planned study is scheduled to begin during autumn 2018, will last for one school year and has currently an experimental group of more than 60 schools with more than 2000 students enrolled. The study is performed by using several different robotics platforms and unique lesson plans that are designed in cooperation with different mathematics teachers.

The authors hope that this will shed light to the question of whether the use of educational robotics in math lessons is feasible and justified on a larger scale.