Keywords

1 Introduction

The Bebras Challenge is a great opportunity for every pupil and student to get in touch with computer science and computational thinking. The Slovak version of Bebras, called iBobor (as in “informatics beaver”) is widely known throughout the country as in the school year 2018/19, 77,928 pupils and students from almost 1,000 schools (both primary and secondary) took part in the challenge. The iBobor’s aim is not only to bring some of computer science concepts to schools, but also to inspire teachers and to give them a chance to teach parts of the informatics curriculum that are not contained in the textbooks. This is also the aim of our research presented in this paper, where we were looking for a way how Bebras tasks could be used in Slovak schools to help pupils learn graph structure topic more in depth and more precisely.

1.1 Context

To better understand why we chose Bebras task as a way to teach some topics, it is important to state that while Informatics is mandatory subject from third grade (8–9 years old pupils) to eighth grade (13–14 years old pupils) with one lesson per week in Slovakia, more than 40% of informatics lessons in lower secondary education (fifth to ninth grade) are taught by teachers with different specialisation (based on data from Slovak School Inspection). These teachers therefore, in many cases teach more digital skills and are not willing to try to teach programming or more specialised areas of computing. This problem is interconnected with another one – there are no official up-to-date textbooks. The old ones were created in 2005 and the National Educational Programme (NEP) [1] has been changed since the books’ publication. Therefore there are some areas which these textbooks do not cover. This leaves a lot of work to teachers – they need to adjust old materials, create their own or search for materials from different teachers or even countries and adapt or translate them for their students. From teachers’ feedback to Bebras we know that many of them use Bebras archive throughout the school year to teach some informatics areas or just to make it easier for them and more fun for their students. Based on this we assumed that Bebras seems like a good starting point to create new learning and teaching materials, as many pupils and teachers are familiar with the challenge which creates a higher chance of them using it.

Which topic to cover stemmed from experience of one co-author who is a teacher in lower secondary school. While we had many materials for each area from NEP, the biggest issue was with the part Structures – Graphs. We were not able to find a good materials which were suitable for pupils of age 10 to 12. What were we looking for? In NEP, there is stated that pupils at the end of the sixth grade (12 years old) should know: “to orientate in a simple structure (searching and obtaining information from structure based on some criteria); to organise information to structures (creating and manipulating with structures with data and simple relations, e. g. tables, graphs, sequences of pictures or numbers); and to interpret information from structures (deducting existing relations from data in structure, retelling information in structure using own words)” [1].

Another reason for using task from Bebras was categorisation of graph task created in Slovakia [2] based on tasks which were used in Slovak contest. One of the main objectives in this categorisation are methods and strategies used in solving process, which are (1) trying all possibilities, (2) the “look-see” method, (3) graph search with constraints, (4) uncovering a strategy, (5) creating a strategy. Connecting this categorisation with NEP, we focused on the first three categories, as the rest of them are more suitable to algorithmic thinking and their main goal is not to work with structures.

1.2 Literature Overview

While looking through the literature concerning the use of Bebras tasks in schools we found many interesting ways – for example adapting Bebras-like tasks into a computational thinking test or tests to evaluate their knowledge and skills from the computing [3]. There were many articles where authors were describing what can Bebras teach, using categorisation of the tasks, e.g. [4, 5]

Valentina Dagiene and Gabriele Stupuriene’s [6] way of bringing Bebras to school unplugged and not as a part of evaluation or assessment was adapting Bebras tasks to playing cards, which can be used in many ways in lessons. One of them is letting pupils solve the task individually and later grouping them and encouraging them to talk about their ideas. These cards were used in primary school level and increased motivation of both pupils and their teachers. On one hand they helped pupils to think about concepts, but on the other hand, there were some misconceptions which partially came from some teachers’ lower qualification in informatics.

This all led us to question whether pupils of this age can gain a deep understanding of some concepts through a small task which can be usually solved in 3 min. Do they learn what we think they do? Can they find similarities in different contexts? A teacher is a crucial part in this constructionist process and while we also deal with teachers who are not qualified in informatics, we need to create some materials which can guide pupils through them, making smaller steps and then trying to generalise what they learnt. To better distinguish between steps, we find Bloom taxonomy of learning objectives [10] to be the best alternative, as slovak teachers are (more-or-less) familiar with it and it is used in many textbooks or teachers’ materials.

An inspiration how to use task from competition in school was found in paper about a Slovak contest for talented students in lower secondary education PRASK [7]. The author, Michal Anderle showed how these tasks can be adapted to high school lessons and why it is important to divide them into subtasks. He mixed individual, pair and group work in one lessons and we believe this is a good approach.

2 Research Methods

Our research is a part of long going mixed methods research [8]. The main goal of this paper was to find out how can be Bebras tasks transformed and implemented into school informatics with emphasis on enhancing pupils’ skills in orientating, interpreting and organising graph structures. Which can also enhance their computational thinking skills, as abstraction and generalisation. Our research is divided into several phases which have different ways of collecting data, and also their analysis (qualitative and quantitative). In this paper we discuss mainly three phases (see below) and one pre-phase which was an important part of creating the whole research idea.

2.1 Zeroth Phase

In this phase we analysed the National Educational Programme in Informatics (NEP) [1], as one of the main documents used in creating curriculums and lesson plans, and we were looking for suitable tasks from Slovak Bebras contest which could satisfy its requirements. We identified tasks which could be used in schools as a learning material, as they contain all processes needed. Theoretical analysis of documents was used in this phase. We also used graph task categorisation from Budinska and Mayerova [2], which was the result of open coding of tasks. And this results in finding graph tasks suitable to use in schools.

2.2 First Phase

In previous phase a group of tasks was chosen, and in this phase we analysed contestants’ results from these tasks to better understand if tasks were easy or difficult and if there are any problems in tasks’ text, pictures or proposed answers. For analysing data, the quantitative methods were used. We used statistical methods – both basic descriptive statistics (percentage of correct answers for each year) and statistical hypothesis test (chi-square test, Pearson standardised residuals [9] in which we tested differences between gender and school grade groups.

2.3 Second Phase

Based on results from previous phase, we knew only global results of these tasks but we wanted to see how pupils solve them and if our hypothesis about what was causing the errors were true. Therefore, the second qualitative phase was conducted in October 2018 in the fifth (7 pupils, 3 boys) and the sixth grade (10 pupils, 4 boys) of one lower secondary school. We chose three tasks from Bebras, all of them were easier with higher success rate but still with a good potential to learn basics of graph theory. They were proposed in a form of a worksheet.

Pupils solved worksheet individually and for each task they were asked to describe how they found the answer. After individual solving, each class was grouped into two groups – boys and girls as one of our aims was to better understand what could have caused the gender-specific differences in solving these tasks. Each group had time to discuss their answers and to find one that all of them think is correct. This is an example of focus group [8], where there is no moderator but instructions in worksheets take his role. All pupils have a chance to say something, and their goal is to find an answer they all agree on.

Discussions were recorded (each group had their own recorder) and later rewritten and analysed using axial coding [8]. To make sure all pupils in the group understand how to solve these tasks, another worksheet was made with only slight changes for each task. Teacher motivated pupils that if (and only if) pupils from the whole group would have correct answers for all three tasks in it they all would get bonus points in Informatics as a reward. Both worksheets were analysed using codes and interjoining it with information from the recordings. To ensure the triangulation of the data the researcher (teacher) took field notes which provided us with more insight to what was happening (e.g. when pupils were showing each other something in worksheets).

2.4 Third Phase

In the third phase, which was taking place in May 2019, we created worksheets for each of the originally tested tasks. We were taking into account the results of previous phase and trying to create tasks in each stage of Bloom taxonomy of learning objectives [10]. To address some problems with original tasks, we proposed some introduction to worksheets, which should be done together with the whole group. We showed worksheets to two groups of primary and lower secondary school teachers and they gave us a lot of ideas about methods that could be used in lesson plans.

First test of first worksheet was made in May 2019 in the same classes as research from previous phase. While most of them saw these tasks before, we saw from their reactions that many of them had no deeper insight into graph tasks, but it is possible, that some pupils created good mental models of family trees. The worksheets adapt only a concept of original task and not the task itself, so they could be suitable for pupils who are new to the topic and others could challange their mental models. From this research we have field notes from the teacher and student products – filled worksheets. All of them were analysed, coded, and evaluated using a point system. Due to problems with time at the end of the school year we were able to test only one of three worksheets, as one testing takes approximately one lesson and these classes have only one informatics school lesson per week.

3 Results

In this article, we focused on three tasks which were analysed, tested with pupils and based on the results, worksheet for each task was created. In this part we present whole analysis, observations and worksheets, separately for each task to make it easier to understand.

3.1 Family Relationships Tree

The first task we chose for this research was the task called Family Relationship Tree that was used in the competition in the school year 2013/14 in the category Benjamin (in that time 5th to 7th grade). This task had been proposed as easy and its real success rate was 48.8%. As can be seen in Fig. 1, there is a graph showing family relationships, that the lower secondary school pupils should be familiar with. For finding the right solution, the direction of the arrows is important, indicating who has a family relationship with whom.

The right answer was an arrow pointing from Maria to Tomas. 46.4% of boys and 51.8% of girls answered this task correctly, which is a statistically significant deviation (Pearson’s coefficient was 7.08 for girls). The difference between boys and girls success rate was significant for all age groups of competitors, but from the results it was also visible that with increasing age the error rate of competitors has decreased.

Based on the graph task categorisation, we thought that pupils mostly use “look-see” method, therefore they obtain information from graph structure and then they interpret them.

Fig. 1.
figure 1

The task Family relationships tree from second phase worksheet

Full size image

3.2 Results of the Second Phase

When working alone, pupils spent either very long time on the task Family Relationship Tree, or not enough time and they did not check their answers. In all four groups, pupils came to the conclusion that Maria was the grandmother. The boys in the sixth grade, even originally just circled the name Maria, so the teacher had to coordinate their solutions, pointing out that they should select an arrow. In the end, both boys’ groups chose an (wrong) answer, an arrow pointing from Lucia to Maria. It has been said several times that “Lucia is the grandmother of Maria” together with “Maria is the grandmother”. It is clear from both debates that some pupils understand what the direction of arrows means, but it seems as if they did not combine these two contradictory pieces of information. In both boys’ groups they spent more time discussing family relationships (“Lucia is Mary’s granddaughter because...”) and they were less focused on the direction of the arrows. Girls’ groups have been more focused on the direction of arrows than family relationships. In both girls’ groups, they correctly read from the graph that Mary has (at least) two grandchildren – Lucia and Thomas. Both groups discussed a lot whether the arrow direction indicates “the relationship of whom or relationship from to”, while trying to base their opinions on the rest of the relationships shown in the graph. Both girls’ groups found the right answers.

The validation task for the Family Relationship Tree was named The Clever Family and it consisted of graph constructed with the same rules as in the previous task. Pupils were asked to name one relationship in the blank arrow. Interestingly, three out of four groups named it correctly – both girls’ groups and the fifth grade boys’ group. In the sixth grade boys’ group only one pupil had correct answer, three other boys in this group all had incorrect answer “daughter”, that means they again changed the meaning of the arrows’ direction.

Based on this observation we identified the main problems in this task – (1) the meaning of arrows direction is ambiguous, (2) boys in this groups had problems with naming the family relationship names, (3) the pupils intuitively understands who is in which relationship with whom but they did not pay attention to direction of this relationship. We addressed these problems in our proposed worksheet.

3.3 Results of the Third Phase

The worksheet consists of 9 tasks, each one focusing on one Bloom cognitive development stage. Before working on the worksheet there is a time for talking about relations name in the family. The names of relations in the family should be written down on a blackboard, as it helps pupils to focus on the task and not to find the right relation (therefore, it deals with the problem (2)). Our solution of problems (1) and (3) was gradation of the tasks. First three tasks used one graph and in the text we explicitly named two relationships from it (e.g. Adam is Simon’s son). In the first task pupils should write down the relationships which are directly visible (and one is also written in the text). In the second task, they need to complete the relationships which are not directly mentioned (the opposite or missing relations). In the third task pupils were asked to draw two specific relationships to the graph and create one on their own. In the fourth task we tell them three relationships and they need to choose one of four graphs which represents them. In the fifth task pupils draw graph based on written relationships. The sixth and the seventh task are very similar – we used the same graph and were asking for the same relationships, but in the sixth task we drew the graph and in the seventh task we wrote down the relationships. We wanted to find out if there is a difference between these two ways of solving the problem and we also asked pupils which one is their preferred one and why (the eighth task). The last, ninth, task was to create own relationships graph, it could be based on their family tree or they could imagine one.

On average, after scoring each answer, fifth grade pupils got 79% of points, while sixth grade pupils got 85%. The lowest success rate was in the third, fifth and ninth task. In the third and the fifth task pupils were drawing arrows and most of their mistakes were based on wording – while Hana is Simon’s wife was incorrect in 5 times, Lenka has a brother named Albert was incorrect 10 times. We chose the wording on purpose because both ways are used in the real life but it changes the direction of the arrows in our graph and we wanted pupils to understand it. Because this was one of the biggest issues in our worksheet, in the next version we would like to add one task were we explicitly ask pupils to write down what the direction of the arrow means. This could help them to think about it and to validate their intuitive grasp of the concept.

In the sixth and the seventh task there were no significant differences between pupils’ results, but we saw that they needed more time to answer the task without the graph and a lot of them draw their own graph while solving it. Based on pupils’ answers in the last task where they were drawing their own family graph we can see that they only had a little problem with it, but they were losing points for forgetting to write two relationships in sentences (e.g. My mother is Eve.). Some of them changed a direction of a few arrows, and another group was using only lines (not arrows) and naming relations such as “siblings”, “couple”, etc. This could arouse the discussion with the whole group and the teacher about representation of the structure, the logic behind it and even about how computers store some data. All of this will be added to the lesson plans.

4 Tram Lines

The second task analysed in this research was the task from the Benjamin category (fifth to seventh grade) in the school year 2015/16 called Tram Lines. The picture shows a map of tram lines and the contestants were supposed to find out what tram a boy used, see Fig. 2. The description of his route (turning and final stop) has been described in the text. 51.9% of the girls and 49.6% of the boys solved this task correctly. This difference is moderately significant (Pearson coefficient 3.38 in favor of girls). From the results it was visible that while in the fifth grade the differences between boys and girls were negligible, in the seventh grade they were markant. Also, the success rate increased significantly with age. Pupils had to use a textual description of the route, which contained several conditions. Therefore, we consider searching a graph with constraints as a suitable method for solving this task. From the NEP point of view it is connected with obtaining information according to specified criteria.

Fig. 2.
figure 2

The task Tram Lines from second phase worksheet

Full size image

4.1 Results of the Second Phase

When solving the Tram Lines task, many pupils initially did not understand the picture. After explaining that white rectangles are stops, they understood what to do. They also gave hints to each other with examples from the real environment familiar to them. Some pupils guessed the answer, the rest got it right. In this task pupils could not easily explain why their answer was correct. When they explained it to each other, they traced the route with their fingers or pencils. Therefore, they were following steps written in the text on the graph representation. The sixth grade boys had the biggest problem with explanation to this task, they all guessed the answer and were not able to tell why. It turned out they used method of exclusion to find the correct answer when they were working together. The problem in this task proved to be that the starting stop is not counted and therefore many pupils got stuck at the first statement. Finally, all the groups agreed on the correct answer.

The variation in the second worksheet was the task named Trams in Beartown, in which the pupils were supposed to determine which trams the boy was allowed to go by. There were no differences between boys and girls, and both groups solved the task correctly. The difference was only between the fifth and sixth grade – in the fifth grade only one girl found two possible lines, but in the sixth grade there were six (out of ten) answers with the two lines. Wrong answers did not occur in this task. The main problems we found were that (1) pupils were not able to understand a graph structure quickly, (2) some wording could be ambiguous (e.g. “initial stop”, “turn”) and (3) younger pupils were not looking for more than one correct answer.

4.2 Proposed Worksheet

In the worksheet, tasks were created using Bloom taxonomy. Each subtask is trying to reach one objective. As we were not able to test this worksheet with pupils, we describe only tasks and not their results. To deal with problems with Tram Lines task, we slightly changed the structure and made it more similar to real-life line maps – that means we added places and names into it. In the first task pupils need to find basic concepts (“stop”, “initial stop”) in the graph structure. This could help with problems (1) and (2). In the next task they count stops of some of the trams – so they need to find which line in graph represents which tram, in the third task they need to decide which tram to take based on some criteria. There is implicitly stated that sometimes there are more options available and then there is more space for pupils to fill in their answers. In the next task they need to choose the better line. The synthesis is represented by the task where pupils design their own tram lines in the “city”, and the last task is to discuss why their map is better than the proposed one and what they think is important when creating such a map.

In the lesson pupils could talk with their teacher about why is it sometimes better to use this kind of diagram instead of the real map. They could also find some maps of public transport from the different cities and towns, and talk about how they think the Internet search tool for transportation could work and how computers know where you should transfer.

5 Bracelet Machine

The task, originally called Mother’s Day, but in our worksheet changed to Bracelet Machine, was the third and final task in the research described in this paper. It was used in 2013/14 in the Cadets category (eighth and ninth grade), with 82.0% boys’ success rate and 85.6% girls’ success rate. The gender difference was statistically significant with a Pearson coefficient of 5.6 in favor of girls. Gender differences are equally significant in both age groups. Although the task was designed for a higher age category, its very good results and a small error rate convinced us that this task could be suitable for younger pupils too.

The task uses a finite state machine model (considerably simplified) and the contestants need to comprehend the rules of making bracelets, which are shown in the graph, see Fig. 3. An example was used to describe how bracelets were made. Since the task contained four possible answers, it was advisable to resolve it by eliminating each option, or in other words, by trying all the options. In this task, the pupils manipulated with the data (pictures) in the graph and, based on the rules, constructed the results (bracelets).

Fig. 3.
figure 3

The task Bracelet machine from second phase worksheet

Full size image

5.1 Results of the Second Phase

The Bracelet Machine task was the most problematic for many pupils in our testing groups – many of them did not know what the rules in the picture meant and how to interpret them. The most common explanation for choosing the answer A was: “She went through the bottom path, now she’ll go through the upper.” Some pupils chose answer B with the argument that it contains all the shapes which were in the picture. The argument for C was that shapes were in the right order. The boys’ groups chose a strategy of exclusion, in discussion in the sixth grade one boy clearly defined the rules for bracelets. In each group it was said that the loop under the pink diamond meant “that it could go twice this way,” none of the groups thought about going more than twice.

It was clear from the discussions that the pupils understood the graph only intuitively and could not fully explain why their answer was correct. This has proven to be a problem in the solution of a similar task in the next worksheet called Bracelets. Only slight changes were made to the original graph – we let them write down (or more accurately draw) their own bracelets made with “machine” from the graph. We were interested if they understood the rules in the graph and how they would work in looking for two different answers.

Pupils who did not engage in the groups discussion of previous task or it was clear from their words they did not understand it, had problems with solving the Bracelets task. There was a pupil in each group except the fifth grade girls, who just redrawn the diagram – they considered it a bracelet, as all of them later explained, “the bracelet is circular”. All fifth grade girls wrote more than two options, in every case at least two of them were correct. The common mistake was to miss the shape from the beginning or end of the bracelet, or to repeat the shape that did not have a loop above it. The remaining pupils had the right answers – they all chose both possible ways and if there was a loop on the way they used it. The problems we identified were (1) more difficult comprehension of rules from graph for this age group and (2) not clear understanding what the loop means.

5.2 Proposed Worksheet

In proposed worksheet we addressed these problems with small steps which pupils need to take in order to understand the structure. Firstly they see some bracelets which were made by the machine and they need to draw the “way” how machine was creating them – to address the second problem we used loop zero, one and three times. In the next task they are asked to fill missing piece into bracelets. Then they decide which bracelets were made by machine in the picture, and in the next task they are shown three machines and six bracelets and they connect each bracelet to related graph. In synthesis we let them write their own bracelets for the graph and then they are supposed to write down what are the rules for the bracelets (with what shape it starts, with what shape it ends\(\ldots \)). To make it more interesting for more motivated pupis an additional task is at the end of the worksheet – to create their own machine (it could create bracelets, funny words or whatever they want).

The discussion following the worksheet could be about simplifying the rules to follow by drawing them into diagram like this one. With pupils, who did not have problems with the worksheet, the teacher could have a discussion about determinism and nondeterminism (e.g. what would the machine do if there are two possible ways each starting with the same shape).

6 Conclusion

In this paper we presented the results of using Bebras graph task in lower secondary school as a part of the learning process. We chose three tasks which seemed to meet the criteria from the National Educational Programme, analysed their results from the competition and let two groups of pupils from fifth and sixth year solve them. While analysing methods they used, errors they made and misconceptions they gained, we were able to identify the most significant problems and we tried to overcome them with creating worksheet for each task. Subtasks in worksheets are created with Bloom taxonomy in mind, so for each cognitive objective there is at least one subtask.

Even though we could not test all three worksheets, based on the results of the first one, it appears to be a good approach to teach graph data structures. We are aware of the small group of participants, and we are planning to test them with different setting groups. To find them, we presented the worksheets to the two groups of primary and lower secondary school teachers and they gave us more ideas and insight to what they need. They liked the idea and they agreed that it is not so easy for them to use Bebras tasks in lessons if they do not have a good understanding of the informatics concept behind it. Also, some ideas about post-worksheet discussion arose from the results and teachers opinions, as well as many different uses of worksheets – it can be done individually, in pairs, in groups or even with a different approach for each task. Some of the teachers are willing to participate in later rounds of the research starting in the new school year, which could bring a new perspective to all of it.