1 Introduction

Informatics in primary education relies upon the introduction of digital devices to provide opportunities for (critical) students’ engagement instead of the mere handling of such devices. Informatics education emphasises access beyond a superficial understanding of predefined procedures, including adaptation, configuration, construction, and design (Bergner et al., 2023). As a result, informatics education predominantly focuses on understanding the underlying principles and concepts beneath the user interface. These essential insights are necessary to construct and elucidate the operation of digital systems, enabling individuals to design and use them more effectively. In place of the term computer science (CS), many countries, especially within Europe, instead use computing or informatics, or add technology-based components (e.g., digital literacy, digital competence, digital culture, digital hygiene). However, in the current study the term ‘informatics’ refers to the wider discipline (Dagienė et al., 2022).

Regarding future learning potential and opportunities, several studies have pointed to the value of introducing informatics and computational thinking (CT) during early-stage learning (del Olmo-Muñoz et al., 2020; Kwon et al., 2022). Many countries are already involved in updating their primary education curricula to include informatics concepts or computational thinking skills (Bocconi et al., 2022; Dagienė et al., 2019, 2022), and based on previous research most countries face similar challenges in areas such as curriculum design, teacher education, scope of the content to be taught, and issues encountered with implementing these interventions.

Teachers can be challenged when faced with teaching new technology-related content. Through the exploration of technological knowledge, Vivian and Falkner (2019) identified how primary school teachers can utilise non-digital materials in an ‘unplugged’ manner as a means to the embodiment of informatics knowledge and skills. However, other studies (e.g., Kravik et al., 2022) have also revealed gaps in teachers’ knowledge in areas such as programming and informatics, as well as with their pedagogical and didactic skills and teachers have clarified that they are in need of further training in order to teach technology-focused courses properly and, as a result, to develop their professional confidence. Forlizzi et al. (2018) noted that at the primary school level, learners are encouraged to ask questions and to discover and explore some of the basic ideas and concepts of informatics in their everyday lives through engagement with either plugged or unplugged activities. Unplugged activities, which teach the basics of informatics without the use of electronic devices, aim to concretise abstract informatics concepts and make them more tangible. This approach enables younger learners to discover solutions based on logical thinking and to understand informatics concepts in a more natural way through the application of everyday thinking (Battal et al., 2021; Dağ et al., 2023; del Olmo-Muñoz et al., 2020; Tonbuloğlu & Tonbuloğlu, 2019). Meanwhile, plugged activities are performed primarily with some form of technology (e.g., computers, robots, or other physical devices) (del Olmo-Muñoz et al., 2020).

Due to the necessity of informatics education at the primary school level, curricula reform, and the identified need for primary school teachers’ didactic competence development in order to teach this subject with sufficient proficiency, there exists a need to explore which pedagogical approaches can effectively integrate informatics and CT within primary education. Approaches such as unplugged activities, which enable students to learn informatics concepts without the involvement of technology, are highlighted as effective but require further exploration and development. Based on research publications, we can generalize that it is a crucial to emphasize moving beyond superficial understanding in order to develop a critical engagement with digital devices in current informatics education research. It is essential, therefore, to ensure that students not only learn how to use technology but also understand the underlying principles and concepts related to the design and effective use of digital systems. It is obvious that there are too many topics and skills to teach which requires the application of different pedagogical approaches for different ages and content. Hence, this research study aims to fill the gap in computing pedagogy by investigating the effectiveness of both unplugged and plugged pedagogical approaches on primary school students’ learning of informatics concepts.

2 Background of the study

Teaching informatics concepts is not new, but teaching these concepts to children, even in kindergarten, is a fact that most countries have experienced not more than a decade. Besides, ‘what to teach?’ we have to answer the question ‘how to teach?’. There are numerous pedagogical approaches for different ages, when we talked talk about teaching informatics (Coleman 2021). These pedagogical approaches are generally referred to as unplugged and plugged, depending on the use of technology tools as well as well-known instructional applications (Olmo-Muñoz et al., 2020). Hence, ‘active learning’ forms the basis for computing pedagogy since students need to learn through hands-on practices.

2.1 Informatics concepts for primary education

Curricula between countries may vary, but certain fundamental computer science areas, such as programming, problem solving and algorithms, abstraction and data representation, data management, and security, are critical elements which cannot be omitted to ensure that educators do not miss out on the most essential computer science skills (Dagienė et al., 2021). These basic computing competencies are essential in order to show diversity and difficulty levels across learner age groups (see Table 1).

Table 1 Competences according to student age (based on Dagienė et al., 2021)
Full size table

Computer science is the foundation for computational thinking, and CT is a way of thinking that can be applied to CS as well as to numerous other areas. Hence, through this two-way interaction, CT can help students develop a variety of skills, including problem-solving skills, critical-thinking skills, creativity skills, communication skills, and collaboration skills. These skills are considered essential for success in today’s world, where technology plays an increasingly important role (Bocconi et al., 2022). Moreover, CT is used in primary education as a compelling alternative to traditional computer science, fostering early technology literacy, problem-solving skills, and creativity in young learners. CT is used to help prepare them for future digital demands, whilst making learning both engaging and relevant (Bocconi et al., 2022; Lee et al., 2022).

As an example, Zhang et al. (2020) specified CT skills progression from grades 1 through to 9 in Sweden. In grades 1 to 3, students use simple commands such as directions and steps or repeat commands several times in order to compose more complex sequences. In grades 4 to 9, students use sequences to perform more complex tasks, such as setting values for different variables or undertaking arithmetic calculations.

In Lithuania’s renewed curriculum (entered into force in September 2023), the new subject of informatics replaced that of information technologies, with informatics being introduced starting from the primary school level. The primary school (grades 1–4) informatics curriculum includes six areas and different levels of scope for achievement: ‘algorithms and programming’ accounts for most of the content and time (around 40%); ‘digital content creation’ and ‘data mining and information’ are roughly equal (20% each); 7–10% for ‘technological problem solving’, ‘virtual communication and collaboration’, and ‘safe behaviour’ divide the remainder in half (Dagienė et al., 2022; Stupurienė & Gülbahar, 2022).

2.2 Pedagogical approaches for teaching informatics in primary education

Recent research studies and classroom implementations have established that different ages have different cognitive abilities, meaning that informatics teaching methods, content, and learning strategies must be adapted accordingly (Dagienė et al., 2021; Saxena et al., 2020), especially for primary level education. Researchers usually indicate the use of unplugged and plugged activities, with activities based on real-life situations, ideally engaging all the senses and stimulating the complete individual (Tonbuloğlu & Tonbuloğlu, 2019). Unplugged activities may also be considered an appropriate method for the teaching of informatics (Saxena et al., 2020; Weigend et al., 2019). Engaging in game-based activities allows learners to become familiar with and comprehend informatics fundamentals using simple materials such as paper, pencils, paint, rope, cards, and balls. On the other hand, since plugged activities involve learning the functionalities of computer programming tools and can increase learners’ cognitive load, unplugged activities may have a greater impact on the development of computational thinking skills than plugged activities (Zhang & Gary, 2023). The literature also suggests combining unplugged and plugged activities in the classroom environment (del Olmo-Muñoz et al., 2020) so as to benefit learners from both these pedagogical approaches. Hence, students’ skills and knowledge have been the subject of prior research where informatics-related concepts and skills have been investigated using two main approaches in schools: computer programming exercises (plugged activities) and unplugged activities.

In a study conducted by del Olmo-Muñoz et al. (2020), students utilised various manipulative materials in unplugged activities, including different stencils and plastic cups, and also used reflection journals. They also alternated performing as robots, with their actions based entirely on a given algorithm. For the plugged activities, tablet personal computers were used in order that students could work individually during those tasks. Their research led the authors to conclude that informatics taught in early years learning was best addressed through a mixed approach, combining the use of both unplugged and plugged activities. Moreover, providing instruction through unplugged activities or a combination of unplugged and plugged activities is seen as not only advantageous in terms of skills acquisition, but also helps to enhance student motivation (del Olmo-Muñoz et al., 2020). In another study, Saxena et al. (2020) aimed to assess the feasibility of fostering computational thinking in early education, with a focus on developing three essential skills: pattern recognition, sequencing, and algorithm design. In their study, the researchers created a series of unplugged and plugged activities to achieve this, with unplugged activities that utilised tangible materials to offer students a more hands-on and concrete experience of informatics-related concepts. The goal was to equip students with the necessary language and understanding required for subsequent CT learning. Their approach aimed to establish a strong foundation for a plugged activity, which involved using a ‘Bee-Bot’ digital device. Their study’s results indicated that K-2 (ages 4–5 years) and K-3 (ages 5–6 years) students generally demonstrated proficiency in pattern recognition, sequencing, and algorithm design (Saxena et al., 2020).

In a study by Tonbuloğlu and Tonbuloğlu (2019), it was shown that unplugged coding activities positively impacted the development of students’ CT skills, with significant improvements seen in creativity, algorithmic thinking, collaboration, and critical-thinking skills. Various teaching methods (e.g., the fishbone method) and techniques (e.g., worksheets, puzzles, scenarios, data tables, and flow schemes) were employed in teaching classes, and students reportedly enjoyed the activities due to their engaging nature and real-life relevance. The observation and daily data analysis showed that students generally exhibited high levels of motivation and class participation during these unplugged coding activities. However, some students encountered difficulties perceiving the relationship between informatics and mathematics and analysing given problems in both disciplines. Factors that contributed to demotivation amongst the students were due to cognitive challenges in connecting the lessons learnt with informatics, since they felt that they had not learnt anything substantial by the end of the class. These factors may be attributable to the need to witness student improvements from the activities, since some of the unplugged activities reportedly lacked a strong foundation in the subject. Additionally, students tended to understand the function of informatics, but would often perceive the computer itself as the central aspect of the science.

Other researchers focused only upon the empirical proof regarding the efficacy of the unplugged approach in fostering informatics-related skills (Brackmann et al., 2017). Study results have demonstrated a substantial improvement in CT skills among students in experimental groups who engaged in unplugged activities (with most material created by the authors themselves based on the book ‘Hello Ruby’ and the board game ‘Code Master’) compared to their peers in control groups who did not participate. This provides strong evidence that the unplugged approach can be an effective method for developing CT skills, where the pedagogical focus was on storytelling and playing games.

Similarly, in research published by Sun et al. (2021), the authors concluded that unplugged activities can serve as a potent method for nurturing students’ CT skills, whereas Dağ et al. (2023) revealed a statistically significant enhancement in participants’ CT abilities, particularly in areas such as algorithmic design, abstraction, evaluation, decomposition, and generalisation, as a direct result of the unplugged coding course. Additionally, their results indicated that the CT skills of elementary school students were not influenced by sociodemographic factors.

In research by Kalelioglu and Sentance (2020), instructional approaches (and methods) employed by educators who utilised tangible tools within educational settings were investigated. The approaches used were categorised into two sets: broad (such as presentations, demonstrations, storytelling, and drama) and programming instruction (including activities like designing flowcharts, identifying and rectifying errors, and hands-on experimentation).

As previously mentioned, the literature has shown that both plugged and unplugged activities have been successfully used to teach informatics concepts and to motivate students. Unplugged activities at the primary education level are usually represented as role-play or hands-on activities, whereas plugged activities require technology mediation.

2.2.1 Role-playing approach (unplugged)

The role-play (RP) pedagogical approach is a dynamic process where participants engage in the imaginative and spontaneous development of hypotheses and theories related to the actions of individuals involved in an event or a sequence of events (Chisum & Turvey, 2011). This may involve participating in a role-play game, acting out, or performing the part of a person, item, or character. Role-play stands out as the most effective approach for the development of the crucial skills such as initiative, communication, problem solving, self-awareness, and teamwork. Incorporating role-play into the teaching/learning process not only fosters enthusiasm and enjoyment, but also boosts motivation, leading to overall benefits in the learning experience (Khan, 2017).

2.2.2 Hands-on approach (unplugged)

According to Ekwueme et al. (2015), the ‘hands-on (HO) approach is a method of instruction where students are guided to gain knowledge by experience. This means allowing the students to manipulate the objects they are studying’ (p. 47). This is an experience-based activity where students abstain from using computers (going unplugged) and, instead of creating programs, make use of materials such as Lego blocks, paper and pencil, or other items present in their surroundings (Weigend et al., 2018). There are numerous hands-on activities with cards, scales and weights, balls, etc., and each are designed to stimulate the thoughts of students (Nishida et al., 2008). Engaging in hands-on activities enhances academic achievement, practical aptitude, and various essential proficiencies in informatics (Hsiao et al., 2023).

2.2.3 Technology-mediated approach (plugged)

The technology-mediated (TM) approach is a plugged approach that stresses that digital technology plays a mediating role in the teaching and learning process participants’ achievement of learning goals (Bower, 2019). Within the context of the current study, a computer and software programming tools were utilised through this approach to manipulate objects and aid the participants’ learning of informatics concepts. Examining how a technology-mediated approach is used together or in parallel with unplugged approaches is a subject of current and future research. For instance, a recent study by Sigayret et al. (2022) found that a technology-mediated (computer-based) approach appeared more effective for fifth-grade students to master computational concepts: the plugged group performed significantly better than the unplugged group.

2.3 Purpose of the study

Based on the findings in the literature, both plugged and unplugged activities were selected for the current research study, and implemented through three pedagogical approaches, namely role-play, hands-on, and technology-mediated, in order to compare and contrast among these varied approaches to teaching informatics and CT. The aforementioned three approaches were selected since their effectiveness is recognised in the literature, and are widely used at an early age (Dağ et al., 2023; Lu et al., 2023; Sigayret et al., 2022; Tonbuloğlu & Tonbuloğlu, 2019). Moreover, the selected approaches made it possible to create or redesign appropriate activities for all five age-appropriate informatics concepts chosen, each of which are discussed in more detail in the following.

Hence, the purpose of the current study is to explore the effectiveness of both unplugged and plugged pedagogical approaches on primary school students’ learning of informatics concepts. Plugged and unplugged activities for students were designed considering five basic informatics topics, namely: command and sequence of commands; sorting algorithms (Bubble sort); sorting network algorithm; searching algorithms (Binary search); and shortest path algorithm. These topics were selected since they are common and widely known as forming the basics of the informatics discipline (Dagienė et al., 2019, 2021; Stupurienė & Gülbahar, 2022).

Based on this scope, the research questions addressed in the current study are as follows:

  1. (1)

    To what extent do pedagogical approaches affect students’ performances?

  2. (2)

    What are the students’ perceptions of activities performed using a particular pedagogical approach to learn informatics concepts?

  3. (3)

    What are the teachers’ perceptions of the pedagogical approaches to teach certain selected informatics concepts?

3 Methodology

This study was designed as a mixed-method research, with a concurrent embedded design employed to investigate the effectiveness of three selected pedagogical approaches for the teaching of informatics. The reason for preferring a mixed-method style of research was to enrich our understanding of the topic of study by using data from different sources as well as comparing and checking the phenomenon under investigation through different perspectives. Based on the initial quantitative data, qualitative findings from the participant students’ and teachers’ perceptions yielded complementary insight on the differences between the three selected pedagogical approaches.

3.1 Working group

The participants of the study were fourth-grade primary education students aged (9–10 years old) from a public school located in Vilnius, Lithuania, who each held no prior experience in informatics gained through formal education. The school was selected since it was both accessible and convenient, had three parallel classes, and the students had not previously participated in any Bebras national informatics and computational thinking challenges (for details, see Sect. 3.2). Of the 65 students who attended the sessions, a total of 55 completed the pretest and posttest measures and were thus included in the study. The sample distribution according to pedagogical approach and gender is presented in Table 2.

Table 2 Distribution of participants according to pedagogical approach and gender
Full size table

After the approval of the university ethics committee and permission to conduct the study, signed parental consent forms were collected, since the participants were minor students. Parents allowed their children to participate in the study on a voluntary and unpaid basis. To ensure confidentiality, all personal identifiers were removed from the data set and replaced with unique codes. Data was stored on a secure server with access limited to the research team.

In total, 55 students and three teachers formed the working group for the current study. Quantitative data were gathered from the student participants only, whereas qualitative data were gathered from both the students and the three participant teachers.

Three different instructors, consisting of two preservice primary teachers and one researcher, led the study’s activities in three groups (parallel classes), with the same instructor for each group delivering all five sessions to the same class. Prior to each session, the three instructors and three class teachers discussed the implementation of the activities in detail. The class teachers actively participated in the activities with the students and provided immediate assistance to both the instructors and the learners where needed. It is worth noting here that prior to the current initiative, the classroom teachers lacked experience and background in the teaching of informatics.

The involvement of classroom teachers and the observing of activities created a unique form of professional development that helped to improved their teaching practices. The competences of the preservice teachers were seen to improve during the preparatory phase prior to undertaking the activities, as well as during the implementation process itself. This collaborative approach not only fostered a supportive learning environment, but also contributed to the overall development of the teaching methods used.

3.2 Data collection

For more than two decades, the international informatics and computational thinking Bebras ChallengeFootnote 1 has accumulated extensive experience in creating and using thousands of short, concise tasks covering a wide range of informatics topics aimed at K-12 students (Dagienė & Dolgopolovas, 2022). In addition, Bebras tasks serve as a valuable tool for the assessment of CT skills (Masiulionytė-Dagienė & Jevsikova, 2022; Román-González et al., 2019). Thus, in order to assess the students’ understanding of the aforementioned informatics concepts, both before and after the implemented activities, appropriate Bebras tasks were specifically selected. When solving these short tasks, students are required to consider a wide range of informatics concepts and principles, such as information, data, algorithms, data structures, data processing, and programming constructs.

The selected tasks were chosen from a pool of tasks used in several previous national Bebras Challenges, ensuring that the tasks were age-group appropriate and based on the selected informatics concepts. Each task has a well-clarified part that explains how the task relates to informatics and computational thinking. Three international experts (researchers in CS education) agreed on the relevance of the tasks in terms of informatics concepts. As the participant students had not previously participated in a Bebras Challenge, the scenario of their having prior knowledge of the tasks was avoided. Since there was a small number of tasks utilised in the study, the pretest and posttest tasks differed in order to prevent recall of correct answers by the participants.

The instruments used for data collection on the students’ outcomes and teachers’ reflections are described in Table 3.

Table 3 Description of data collection instruments
Full size table

The participant students were tasked with completing the questionnaire after the intervention, and the teachers were requested to maintain a weekly reflection diary. The content validity of the student questionnaire and the teachers’ reflection diary were ensured through being codesigned with three inservice teachers, two preservice teachers, and one of the researchers; all of whom had diverse experience and varying levels of expertise in the field of education. As experts in the field, this approach ensured that the questions used were both clear and concise.

3.3 Implementation process

Based on the main objectives of the study, Fig. 1 describes and illustrates the process implemented during the intervention. In the first phase, the participant students completed the pretest in order to confirm their prior understanding of the selected informatics concepts. The second phase consisted of five classroom sessions (with one 45-min lesson held per week) delivered to the three groups of students. During the third phase, the students completed the posttest. The process was finalised with the students providing their reflection (via a questionnaire) on the five implemented activities.

Fig. 1
figure 1

Design of the learning sessions

Full size image

All phases of the intervention were implemented between October and December of 2022. It was considered essential to take note of any pre-Christmas events taking place during the implementation period since there were many other activities that took place at the school at that time which had the potential to impact upon the students’ levels of concentration.

The application of five instructional sessions (see Table 4) enabled the participant students to develop skills related to the five selected informatics concepts. The design of the activities implemented during the sessions and the pedagogical materials were based on part of the ‘CS Unplugged’ program,Footnote 2 and were adapted and translated into the Lithuanian language and context by the study’s three authors. A detailed description of all 15 activities (five sessions, with a different approach applied to each of the three groups) is provided in Appendix E. Each session started with a short presentation of the informatics concept being targeted, with the same information provided to all of the participating students, plus and an introduction to the activity.

Table 4 Short description of five instructional sessions
Full size table

3.4 Data analysis

3.4.1 Quantitative

To answer the first research question, statistical methods were employed to analyse the collected students’ quantitative and qualitative data. Results represent the students’ performance and success in solving the pretest and posttest Bebras tasks. Then, the students’ results values were calculated as a sum of the tasks solved in the pretest (ranging from 1 to 5) and the posttest (ranging from 0 to 5).

Qualitative data on the students’ perception of their learning experience was analysed based on thematic analysis (as described in this subsection). The reflection value (ranging from 1 to 9) was derived from the students’ answers to three open-ended questions (see Appendix C) as a sum of three values:

  1. (1)

    Would you like to participate in more activities like this, and why? (1 ‘no, because…’, 2 ‘yes, because…’);

  2. (2)

    What did you learn from these activities? (from 0 ‘nothing’ to 4. See Sect. 4.2. for answer categories);

  3. (3)

    Where could you use this new knowledge in your everyday life, and why? (from 0 ‘nothing’ to 3. See Sect. 4.2. for answer categories).

Due to the relatively small sample, rank-based distribution-free non-parametric measures were utilised:

  1. (i)

    Mann–Whitney U test was used to compare differences between two independent samples (grouped by gender, results values, and reflection values);

  2. (ii)

    Kruskal–Wallis test was used to compare differences between more than two independent samples (three groups with different pedagogical approaches applied);

  3. (iii)

    Wilcoxon signed-rank test was utilised to compare differences between two related samples (pretest and posttest results values);

  4. (iv)

    Spearman’s criterion was used to find correlations between the results and reflection variables.

Interquartile ranges were visualised using boxplots. The significance level was set to 5%. For statistical analysis, IBM’s SPSS Statistics 28 software package was utilised.

Additionally, the students’ responses to four multiple-choice questions were collected at the end of the intervention. The students ranked the five activities (related to the five selected informatics concepts). MS Excel was used for this stage of the analysis.

3.4.2 Qualitative

In order to answer the second and third research questions, the students’ and teachers’ open-ended question responses were analysed using a variant of thematic analysis called ‘structured tabular thematic analysis’ (ST-TA), which provides an adaptable technique for working with short qualitative data in a relatively structured way (Robinson, 2022). Short-text research is considered to be richer based on the variety of the answers given rather than the depth of the answers. Inter-analyst agreement and discussion was used throughout the data analysis process to ensure reliability of the process. The three researchers (authors) were involved throughout and worked separately to analyse the student data manually so as to ensure the credibility of the process. All three researchers then discussed and resolved any disagreements. The level of agreement reached between the coders was measured using Miles and Huberman’s (1994) formula (i.e., Reliability = number of agreements / total number of agreements + number of disagreements). Agreement ranging from 82 to 89% was reached between the coders, indicating that the codes and categories used were reliable (Saldaña, 2009). The same process was followed for the teachers’ data with the use of MS Excel and two researchers as coders, which yielded an agreement level of 83% between them.

4 Results

4.1 Students’ performances

Considering the three different pedagogical approaches, the Wilcoxon tests revealed no significant differences between the pretest and posttest results for the technology-mediated (TM) approach (Z = -0.347, p = 0.728). However, significant differences between the pretest and posttest results were found for the role-play (RP) (Z = -3.000, p = 0.003) and hands-on (HO) (Z = -2.223, p = 0.026) approaches, and the results for these approaches were higher at the pretest than the posttest. Possible reasons for these unexpected results are discussed in Sect. 5.

A Kruskal–Wallis H test revealed that a statistically significant difference existed between the students’ results (number of solved tasks) for the three pedagogical approach groups (χ2(2) = 21.44, p < 0.001), with a mean result of 22.42 for the RP approach, 18.03 for the HO approach, and 40.38 for the TM approach. Also, the same test revealed a statistically significant difference between the students’ reflections according to the three learning approach groups (χ2(2) = 10.208, p = 0.006), with a mean result of 18.44 for the RP approach, 32.66 for the HO approach, and 32.64 for the TM approach. The students’ attitudes towards the Bebras activities did not always correlate with the results of the tasks solved. For the RP approach only, Spearman’s rho 0.305, at the 0.05 level (2-tailed) of significance, revealed that a moderate correlation (Dancey & Reidy, 2007) existed between the actual task-solving results and the students’ reflections.

On average, students from the RP approach group performed better than those from the HO approach group (see Fig. 2), but the reflection values (see Fig. 3) of the students from the RP approach group were lower. Results values for the RP approach ranged from 1 to 6 (median = 5, more dispersed data), while the values for the HO approach ranged from 1 to 8 (median = 4, less dispersed data) (Fig. 2). Reflection values for the RP approach group ranged from 1 to 8 (median = 5.5), while the values for the HO approach group ranged from 4 to 9 (median = 6.5) (Fig. 3), and it means that half of students perceived their learning experience quite high. Students from the TM approach group achieved the best results (values ranged from 4 to 10, median = 7) (Fig. 2), and their reflection values were, on average, at the same level as those from the HO approach group. The students’ results from the HO approach group were the lowest among the three groups, but they reflected positively about the Bebras activities that they had undertaken.

Fig. 2
figure 2

Distribution of Results (solved tasks pretest/posttest) for all students based on approach

Full size image
Fig. 3
figure 3

Distribution of Reflection values of all students based on approach

Full size image

Also, the results values and reflection findings were calculated according to the prism of the student’s gender, with results from 26 males and 29 females. A Mann–Whitney test revealed no significant differences between the gender groups for either the reflection (U = 344.500, p = 0.570) or the results (U = 326.000, p = 0.384). As previously mentioned, the students’ attitudes (reflection) toward the Bebras activities did not always correlate with the results of the tasks that they had solved. Only the male students from the TM approach group had a Spearman’s rho (0.641, at the 0.05 level, 2-tailed) that revealed a significant correlation (Dancey & Reidy, 2007) between the results of solving the tasks and the students’ reflections.

4.2 Students’ perceptions of the performed activities

While analysing quantitative data from the students’ ranking of the five activities based on informatics concepts in terms of their enjoyment and level of perceived difficulty (see Figs. 4, 5, 6), these terms were related to an emotional and subjective assessment that was dependent upon the student participants’ personal feelings and attitudes. It may be observed that despite each activity having a quite diverse ranking among students from the same approach group, there were certain notable trends. Overall, students from the TM approach group were more positive than those from the other two groups, and that they considered four of their activities to be easy and enjoyable to undertake.

Fig. 4
figure 4

Students’ ranking of five activities from RP approach

Full size image
Fig. 5
figure 5

Students’ ranking of five activities from HO approach

Full size image
Fig. 6
figure 6

Students’ ranking of five activities from TM approach

Full size image

Students of the HO and TM approaches enjoyed the first activity on the command concept, while students from the RP approach liked the same activity the least. The third activity, related to the concept of sorting network algorithm, was perceived as the hardest and too difficult for students from the TM approach group. On the other hand, this concept, unless objectively difficult for primary school students, was either the most liked or second most liked by students from the RP and the HO approach groups.

The participant students also reflected on the Bebras activities they undertook by responding with textual answers to three open-ended questions. The first open-ended question was: Would you like more activities like this, and why? (Appendix C). The students’ answers were coded under two main categories: yes and no (see Fig. 7). The responses of those students who answered ‘yes’, that they would like more such activities, were subdivided by meaning. In total, 67% of the students found the Bebras activities to be interesting, and 16% found them useful. On the other hand, 16% of the students responded ‘no’, that they would not like to undertake similar activities.

Fig. 7
figure 7

Results of thematic analysis on students’ perception of activities

Full size image

The students’ answers to this question were analysed based on the three pedagogical approaches. Students from the RP approach group responded both positively (55%) and negatively (45%), with 44% of the students stating that the activities were considered fun and exciting. However, the activities were perceived as boring by 28% of the students in that same group and hard by 17%. In the second group, which worked according to the HO approach, 69% of the students mentioned the fun and exciting nature of the Bebras activities and 25% mentioned their usefulness, but only 6% found the activities to be too tricky. Meanwhile, in the third group with the TM approach, all of the students responded positively and emphasised the fun and exciting nature of the activities (86%), with less emphasis on their usefulness.

The second open-ended question asked: What did you learn from these activities? (Appendix C), relating to their perceptions on what they had learnt from undertaking the Bebras activities. The structured tabular thematic analysis results show that seven categories emerged from the students’ responses (see Fig. 8), where the most answers were in the category ‘Some informatics concepts’, including a combination of informatics concepts like commands and sorting algorithms, etc. A total of 67% of students from all three pedagogical approach groups mentioned at least one of the five topics they studied during the tasks (split as 66% of students from the RP approach group, 88% from the HO approach group, and 52% from the TM approach group). Around 28% of students from the RP approach group revealed having learnt about sorting algorithms, and 16% mentioned commands and sorting algorithms. Meanwhile, 31% of students from the HO approach group stated that they had learnt about sorting algorithms.

Fig. 8
figure 8

Results of thematic analysis on students’ perception of what they learnt from activities

Full size image

A total of 29% of the students from the group involved in activities with computers (TM approach) highlighted that they had learnt how to use a computer, and 14% of students from the same class emphasised having learnt the basics of programming. One student from the class who had learnt by following the hands-on (HO) approach recognised that they now understood how to use a computer, even though they had not used one during the actual Bebras activities. Around 5% of all students from all three student groups found similarities in the activities to mathematics (notably, all of the students who worked with the role-play (RP) approach), and the same number of students confessed to having learnt nothing from the Bebras activities (although there were none from the TM approach group). Meanwhile, two students from different approach groups focused on their ability to work collaboratively rather than having learnt any specific content.

The third question was: Where could you use this new knowledge in your everyday life, and why? (Appendix C). The participant students’ answers were classified under seven categories (see Fig. 9). The category that contributed the most related to the possibility of using their new knowledge about informatics, programming, and robotics, and this was mentioned by 29% of students, mostly from the TM approach group. A total of 20% of students addressed a specific topic (from the five informatics topics covered during the sessions) and a specific practical context (most students from the HO approach group). A total of 20% of all of the students (Fig. 9) revealed being able to use their new knowledge in other tasks and in various games. At the same time, 11% of the students mentioned that their new knowledge could be helpful and practical in other disciplines such as mathematics or physical culture. Interestingly, as in the second open-ended question, four of the students highlighted their ability to collaborate and help teach their peers. Five students from the RP approach group declared having learnt nothing or did not know where to apply/use their new knowledge. Interestingly, two of these same students answered in the previous question that they had learnt sorting algorithms or understood all five topics.

Fig. 9
figure 9

Results of thematic analysis on students’ perception of new knowledge use in daily life

Full size image

4.3 Teachers’ perceptions of the activities implemented with different pedagogical approaches

From the structured tabular thematic analysis, it was revealed that the participant teachers’ answers (57 coded segments) were divisible between two central themes: advantages and drawbacks (see Fig. 10). Under the advantages theme, the category with the highest number of codes pertained to students’ motivation (44% of the coded segments), which consisted of four subcategories: Interest and engagement (19%); Praise and encouragement (11%); Success (9%); and Collaboration (5%). The subcategory perceived the most by all of the teachers was interest and engagement. Teachers reflected that the students had been motivated by the interesting explanations given about the new topics, the presentation of the activities and the practical demonstrations, as well as the unusual nature of the activities. Teacher 3 observed that the ‘Students eagerly carried out all of the tasks with interest’.

Fig. 10
figure 10

Thematic analysis of teachers’ perceptions of activities based on pedagogical approach

Full size image

The second most frequently perceived subcategory for the teachers was praise and encouragement. Two of the teachers (except Teacher 2), mentioned the students having been motivated by praise and encouragement in the classroom. All three of the teachers emphasised that the experience of success in completing a task or part of a task had notably motivated the students. For example, Teacher 2 mentioned that ‘The students were motivated by the fact that they had understood the application of the algorithm and had actively participated in the activity’. All three of the teachers had highlighted the importance of collaboration, having stated that the students appeared motivated by the immediate communication and cooperation between them when undertaking the activities. Additionally, this was evident from the group work and the creative elements of the activities.

Within the advantages theme, the second most perceived category was the development of skills (32% of the coded segments), which consisted of three subcategories: Other skills (including comparison, analytical thinking, and active listening) (14%); Social and communication skills (11%); and Logical thinking (7%). Other skills, as the most perceived subcategory by all three teachers, reflected on how students’ own participation in these activities had helped the students to analyse, compare, and calculate. Also, the students had been encouraged to observe and take careful note, to discover connections, and to correct their mistakes. The students’ knowledge from maths lessons had enabled them to grasp the concept of algorithms and thereby enjoy the tasks more quickly. Teacher 3 observed that it had ‘Encouraged the students to listen carefully, observe, try interesting activities, and obtain quick results’.

All three teachers mentioned that attentiveness and not only subject but also the social and communication skills of the students had improved. In all three classes, the students had worked collaboratively in groups and in a friendly manner. In each group, some of the students had taken the initiative and offered advice to their peers or had corrected their mistakes. Both Teacher 2 and Teacher 3 had emphasised that the students’ logical thinking had developed during the activities.

Within the drawbacks theme, the most perceived category was difficulties for several students (11%). Both Teacher 2 and Teacher 3 had observed that not all of the students had managed to successfully grasp or complete the tasks immediately. Teacher 2 described that ‘All of the students were able to do it, that no help was needed because they all thought it was easy to write the number sequences, but then they realised that not everybody could do it’. The same two teachers also revealed that the students’ lack of attention had been evident to some degree, especially when they were required to listen to the explanations being given, as the students had eagerly wanted to try things out in practice as soon as possible.

Concerning time issues as a category under the drawbacks theme, the same number of segments were coded as for the advantages category. On the positive side, two of the teachers (1 and 3) had emphasised that there had been plenty of time in general for the students to undertake the activities, with most able to attempt the activities several times over. However, on the negative side, all three of the teachers mentioned there having not been sufficient time for some of the students, with Teacher 3 mentioning that: ‘It was difficult to provide timely help to students who were struggling’.

5 Discussion and conclusion

This study explored the process of primary school students’ learning of informatics concepts through three different pedagogical approaches, employing a mixed-method research construct that considered data from both students and their teachers.

5.1 Discussion

In contrast to what was predicted based on results in the current literature (Dağ et al., 2023; Sun et al., 2021; Tonbuloğlu & Tonbuloğlu, 2019), from analysis of the study’s findings based on the students’ pretest and posttest (on sets of Bebras tasks) results, the posttest failed to show any significant increase. The posttest values for the TM approach did not differ from the pretest, and for both the RP and HO approaches, the posttest results were actually lower than in the pretest. Since the pretest and posttest were each comprised of Bebras tasks, it is a crucial to note that Bebras tasks are real challenges for students since they are created by meeting various criteria deviating from typical classroom-based tasks that the students will have previously encountered. In Bebras tasks, many concepts are not expressed directly, which is deliberate in order to stimulate computational thinking, but it is expected that students will recognise in the future that certain informatics concepts had been learnt from undertaking Bebras tasks when they became more acquainted with the concepts that they represented. A similar pattern of results was also obtained in a study conducted by Tonbuloğlu and Tonbuloğlu (2019), in which it was revealed that although there were positive effects seen with various CT skills, there was no significant improvement in the participant students’ problem-solving abilities. The students displayed consistent levels of motivation and engagement in unplugged coding activities, yet struggled with certain concepts that required algorithmic thinking skills. Their study’s findings underscored the effectiveness of various teaching methods and the appeal for students of activities tied to real-life situations, while noting occasional issues with scheduling and teamwork in a crowded classroom setting. In the current study, the number of students in each of the three classes averaged 22. However, the researchers reached a consensus that in order to achieve the intended advantages, the class sizes should not actually exceed 18 students (Singer, 2022). It was notable from the interventions that the provision of individualised support to the students was challenging.

Based on the current study’s findings, the students’ perceptions of the Bebras activities did not always correlate with the actual task problem-solving results. However, it may be concluded that the students valued the process of learning facilitated by the approaches provided and the enjoyment that they derived from undertaking the activities, irrespective of whether or not they managed to successfully solve each Bebras task. These findings are consistent with those of Tonbuloğlu and Tonbuloğlu (2019), who found that students maintained a high level of motivation and consistently attended class during the unplugged activities. Furthermore, the same study’s results also suggested that students enjoy activities based on their inherent interest and the activities’ relevance to everyday life.

Similar to the study conducted by del Olmo-Muñoz et al. (2020), which found the unplugged teaching method provided the same benefits in terms of informatics learning regardless of gender, this method benefits females in terms of their motivation to learn. The current study’s results revealed no significant differences between gender groups in terms of the students’ results and reflection. Only for males from the approach TM (plugged) group was there a moderate correlation between the results of task solving and the students’ reflections. In this same group, seven of the 10 male students mentioned the words ‘computer’, ‘programming’, and ‘robotics’ in their open-ended answers. This result ties well with previous research in which plugged activities utilised technology; a phenomenon illuminated in the findings of Küçükaydın and Çite (2023), who demonstrated the efficacy of integrating technology into lessons in order to enhance students’ CT abilities. Integrating technology in the primary school classroom not only facilitates the visualisation of abstract concepts, but also nurtures creativity in students, encouraging a discerning perspective on events, and helps foster the application of technological solutions to problem solving.

Comparing the students’ results across the three student groups in the current study, in which different pedagogical approaches were applied, it can be seen that statistically significant differences were found in terms of the tasks solved. Considerably better results were demonstrated in the group learning scenario where students used computers in their activities. The current study involved fourth-grade (final year) primary school students, and this finding may indicate that starting from a certain age, informatics concepts should be taught with computers rather than just through unplugged activities. Overall, these findings are similar to those of del Olmo-Muñoz et al. (2020), in that it is more appropriate to work with informatics at the early primary education level using a mixed approach that combines both unplugged and plugged activities rather than only one. Additionally, research findings by Saxena et al. (2020) involved creating a series of unplugged activities which were strategically designed to provide students with the necessary language skills for their future computational thinking. Their aim was to establish a stronger foundation for subsequent plugged CT activities involving digital tools. Based on this and following the results of the current study, it may be recommended to start the teaching of informatics concepts through unplugged activities. The authors claim is also supported by research by Munasinghe et al. (2023), who stated that using unplugged activities prior to programming with digital devices can result in improved student outcomes within an equivalent timeframe.

In the current study, students’ perceptions of activities differed significantly across the three student groups according to the pedagogical approaches used. The least positive student perceptions were received from those who learnt in the role-play (RP) approach group. However, despite more negative perceptions from the students, the RP approach group appeared to perform better in their results than the second group which worked with hands-on (HO) activities. These findings can be said to align with the results of a comprehensive literature review conducted by Lu et al. (2023) across various grade levels. Their study revealed that role-play had a more significant impact on CT than game-based learning involving action games, simulation games, puzzle games, or adventure games. Another finding from the current study showed that despite the results from the HO approach group being the lowest among the three, students from the HO approach group appeared positive in their perceptions. This finding aligns directly with previous studies (e.g., Kalelioglu & Sentance, 2020) in that the use of a hands-on approach can be fun for students. Furthermore, engaging in learning by doing encourages breaking away from everyday routines and the rigidity of the school learning environment (Weigend et al., 2019).

The teachers’ perception of the pedagogical approaches employed in the current study confirmed the findings (Lee et al., 2022; Tonbuloğlu & Tonbuloğlu, 2019), that unplugged classroom activities can be implemented in a way that stimulates both students’ interest and motivation. According to the results of the current study, plugged activities were conducted in a way that piqued the students’ interest and made learning more engaging and relevant. On the other hand, the perception of two of the teachers (plugged and unplugged approaches) led to a similar conclusion (Tonbuloğlu & Tonbuloğlu, 2019) when some students encountered difficulties and could not grasp or complete the tasks immediately. This was concluded to partly relate to the students’ lack of attention, especially when listening to explanations, and in facing difficulties in connecting theory to practice. The teachers also highlighted the development of various student skills as clear advantages of the Bebras activities. This supports the findings (Bocconi et al., 2022; Tonbuloğlu & Tonbuloğlu, 2019) that skills such as logical thinking, problem solving, communication, and collaboration are essential for success in today’s world.

5.2 Practical Implications

The results show that all three approaches are appropriate for the students and can be applied to teach informatics in primary education. However, some topic-related practical recommendations for classroom activities can be suggested.

For the topic command and sequence of commands, there is only a need to allocate more time for the activity to allow students to fully engage in the creative process of creating their own command sequences. The introduction to the activity should be short to maintain student engagement and maximize practical learning time. On the other side, it is essential to provide timely help and acknowledge successful task completion to motivate and engage students. As well as highlight the development of logical thinking, attentiveness, and social communication competencies through the activity. Also, the activity's potential for error detection should be emphasized as a valuable learning experience for students.

While working on sorting algorithms, students faced almost no difficulties and just needed to continue to encourage group work that allowed students to take on different roles, fostering teamwork, peer teaching, and a sense of shared accomplishment. To continue to leverage familiar and enjoyable elements like Lego bricks, novel activities, and group dynamics to sustain students' motivation and interest. Also, it is important to emphasize the development of attentiveness, concentration, logical thinking, and problem-solving skills through this activity.

For the sorting network algorithm, based on reflection from teachers, it is essential to incorporate opportunities for students to repeat the activity for reinforcement, especially to enhance their comparison of numbers. Also, time constraints should be considered, and sufficient time should be allocated for students to complete the task. It is essential to encourage students and create an environment where students can experience success to maintain high motivation levels. Furthermore, to address the challenge of providing one-to-one support to multiple students simultaneously, ensuring that struggling students receive adequate assistance.

For the topic searching algorithms (Binary search), all three approaches suited very well, and need to continue using attractive and inclusive explanations, demonstrations, encouragement to keep students engaged. It is a crucial to emphasize the benefits of the activity in teaching comparison skills and fostering critical and logical thinking among students, also focusing on helping students understand the practical application of algorithms. To continue to encourage a collaborative environment where students who complete tasks faster can assist their classmates, fostering teamwork and mutual support.

For the shortest path algorithm, need to continue to ensure that topics are presented and explained clearly to help students grasp algorithms quickly, leveraging their existing knowledge from related subjects. Moreover, this activity facilitates group work where students can help and explain concepts to each other, fostering a friendly and supportive learning environment. On the other hand, it is important to accommodate individual preferences within the group setting, allowing students who prefer working individually to do so occasionally. It is important to implement strategies to address issues with attentiveness, ensuring all students remain engaged throughout the activity. Also, sufficient time should be allowed for activities, allowing students to explore, practice and repeat tasks to reinforce learning. Finally, it should be emphasized that this activity fosters the development of logical thinking skills through tasks requiring analysis, comparison, and calculation.

5.3 Conclusion

The current study highlighted the impact of pedagogical approach when it comes to primary school students’ comprehension of informatics concepts. Three different approaches (two unplugged and one plugged) were applied in teaching selected informatics concepts, and then assessed based on the participant students’ results, reflection, and gender groups.

Based on the results, it can be highlighted that:

  1. (i)

    The students valued the learning process and stated that they had fun and enjoyed it, even if they could not solve the Bebras tasks in pretest and posttest. The activities foster primary school students’ interest in informatics, gradually building the basis for further learning.

  2. (ii)

    More advanced concepts, as the sorting network algorithm in the third learning session, involving implementation with a computer in a coding environment, may have proved to be too difficult for the students in their age group. Such activities could be introduced after engaging in unplugged activities on the same concept.

  3. (iii)

    All three pedagogical approaches were considered suitable for teaching the selected informatics concepts at the primary school level, and were achieved with the students’ interest. However, unplugged activities can serve as a stepping stone prior to switching over to plugged activities during primary education. However, for male students in the age group studied, the plugged pedagogical approach produced superior outcomes than the unplugged approach.

5.4 Limitations and Future Research

While the research contributed valuable insights on this topic of study, it is a crucial to acknowledge several limitations that should be considered with regards to the finding. First, the activities were performed with a relatively small sample, although small-sample findings can also be considered useful in the planning of future larger scale interventions aimed at developing a deeper level of student engagement in informatics education that positively affects the participants.

The current study explored the effects of three distinct pedagogical approaches, each with its respective strengths and weaknesses. While differences were observed in terms of the Bebras tasks solved by students and their reflections, it is essential to consider that the effectiveness of any teaching and learning approach may be dependent upon various contextual factors, including the skills of different teachers and instructors, the classroom dynamics, and the individual and collective students’ preferences. Factors such as student background, prior exposure to technology, and individual learning styles could have impacted the outcomes of the intervention but were not extensively addressed in this study. Furthermore, activities were implemented just prior to Christmas, a time during which many other activities took place at the school, which may have potentially have impacted upon the students’ levels of concentration.

Contrary to expectations drawn from the existing literature, the posttest results from the current study did not reveal any significant improvement in the students’ performance. Despite having observed positive effects in other studies, the current study revealed only limited improvement in the students’ results. This finding may be attributable to various factors, including the design of the tasks and the specific focus of the interventions. The study employed Bebras tasks in order to gauge students’ understanding of certain informatics concepts. However, it is important to recognise that Bebras tasks may not entirely align with the conventional classroom curriculum. Additionally, the effectiveness of different teaching approaches may also be influenced by class size, suggesting that smaller class sizes may be more beneficial for individual support and learning outcomes.

The study involved fourth-grade students, and the findings suggested that informatics concepts might be more effectively taught using computers at that age. However, the findings should be interpreted within the context of this specific age group, and the results might not be generalised to other grade levels. The study’s findings pertain to a specific context and a specific set of pedagogical approaches. The generalisability of these findings to different educational settings and diverse student populations may vary.

To address the limitations of short-term interventions, longitudinal studies could be undertaken as a means to assessing the longer-term impact of different pedagogical approaches on students’ retention of informatics concepts. Such a research exercise could provide greater insight into the durability and transferability of learning outcomes. Qualitative methodologies such as think-aloud protocols or cognitive interviews could be incorporated to elicit a deeper insight into the cognitive processes underlying student performance. This would enable a finer-grained examination of how different pedagogical approaches influence problem-solving strategies. Extending the investigation to different age groups within the primary education spectrum could elucidate age-specific variations in the effectiveness of pedagogical approaches. This could also contribute to more comprehensive informatics education at the primary education level.