Keywords

1 Introduction

Across the world, governments have developed guiding policies to support efficient digital innovation and successful implementation of digital technologies at the school [32]. Investment in teacher training, teachers’ digital competence, and access to digital learning resources (DLRs) have increased. It is also already well established that the way technology is blended into teaching and learning practices is crucial for ensuring that its use can lead to better student outcomes [1]. Quite often, increased access to technology does not change teaching and learning fundamentally, and learning gains remain unimpressive [2]. The evidence of the impact of technology-enhanced learning practices on students’ learning is ambiguous, reported successes of implementing digital innovation at schools are often small-scale [3], not always sustainable [4] and frequently, learning technologies are used to replicate existing practices in school [5] instead of changing education more fundamentally [6]. OECD results indicate that although students who use digital technologies at school often have better learning outcomes compared to students who use technologies infrequently, students who use digital learning technologies at school very frequently perform significantly worse at most of the included learning outcome measures [19]. However, these findings do not provide evidence on how learning technologies are integrated into teaching-learning.

It is evident that making technology available for educational institutions is just one from several aspects, as the adoption of such innovation depends on different individual and organisational aspects [7]. Research has shown that students’ experiences with new technologies are dependent on the choices teachers make about the technologies, which in turn reflect teachers’ skills, pedagogical values, philosophies, and curriculum approaches [8]. It is quite clear that the way teachers are integrating learning technologies and especially DLRs into their pedagogical practice will have an impact on students’ learning experiences. Teachers’ technology-related teaching skills are closely linked to multifaceted and complex technology-enriched learning activities [33]. There is evidence that teachers, on the other hand, need support and training to create meaningful learning designs (LD) to uncover the potential of learning technologies impacting student learning [9]. This is particularly important if a textbook, which usually gives clear guidance to teachers on how to teach, is replaced by DLRs which are usually more flexible in their application.

There have been studies focusing on the adoption of technologies in the classroom from teachers’ perspective, but there is still much we do not fully understand about secondary education students’ experiences when introducing new technologies in teaching and learning [11]. Though the students already exhibited some behaviours which can be productive for learning (e.g., easiness using digital tools), it has been argued that teachers fail to provide a technology-rich environment that can foster students’ engaging experiences with digital learning technologies [32].

This study is motivated by an understanding of how students experience when teachers introduce new technologies in the learning process. Based on a national-level initiative to launch DLRs we analyse teachers’ practices and students’ experiences to answer the following research questions: RQ1. How did teachers adopt the DLRs in their pedagogical practices?; RQ2. How did teachers’ pedagogical practices impacted students’ perceived experience with DLRs in the learning process?

2 Theoretical Underpinnings of DLR Implementation

Learning in a digitally-enhanced environment means that the teacher uses learning technologies to foster students’ learning through a variety of (personalised) instructional methods, challenging content, and feedback through formative assessment to ensure all students reach their potential [12]. One of the technologies that has great potential to transform student learning experiences is DLRs (e.g., e-textbooks, interactive materials, digital tasks). DLRs have become an essential part of learning environments where teachers and students work together [13].

There is no universally accepted definition of DLRs, synonyms used to describe the practices of learning with digital content: digital textbooks, e-textbooks, digital learning materials, digital learning resources, open educational resources, digital learning objects, etc. A spectrum of characteristics has been proposed by different authors and initiatives to describe these concepts technically or instructionally. The efficient implementation of DLRs in teaching and learning could be seen from two perspectives: First, it is important to consider how the tasks are designed and whether they activate students’ thinking (the instructional design aspect). Second, no matter how tasks are designed, teachers have multiple options in terms of the pedagogical approach the tasks are embedded in (learning design aspect).

Concerning instructional design, DLRs could follow a behaviourist approach - for effective learning one needs appropriately presented material to initiate the desired responses [14]. From the cognitive perspective, instructional design approaches could be followed by emphasising the importance of learning by employing whole problems to avoid fragmentation and encourage the integration of knowledge, skills, and attitudes [15]. From a constructivist perspective, a learning object is a resource to mediate learning activity leading to learning outcomes while students’ knowledge is constructed, transformed, and applied through active engagement [16].

The second aspect, how the teacher implements the design, is as important as the design of the task. Currently, teacher-centred lessons dominate and often aim at knowledge transmission and promote mere rote learning, but educational practices should enhance active learning by emphasising the interest, motivation, and engagement of the learners [17]. The way teachers integrate DLRs into practice, by taking into account subject-related aspects, and an understanding of how students learn, can fulfil the potential of DLRs in teaching and learning [18]. Some of the authors have proposed using the ICAP framework [34] as a systematic approach to differentiate the levels of students’ cognitive engagement while interacting with digital technologies. Despite the trend for teachers’ pedagogical practices to become more diverse, passive learning approaches are fostered more often compared to approaches in which learners are active, constructive, and interactive [35].

Evidence regarding the effects of technology use on student outcomes, however, paints a rather sobering picture. For example, on analysing the relationship between students’ access to technology and their results on PISA tests, it was found that students without access to computers in mathematics class achieved better results on both the paper- and computer-based assessments [19]. One reason for this discrepancy between prediction and reality is that the fields of educational technology, educational research, and educational practice have largely remained detached from each other [20]. Teachers, confronted with rapidly changing technology for the classroom, but supplied with very little guidance about its use, and insufficient time to experiment with it, either resist change or adopt technology only to use it in ways they are already accustomed to, treating it as merely a substitute for conventional resources and methods [21]. National-level investments can ensure that teachers have access to high-quality DLRs based on the national curriculum. However, the pedagogical design of the materials and knowledge practices around learning technologies are decisive in deciding whether students benefit from the technology or not. The aim of the present study is therefore to uncover teacher practices around DLRs and to find out how students experience the usage of these resources.

3 Research Design

3.1 Research Context

This study reflects experiences from the national level piloting experience of DLRs in Estonian secondary schools. Estonia is well-known for its educational innovation and widespread implementation of technologies for teaching and learning [10]. The education system of Estonia is decentralised and teachers have autonomy in deciding how to deliver educational content to achieve set learning outcomes. The nationally implemented Digital Turn program was a strategy to provide Estonian schools with DLRs, providing teachers with a variety of ways to enrich the learning process [10].

During the project’s lifespan, the team of Estonian researchers, didactics, and practising teachers from different subjects developed nearly 6000 tasks to cover the national curriculum. The instructional design of the DLRs was based on Merrills’ task-centred instructional design model [15]. According to this model, DLRs should be designed at different levels which enable students to be engaged in solving real-world problems, and activate existing knowledge as a foundation for new learning, new knowledge is demonstrated to the learner, can be applied by the learner, and is integrated into the learner’s world. Based on this model, tasks were developed inspired by four different types of instructional interaction (Tell, Show, Ask and Do). The technical infrastructure for authoring and storing DLRs was built on Drupal Content Management System enhanced with H5P module, allowing easy generation of interactive DLRs from HTML5 templates. It enabled the teachers to use more than 40 different types of interactive resources (e.g. multiple-choice, fill-in-the-blanks, drag-and-drop, interactive video) in line with selected LDs and make the finalised resources available to all interested users through the national repository called eKoolikott.

LD - plans laying out instructional activities and experiences - for implementing the DLRs were created by the project team to foster the effective combination of teacher practices, DLRs, and students’ practices. Instead of using DLRs to replace textbooks, teachers were guided to implement Student-Centred Learning (SCL) LDs created by the teachers, researchers, and didactics to fulfil the potential of the developed DLRs. The following scenarios were designed: (a) Flipped Classroom. Before the lesson, the student gets familiar with the basic concepts using DLRs suggested, and in class, they apply new knowledge in solving vital problem situations; (b) Project-based learning. Students in groups work on different activities, some of which require individual work with the DLRs to produce the collaborative outcome of the project; (c) Task-based learning. Students solve increasingly complex tasks while learning a new topic, relying on DLRs. Once the tasks given by the teacher have been solved, the students themselves work in pairs to create new tasks and give them to other students to solve. (d) Game-based learning. Students participate in a game with predefined rules the aim of which is to find and apply new knowledge while solving tasks. Some of the tasks are created by the students themselves and most of the tasks require the answer to be provided as a digital artefact. Before the piloting phase, teachers received a short training (4 h) to understand the pedagogical ideas behind the DLRs and innovative learning scenarios, technical aspects of DLR use, and the possibilities of mixing the DLRs and re-designing learning scenarios. Each teacher was asked to pilot the DLRs in their class at least three times during one month.

3.2 Sampling, Data Collection, and Analysis

Teachers were recruited voluntarily to pilot the developed DLRs through an open call for participation among Estonian secondary schools. 21 teachers from 17 schools applied: 8 for mathematics, 7 for science, 4 for social sciences, and 2 for music and art. They piloted the DLRs with their students from grades 10 and 11. Altogether, data was collected from 1200 students in the piloting phase. Once the data quality was checked, we analysed the data of 683 students and 21 teachers. A mixed methods research was carried out:

Teachers’ Reflections.

To understand how teachers employed DLRs (i.e., using different instructional strategies), we asked them to fill a report after each piloted lesson describing how they designed classroom activities. Teachers’ reflections were coded independently through thematic analysis by two of the authors of the paper. A second iteration in the data analysis included two authors discussing the themes and categories until reaching an agreement.

Students’ Questionnaire.

After the piloting experience, students filled a web-based survey including questions related to (a) demographic information (gender, grade, age); (b) earlier experience with DLRs and other TEL practices; (c) piloting experience (pedagogical approach) with DLRs and perceived usefulness; (d) challenges experienced in the process (open-ended questions).

Open-Ended Questions.

A thematic analysis was made to analyse the challenges that students experienced while using the DLRs. Two of the authors read and coded the dataset to identify whether and how students struggled with DLR use. Then both authors reviewed and discussed the codes until reaching agreement.

Likert-Scale Questions.

These items required students to rate their agreement with given statements on a scale of 1 to 5 (e.g., I liked the way the teacher organised the testing of DLRs). We used SPSS version 21.0 to extract the underlying factors through an exploratory factor analysis, with Principal Component Analysis and Varimax with Kaiser Normalisation for both sets of items (the first one regarding the context of the use of different technological devices and the second regarding students’ attitudes towards the use of technology in the classroom). Missing data were treated through pairwise analysis. The analysis returned a 20-factor solution (12 factors for the first set of items and 8 for the second), explaining 65 and 64% of the variance (respectively). Coherently with our interest to uncover different teacher practices in the use of DLRs, we grouped these factors into teacher-led practices and student-centred ones. In each of these, we also included factors that addressed earlier experiences with these teaching strategies. Moreover, we added two factors related to student experiences (perceived satisfaction and perceived usefulness):

Teacher-led practices (during piloting and earlier):

  • Teacher-instructed piloting experience: describes students’ experience of using DLRs only according to teachers’ instructions.

  • Individual DLR piloting practices at school: describes the usage of DLRs during the piloting by the students individually.

  • Teacher-initiated earlier usage of DLRs: describes students’ earlier experiences of DLR use and information retrieval after teachers’ instructions.

Student-centred practices (during piloting and earlier):

  • Collaborative and student-initiated instructional piloting experience: describes the usage of DLRs during the piloting in groups or pairs, but students’ own initiated usage of DLRs at home was also loaded here.

  • Students’ earlier experience with TEL practices: describes students’ earlier experience such as searching for information and materials from the web on their own or teachers’ initiative and solving e-tests.

  • Students’ earlier experiences with DLRs: describes items regarding students’ earlier experience of using DLRs for learning and their habit of searching for relevant DLRs on their initiative.

Effects on the student experience

  • Perceived satisfaction of piloting: describes the students’ satisfaction with the quality of DLRs, level of the difficulty of the DLRs, and the instructional practices implemented by the teacher around the DLRs.

  • Perceived usefulness of DLRs: describes the students’ willingness to use the DLRs in the future, wish that DLRs will be used again in the future by their teacher, and perception that DLRs help them to learn better, organise learning flexibly, and learn new topics faster on their own.

Once we chose the variables for this study, we modelled them through Partial Least Squares (PLS) analysis, in SmartPLS 3.0 software. We used it as a tool for formative measurement of the latent variables due to the exploratory nature of this study [22]. PLS-based Structural Equation Modelling (SEM) is a frequently used technique for estimating path coefficients in structural models [22]. We built a Reflective Model, tested the measurement model (validity and reliability of the measures), and examined the structural model [22, 23]. We tested the significance of the path coefficients, loadings, and hypotheses through a bootstrapping method (10000 resamples) at the 5% significance level [22]. In line with the research questions above, we assumed that the organisation of the instruction would be an important predictor of the students’ experience in using DLRs (see structural model in this analysis in Fig. 1). However, no further assumptions were made about specific and differential effects.

Fig. 1.
figure 1

Hypothetical model.

Full size image

4 Results

4.1 Adoption of DLRs into Pedagogical Practices

21 teachers piloted the DLRs during two months in 196 lessons in art and music, 60 in history, 41 in natural sciences, and 76 in mathematics. The granularity of these resources varied significantly, covering activities from 10 min (in arts) to 90 min (in maths and science). Analysis of teachers’ descriptions of LDs indicated that although they participated in training regarding how to design SCL with the support of DLRs, 90% of the LDs simply replaced textbooks with DLRs, and the potential to activate students through SCL practices remained underused.

For instance, in all the domains (arts and music, natural sciences, mathematics, and social sciences), the majority of teachers used a similar instructional approach: first, they introduced the new topic, after that students interacted with DLRs, and finally, students solved tasks individually or sometimes in pairs (e.g. example from art teacher: “Initially, I introduced the topic on the basis of my own slides, then I let the students read the DLRs, they performed the tasks and finally they analysed the painting”). History teachers tended to mainly watch historical resources ( videos, film clips), followed by individual work with DLRs to analyse, reflect and make connections with the content of the video material (“We watched together the material on the screen and discussed it, students read the two materials independently and solved one self-assessment test”). In itself, it is good practice to guide students to work with historical resources, but behind these materials and LDs also lay the potential to guide students to synthesise knowledge and solve problems. Mathematics teachers especially reported the usage of DLRs for individual solving of tasks in school during the lesson and also, sometimes, at home (“We had a repetition lesson before the test with the DLRs”). This may come from the fact that maths, by its very nature, is quite drill and practice-oriented, and much of the material allowed for this pattern of use. A similar trend was also observed in the natural sciences, where the dominant instructional practice was frontal teaching. For instance, the teacher introduced a new topic (often presenting DLRs on the screen), followed by the students’ individual work with the DLRs and a joint discussion (“Individual work in a computer [assisted] class: getting acquainted with a new topic and repeating what has been learned before”).

Next, we analysed in-depth the 10% of the LDs that integrated SCL elements. For instance, one science teacher asked students to work with the DLRs in pairs, debate the strengths and weaknesses of certain aspects of energy resources and come up with a joint poster introducing resources with their benefits and disadvantages. Task-based or problem-based learning scenarios were also designed by some science teachers ((“Students had to get acquainted with the world’s forest types and deforestation as a global problem, and the world’s forests and their importance. Each student had to prepare multiple-choice questions on each topic. At the end of the lesson, we answered the questions prepared by the students together”); (“At first students got acquainted with the topic and solved the tasks by the river outside, after getting acquainted with the environmental topics, the students went to the school surroundings by the river and answered questions about the environmental topic at a selected point”)). One history teacher also attempted to engage students in debate, she started the lesson by introducing a new topic, then used DLRs to enhance students’ oral argumentation skills by encouraging panel discussions, and at the end of the class students individually repeated what they had learned during the class with the support of DLRs.

It can be concluded that a majority of the 21 teachers mainly perceived DLRs as materials that could support traditional textbooks, which just provides additional advantages such as different types of media (video materials e.g.), level of interaction, and instant feedback for the students. Novel DLRs were used for traditional purposes such as learning new topics, preparing for tests, and repetition and validation. Despite the training and the possibility to redesign and adapt the LDs and DLRs, teachers did not use this opportunity.

4.2 The Impact of Pedagogical Practices on Students’ Perceived Experiences

This section reports the test results for the measurement and the structural model.

Measurement Model.

We assessed the measurement model through several measures. All factor loadings exceeded the threshold of 0.6 [24]. The Variance Inflation Factor (VIF) statistics (to assess multicollinearity) of the indicators were below five (5) [25]. The Cronbach’s alpha and Composite Reliability (CR) were within the accepted range of 0.7–0.95 [22], while the convergent validity of the items, based on the average variance extracted (AVE) measures, was above 0.5. We assessed discriminant validity through the analysis of cross-loadings, the Forner-Lacker criterion, and the heterotrait-monotrait (HTMT) criterion [27]. Regarding cross-loadings, all the items’ outer loadings were greater on their respective constructs than their cross-loadings on other constructs [28]. We established discriminant validity through the assessment of the Fornell-Larcker criterion observing that the square root of the AVE of each construct was higher than its correlation with other constructs [28]. The assessment of HTMT shows values lower than 0.90 indicating a satisfactory discriminant validity [27]. All the aforementioned measures and descriptives are available in a live hyperlinkFootnote 1.

Structural Model Evaluation.

To assess the hypothesised relationships we considered the following criteria:

Model Fit.

We established a model fit through an acceptable Standardised Root Mean Square Residual (0.085) [29] and confirmed that the original value of d_ULS (i.e., the squared Euclidean distance) and d_G (i.e., the geodesic distance) is smaller than the upper bound of the bootstrap confidence interval [30].

Goodness of Fit (MOdel’s Predictive Capabilities).

We assessed the coefficient of determination (R2), beta, corresponding t-values, and statistical significance (p) via bootstrapping procedure. We also assessed the predictive relevance through effect sizes (f2) [26]Footnote 2. Results showed an R2 value of 0.182 (p = 0) for the outcome variable “Perceived usefulness of DLRs”, demonstrating a moderate percentage of variance explained by the model (18.2%) and 0.121 for the outcome variable “Perceived satisfaction of piloting”, showing a weak percentage of variance explained by the model (12.1%) [31]. We obtained a small but significant contribution of the variables whose hypotheses were confirmed [31]. The bootstrap results are illustrated in Fig. 2 and available onlineFootnote 3.

Path Analysis.

According to Fig. 2, the perceived usefulness of DLRs (3.1) was positively (and significantly) impacted by all of the student-centred approach variables (2.1, 2.2, and 2.3), and the individual piloting of DLRs at school (1.1, traditional approach). Yet, the teacher-initiated usage of DLRs (1.2) and teacher-instructed piloting experience (1.3) had a negative but non statistically significant effect on the perceived usefulness.

Fig. 2.
figure 2

Structural model with statistics

Full size image

Moreover, the Students’ earlier experiences using DLRs (2.3), Individual DLR piloting practices at school (1.1), and Teacher -instructed piloting experience (1.3) had a positive and statistically significant effect on the perceived satisfaction of piloting (3.2). On the other hand, while the variables Collaborative and student-initiated instructional piloting experience (2.2) and Students’ earlier experience with TEL practices (2.1) had a positive but non statistically significant effect on the perceived satisfaction of piloting (3.2), the Teacher-initiated usage of DLRs (1.2) had a negative but non-significant effect on the outcome variable.

4.3 Challenges and Opportunities Perceived by the Students

We identified from students’ reflective responses (n = 187) that they mainly faced technical and methodological challenges during the piloting.

Technical Difficulties.

A majority of students reported technical difficulties while interacting with the DLRs. The biggest category under this theme was related to small screen size and navigation issues. Next, the lack of possibility to save responses and continue working on the assignment later, the lack of opportunities to zoom in on the content. Students also faced issues with an internet connection, due to which some materials were not accessible, answers were not saved and the overall experience was not satisfactory.

Methodological Aspects and Teaching Practices.

Students appreciated video-based and other illustrations next to the text-based materials, which helped them to better understand the content. However, students also felt that there were too many ‘Tell’-type questions based on multiple-choice templates and it was not motivating for them (“I ended up putting answers in arbitrarily to get the work done, but didn’t think through any of the tasks or master the topic. Should therefore reduce the number of tasks with multiple answers.”). Students reported that overall the experience was interesting and positive, but such materials could be used for rehearsal and anchoring the material and not for replacing textbooks and teachers’ explanations (“It was useful more as an introduction and refresher, but not as a deep consolidation of the topic”). Based on the students’ open responses it can be concluded that they did not perceive individual learning with DLRs as an efficient way to learn, especially when it was a new topic that students had to acquire on their own (“I can say that I understand better when the teacher explains in front of the class. I didn’t like learning independently with DLRs”). About 10% of the students who participated in the pilot pointed out that the pedagogical potential of the DLRs was not clear to them. From the students’ responses, it seemed that they felt the DLRs were created to replace textbooks and diminish teachers’ roles, which was not the goal of the pilot.

5 Discussion, Implications, and the Conclusion

This study described findings from a national-level initiative, carried out by the Estonian Ministry of Education, during which interactive DLRs were made available for secondary school teachers. It is known that the adoption of innovations needs teachers to scaffold to build a shared understanding of the innovation and create new pedagogical practices that effectively embed technological innovations [9].

First, the aim was to understand from teachers’ reflections how they integrated DLRs into their pedagogical practice. Analysis of lesson descriptions demonstrated that most of the learning activities (nearly 90%) focused on teacher-led activities, where novel, pedagogically-meaningful tasks were mostly implemented to replace the textbooks and rarely to activate students in different ways. Despite an introduction to SCL scenarios before the pilot process, teachers found DLR-supported SCL complicated or irrelevant to implement. These results are well aligned with earlier research indicating that teachers tend to use passive learning approaches in technology-enhanced learning environments [35]. This in turn demonstrates once again the importance of supporting teachers in adopting innovations, for instance by providing pedagogical support throughout the implementation of the program, and constantly monitoring and giving feedback [7]. Without such support, it could be that investments are done at the national level, but the full potential of novel technologies remains uncovered and they are only used to replace traditional teaching methods [2].

Second, we aimed to explore students’ piloting experience to understand to what extent their earlier experience of TEL practices and different instructional practices during the piloting affected their perceived usefulness of the DLRs and overall satisfaction with the piloting. Results indicate that students’ earlier experience of searching for and interacting with information resources and DLRs and solving e-tests and quizzes independently outside of the classroom had the largest positive effect on students’ perceived usefulness of DLRs, but only if this happened to the students’ initiative. When this was encouraged through classroom use under the teachers’ guidance, it did not lead to increased perceived usefulness of DLRs and satisfaction with the piloting experience perhaps due to the students’ passive role [11].

Prior experience with DLRs and TEL practices and the experiences during the piloting period with different pedagogical practices contributed significantly and independently to the students’ perception. Both individualised use of DLRs, as well as collaborative and student-initiated use, led to some positive perceptions. Whereas individual practices led to higher student satisfaction, collaborative practices led to higher perceived usefulness. Teacher-initiated use was not related to any student perceptions probably due to the fact that individual interactions with the DLRs are quite straightforward and close to the teaching style students experience in their everyday learning process [2]. In the normal classroom a variety of approaches are used from passive to active learning, constructive and interactive, and all of these approaches are needed depending on the learning goals, but the crucial thing for the teachers to understand is the balance between approaches [32]. We found that students, similar to teachers, need to adapt their learning to a situation where DLRs afford different types of practices. For instance, some students with previous experiences with TEL practices were able to organise their learning differently, or were unsatisfied, in case the teacher was using the resources in a more teacher-directed manner. Students with fewer experience in the use of DLRs may have had more difficulty adapting and were then satisfied with a more traditional use by the teachers. Survey data did not enable us to understand instructional design around the DLRs - we know the DLRs were mainly used individually, but it is not known whether the teacher used the materials for rehearsal, acquiring new content knowledge individually, or as supportive materials in problem-solving tasks and therefore we cannot draw conclusions regarding whether some pedagogical practices lead to higher perceived usefulness and satisfaction of the students.

Even though the teaching practices remained mainly traditional (as reported by the teachers in the previous section), some of the students were satisfied with them. Again, we can assume this is because students have been learning in this manner and are therefore comfortable with it [11]. The students who had been previously guided by teachers to find additional information and resources from the web during the lessons did not perceive the piloted DLRs as useful and did not rate the overall experience as satisfactory. These students have been used to using digital resources, which may not have all been designed for learning, but this experience allowed them to find materials they judged useful. Such results could mean that to some extent some of the students are used to some aspects of SCL practices (finding their resources at their own pace), which hides certain risks if information resources are learning materials. But at the same time it indicates that during the piloting, students may not have experienced SCL practices where they could choose strategies to support their learning. It is important to note that though it is not possible to conclude what kind of piloting experiences lead to a more efficient learning experience, data suggests that variety in teaching practices is needed - to motivate more self-directed learners, but also to scaffold those students who are used to learning traditionally, where technology is merely used for replacing existing resources [11]. A balance between those aspects is needed, hence pedagogical support can aim to train teachers towards a diversity of instructional practices that support student needs [7, 9].

Finally, we aimed to understand students’ experiences of the pilot and what kind of challenges and opportunities they perceived. Students faced difficulties with working with small mobile devices and the need to improve the navigation of the DLRs was noted. Students used many different kinds of devices, which must be taken into account when designing the DLR interface. More importantly, students emphasised in their responses that there is a need to think about the pedagogical practices to enhance a deeper learning experience and amplify the teacher’s role [2].

Although our study was not very long - about 1,5 months, future national initiatives that promote SCL may introduce the novel DLRs step-by-step and systematically through long-term introductions. External and temporary intervention inviting teachers and students to use new technologies and approaches, without preparing them and creating “ownership” of this innovation, is not welcomed by a relatively large share of students, as it creates extra efforts without perceived benefits to learning (from their perspective) [6, 11]. The results pinpoint the importance of dialogue between teachers, students, researchers, and developers of TEL innovations because, without a shared understanding of the capabilities and role of digital innovations, the potential to transform teaching and learning cannot be reached [21]. Especially now, when we have experienced emergency remote teaching due to the COVID-19 pandemic, the importance of learning technologies and the role of the teacher in their effective use is apparent. To scale up the materials developed under this project, while at the same time ensuring that teachers’ use practices are pedagogically sound, there is a need for further awareness-raising among teachers [5]. One way to do this is to follow a teacher’s professional development intervention as proposed by Ley et al. [9]. We need to make sure that the practice around learning technologies is student-centred and offers fundamentally new perspectives for students and at the same time enhances the scale of the created DLRs.