Cognitive processes

Cognitive processes are defined as the mental processes by which knowledge is acquired and understood through thought, experience, and the senses (Anderson et al. 1995). According to Woolfolk (2005), cognitive processes can be simple, such as obtaining, storing and memorizing learning concepts, or complex, such as transforming and using learned concepts outside of their original context. Anderson and Krathwohl (2001) have developed a taxonomy to help label the cognitive processes used for learning, teaching and assessing through which teachers can monitor, assess and understand the complex cognitive processes of students. Through an understanding of students combined with the appropriate use of the taxonomy, teachers can identify and label weaknesses both in students’ attainments and the instruction itself. This taxonomy includes six levels (listed in order of increasing complexity): (1) Remember—retrieve relevant knowledge from long-term memory; (2) Understand—construct meaning from instructional messages; (3) Apply—carry out or use a procedure in a given situation; (4) Analyze—break material into its constituent parts and determine how the parts relate to one another and to an overall structure; (5) Evaluate—make judgments based on criteria and standards; and (6) Create—put elements together to form a novel, coherent whole or to make an original product. This taxonomy for learning, teaching and assessing has been widely used in education to examine learning outcomes, and its application in technology-aided learning has been extensively researched (Kuo et al. 2012; Shadiev and Huang 2016). For example, Azar (2005), and Kocakaya and Gönen (2010) evaluated students’ levels of cognition by analyzing physics questions both from a high-school examination and from university entrance exams and then comparing the evaluation scores.

One important issue to consider in the learning process is whether students are engaged not only in simple cognitive processes (e.g. obtaining and retaining new knowledge) but also in more complex ones (e.g. applying new knowledge in different contexts). According to the Cognitive Diffusion Model (Hwang, Chen, et al. 2014; Hwang and Shadiev 2014; Shadiev, Huang, Hwang, and Liu, in press), students’ cognitive processes may be distributed into six levels throughout three different learning periods. In the first period, which takes place before the learning process (i.e. pre-learning), students’ cognitive processes are at the lowest level. During this period, students have prior knowledge which they must recall. The second period takes place after learning (i.e. after learning). During this period, students continue to be taught new knowledge and receive related assignments and examinations—which most students are able to understand and do. However, only a few students can apply what they have recently learned to solve problems in real-life situations. According to the model, a chasm exists between the lowest (i.e. Remember and Understand) and highest (at least Apply) cognitive levels. “Crossing the chasm” happens when students’ cognitive processes move from the lowest to the highest levels. If the chasm is crossed, then students are able to understand and apply new knowledge to solve daily life problems. This is when the third period takes place (i.e. crossing the chasm).

It is very important for educators to find a way to enable students to cross the chasm. Usually, in a traditional context (sitting at their desks in a standard classroom using paper and pencil), learning takes place, but newly-acquired knowledge is not applied. As a result, “Crossing the chasm” rarely occurs in a traditional context (Hwang et al. 2014; Hwang and Shadiev 2014). To enable students to cross the chasm, curriculum should be modified in the following ways: 1) the instruction should focus not only on teaching basic knowledge and exam preparation but also on the application and analysis of new knowledge; 2) students should learn both at school and outside by applying and analyzing new knowledge in a wide range of daily life situations; 3) knowledge application and analysis should be linked to an authentic environment (Scardamalia and Bereiter 1994).

The main focus of our intervention is to enhance students’ cognitive processes. By following the Cognitive Diffusion Model along with its useful guidelines, our intervention specifically aims to raise students’ cognitive processes from the simple to the complex. That is, our intervention aims to ensure that, apart from learning basic concepts in school, students learn how to apply and analyze new knowledge to solve daily life problems. To do so we employ a taxonomy for learning, teaching, and assessing to examine students’ cognitive processes, their distribution during different learning periods and the transition of cognitive processes across the three learning periods. Although enhancing students’ cognitive processes is very important, little attention has so far been paid to it, despite the taxonomy’s having been adopted in many studies. The Cognitive Diffusion Model was proposed along with useful guidelines to ensure that students learn how to apply new knowledge to solve daily life problems with technological support. However, no empirical evidence has yet solidly affirmed the validity of this model.

Practice in an authentic environment with familiar context

The pedagogical usefulness of practice is emphasized in the related literature (Scardamalia and Bereiter 1994). Practice involves repeatedly and regularly using skills in order to improve and master them. Storytelling is an instructional approach that has the potential to facilitate language practice (Nilson 2010; Wang et al. 2009). Storytelling has been described as a highly-effective instructional method because it surrounds students with the target language and enables them to communicate intentionally by using narrative sentences (Guha et al. 2007; Wang et al. 2009). The educational value of using storytelling for language practice has been emphasized in the related literature, where it has been pointed out that it can facilitate the development of students’ speaking, writing and other language abilities (Nilson 2010). Wright (1995) suggests that storytelling helps develop language expression, logical thinking, imagination and creativity in students. Learning through storytelling is generally student-centered and helps students communicate their own stories effectively when the topic is related to their daily life and personal subject matter (Kim 2014).

The relationship between life-related scenarios and language acquisition has been emphasized by Hwang, Huang, et al. (2014), Hwang et al. (2016a), Hwang et al. (2016b), and Nilson (2010). According to Lai and Gu (2011), an authentic environment with familiar context is one important prerequisite for effective learning due to several critical characteristics. First, it provides authentic contexts that reflect the way the knowledge will be used in real life. Second, it provides authentic activities that have real-world relevance, ideally ones which present complex tasks to be completed over a sustained period of time. Third, it creates an opportunity for sharing learning experiences and accessing the experiences of learners regardless of their level of expertise. Finally, it promotes reflection and enables authentic learning assessment within the tasks (Nilson 2010). In such an environment, students are more inclined to learn as they apply new knowledge to solve real-life problems which they are likely to encounter frequently as they occur in a familiar, natural context (Golonka et al. 2014).

Informed by related studies, the authors have designed learning activities based on the storytelling instructional method. Our activities focus on both knowledge acquisition and its practical application and analysis. In this study, students first learn in class and then analyze and apply their new knowledge in an authentic learning environment with familiar context.

Mobile-assisted learning

Mobile technology can offer a seamless learning experience, i.e. it can be used anytime and anywhere (Kim et al. 2015) and it may create an authentic learning environment within a familiar context rich in resources for learning (Kim and Kim 2012). According to Wang et al. (2009), mobile technology aids both formal learning in the classroom and informal learning outside class.

Several studies have already been carried out in which students learn social studies (Chu 2014; Hwang et al. 2013), science (Hung, Sun, et al. 2015a; Wang et al. 2015), English as a foreign language (Golonka et al. 2014; Lan et al. 2007; Oh et al. 2014; Shadiev et al. 2015), and other subjects with the assistance of mobile devices. These studies have demonstrated that mobile learning technology can facilitate learning.

Using mobile multimedia tools, students can create learning materials in an authentic environment. Utilization of multimedia aids such as pictures and audio for learning tasks makes learning more interactive and richer in information; they also tend to make participation more engaging (Golonka et al. 2014). Furthermore, multimedia objects in learning stimulate the imagination of students and help elicit meaningful output (Caldwell 1998); by using multimedia aids students are able to practice the target language repeatedly and regularly and pursue diverse learning goals which increase the richness of their language experience. Harmer (2007) suggests that when students record their speech they (and their teachers) can then listen in order to evaluate language performance and gauge progress. Golonka et al. (2014) claims that sharing homework with peers allows further reflection, discussion and collaboration. In addition, sharing homework increases practice opportunities and helps students to engage in EFL contexts. For example, students can listen to classmates’ audio files and thereby hear a variety of speech styles.

Mobile technology, such as tablet PCs, can be actively used in EFL learning. The potential advantages of tablet PCs in comparison to other mobile devices (e.g., cell phones or PDAs) are their bigger screens, more powerful memories, and enhanced data processing speeds. Reviewing related studies reveals that many focus on EFL learning through game-based learning. For example, Hung, Young, et al. (2015) carried out an experiment in which a game-based learning environment was constructed using tablet PCs. Elementary school students played the Crossword Fan-Tan Game: under controlled conditions they used a conventional approach (markers with different colored paper) and in an experimental condition they used tablet PCs. Hung et al. (2015b) examined how the game facilitated students’ learning. The results of the study indicate that low-achieving students in the experimental group had a better learning performance and a more positive attitude than those in the control group. Kim and Kim (2012) used a tool they call the Digital Mind Map to help elementary school students learn English vocabulary with visual support, after which both improvement in vocabulary and level of class satisfaction were measured. The results led Kim and Kim (2012) to conclude that a Digital Mind Map class is an effective platform for English vocabulary learning. Students who learned with the mind map learned significantly more. Students stated that the Digital Mind Map helped them rapidly find vocabulary words and connect them with already known vocabulary as well as memorize new words.

Our literature review shows some benefits of using tablet PCs for EFL reading. Lin (2014) and Oh et al. (2014) explored the effects of general tablet PC-based English learning. In both studies the students in the experimental group learned with tablet PC-based materials while students in a control group learned with different materials (traditional paper-based materials in Oh et al. and with desktop PC-based materials in Lin). Both studies found a statistically greater improvement in reading performance among the tablet PC-based instruction group than for the desktop PC-based or traditional instruction group. Furthermore, the tablet PC-based instruction group showed greater appreciation of the reading program than their PC counterparts (Lin 2014) and the tablet PC-based instruction resulted in more learner autonomy than traditional instruction.

Two other studies employed tablet PCs for collaborative EFL learning. One was conducted by Lan et al. (2007) with elementary school EFL students. Two classes participated in the study: one class learned in a traditional EFL setting whereas the other class learned using a mobile-device-supported peer-assisted learning (MPAL) system installed on tablet PCs. The results show that students who learn with tablets attend more closely to the reading tasks and collaborate more than the students who learn without tablets. In another study, carried out by Shen and Chern (2014), students read and wrote digital stories. Both students and teachers reported positive attitudes towards using tablet PCs in a class for EFL learning, stating that tablet PCs enabled students to learn collaboratively, a process which enhanced their engagement. However, some issues associated with tablet PCs-based reading and writing have also been reported (e.g. stability of wireless environment and screen-sensitivity problems).

Several other studies have focused on EFL learning in informal settings outside of class. Chen (2013) investigated how students use tablet PCs as an EFL tool in informal learning situations outside of the classroom and also monitored students’ perception towards the technology. According to Chen (2013), tablet PCs are ideal tools for creating an interactive, collaborative, and ubiquitous EFL learning environment. Furthermore, students indicated high satisfaction, and had a positive attitude towards the usability and effectiveness of tablet PCs used as EFL tools. The authors of the present research explored the effects on learning achievement and cognitive load of learning outside of class with and without tablet PCs and found that students who learn with tablet PCs outperform those who learn without them on the post-test (Shadiev et al. in press). What is more, learning activities using the tablet learning system seem to cause less cognitive load for the students than when learning without technological support.

Dashtestani (2015) and Gabarre et al. (2014) attempted both to identify how students use tablet PCs for EFL learning and to gauge their attitudes towards using them. For this purpose, classroom observations, field notes, questionnaires and interviews were used. According to their results, most students had a positive perception of mobile learning and the use of tablet PCs for EFL learning. Students listed the benefits of mobile learning as including opportunities for ubiquitous learning and access to the Internet, use of multimedia in the classroom, and portability.

Our literature review shows that relatively few studies have been carried out which focus on EFL learning with the support of tablet PCs, and little research has been performed which examines enhancing cognitive processes in tablet PC-based language learning environments. Particularly, little is known about how to promote the cognitive processes of students from the simple to the complex in a technologically supported environment. Guided by the related research, we have designed learning activities and developed a learning system, Virtual Pen for Tablet PC (VPenTPC), which enables students to participate and utilize their new skills. Students learn and apply new knowledge by taking pictures of objects in a learning environment, describing them, and sharing their homework with peers. Assuming that this will enhance the high level cognitive processes of students, this study aims to test the effectiveness of learning activities which employ VPenTPC.

Learning strategies

Learning strategies are procedures that students use to succeed in tasks (Lee 2002). Lee (2002) and Oxford (1990) classify learning strategies into three categories: cognitive (to associate new information with existing information in long-term memory and to form and revise internal mental models), metacognitive (to plan, arrange, focus and evaluate the students’ own learning processes), and social (to interact with others and to manage discourse).

According to Oxford (1990), cognitive strategies include: (1) organization—grouping and classifying words, terminology, or concepts according to their semantic or syntactic attributes; (2) summarizing—intermittently synthesizing what one has heard to ensure the information has been retained; (3) imagery—using visual images to understand and remember new verbal information; (4) elaboration—linking ideas contained in new information, or integrating new ideas with known information; (5) rehearsal—repeating the names of items or objects to be remembered; (6) making inferences—using already learned information to guess the meaning of new linguistic items, predicting outcomes, or completing missing parts; (7) deduction—applying rules to the understanding of language; and (8) transference—using known linguistic information to facilitate a new learning task. Going beyond cognitive, Oxford (1990) also lists the following metacognitive strategies: (1) planning—organizing either written or spoken discourse; (2) monitoring—reviewing attention to a task, comprehending information that should be remembered, or producing it on the fly; (3) evaluating—checking comprehension after completing a language activity, or evaluating language output; (4) selective attention—focusing on special aspects of learning tasks. According to Oxford (1990), social strategies include: (1) cooperation—working with peers to solve a problem; (2) questioning for clarification—eliciting additional explanation from a teacher or peer; and (3) self-talk—using mental redirection of thinking to assure oneself that a learning activity will be successful.

Following the general recommendations of Lee (2002) and Oxford (1990) and drawing upon previous related studies, we have developed a questionnaire to explore the learning strategies students use and how frequently they use them.

Several issues are identified in the literature review. For example, little attention has been paid to enhancing students’ cognitive processes from the simple to the complex during EFL learning in tablet PC-based learning environments. Informed by the related literature, we have designed the present study, which attempts to address certain identified issues. First, based on the storytelling instructional method, we designed learning activities which focus on both knowledge acquisition and its practical application and analysis. Students learn in class first, and then apply and analyze new knowledge in an authentic learning environment with familiar context. Second, we develop and employ a learning system, Virtual Pen for Tablet PC (VPenTPC), which enables students to participate in the required learning activities. Students learn and apply new knowledge by taking pictures of objects found in an authentic learning environment, describe them, and then share their homework with peers. Aiming to enhance the high level cognitive processes of students, this study tests the effectiveness of those learning activities which VPenTPC supports. We also investigate changes in the cognitive processes of students during learning activities and attempt to determine precisely which strategy is being used as part of the of cognitive processes occurring during different stages of learning. Furthermore, this study explores the learning strategies students use and the frequency of that use. Finally, this study evaluates students’ acceptance of technological support in assisting learning. The following research questions are addressed in this study:

1. Do students who participate in learning activities supported by annotation, recording, assistance, and the sharing functions of VPenTPC perform better in cognitive processes than those who study without such support?

2. Do changes occur in the cognitive processes of students between the first and last lessons?

3. What learning strategies do students use, how frequently, and what is the relationship between learning strategies usage and learning performance?

4. What is the distribution of students who reach the highest level of cognition during different learning periods?

5. What are the perceptions and behavioral intentions of students towards VPenTPC based on the questionnaire and interview surveys?

Methods

Participants

Two classes with a total of 58 junior high school students participated in this study (Table 1). All students are non-native English speakers studying English as a foreign language. Most students were thirteen years old, with four to six years’ experience using computers. Additionally, most students had had no more than three years’ experience using tablet PCs. One class with 30 students served as the control group, and the other class, with 28 students, served as the experimental group. Learning content, tasks, and objectives were exactly the same for both groups. The only difference was that students in the control group studied learning content and completed tasks using a textbook, pen, paper and digital camera while the experimental group used a VPenTPC.

Table 1 Participants’ profile
Full size table

Experimental procedure

Figure 1 shows the procedure used in this study. A pre-test was conducted in the first class. Both groups received the same number of hours of English instruction: three one-hour lessons a week. The curricula in the two classes were presented by the same instructor. After each class, students participated in learning activities. The control group completed the activities using traditional textbooks while the experimental group used VPenTPC. The learning activities were comprised of three tasks, each of which lasted for two weeks. The students’ cognitive processes were measured for each learning task. In the first class, every student in the experimental group was given a tablet PC and taught how to use it. The experimental group was provided with immediate assistance in troubleshooting technical problems during the experiment to reduce any negative effects the unfamiliar technology might cause. Students were asked to fill in a questionnaire about the learning strategies that they used and return it after completing each task. During the last class a post-test was given to all students; the experimental group had a supplementary perceptions questionnaire and interviews.

Fig. 1
figure 1

Experimental procedure

Full size image

Learning activities design

In this study students were presented with specially designed learning activities that helped them (1) to learn the basic curriculum and (2) to apply and analyze it. The learning activities were based on the storytelling instructional method (Wang et al. 2009); students were asked to tell stories through introducing, describing and explaining objects found in an authentic environment with familiar context. Three topics, with related grammar, from the textbook, such as (1) “Which do you like – Healthy diet,” (2) “How much/many do we need,” and (3) “We were in different classes,” were covered in the learning activities. The learning activities included three tasks, and each corresponded to one topic.

Task 1

My meal and food critic In this task, each student is required to take a photo of his/her daily three meals (i.e. breakfast, lunch and dinner) and introduce them. After this, each student must act as a food critic and comment on the meals of his/her partner. The learning objective of this lesson is to learn how to express personal preferences for food and other things.

Task 2

Make my own salad! In this lesson, students are supposed to learn how to say countable and uncountable nouns used in preparing a dish. Each student is asked to help their parents with food shopping, focusing on the ingredients which the students would later include in a salad or other dish, each of which they must photograph. Students then prepare the dish, using the newly purchased items, after which they take a photo of the dish and explain their recipe and cooking steps. Finally, each student is asked to write/tell how the dish is different from his/her partner’s.

Task 3

Do you often clean your room? In this task, students take a photo of their room before and after cleaning it. Students then describe how the room is different before and after being cleaned. Students are also asked to review their partner’s photos and give comments on their cleaning job. Students can learn how to use past-tense BE verbs to describe things.

After each task, the instructor monitors the learning progress of the students to ensure that they have completed the tasks, reviewed their partners’ work, and left comments for each other. According to the instructor, all students completed everything. The VPenTPC recorded all the students’ annotations, but only the instructor could make sure the content met the class criteria.

Virtual pen for tablet PC

The researchers developed this learning system to help students carry out their learning tasks. The VPenTPC interface is shown in Fig. 2. The VPenTPC features the following four main functions:

Fig. 2
figure 2

VPenTPC interface

Full size image

1. Annotating

Students can annotate important parts of learning materials on their tablet PCs. For example, students can write about an object by creating a textual annotation. In addition, students can take photos of objects and attach photos to an annotation.

2. Recording

When students orally give a description of an object, they can record their own voice and play it afterwards. Students can also record the instructor and listen to the recording after class.

3. Assistance

Assistance, such as Dictionary, is provided by VPenTPC. Students can find translations and the meanings of new words.

4. Sharing

Students can share their own annotations, photos and audio files with peers. Such an approach allows students to review their peers’ annotations.

Measures

Pre-test, post-test and learning tasks scores

We evaluated students’ level of cognition before and after the experiment using a pre- and post-test. Each test contained thirty items. The two tests’ items were created by an experienced junior high school teacher, based on the learning material and activities of this study. The tests are provided in Table 2. The items for both tests are similar in structure but different in content. This study adopts Anderson and Krathwohl’s (2001) taxonomy to measure the cognitive levels of students, focusing on the first four levels, i.e. “Remember,” “Understand,” “Apply,” and “Analyze.” Items 1–8 of the test measure the “Remember1” level whereas items 9–14 measure the “Remember2” level. Items 15–24 measure the “Understand1” level while items 25–29 measure the “Understand2” level. Item 30 measures students’ “Analyze” level.

Table 2 Test items examples
Full size table

We divided students into two groups based on their pre-test scores: a lower ability (the last eleven participants of the rank) and higher ability (the first eleven participants of the rank) groups. The Mann–Whitney Test was used to determine that the ability of students in the lower ability group (M = 17.45, SD = 5.05) was lower than that of students in the higher ability group (M = 41.64, SD = 12.44), U = 0.000, Z = −3.983, p = 0.000). The small class sizes prevented us from creating three groups; therefore the bottom students in the high ability group and the top students in the low ability group could potentially have close scores. In our study, only two students from each group had relatively close scores (i.e. 24 vs 22). Nevertheless, we acknowledge such categorization as a limitation of our study.

Students’ scores on each learning task were also measured. For this, we collected students’ completed tasks and coded and scored them. Scores represent the cognitive processes of students.

Actual usage of VPenTPC functions

A VPenTPC automatically retains all of the students’ inputs and requests in its database. A screenshot with details of the actual usage of the VPenTPC functions is presented in Fig. 3. The first column in the figure shows student IDs, the second column when students updated their annotations for the last time, the third and fourth columns how many textual or audio annotations students created on top of text content in the textbook (“Text” inside of square brackets), the fifth and sixth columns how many textual or audio annotations students created on top of image content in the textbook (“Image” inside of square brackets), the sevenths column the number of annotations with photos, the eighth column how many lectures the students recorded, and the last column how many times students listened to the lecture. Please note that we did not focus on “Parents recording” and the “TTS” functions in this study, and therefore do not discuss them in this paper.

Fig. 3
figure 3

Actual usage of VPenTPC functions

Full size image

Learning strategies usage questionnaire

In order to explore what learning strategies students use and how frequently, we developed a questionnaire following the general recommendations of Lee (2002) and Oxford (1990) and drew upon previous related studies (Hung et al. 2015b; Golonka et al. 2014; Kim and Kim 2012; Lan et al. 2007; Oh et al. 2014; Shadiev et al. in press). Four cognitive strategies were targeted in our questionnaire: (1) organization, (2) summarizing, (3) imagery, and (4) elaboration, as well as three metacognitive strategies: (1) planning, (2) monitoring, and (3) evaluating. We did not include (5) rehearsal, (6) making inferences, (7) deducing, and (8) transfer cognitive strategies, selective attention metacognitive strategy or any of the social strategies in the questionnaire as the learning tasks did not require these operations. The following are a few examples related to using cognitive and metacognitive strategies from previous related studies: imagery and summarizing—students took pictures of objects and described them (Golonka et al. 2014; Shadiev et al. in press); organization—students use the electronic dictionary to complete tasks (Hung et al. 2015b; Kim and Kim 2012; Oh et al. 2014); monitoring and evaluation—students share content with peers for further reflection and to assist each other in completing tasks (Hung et al. 2015b; Lan et al. 2007); elaboration—students improve their created content based on peer feedback (Golonka et al. 2014; Shadiev et al. in press). Using a dictionary is evidence of an organization strategy as students look up unfamiliar vocabulary to complete tasks. Sharing content with peers indicates monitoring and evaluation strategies because students review peers’ content in order to monitor, evaluate, and compare it with their own content and learning progress. A questionnaire was distributed to students before each task and students were asked to use it to record each time they used a particular strategy. Students returned the completed questionnaires right after they finished a task.

Perception of the VPenTPC questionnaire

Another questionnaire survey evaluated the experimental group’s perception of VPenTPC. The design of the questionnaire was informed by previous related studies (Kuo et al. 2012)and includes twenty-one items in four dimensions: (1) Perceived ease of VPenTPC use—the degree to which a student believes that using VPenTPC is free of physical and mental effort; (2) Perceived usefulness of VPenTPC—the degree to which a student believes that using VPenTPC for learning enhances his or her learning; (3) Perceived satisfaction—the degree to which a student is satisfied with VPenTPC for learning purpose; (4) Behavioral intention of using VPenTPC—a major determinant of whether a student would continue to use VPenTPC in the future or not.

All items of the questionnaires (i.e. learning strategies usage and perception of the VPenTPC) were reviewed by an experienced junior high school teacher and two experts in the field of mobile assisted language learning to ensure content validity. These reviewers provided some comments which we used to revise questionnaire items.

Interviews

One-on-one semi-structured interviews were conducted with ten randomly selected students. Interviews explore students’ learning experiences and provide insight into their perceptions regarding the usefulness of VPenTPC for learning. Each interview lasted for 20 min, and students were asked “Was VPenTPC useful for learning? If yes, please explain why,” as well as to “Please describe your learning experience”.

Since we aimed to explore students’ experiences learning with technology, including their learning strategies and perceptions, only students from the experimental-group were asked to complete the questionnaire and participate in the interviews.

Data analysis

The following data analysis was carried out:

(1) We evaluated students’ pre-test and post-test results. Each correct answer to items 1–29 is scored as “1,” while each incorrect answer receives a “0.” Item 30 is an open ended question. Students’ answers to item 30 were coded by three raters using a sentence as a coding unit and scored on a 29-point scale (with 29 as the highest score). A sentence that shows a student’s ability to grasp and to interpret the meaning of material represents “Understand” level. It was scored from 0 to 9 based on the amount of material that the student grasped and interpreted. A student answer showing the ability to use learned material in new and concrete situations indicates the “Apply” level. It was scored from 10 to 19 based on the amount of material that the student used in new and concrete situations. The “Analyze” level was represented by an answer that shows the ability to break material into its constituent parts and determine how the parts relate or interrelate to one another and to an overall structure. It was scored from 20 to 29. We employed analysis of covariance to test the difference in the level of cognitive processes between the control and experimental groups on the post-test, using the pre-test as covariate (Sect. 6.1).

(2) We evaluated students’ cognitive processes when engaged in the learning tasks. For this, we collected students’ completed tasks and then coded and scored them in the same way as with their answers to item 30 of the pre-test and post-test. The inter-rater reliability of the assessments (the pre-test, post-tests, and cognitive processes on learning tasks) was ensured by resolving notable differences in the coding through discussion among the raters. Inter-rater reliability of the assessment was evaluated using Intra-class correlation coefficient (ICC). The average ICC measure was more than 0.900, indicating high reliability. We employed one‐way repeated measures ANOVA with a Greenhouse-Geisser correction to examine the cognitive development progress of the experimental group from tasks 1 to 3 (Sect. 6.2);

(3) We counted the number of times each strategy was used to explore what learning strategies students used and how frequently (Sect. 6.3). To ensure the reliability of this instrument, the researchers crosschecked each student’s answers with the questionnaire and their recorded actual usage of the VPenTPC. That is, the researchers checked whether a particular action which corresponds to a learning strategy that a student indicated in the questionnaire had actually been recorded by the VPenTPC, e.g. the learning behavior associated with describing an object in textual annotation using VPenTPC corresponds to the summarizing strategy used when writing about the object.

(4) We employed a Pearson product-moment correlation coefficient to investigate the relationship between the learning strategies usage and the post-test scores of all students as well as the post-test scores of students with different learning abilities (Sect. 6.3).

(5) We used an independent samples t test to determine the difference between lower and higher ability students with respect to their learning gain; learning gain here refers to the difference between the pre and post-test results (Sect. 6.3).

(6) We examined the distribution of the control and experimental groups’ highest levels of cognitive development in the pre-test and post-test to evaluate the Cognitive Diffusion Model (Sect. 6.4).

(7) We explored perceptions of students towards VPenTPC based upon their responses to the questionnaire (Sect. 6.5). Twenty-eight valid answer sheets were obtained from twenty-eight students. Students’ responses to the items were scored using a five-point Likert scale, anchored by the end-points “strongly disagree” (1) and “strongly agree” (5). Cronbach α was employed to assess the internal consistency of the survey. The values exceeded 0.80 in all dimensions, demonstrating the reliability of the items.

(8) We explored students’ learning experiences to gain insights into their perceptions of the usefulness of VPenTPC for learning based upon the interview data (Sect. 6.1, Sect. 6.3, and Sect. 6.5). All interviews were audio-recorded with the students’ permission and then fully transcribed for analysis. The text segments that met the criteria of providing the best research information were highlighted and coded according to an open coding method of analysis described by Strauss and Corbin (1998). The codes were then sorted into categories; codes with similar meanings were aggregated. Established categories formed a framework to report findings pertinent to the research questions. Table 3 presents derived categories, codes and code definitions. The inter-rater reliability of interview data was also evaluated using Cohen’s kappa. The result exceeded 0.90, indicating its high reliability. A Statistical Package for the Social Sciences (SPSS Inc.) program was employed for statistical analyses in this study.

Table 3 Interviews data coding
Full size table

Results and discussion

Do students who participate in learning activities supported by annotation, recording, assistance and the sharing functions of VPenTPC perform better in cognitive processes than those without such support?

The means and standard deviations of the students’ pretest and post-test scores are shown in Table 4. There is no significant difference between the control and experimental groups on the post-test for “Remember 1,” F(1, 55) = 1.504, p = 0. 225, partial eta-squared = 0.027, or “Understand 1,” F(1, 55) = 1.588, p = 0.213, partial eta-squared = 0.028. However, the experimental group outperforms the control group on the post-test items related to “Remember 2”, F(1, 55) = 7.075, p = 0.010, partial eta-squared = 0.114, “Understand 2,” F(1, 55) = 8.876, p = 0.004, partial eta-squared = 0.139, and “Analyze,” F(1, 55) = 11.173, p = 0.001, partial eta-squared = 0.169. Partial eta-squared values show that the effect size is medium for “Remember 2” and large for “Understand 2” and “Analyze;” that is 11.4 % (Remember 2), 13.9 % (Understand 2), and 16.9 % (Analyze) of the variance can be accounted for by differences among the groups.

Table 4 Results of the pre-test and post-test and analysis of covariance
Full size table

This finding is interesting since there are both significant and insignificant differences in student scores across groups for various items on the post-test. The following is our explanation of these results. Test items related to the “Remember” and “Understand” cognitive process levels have been organized into two difficulty levels. For “Remember 1” items, students were asked to match the English word with the correct Chinese meaning, whereas for “Remember 2” items, students wrote down the Chinese meaning of each English word. Similarly, for “Understand 1” items, students filled in the blank, whereas for “Understand 2” items, they needed to write down sentences that (1) form a question based on a sentence, (2) change a positive statement into a negative one and (3) translate a sentence from Chinese into English. Students in both groups obtained high scores in “Remember 1” and “Understand 1,” probably because the items were very easy to complete and most students already had some prior knowledge of the material. However, as the difficulty level of the items increased, the control group’s performance decreased while the experimental group’s performance did not, suggesting that learning activities supported by VPenTPC significantly promoted the cognitive processes of the experimental group. The experimental group showed better learning, recall, and understanding of the new vocabulary and sentence structures, and were better able to change sentences into questions and negative forms.

In the items related to the “Analyze” cognitive process level, students were asked to write and compare sentences about themselves when they were in the first grade of elementary school and at the present time. The experimental group performed this task significantly better than the control group, suggesting that learning activities supported by VPenTPC significantly facilitated students’ gaining and understanding knowledge as well as their application and analysis of it.

The following may be some of the reasons behind these findings, as revealed in the interviews with students. First, using functions of VPenTPC such as annotating and recording, students were able to take pictures of objects and then either record their own voice or write descriptions of them. Students showed a tendency to review pictures, to listen to their own recorded files, or read their notes afterwards. If the quality of the content was unsatisfactory (e.g. incorrect grammar or pronunciation), students wanted to make improvements. Similar reasons for using multimedia tools for language practice have been reported elsewhere. For example, Golonka et al. (2014) and Oh et al. (2014) suggest that EFL students take advantage of technology in their studies in the same way to practice the target language repeatedly and regularly. Harmer (2007) reports that students benefit from using technology by listening to their recorded audio files and evaluating their own language performance. Students in the control group could also take pictures and describe them by recording their own voice or by writing notes. Using the VPenTPC, students can attach annotations (photos, recorded files and written notes) to related parts of the learning material and then present the material on the same screen (Fig. 2). That is, students can see how their annotation and the learning materials are connected, giving them a clearer picture of the whole learning scenario with an appropriate explanation of it. Later, when students want to review their tasks, they can find important concepts and related annotations easily. In contrast, students in the control group did not have their annotations and the learning material organized in this way. Therefore, it was not easy for them to find their annotations, nor could they build connections among them and the learning material in the same, seamless manner.

Second, the VPenTPC sharing function enables students to share their recorded files with peers and listen to the recorded files of peers who perform well to get inspirational ideas or study how peers accomplish assignments in order to complete or improve their own assignments. Through sharing, students can also exchange meaningful comments with each other. In our experiment, some students gave reflective comments and suggestions to peers who did not complete the homework correctly. Such comments can be useful for improving homework. Students thought highly of the sharing mechanism of VPenTPC as they were able to learn from others as well as locate and revise their own homework mistakes. Golonka et al. (2014) and Lan et al. (2007) argue that through multimedia aids students can access more diverse objects in their learning environment, thus increasing the richness of their language experience. Golonka et al. (2014) further suggests that sharing may engage students in EFL contexts and allow reflection and student discussions about shared learning material. Although, students in the control group could also share their photos and oral or written notes, sharing using VPenTPC was more efficient, partially because VPenTPC allows students to review annotations of an unlimited number of peers, something not possible using a traditional approach. Another reason is that VPenTPC enables students to view peers’ annotations (photos and their audio or written descriptions) and see how they are anchored to the learning material. That is, students are able to see the whole learning scenario and the relationship among annotations and the learning material. Therefore, using VPenTPC helps students to review shared annotations of an unlimited number of peers and understand them better.

Third, VPenTPC allows students to record lectures of the instructor using its recording function. According to interviews, playing recorded lectures on a Tablet PC helps students to recall them or rehear the instructor’s pronunciation of the learning material. This was particularly useful outside the classroom where students could not consult their instructor and ask questions. Students in the control group could also record lectures. In contrast to traditional approaches, VPenTPC enabled students in the experimental group to attach recorded lectures to the related learning material. Later, experimental-group students could easily find the learning materials and attach them to the recorded lecture.

Finally, students report that the VPenTPC’s function of assistance is very handy. In particular, the built-in dictionary helps students when they are outside of school and need to translate unfamiliar vocabulary to complete assignments. Moreover, when using the dictionary feature, students can find multiple meanings of a word and see how it can be used in different contexts. Hulstijn et al. (1996) argue that the use of a dictionary positively affects vocabulary learning. Students look up target words in a dictionary during the reading session in order to find word meanings and to understand the main idea of texts. According to Hulstijn et al. (1996), students who consult a dictionary when reading challenging texts tend to understand them better and remember more vocabulary. Of course students in the control group could also use a dictionary. However, some students find dictionaries too cumbersome to carry with them everywhere, or forgot to bring them outside of school to complete tasks. These are not issues when the dictionary is embedded into the VPenTPC, making it always available when students want to learn using tablet PCs outside of school.

According to students, all of these learning factors lead to more frequent EFL language practice as well as to better language output. In addition, students claim that when they learn with tablet PCs, they can communicate in the target language with less anxiety about making mistakes (Golonka et al. 2014; Lan et al. (2007). Therefore, students mentioned that, compared to traditional approaches, the learning activities could be completed more efficiently and they had more opportunities for language practice when using VPenTPC.

These findings about the benefits of the learning activities and multimedia supported learning are in line with other studies (Harmer 2007; Hung et al. 2015b; Kim and Kim 2012; Lan et al. 2007; Oh et al. 2014). However, in contrast to other related research, we designed the learning activities specifically to be supported by VPenTPC and have focused on enhancing the cognitive processes of students, particularly when applying and analyzing new knowledge to solve daily life problems in an authentic environment.

In Table 5 we report recorded actual usage of various functions of VPenTPC by experimental-group students in order to add rigor to our reported findings. The table shows that the function which experimental students used the most was text annotations (552). Recorded audio is second (259), taking photos is third (294), listening to recorded lectures is fourth (89) and recording lectures is the last (34) in students’ list of preferences of VPenTPC usage. Unfortunately, VPenTPC was not designed to record the number of students’ reviews of their own or peers’ text annotations, listening to their own or peers’ recorded audio, or re-writing and re-recording content and re-taking photos. Therefore, the data related to these learning behaviors was not available for analysis. This limitation will be addressed in a future study.

Table 5 Usage of the system features by experimental students
Full size table

Do changes occur in the cognitive processes of students between the first and last lessons?

According to the statistical results, the mean scores for the cognitive processes of students on task 1 (M = 19.07, SD = 7.57), task 2 (M = 20.64, SD = 6.86), and task 3 (M = 21.71, SD = 7.09) differed significantly: F(1.309, 35.355) = 21.366, p = 0.000. Post hoc tests using the Bonferroni correction reveal that the students’ scores significantly increased from task 1 to task 2, p = 0.005, and then from task 2 to task 3, p = 0.000, strongly suggesting that learning activities supported by VPenTPC enhance the cognitive processes of students. Kuo et al. (2012) suggest designing technology-based instruction in a way that guides and encourages students to use educational technology more regularly. This enables users to identify the strengths and limitations of the technology and then to fully utilize it for their learning. The results of this study show that students quickly got used to VPenTPC for the learning activities from task 1 to task 3, and their cognitive processes consistently improved.

What learning strategies do students use, how frequently, and what is the relationship between learning strategies usage and learning performance?

Table 6 shows the Learning strategies usage questionnaire results: the different strategies that students used in this study (Learning strategy column), how many times students used each strategy (the numbers in the Lesson1, Lesson2, and Lesson3 columns), top ten most frequently used strategies (the numbers with asterisks in the Lesson1, Lesson2, and Lesson3 columns), and the rank of the top ten frequently used strategies (the numbers in parentheses). According to Table 6, students employed thirty different learning strategies when using technology for learning. The most frequently used cognitive strategies include: (1) taking a photo of objects (imagery), (2) writing and (3) speaking about objects (summarizing), (4) using the included electronic dictionary (organization), and (5) improving and re-writing their information about the objects (elaboration). That is, students took photos of objects in an authentic learning environment (Strategy 1) and they wrote (Strategy 2) and spoke (Strategy 3) about them while using a dictionary (Strategy 4). Finally, students improved their homework (Strategy 6). The top metacognitive strategies include: (1) preparing and listening to audio documents (planning), (2) reading their own and (3) their partner’s writings, (4) reviewing their partner’s photos, and (5) listening to their partner’s audio files (monitoring). That is, students prepared and listened to their recordings about objects (Strategy 9). Students then often read their own (Strategy 10) and partners’ (Strategy 11) writings about objects, listened to partners’ recorded audios (Strategy 16), and reviewed partners’ photos (Strategy 13). All these strategies were consistently and frequently used in all three lessons. However, the number of strategies used decreased in Lesson 3. This may be because the size of the project was reduced because it was the end of the semester and students were busy preparing for final exams. With this in mind, the instructor only required the students to complete one part of the task (i.e. describe their rooms). As a result, students used fewer strategies in the third task. Based on these results, this study suggests that the abovementioned strategies are useful for learning. While cognitive strategies help students to complete their homework, metacognitive strategies assist them in improving the content of their homework. The results suggest that metacognitive strategies at the evaluation level are less likely to be used by students. In the interviews, we found that not all students knew the various strategies. Therefore, we suggest that the instructors teach students a variety of these strategies, emphasize their importance, and encourage students to use them frequently.

Table 6 Learning strategies
Full size table

In this section, we will explore the relationship between learning strategies usage and the post-test scores. According to the results, there is no significant correlation between the two variables, r = 0.155, p = 0.480. However, a significant correlation exists when considering students of different learning ability; strategies used by students of low ability are significantly correlated with the post-test scores, r = 0.758, p = 0.007. We also compared the difference in learning gain between low and high ability students, and found that the learning gain of low ability students (M = 19.64, SD = 10.68) is significantly higher than that of high ability students (M = 1.45, SD = 10.33), t = 4.059, p = 0.001, suggesting that lower ability participants take better advantage of learning strategies while engaged in learning. In the interviews, some lower ability students stated that they prefer to write and speak aloud about objects (Strategies 3 and 4) first, and then read peers’ writings and listen to peers’ recorded files (Strategies 11 and 16) as they feel this helps them to improve their own homework (Strategies 7 and 9). Pearson’s results support this finding; a significant correlation was found between strategy-usage and the post-test scores of low ability students. Therefore, the researchers conclude that the deeper engagement of low ability students in using learning strategies leads to a significant learning gain. We also suggest that students be taught learning strategies and how to make better use of them.

What is the distribution of students who reach the highest level of cognition during different learning periods?

Figure 4 shows the distribution, i.e. the number of students across both groups, that reached “Analyze” as their highest level of cognition, which is derived from the results of measuring the last item (i.e. item 30) of the pre and post-test. To highlight the distribution more precisely, the distribution axis is divided into 5 interval units based on the post-test scores: (1) 0 (no Analyze level); (2) 1–9 (lowest Analyze level); (3) 10–19 (low medium Analyze level); (4) 20–24 (high medium Analyze level); and (5) 25–29 (highest Analyze level). According to the figure, the distribution changes based upon different learning period (i.e. pre-learning and post-learning) and instructional approach (i.e. learning activities supported with VPenTPC vs without technological support). The distribution of the highest cognitive level is low during the pre-learning period, but for both approaches becomes higher after a period of learning. After learning, the distribution of the highest cognitive level is lower in the traditional learning environment than in that supported by VPenTPC. All students in the experimental group (100 %) reached the lowest Analyze level. By Contrast, only half of the students in the control group (53 %) reached this level. Our findings suggest that learning activities supported by VPenTPC are beneficial to the experimental group, and so increase the cognitive processes of students, particularly in the higher levels. According to the Cognitive Diffusion Model (Hwang et al. 2014; Hwang and Shadiev 2014), cognitive processes of students are differently distributed into six levels, and the distribution is differently shaped during different phases of learning. The model suggests that while the cognitive processes of students are on the lowest level in the pre-learning period and on a higher level after the learning period, they are at their best after a learning period in which the learning process is supported by technology.

Fig. 4
figure 4

Cognitive processes distribution

Full size image

This study has attempted to test the feasibility of the Cognitive Diffusion Model and to promote student cognitive processes. Students were encouraged to use English in their daily lives (e.g. to apply new knowledge when they visit a supermarket or local convenience store). The results show that the experimental group’s cognitive processes raised through learning activities supported by VPenTPC.

The researchers designed learning activities with authentic support to facilitate cognitive processes of students at the Analyze level. However, according to the results, less than 30 % of the students in the control and experimental groups reached this level. In the interview, the instructor mentioned that the national English curriculum focuses mainly on simple language forms, sentence patterns, and basic daily conversation; these skills are already hard enough to master for average students. According to the instructor, only a few students, who might have had a more privileged English learning environment since childhood are likely to have such skills and be able to analyze new knowledge. This is an issue that needs to be considered in future related studies.

What are the perceptions and behavioral intentions of students towards VPenTPC?

The Perception of students towards VPenTPC as indicated in their questionnaires is distinctly positive; most items were ranked high: perceived ease of VPenTPC’s use (M = 4.24, SD = 0.8), perceived usefulness of VPenTPC (M = 3.91, SD = 0.78), perceived satisfaction (M = 4.04, SD = 0.78), and behavioral intention of using VPenTPC (M = 3.81, SD = 0.61). This data shows that, in general, students felt that it was easy to use VPenTPC and that it was useful for learning. In addition, most students indicate both that they were satisfied with VPenTPC and that they were highly motivated to use it for learning in the future.

In interviews, students agreed that VPenTPC was easy to use. Students also stated that VPenTPC made learning more interesting and fun. Caldwell (1998) suggests that multimedia aids can engage students in learning, stimulate their imagination and lead them to meaningful output while simultaneously decreasing anxiety. In this study, interviews with students show that most of them agreed that creating and using multimedia content made learning more interactive, information rich, and engaging.

In the interviews, both students and the teacher approved of VPenTPC’s usefulness for learning. The teacher said that VPenTPC was useful for monitoring students’ learning progress. VPenTPC enables the teacher to easily find out who did not complete homework, enabling the teacher to provide assistance to those students. Since students know that their learning progress is monitored, they are more likely to complete assignments correctly. Some students indicated that feedback received on homework was useful. This finding is in line with other related studies. For example, Wang et al. (2009) claim that the monitoring features of the mobile learning system are useful for ensuring student accountability. Monitoring features also enable the instructor to easily keep track of students’ learning without too much delay, thus facilitating the instructor’s ability to supervise students’ learning activities and provide appropriate guidance.

However, in the interviews students mentioned two issues with VPenTPC. First, the tablet PC is too big to carry around and they prefer to use mobile phones for these activities. This issue needs to be considered in the future by teachers and researchers planning to use this type of system. Second, the instructor mentioned that not all students could complete their homework efficiently at the beginning of the experiment as they were not technologically proficient. In this study, one week of tablet PC orientation was carried out at the beginning of the experiment, as in most previous related studies (Hung et al. 2015b; 2014; Kim and Kim 2012). Perhaps, more than a week is needed for some students to get acquainted with the technology. Furthermore, learning activities need to be designed in a way that guide and encourage students to use the technology more regularly, enabling students to identify its strengths and limitations and then fully utilize it for learning.

Conclusion

This study has four main findings. First, although there was no significant difference between the control and experimental groups on the post-test items related to “Remember 1” and “Understand 1” levels, the experimental group significantly outperformed the control group on items related to the “Remember 2,” “Understand 2,” and “Analysis” levels, suggesting that learning activities supported by VPenTPC significantly facilitate students’ gaining and understanding knowledge as well as their application and analysis of it. Second, students made significant improvement in cognitive processes from Lessons 1 to 3, suggesting that learning activities supported by VPenTPC enhance the cognitive processes of students after they get acquainted with VPenTPC and learn to fully employ it during learning. Third, thirty learning strategies were used by the students, from which we distinguished the top ten most frequently used cognitive and metacognitive strategies which are important for learning. Fourth, the learning gain of lower ability students is significantly higher than that of higher ability students, suggesting that students in the lower ability experimental group made better use of learning strategies. Finally, most students had a positive view of VPenTPC and said they were highly motivated to use it for learning in the future, suggesting acceptance of the technology by most students.

Based on these results, we have several recommendations for teaching and research in this field. First, we suggest designing and implementing appropriate learning activities supported by VPenTPC in order to help students learn in class and then apply and analyze new knowledge in a wide range of daily life situations. Particularly, we emphasize the importance of creating and sharing multimedia learning content as well as reviewing peers’ content to enhance high level cognitive processes. Our approach is practical and feasible as students are able to learn and participate in language learning activities using VPenTPC. VPenTPC includes all of the content of a traditional textbook while also providing various multimedia tools to make language learning more interesting and efficient. However, our approach requires that students have tablet PCs with the VPenTPC installed.

We also suggest that educators give students sufficient time to get acquainted with the technology and encourage them to use it regularly. This will enable students to identify the strengths and limitations of the technology and fully utilize its learning potential.

The instructors need to emphasize the most frequently used learning strategies in order to facilitate student language learning. In addition, lower ability students should be encouraged to participate in learning activities supported by VPenTPC, as they benefit from them the most.

It is further suggested to extend the proposed novel approach and apply it to other domains. That is, in addition to the domain of foreign language learning, our approach can be applied to domains such as biology or mathematics. Within the extended approach, students may acquire conceptual knowledge in class and then apply and analyze it outside of school.

Finally, this study suggests that the technology can help instructors to become aware of the learning behavior and progress of students. The educators then may encourage passive students to be more active, teach useful strategies to students who perform poorly, and scaffold the language learning of students when necessary.

Several limitations of the present study have already been mentioned. They are: (1) we created two groups of different language abilities whose scores were close due to the small number of participants, (2) reaching “Analyze” level by most students, (3) VPenTPC could not record some important learning behavior; and (4) limited duration of tablet PC orientation. All of these shortcomings will be addressed in a future study. For example, we will involve more students in the future in order to have groups of different language abilities whose scores are not close. To deal with the second limitation, we will introduce additional scaffolding mechanisms into the learning activities (e.g. students of higher ability may help students with lower ability during learning) or focus on students from upper grades. In addition, we will improve the VPenTPC learning system and add additional functions, e.g. recording the number of students’ reviews of their own or peers’ text annotations, listening to their own or peers’ recorded audio, or re-writing and re-recording content and re-taking photos. Regarding the fourth limitation, we will extend tablet PC orientation from one week to a few weeks.

In the future study, we will also plan to extend our learning activities from individual to collaborative. In this study, students acquired knowledge in class and then applied and analyzed it in authentic learning environment using technology. Many students cannot perform high level cognitive processes due to their age and the absence of required skills. Therefore, in the future we may group high and low achievement students together so that expert students with sufficient prior knowledge and skills can guide lower achieving students through the learning activities. Additionally, the instructor may design collaborative tasks that require contribution from all students. In this way the cognitive processes of all students can be raised to higher cognitive levels.