1 Introduction

The positive impact of feedback on student learning has been well evidenced in the last decades (Xu & Carless, 2017). Learners use feedback to deepen their understanding of the problem on the one hand and to optimize their learning style on the other (Carless, 2015; Carless & Boud, 2018). Currently, teachers and peers have become the primary providers of feedback, and the feedback they provide can continuously help students improve the quality of their learning (Carless & Boud, 2018; Li et al., 2020). With the massive expansion of higher education and increased class size, it will become increasingly difficult to implement effective feedback activities in the classroom (Evans, 2013). For example, in Chinese higher education, many university English classrooms have large class sizes, with more than 50 students in each class, and each teacher teaches two or three classes (Jin & Cortazzi, 1998).

In this case, it is difficult for teachers to give timely feedback to all students in the classroom about their English writing. Without timely feedback on problems in writing, it will be difficult for students to improve their writing skills and motivation to write (Gibbs & Simpson, 2004). One way to solve this problem is to shift from teacher feedback to peer feedback (Chang, 2015). As a result, a growing number of higher education researchers have focused on applying peer feedback to teaching in different subject areas, including science, mathematics, English, and English as a second/foreign language (ESL/EFL), and they often compare and contrast the differences and effectiveness between peer feedback and teacher feedback (Wu et al., 2022).

Research results about ESL/EFL show that teacher and peer feedback play an important role in students’ English writing process (Yu & Hu, 2017). On the one hand, students find teacher feedback helpful in improving their writing, but teacher feedback is a form of corrective feedback based on scores (S-TCF). Apparently, it is difficult for students to know the problems in writing by using scores as feedback information, and this feedback also has a delayed nature (Yu et al., 2020). On the other hand, by using peer feedback, students believe that the diversity of feedback can be increased and that both the giver and receiver will have a good learning opportunity and valuable feedback information (Storch, 2005). However, in actual classroom learning, it may not be easy to find an ideal learning partner to provide effective feedback. To solve this problem, an automated writing evaluation system (AWE) plays a dual value and potential as an application of artificial intelligence.

Automated Writing Evaluation (AWE) involves computer-generated scoring and feedback on writing. The core component of AWE is a scoring engine that generates automatic scores based on techniques such as artificial intelligence, natural language processing, and latent semantic analysis. Many of today’s AWE systems also include feedback on writing. For example, the Criterion ® system and the My Access! System (Grimes & Warschauer, 2010). AWE feedback can evaluate aspects such as word or sentence syntax to help students gain accuracy in their writing. It has been shown that the AWE system has significant advantages over human reviewers: its ability to provide immediate commentary on student writing, multiple opportunities for students to revise, and the usability of holistic and scored reports (Zhang, 2020). The AWE system can be of practical value as an ideal learning partner. However, there are also very controversial issues regarding the AWE system. For example, some researchers have argued that AWE feedback tends to make students focus more on surface features of writing (e.g., grammar and spelling) than on content ideas (Cheville, 2004). Such corrective feedback hardly reflects problems with writing patterns, context, and thinking about writing (Vojak et al., 2011). There are also criticisms that the AWE system tends to make students form a habit of formulaic writing (Stevenson & Phakiti, 2014). Although AWE has a timely corrective feedback feature, it is the teacher who instructs students on how to use the feedback. To help students use the AWE corrective feedback in a meaningful way, the teacher’s instructional strategies are essential.

To sum up, in order to use the AWE system to help EFL learners improve their expressive skills in writing language, this study applies the Intelligent Writing Critique System (IWCS) to the process of English writing in college. This study proposed an instructional strategy based on intelligent-based cognitive diagnostic feedback (I-CDF), which provides feedback to EFL learners as a kind of cognitive diagnostic information, rather than corrective information. There are significant differences between the I-CDF strategy proposed in this study and the S-TCF strategy. One is that the I-CDF-based strategy provides richer feedback, including the diagnosis of vocabulary, syntax, rhetorical expressions, chapter structure, and discourse intention of writing. The S-TCF-based strategy provides feedback on writing performance only. Secondly, the I-CDF-based strategy provides faster feedback and can provide students with timely diagnostic information. The S-TCF-based strategy provides feedback with a certain lag. Thirdly, the feedback provided by the I-CDF-based strategy is in the form of human-machine collaboration, and the feedback provided to students is more comprehensive. The feedback provided by the S-TCF-based strategy is provided by the teacher only, and the feedback content is homogeneous. Fourthly, the I-CDF-based strategy provides feedback not only for students on their writing problems but also for the teacher to understand the writing level of the whole class, for example. Statistics on grades in different dimensions, statistics on the number of student revisions, vocabulary statistics on writing, and so on. The feedback provided based on the S-TCF strategy is generally aimed at the students, not the teacher. Overall, this study expects to utilize the advantages that I-CDF feedback has to further optimize student writing.

Therefore, this study will verify whether the proposed instructional strategies can promote learners’ writing performance and what is their level of self-efficacy. In addition to their writing scores, did their epistemic network structure change with the support of an intelligence-based cognitive diagnostic feedback (I-CDF) strategy? Did their learning transferability improve through intensive training in this strategy? Was there a strong correlation with reading tendencies? We have thought deeply about these problems and have also been inspired by related research. On the one hand, this study is inspired by related writing research. In Liu et al.’s (2023) study, they compared and analyzed the influence of PA-AWE and C-AWE writing strategies on students’ writing achievement, learning motivation and critical thinking from the perspective of peer feedback. This study hopes to improve students’ writing performance from the perspective of cognitive diagnosis, and analyze the differences in epistemic network structure, and transferability of different groups. On the other hand, this study is also enlightened by AWE research. In Dikli and Bleyle’s (2014) study, AWE is mainly used to promote students’ grammar (e.g., subject-verb agreement, ill-formed verbs), usage (e.g., incorrect articles, prepositions), and mechanics (e.g., spelling, capitalization) of writing. This study hopes to use the intelligent writing critique system to evaluate students’ compositions in time and provide them with diagnostic feedback information. Therefore, in order to better improve students’ writing performance. At the same time, the instructional strategies or interventions that were designed in these studies did not take into account the analysis of the differential effects on the structure of students’ epistemic networks. Although the aforementioned studies can easily evaluate a certain element of students’ specialized cognition, such as the degree of mastery of the learned knowledge, through traditional paper-and-pencil tests, questionnaires, and interviews, it is difficult to analyze the structural characteristics of the elements of the learned knowledge, and it is difficult to make an in-depth analysis of the internal connections between the learned knowledge. In contrast, the association between the knowledge elements represented by the content of students’ writing with the help of epistemic network analysis and the modeling characterization of the connection structure between the knowledge elements can represent the cognitive schema and structural features of individual or group students more comprehensively. In this study, we used ENA to analyze the differences between the epistemic network structures in students’ writing content, so that we can, on the one hand, understand what kind of epistemic network structure characteristics the I-CDF and S-TCF form on students’ writing content, and thus optimize students’ writing content. On the other hand, it is also possible to find out the differences in students’ epistemic network structures and to adjust and optimize teaching feedback strategies through this difference. This is the significance and worth of using students’ writing content for epistemic network structure analysis in our study. Ultimately, to verify the effectiveness of the proposed instructional strategy, this study focused on the following questions (RQs).

  • RQ1: Does I-CDF strategy improve students’ English writing performance compared to S-TCF strategy?

  • RQ2: Does I-CDF strategy increase students’ self-efficacy compared to S-TCF strategy?

  • RQ3: Does I-CDF strategy promote students’ transferability compared to S-TCF strategy?

  • RQ4: Does the epistemic network structure differ between the I-CDF and the S-TCF in students’ writing?

  • RQ5: Does I-CDF strategy correlate more strongly with reading tendencies than S-TCF strategy?

  • RQ6: Does I-CDF strategy result in higher student satisfaction with English writing learning than S-TCF strategy?

2 Literature review

2.1 Cognitive diagnostic feedback

In educational and psychological research, the feedback can convey, identify and make a correction (Whyte et al., 1995), which provides students with information about effective aspects for improvement at the end of an assessment task (Hattie & Timperley, 2007). In learning assessment, providing timely feedback based on students’ learning status can effectively promote students’ learning (Wiliam, 2011). Cognitive diagnostic feedback (CDF) was generated with the development of psychometrics. Unlike evaluative and corrective feedback, cognitive diagnostic feedback values students’ knowledge structures and processing skills, offering feedback to students about cognitive advantages and disadvantages (Jang, 2009). On the one hand, cognitive diagnostic feedback provides teachers with a formative assessment framework to discover students’ potential abilities in different subject areas. It provides them with guidance to improve instruction (Leighton et al., 2004).

On the other hand, students receive diagnostic feedback to understand cognitive problems, such as specific knowledge, skills, or strategies (Birenbaum et al., 1993). Therefore, cognitive diagnostic feedback is widely used as an instructional strategy in different subject areas, including the physical disciplines of voltage and current (Ohm’s law) and buoyancy (Zhan et al., 2019; Gao et al., 2020), mathematical disciplines of fraction subtraction (Tatsuoka, 1983) and geometric sequences (Hansen et al., 2010), psychological and behavioral science disciplines of spatial rotation (Wang et al., 2018), situational judgment (Sorrel et al., 2016), and psychological disorders (Templin & Henson, 2006), English disciplines of reading skill (Jang et al., 2015) and translation competent (Mei & Chen, 2022).

With advances in information technology and artificial intelligence, intelligent writing assessment has shown great potential for improving EFL writing (Link et al., 2022). Intelligent writing assessment is a technique that uses latent semantic analysis and sophisticated natural language processing (NLP) techniques to provide immediate, intelligent feedback on writing (Jiang & Yu, 2022). On the other hand, it can help learners to improve their writing skills (Wilson & Roscoe, 2020) and reduce writing errors and provide students with more accurate practice (Bai & Hu, 2017; Gao, 2021). However, some research has raised questions about the use of intelligent writing assessments in EFL writing instruction (Wilson et al., 2021): firstly, early intelligent writing assessments were more concerned with feedback on scores. Secondly, the simple use of intelligent writing assessment can lead to poor revision skills, with students tending to focus on the length and complexity of the writing rather than the quality of the writing. Thirdly, with the use of intelligent writing assessment, teachers see only the students’ writing performance and have difficulty in identifying the development of students’ cognitive skills in writing. Based on the analysis of the literature, there is a need for further research on how to apply the feedback provided by intelligent writing assessment as a form of cognitive diagnostic information in second-language writing. There is also a lack of understanding of the epistemic network structures in students’ writing from a micro perspective. Therefore, how to provide personalized feedback on students’ writing and quickly diagnose problems in writing with the help of an intelligent writing assessment approach is an important problem that needs to be addressed in this study. In addressing these questions, it is also expected to improve students’ writing performance and teachers’ writing assessment methods.

2.2 Self-efficacy in writing

Self-efficacy refers to people’s judgments about their ability to organize and execute the course of action required to reach specified goals (Bandura, 1995). It determines an individual’s choices, goals, motivation, perseverance, and expected outcomes (Schunk & Pajares, 2010; Schunk & Usher, 2012). It has been established that self-efficacy has a role in predicting academic achievement across a wide range of domains (Bandura, 1997), and writing is no exception. Self-efficacy in writing refers to learners’ self-judgment of their writing ability (Pajares & Valiante, 1997; Pajares, 2003). The researchers involved have analyzed the role of self-efficacy in writing and provided empirical evidence. They found that self-efficacy in writing was related to writing performance (Pajares & Johnson, 1994), writing quality (Zimmerman & Bandura, 1994), writing goals (Pajares & Graham, 1999), and interest in writing (Bruning et al., 2013). This is because cognitive and affective aspects of writing contribute to writing learning, which can affect learners’ self-efficacy (Hayes, 2000).

Many studies have attempted to explore the relationship between self-efficacy and writing by developing instruments to measure self-efficacy in writing learning environments (Pajares & Valiante, 1999; Shell, 1999). In terms of writing goals and tasks, Teng et al. (2018) conceptualized self-efficacy in the context of EFL writing in a multidimensional manner based on social cognitive theory (Bandura, 1986) and self-regulated learning theory (Zimmerman, 2013). The results of the study indicated that self-efficacy in writing is related to an individual’s intrinsic goal orientation and task values. In terms of writing performance, Zabihi (2018) found that self-efficacy significantly predicted complexity, accuracy, and fluency in L2 writing. In terms of writing problem-solving, Tsao (2021) found that in L2 writing, learners with higher self-efficacy were more likely to solve cognitive difficulties in writing (e.g., rhetorical ability, linguistic knowledge, and writing strategies). Most studies point to a common finding: self-efficacy provides necessary predictions for writing learning and influences writing effectiveness.

Therefore, to explore the effectiveness of the proposed I-CDF strategy, a scale of self-efficacy in writing was designed. On the one hand, it aims to know students’ subjective emotions during the writing process by measuring their self-efficacy, to determine whether they respond positively to the feedback strategy and effectively receive the feedback information. On the other hand, self-efficacy was analyzed to determine whether the designed feedback strategy helped to strengthen students’ independent learning behavior.

2.3 Transferability

Transferability is the application of prior learning to new situations or environments (Lu et al., 2015), which promotes deep internalization of students’ knowledge learning. It helps students apply knowledge flexibly in different contexts, and develop problem-solving skills (Goldstone & Day, 2012). In L2 writing learning, transferability is the learning skill applied from a prior context into a new context (Perkins & Salomon, 2012). This transferability can transfer writing methods and rhetorical principles to other learning content (Spack, 1988). On the one hand, transferability can improve EFL learners’ cognitive skills in different learning tasks and promote their learning efficiency (Pink et al., 2010). On the other hand, EFL learners are helped to become proficient in writing skills through the conversion of writing outcomes and their continuous application (Perin et al., 2017).

In recent years, research on transferability in writing has focused on cross-linguistic transfer, cross-disciplinary transfer, and cross-genre transfer. Firstly, in terms of cross-linguistic transfer, Rinnert et al. (2015) investigated how learners use cross-linguistic knowledge to construct L1 and L2 texts by considering an adaptive transfer from a dynamic perspective as invoking prior knowledge in a writing context. It was finally shown that learners were able to creatively reshape cross-linguistic knowledge to fit a specific writing context. Secondly, in terms of interdisciplinary transfer, many studies have examined L2 learners’ transfer from ESL or EAP courses to content-specific courses to explore whether learners’ performance in writing learning is influenced by their ability to transfer across disciplines. The findings suggest that students’ use of prior knowledge in writing is heavily dependent on their perceived differences and similarities in disciplinary contexts, which affects their writing performance. (Hansen, 2000; James, 2012; Jwa, 2019). Finally, in terms of transfer across genres, many studies have explored the application of transferability skills across different writing genres, such as narrative, expository, and argumentative writing. These studies have shown that students with higher transferability perform better in writing across genres. (Shrestha, 2017; Kim & Belcher, 2018; Wilson & Soblo, 2020).

Therefore, transferability as an essential learning skill is widely used in writing learning, including application between courses (DePalma & Ringer, 2011), languages (Pavlenko & Jarvis, 2002), and disciplines (Jwa, 2019). So, transferability plays an influential role in L2 writing learning. In the study, transferability was divided into near-transferability, which is the transferability in different writing topics, and far-transferability, which is the transferability of learning in different courses. The study aims to explore the effects of different feedback strategies on students’ transferability in L2 writing.

2.4 Epistemic network analysis

Epistemic Network Analysis (ENA) is an analytical method, which is used to explore an individual or group’s epistemic framework by quantifying qualitative data. ENA shows the characteristics of dynamic and coupling (Shaffer et al., 2016). It is an effective method for exploring content analysis and a valuable tool for studying the epistemic structure. Its role is to calculate the number of co-occurrence of each knowledge element in the context of writing by using mixed methods and data visualization techniques, building the networked representation of the writing process, and reflecting the connections between different competency codes. The ENA tool creates an adjacency matrix for each section, quantifies the co-occurrence relationships between individual codes, accumulates, and normalizes the adjacency matrices of different sections into vectors in a high-dimensional space according to the set analysis units. And then, the ENA tool uses singular value decomposition (SVD) to downscale and rotate this high-dimensional space and retain the core information of the original data. With minimal data loss, the ENA tool presents the data of analysis units and groups in a visible two-dimensional space (Fougt et al., 2018). The generated ENA model contains the following information: (1) codes (or nodes), which are people/concepts connected in the ENA model; (2) relationships (or edges), which are the way the codes are related to each other; (3) sections, which are time- or process-based identification units.

In recent years, ENA has been widely used in learning analytics, including collaborative learning analysis, metacognitive learning, gaming learning (Elmoazen et al., 2022). For example, in collaborative learning analysis, Csanadi et al. (2018) compared traditional coding and counting analysis methods with epistemic network analysis (ENA) methods in a computer-supported collaborative learning (CSCL) study. With the help of the ENA method, the temporal co-occurrence of students’ communicative discourse coding was analyzed to understand students’ social cognitive activities during collaborative learning. In the context of metacognitive learning, Pratt et al. (2023) explored the cognitive relationship between visual information strategies and non-linear layout strategies and text comprehension in primary school students using the ENA analysis method. The study characterized students’ complex thinking through ENA, including cognitive elements such as values, knowledge, skills, epistemology and identity, and its findings evidenced the ability of the two strategies to enhance students’ metacognitive awareness and text comprehension. In the context of gamified learning, Nash and Shaffer (2013) used ENA to explore the cognitive framework elements of game design, including the three cognitive elements of gameplay, game concepts and conceptual domains, and found that students’ cognitive developmental trajectories not only approached their tutors’ cognitive trajectories over time but also became progressively similar to their tutors’ cognitive trajectories.

However, ENA studies have focused less on the L2 writing field. Using cognitive network analysis to analyze the cognitive structure of students’ writing texts can effectively reflect the developmental process of students’ writing skills and provide an essential reference for areas such as analysis of students’ writing levels. In the study, the ENA 1.5.2 Web Tool (version 1.5.2) was applied to analyze the characteristics of the epistemic network structure reflected by word frequency in writing (Shaffer et al., 2015), whose purpose is to analyze the differences in the epistemic network structure with different writing feedback strategies.

3 Intelligent-based cognitive diagnostic feedback strategy

In the study, based on the application of IWCS, the I-CDF strategy was designed for writing learning activities. It includes an intelligent writing module, an assignment management module, and a database management module. See Fig. 1. This feedback strategy aims to help students identify writing problems by providing timely cognitive diagnostic feedback, such as vocabulary spelling errors, punctuation misuse, and improperly paired sentences. Also, to timely adjust the instructional writing design by grasping the overall level of learners’ writing, diagnostic feedback needs to be applied to analyze writing feedback, such as error statistics and word frequency statistics feedback. The strategy includes five levels of cognitive diagnosis: the diagnosis of lexical processing, the diagnosis of syntactic processing, the diagnosis of rhetorical expression, the diagnosis of chapter structure, and the diagnosis of discourse intention. Among them, the first two diagnoses belong to low-level cognitive diagnostic feedback, and the last three belong to high-level cognitive diagnostic feedback.

Fig. 1
figure 1

Structure diagram of the intelligent writing critique system

Full size image

The purpose of lexical processing is to diagnose vocabulary to understand that vocabulary is reasonable and accurate with the help of verbal feedback. The purpose of the diagnosis of syntactic processing is to diagnose the fluency of sentences to realize that the articulation and collocation of sentences are fluent with the help of syntactic feedback. The purpose of diagnosing rhetorical expression is to diagnose the regulation of expressions in writing. Feedback on rhetorical expressions helps learners notice that the language expressed is regular, authentic, and formal. The purpose of the diagnosis of chapter structure is to diagnose the logic of structure in writing. The learners would clarify that the structure is logical with the help of chapter feedback. The purpose of the discourse diagnosis is to diagnose the integrity of the content. The learner would understand that the intention in writing is thematically appropriate and innovative with the help of feedback on the discourse. See Fig. 2.

Fig. 2
figure 2

A framework for intelligent-based cognitive diagnostic feedback

Full size image

The writing learning activities were divided into three phases based on the I-CDF strategy. See Fig. 3. The first phase is conceiving and writing. The teacher releases the writing tasks and requirements in the IWCS. The students conceived a writing framework based on their understanding and experience and then created writing text. The second phase is evaluation and feedback. Students must submit their writing texts to the IWCS after completing their writing tasks. In this process, students get timely feedback on their writing scores, good words, good sentences, error tips, revision suggestions, extended sublimation, and excellent examples (see Figs. 4 and 5). At the same time, teachers also get the corresponding feedback data, including error statistics data, word frequency statistics data, and word-pairing statistics data. These feedback data can also provide students with comprehensive and personal feedback suggestions. See Fig. 6. The third phase is reflection and correction. When students receive the appropriate diagnostic feedback, they reflect on the problems in writing and form revised thinking based on the feedback. These three phases can be done iteratively, with multiple rounds of revisions until the writing goals are achieved. The three main features of the I-CDF strategy are also highlighted through writing learning activities. The first is the timeliness feature of the feedback. With the help of IWCS, students can get feedback on their writing quickly. The second is the multi-level feature of feedback. Feedback information includes five different levels of diagnostic information, namely, lexical diagnosis, syntactic diagnosis, rhetorical diagnosis, structural diagnosis, and discourse diagnosis. The students will receive good words, quality sentences, and an excellent model article for reference. The third is the synergistic nature of feedback. Thus, the score is no longer the only evaluative feedback information. Students receive diagnostic feedback on their writing that is generated collaboratively by the teacher and the IWCS.

Fig. 3
figure 3

Intelligent-based cognitive diagnostic feedback strategy model

Full size image
Fig. 4
figure 4

Score-based teacher corrective feedback strategy model

Full size image
Fig. 5
figure 5

Cognitive diagnostic feedback context

Full size image
Fig. 6
figure 6

Cognitive diagnostic feedback comprehensive data

Full size image

The structure of the S-TCF is shown in Fig. 4. Feedback based on the S-TCF is mainly provided by the teacher. The greatest advantage of this type of feedback is that the teacher can know the level of each student’s writing through the evaluation of each piece of writing. However, S-TCF-based feedback is less informative (e.g. including score feedback, word feedback and sentence feedback) and has a longer feedback period. In addition, S-TCF-based feedback is inefficient and does not provide students with timely diagnostic information. It is difficult to provide teachers with a visual panorama of class writing levels and, therefore, to understand the quality of their teaching.

4 Methodology

4.1 Participants

A quasi-experimental design was used in the study. And the participants were first-year undergraduate students at a public university in southeastern China, with an average age of 19 years. They are taught by the same teacher. In the study, the experimental group using the I-CDF strategy (n = 32) and the control group using the S-TCF strategy (n = 30) were selected using a pre-test. In this study, the number of participants in the two groups was randomly selected. During the actual test, two participants in the control group failed to take the test, resulting in two more people in the experimental group than in the control group. However, this does not affect the validity of this study. A similar situation emerged in the studies of Brüggemann et al. (2023), Liu et al. (2023) and Wang et al. (2022). Among these studies, Wang et al. (2022) used AWE and SVVR to support college students’ EFL Writing Performance by dividing the number of randomized participants into an experimental group = 37 and a control group = 39. Brüggemann et al. (2023) in a reading comprehension study, the number of study participants was randomly into three test groups, where N PPT group (pencil-and-paper test) = 65, N CBT group (computer-based test) = 69, and N CAT group (computer adaptive test) = 78. In Liu et al.’s (2023) study, which integrated peer assessment with AWE to promote students’ EFL writing, the study divided the number of randomized participants into an experimental group = 33 and a control group = 31. Overall, these studies showed numerical inequity and randomization of allocation in the selection of study participants, which conformed to the norms of empirical research in education (McMillan & Schumacher, 2010). There is no doubt that the participants in this study also conformed to such characteristics.

4.2 Experimental procedure

Figure 7 shows the experimental design of this study. In the first week, it is the IWCS training phase. Its purpose is to familiarize students with the functions and operations of IWCS. The second week is the pre-test phase of the experiment. The pre-test of EFL writing and translation, the pre-analysis of self-efficacy, epistemic network, and transferability were administered. The third to sixth weeks is the experimental activity phase. Students must complete four writing exercises, each going through three phases: conception and writing, evaluation and feedback, and reflection and correction. Table 1 shows the learning time, learning topics, and learning requirements. Students in the control group used the S-TCF to obtain information about their revised writing. In contrast, students in the experimental group used the I-CDF to get information about their revised writing. In the seventh week, it is the post-test phase of the experiment. After the learning activity, the post-test of EFL writing and translation and the post-analysis of self-efficacy, reading tendency, epistemic network, and transferability were administered. Finally, ten students (5 from each group) from each group were randomly selected to participate in the interview.

Fig. 7
figure 7

Experimental procedure

Full size image
Table 1 Writing samples
Full size table

5 Instruments

5.1 Self-efficacy scale

The self-efficacy was measured using a questionnaire adapted from Redford et al. (Prat-Sala & Redford, 2010). Self-efficacy is the level of confidence students gain in completing this learning task. There were 18 projects in total. In this study, self-efficacy was divided into two dimensions, namely writing skill self-efficacy and writing task self-efficacy. Writing skill self-efficacy is the confidence that learners have in the basic writing skills they use in writing, such as vocabulary and rhetorical skills and sentence transformation skills. Writing task self-efficacy is the confidence that learners have in their ability to complete a writing task. Reliability assessed with Cronbach’s alpha was 0.92 for the task subscale and 0.93 for the component skills subscale, indicating a high level of internal consistency for each subscale.

5.2 Epistemic network analysis scale

The L2 writing vocabulary is a key component of effective writing (Engber, 1995), affecting the writing quality, and reflecting the overall descriptive writing of the learners (Binder et al., 2017). Therefore, ENA analysis of students’ writing word frequency can form a dynamic network model of students’ writing texts and characterize the structure of associations among students’ epistemic words, which can reflect the characteristics and changes in the structure of students’ writing epistemic cognitive networks. In the study, WordsSmith software was used to select the frequency of the writing word and code it. And then the ENA was used to analyze the epistemic structure of students’ writing and explore the effects of feedback strategies on the epistemic structure of students’ writing. The coding scheme in the existing literature does not fully capture the characteristics of the vocabulary in students’ writing. Therefore, our team developed a coding scheme based on Gardner’s approach to vocabulary classification and applied grounded theory to code the vocabulary analyzed by WordsSmith (Gardner & Davies, 2014; Charmaz, 2006). It is important to particularly note that Gardner’s approach is a new approach to vocabulary classification, which provides a new vocabulary list (AVL). Therefore, this study draws on the vocabulary list it provides to classify students’ writing words, thus facilitating coding in the ENA analysis. Eventually, the vocabulary was divided into two categories, including content words and function words. The content words are nouns (SN), pronouns (SP), adjectives (SAJ), adverbs (SAV), and verbs (SV). The function words are crown words (XAR), conjunctions (XCO), and prepositions (XPR). The coding scheme was reviewed and discussed by our team to ensure its appropriateness and consistency. After reaching an agreement, the two researchers applied the coding scheme to independently code 10% of the entire dataset. The inter-rater agreement for this coding was 0.78 (Cohen kappa). After discussing and resolving the discrepancies in the coding results, one of the researchers coded the remaining data. See Table 2.

Table 2 High-frequency words category coding scheme
Full size table

5.3 Interview questionnaire

The interview questionnaire designed for this study was modified from Zhe (Victor) Zhang’s study (2020). For example, “Can you share with me your experience of writing in English? What types of feedback do you receive?” “What is your first impression and overall opinion of the IWCS in providing feedback?” “Do you think that feedback optimizes writing? In what ways? Please give examples.” “How do you use the IWCS to provide feedback to revise your writing?” A represents respondents in the control group, and S represents respondents in the experimental group. Based on the rooting theory proposed by Glaser and Strauss (2017), Creswell (2013), and Miles et al., 2018. The interviews were conducted by a Ph.D. researcher, a master’s researcher, and a professor to categorize, summarize, and validate to infer the key factors between the two groups’ perceptions of writing learning and writing performance. When their classifications conflicted, they revisited and discussed the content to reach a consensus. The interview questions were reviewed by two faculty members with more than 10 years of experience teaching foreign languages at the university level to verify their validity and to check that the wording was understood by the students.

6 Results

6.1 Analysis of writing score

There was no significant difference between the pre-test scores of the experimental and control groups (t = 0.271, p = 0.788 > 0.05). The two groups belonged to a homogeneous level. Also, this study tested the students’ technology acceptance and the results were found to be non-significantly different (t = −1.147, p = 0.257 > 0.05), which indicated that their technology acceptance was at the same level. However, the mean score of the experimental group (ME = 77.61) was significantly higher than the mean score of the control group (MC = 68.72), and the post-test scores of the two groups showed significant differences (t = 8.382, p < 0.05). See Table 3. After a period of intervention with the I-CDF strategy, a significant difference in post-test scores emerged between the experimental and control groups, with the mean writing scores of the experimental group being higher than those of the control group. The results of this data indicate that the implementation of the I-CDF strategy improved students’ writing performance and that students were able to use the diagnostic feedback to help them continuously reflect on their writing problems.

Table 3 Independent sample t-test results of post-test scores
Full size table

6.2 Analysis of self-efficacy

Independent samples t-test was used to analyze the differences in self-efficacy between the experimental and control groups. The results of the pre-test showed that there were no significant differences between the experimental and control groups in writing skill self-efficacy (t = 1.749, p = 0.085 > 0.05) and writing task self-efficacy (t = 1.455, p = 0.151 > 0.05). The results of the post-test showed that the mean value of writing skill self-efficacy (ME = 32.53 > MC = 27.00) and writing task self-efficacy (ME = 33.43 > MC = 28.26) was higher in the experimental group than in the control group. Both were significantly different (p = 0.000 < .05), which indicated that students’ self-efficacy in the experimental group improved significantly after adopting the I-CDF strategy. See Table 4. This indicates that after a period of I-CDF strategy intervention, there was a significant difference in posttest self-efficacy between the experimental and control groups, with the experimental group having higher mean values for writing skill self-efficacy and writing task self-efficacy than the control group. The results of this data indicate that the implementation of the I-CDF strategy was able to increase students’ self-efficacy, and at the same time, students had greater confidence to apply more writing skills (e.g. spelling, wording, etc.) with the help of the diagnostic feedback, and eventually complete the writing task.

Table 4 Independent sample t-test results of post-test self-efficacy
Full size table

6.3 Analysis of epistemic network structure

In the study, the writing texts of the experimental group and the control group were pooled and analyzed. Based on the word frequency data of the writing texts of the two groups, the data were counted. See Table 5.

Table 5 Writing vocabulary frequency statistics table
Full size table

6.3.1 Projection space analysis of epistemic networks

Figure 8 shows the epistemic network projection space of students’ writing vocabulary application in the experimental group (red) and the control group (blue). Each point is the epistemic network centroid of the student’s vocabulary application. The squares are the centroid of the control and experimental groups. The first dimension (ENA1) accounts for 64.4% of the overall variances (variance) in the original data. The second dimension (ENA2) accounts for 22.0% of the overall variance in the original data.

Fig. 8
figure 8

Writing epistemic network projection space for the experimental group (red) and control group (blue)

Full size image

The Mann-Whitney U test compared the projection point distribution in ENA space between the control and experimental groups. Statistically significant differences were shown at the α = 0.05 level on the horizontal axis of ENA space (ENA1) (Mdn = −0.83, N = 8 U = 61.00, p = 0.00, r = −0.91), while no significant differences were observed on the vertical axis of ENA space (ENA2).

Each network (Fig. 8) and the corresponding epistemic network model (Fig. 9) are in the same projection space. Therefore, according to the distribution of epistemic elements in the epistemic network model (Fig. 9), the ENA1 and ENA2 axes were assigned the relevant meanings: (1) The epistemic elements near the ENA1 axis are SN, XAR, XPR, SAV, and SAJ. SN and XAR can be summarized as “conceptual type” descriptions of objects, and XPR, SAV, and SAJ as “rhetorical type” descriptions. Thus, the left-end description space of the ENA1 axis is called the “conceptual expression,” while the right-end description space is called the “rhetorical expression”. (2) The epistemic elements near the ENA2 axis are SP, SV, SAV, and XCO. SP and SV can be grouped into “lexical type” descriptions. SAV and XCO can be grouped into “syntactic type” descriptions. Thus, the top description space of the ENA2 axis is called the “lexical expression” and the bottom description space is called the “sentence expression”. Eventually, ENA1 forms a continuum of “conceptual expression” and “rhetorical expression” from left to right. At the same time, ENA2 forms a continuum of “lexical expression” and “sentence expression” from top to bottom.

Fig. 9
figure 9

Epistemic network model

Full size image

6.3.2 The difference analysis of network cognition

To investigate the differences in students’ epistemic abilities in writing vocabulary among different feedback strategies, an epistemic network analysis method was applied to analyze the use of vocabulary in writing, which valued the epistemic abilities in writing vocabulary of the experimental and control groups.

Table 6 shows the pre-test results for the experimental and control groups. According to the first dimension (ENA1 axis), the result of the t-test showed no statistically significant difference between the experimental and control groups at the alpha = 0.05 level, when the variances were not assumed to be equal. According to the second dimension (ENA2 axis), the result of the t-test showed no statistically significant difference between the experimental and control groups at the alpha = 0.05 level, when the variances were not assumed to be equal.

Table 6 Pre-test of the epistemic network of writing vocabulary for the experimental and control groups
Full size table

Figure 10 shows the epistemic network of writing vocabulary for the experimental group post-test (red) and the control group post-test (blue). Figure 10(a) shows the epistemic network of the experimental group on the post-test. Figure 10(b) shows the epistemic network of the control group on the post-test. And Fig. 10(c) shows the differences in the epistemic network of the experimental and control group on the post-test. In (a), there is a very strong connection between SAJ and SV (connection weight = 0.59), and a relatively strong connection between SAJ, XPR, SAV, XCO, and SV (connection weight = 0.45), while the connection between SP, SN, XAR, and XAR, SAV, SV (connection weight = 0.30) is weaker. The connection between XCO, XPR, and SAJ (connection weight = 0.15) appeared to be the weakest. This shows that students used the I-CDF strategy to focus on rhetorical and syntactic aspects of writing. In (b), there is a strong connection between SN-SP (connection weight = 1.00), SV-SP (connection weight = 0.75), and SN-SV (connection weight = 0.75), a weak connection between XAR and SN, SP, SV (connection weight = 0.50). The weakest connection is between SAV and SP, SN (connection weight = 0.25). In addition, XPR, XCO, and SAJ elements are not connected and are also not connected to any other elements. This shows that students used the S-TCF strategy to focus on vocabulary and concepts in writing. In (c), students in the control group tended to use the appropriateness of vocabulary and the correctness of concept expressions in writing. In contrast, students in the experimental group focused on rhetorical expressions and the logic of sentence structure based on the correctly applied vocabulary and concepts in writing. The results of the analysis showed that, in comparison to the control group, the experimental group students’ epistemic in writing gradually improved, which was more focused on high-level rhetorical expressions in writing. It makes the writing expressions more diversified in the experimental group.

Fig. 10
figure 10

Epistemic network of writing vocabulary for the experimental group post-test (red) and the control group post-test (blue)

Full size image

In addition, a t-test was used to value differences in the post-test writing vocabulary epistemic network between the experimental and control groups. See Table 7. According to the first dimension (ENA1 axis), the results of the t-test showed a statistically significant difference between the experimental and control groups at the alpha = 0.05 level, when the variances were not assumed to be equal. According to the second dimension (ENA2 axis), the results of the t-test showed no statistically significant difference between the experimental and control groups at the alpha = 0.05 level, when the variances were not assumed to be equal.

Table 7 Pro-test of the epistemic network of writing vocabulary for the experimental and control groups
Full size table

The analysis of the above data results shows that after the formal experimental activities were conducted, significant differences in the epistemic network structure of writing emerged between the two groups of students. This also implies that the students’ epistemic network structure of writing improved after a period of I-CDF strategy intervention.

6.4 Transferability analysis

At the end of the writing activity, both groups were tested for transferability. The near transferability was valued by using expository writing scores, and the far transferability was valued by using reading scores. The results are shown in Table 8. The experimental group scored higher than the control group on the near transferability (t = 2.355, p < .05) and also on the far transferability (t = 2.108, p < .05). Thus, these results indicated that the I-CDF strategy was more effective than the S-TCF strategy in enhancing students’ transferability. After a period of I-CDF strategy intervention, a significant difference in transferability emerged between the experimental and control groups. The mean value of transferability was higher in the experimental group than in the control group. The results of the data indicate that the implementation of the I-CDF strategy was able to improve students’ transferability, and that students were able to transfer their knowledge from the diagnostic feedback to other areas of learning.

Table 8 Independent sample t-test results of the near transferability and the far transferability
Full size table

6.5 Reading tendency and writing correlation analysis

To investigate the correlation between students’ reading tendency and writing scores, independent samples t-test was used to compare the reading tendency of the two groups before the writing activity (t = 1.256, p = 0.214 > 0.05). The results showed no significant difference between the two groups in reading tendency. After the writing activity, Pearson Correlation Analysis was conducted between the reading tendency and writing scores in the experimental and control groups. See Table 9. The results showed a significant correlation between reading tendency and writing scores in the experimental group (r = 0.353, p < 0.05), while the control group was not (r = −0.261, p > 0.05). It showed that the students of the experimental group would apply knowledge and experience from previous reading in writing. This was also mentioned in the interview.

Table 9 Analysis of the correlation between reading tendency and writing scores
Full size table

7 Qualitative analysis

In order to deeply analyze and verify the students’ satisfaction with the support of different feedback strategies, 10 students from each group were randomly selected for the interview. Grounded theory was used to code and analyze the interview results.

This study conducted a three-level coding analysis of the interview data based on grounded theory. In the first level of coding, the open coding stage, we transcribed the collected interview recordings into text. Then, concepts with similar connotations in the text were collated and merged to form the initial concepts. In the second level of coding, namely the axial coding stage, the connotations of the initial concepts are explored in depth and the main categories to which these concepts belong are refined. Finally, the initial concepts were divided into 10 main categories: writing structure, interest in writing, ability to write independently, transferability, corrective ability, self-efficacy, confidence in writing, achievement, attitude towards writing, and engagement. In the third level of coding, namely the selective coding stage, the main concepts of the research question were identified from the open and axial coding, and cases were found that illustrated the themes. Thus, the selective coding established the six main concepts to be explored in this paper: writing structure, interest in writing, ability to write independently, transferability, corrective ability, and self-efficacy.

7.1 Results of the experimental group interview

The results of the interview in the experimental group showed that the I-CDF strategy promoted students’ writing learning in the following four main aspects. Firstly, it enabled the optimization of writing structure. Secondly, it stimulated the interest in writing learning. Thirdly, it enhanced the independent ability to write. And fourthly, it promoted the improvement of transferability.

In terms of writing structure optimization, the students who participated in the interview showed a consistent attitude that the I-CDF strategy provided helpful feedback and pointed out the shortcomings of writing promptly. For example, S1 stated, “It is possible to obtain timely information about the shortcomings of writing and revise the writing content in time by using the IWCS for writing learning.” S2 mentioned that “The IWCS provides feedback on content, structure, vocabulary, and language expression, which facilitates the improvement of writing structure and enhances fluency of expression.”

In terms of interest in writing, in comparison to traditional writing classes, most of the interviewees felt more relaxed and enjoyable in intelligent writing classes. In addition, getting timely feedback makes students think that writing is not difficult but further gains confidence in writing. For example, S1 mentioned, “I do not feel much pressure when using the IWCS in writing, and it seems easier than paper-based writing in the past.” S3 stated, “In the past, after completing the writing, the only feedback I got was an individual score, and the teacher gave writing both delayed and single-minded comments. As a result, it was difficult to correct writing in time, causing me to dislike writing.” S4 said, “After studying with the IWCS, my writing score has increased significantly. Now, I am very confident in my writing.”

In terms of the ability to write independently, most interviewees expressed that the I-CDF strategy allowed them to take the initiative to revise their writing in class based on feedback and achieve independent learning. For example, S3 reported that “Correcting writing problems based on feedback from the IWCS provides me with sufficient time for independent study.”

In terms of transferability, most interviewees expressed that the I-CDF strategy not only supported them in writing successfully on different topics and genres but also helped them improve translation scores and reading comprehension, which facilitated the transfer of their learning abilities. For example, S2 expressed, “After weeks, not only do I increase the scores in writing, translation, and reading, but also realize the importance of reading.” S5 mentioned, “I am now able to write high-scoring expository and argumentative writing as well.”

7.2 Results of the control group interview

The results of students’ interviews in the control group showed that the S-TCF strategy mainly improved their word and sentence correction skills and self-efficacy.

In terms of improving students’ ability to correct errors in words and sentences, interviewees agreed that the S-TCF strategy emphasized feedback on using words and sentences to correct errors in writing. A6 mentioned, “Teachers often point out errors in spelling, case, and sentence usage of vocabulary in writing so that I will pay close attention to the use of words and sentences in writing.” A3 also agreed with A6, adding, “Before, I was not sensitive to problems with spelling and capitalization in my writing, but I realized the problem after my teacher often pointed it out. This can affect the readability of the writing.”

In terms of self-efficacy, some students expressed that the S-TCF strategy emphasized the importance of writing scores, but was ineffective. A9 stated, “I would be able to understand my overall level by writing scores. However, I would prefer to have more detailed diagnosis information.” A7 said, “Although I got a good grade in writing and my teacher approved my writing ability. However, I want to reach a higher level of writing and would like my teacher to give me more detailed instructions.”

7.3 Discussion of interview results

Concerning RQ6,(i.e., Does I-CDF strategy promote students’ transferability compared to S-TCF strategy?), it was found that both the experimental and control groups thought that the feedback strategy could facilitate second language writing learning. Students in the control group believed that the S-TCF strategy improved their basic writing skills in words and sentences. But there were still deficiencies in rhetorical expression revision, logical structure revision, and innovation in writing. At the same time, due to the long feedback period, it wasn’t easy to ensure that every student could absorb the teacher’s feedback. After the introduction of the IWCS, the results of the interview in the experimental group showed that the I-CDF strategy could promote writing structure, increase their interest and confidence in writing, enhance their learning independence, and promote their transferability, all of which, in turn improved writing scores. It can be seen that students in the experimental group affirmed the merits of the I-CDF strategy, which was consistent with McNamara et al. (2013), Lv (2018), and wang (2020).

8 Discussion and conclusion

In the study, the IWCS was used to support L2 writing activities, which was different from the AWE system that aims to get writing scores (Yuan, 2021). The I-CDF strategy designed in the study, provided students with timely, specific, diagnostic feedback, supporting them in completing and revising their writing. The results showed that the I-CDF strategy better promoted students’ writing scores, writing transferability, and writing self-efficacy than the S-TCF strategy. In addition, the results showed that in terms of epistemic development, the experimental group focused on writing skills, including rhetorical expression of words and logical structure of sentences in writing, while the control group focused more on the appropriate use of vocabulary and concepts. As a result, students in the experimental group developed better epistemic abilities. It showed that the I-CDF strategy has a positive effect on students’ writing scores, self-efficacy, transferability, and epistemic development.

Concerning RQ1,(i.e., Does I-CDF strategy improve students’ English writing performance compared to S-TCF strategy?),the results showed that the I-CDF not only provided detailed cognitive feedback in writing, which helped students receive precise feedback that enhanced their awareness of the problem-solving, and strengthened their writing expression. It showed that the I-CDF strategy could improve students’ higher-level abilities in writing, which was consistent with Roscoe & McNamara (2013) findings that IWCS for supplemental learning could improve students’ writing scores (Roscoe & McNamara, 2013).

Concerning RQ2, (i.e., Does I-CDF strategy increase students’ self-efficacy compared to S-TCF strategy?), the results showed that the I-CDF strategy had a positive impact on the self-efficacy of the experimental group students. In terms of writing skills, an important reason was that the immediacy of the feedback supported students in getting timely content on evaluation. And then, based on the feedback, they can correct their mistakes and improve their writing content in time, which helps them enhance their writing quality. In terms of writing tasks, the I-CDF strategy can provide diverse feedback. Students not only know what is wrong with their writing but also have access to a wealth of revision advice and materials, including good phrase expansions and excellent writing examples, which enable them to complete writing tasks with high quality quickly. The two important features of the I-CDF strategy stimulate students’ learning motivation and improve their self-efficacy.

Concerning RQ3,(i.e., Does I-CDF strategy promote students’ transferability compared to S-TCF strategy?), the results showed that the I-CDF strategy had a positive effect on both the near-transferability and far-transferability of the experimental group students. The analysis results of the differences in epistemic network structure showed that the trend of students’ cognitive ability in writing in the experimental group ranged from low to high levels. As the students in the experimental group paid attention to rhetorical expression and structural optimization, their writing expression ability was improved, translation ability as well.

Concerning RQ4,(i.e., RQ4: Does the epistemic network structure differ between the I-CDF and the S-TCF in students’ writing?), the results showed that the epistemic network structure of the experimental group reflected the students’ higher-level vocabulary applications, focused on the description of rhetorical expressions and the logical structure of sentences in writing. Students in the control group were more inclined to use vocabulary appropriately in writing and made fewer higher-level vocabulary applications. Overall, the experimental group focused more on higher-level vocabulary applications in writing. Therefore, the experimental group obtained higher writing scores.

Concerning RQ5,(i.e., Does I-CDF strategy correlate more strongly with reading tendencies than S-TCF strategy?), the results showed that the students in the experimental group had a stronger correlation between their writing scores and their reading tendencies. On the one hand, it once again argues for a relationship between reading and writing, which is a positive relationship (Graham & Hebert, 2011). On the other hand, it also reminds teachers that developing students’ reading habits need to be emphasized.

The limitations of the study are as follows. First, due to the short duration of the writing learning activities, the methods proposed in the study may increase students’ writing scores. Still, it can’t accurately explain that they can improve students’ writing ability because developing writing skills takes time. Second, due to the small sample size of the data, some data errors may arise in the data analysis. This will lead to the risk that the study conclusions will be unreasonable. Therefore, in future studies, we need to increase the sample size of the study.

In summary, the contribution of this study is to show that the I-CDF strategy with IWCS can enhance students’ writing scores, writing transferability, and writing epistemic network structure. At the same time, students undergo a significant change in their writing knowledge, writing methods, and writing attitudes. To some extent, there are still some shortcomings in this study. For future research, we not only need to design a more extended period of experimental activities but also to further compare the differences in grades and school districts. In addition, it is also worthwhile to study the effects of the I-CDF strategy on different writing genres, such as argumentative and explanatory writing. In conclusion, we believe that this study not only provides a good case study for further research on the teaching of English writing in Chinese universities, but also brings certain insights, including the integration of intelligent technologies into classroom writing teaching models, the use of ENA to analyze the epistemic network structure of students’ writing learning, and the rational use of feedback information by teachers to diagnose and assess students’ writing.