Dynamic Assessment: Co-constructing the Future with English Language Learners

Abstract

All forms of assessment are concerned with interpreting individuals’ performances not merely for the sake of describing those performances but for employing them as a basis for making claims about the knowledge and abilities believed to underlie them. Dynamic Assessment (DA) is a framework that challenges more conventional views of performance and the evidence of abilities most appropriate to forming generalizations regarding abilities. Specifically, DA requires the integration of teaching into assessment activity for the purpose of understanding learner responsiveness. Based on the theoretical writings of L. S. Vygotsky, DA proponents consider learner independent performance of tasks to reveal abilities that have fully formed whereas learner responsiveness to support that is offered when difficulties arise indicates abilities that may not have fully developed but are emerging. In this way, DA offers a developmental diagnosis that does not predict learner future functioning solely on the basis of past development but that instead begins to construct a future with learners during the assessment itself. This chapter considers the major theoretical underpinnings of DA and the models and principles elaborated in the extensive DA research literature concerned with cognitive abilities and general education. Discussion then turns to implementation of DA with L2 learners. Two studies are presented in detail that emphasize DA’s potential in both formal language testing situations and instructional contexts.

Introduction

Perhaps the greatest challenge in educating English language learners is arriving at an appropriate understanding of their past in order to take stock of their current abilities and needs and help them move forward. In this regard, assessment, broadly conceived as an activity of gathering information about what learners know and can do (Bachman and Palmer 1996), plays a central role. Of course, assessment can denote a wide range of practices, from requiring students to sit for externally designed standardized tests to student creation of portfolios of their work and from quizzes created by teachers targeting outcomes from specific lessons to student in-class presentations. Traditionally, these practices have been categorized as instances of either summative or formative assessment (Dixson and Worrell 2016). Gardner (2010) explains that summative assessment frequently involves the use of standardized procedures for administering the assessment and for interpreting performance and that they may be used for determining whether learners need particular services, can advance to higher levels of study, are ready to matriculate to university, qualify for employment opportunities, etc. Dixson and Worrell (2016, p. 156) add that “summative assessments are almost always graded, are typically less frequent [than formative assessments], and occur at the end of segments of instruction.” These authors continue that formative assessment, in contrast, “encompasses a whole host of tools that provide feedback to teachers or students to help students learn more effectively” (p. 154). Information from formative assessments may be used by teachers as they plan subsequent learning objectives and determine whether learners are prepared to move on to new material (Sadler 1989; Shepard 2006). In addition, formative assessments provide information to students, administrators, and other stakeholders that can be useful for supporting continued learning (Torrance and Pryor 1998).

The summative-formative dichotomy offers a quick and perhaps appealing categorization of assessment possibilities, one that has been extended in the conceptual proposals of Black and Wiliam (1998). These authors put forward the now well-known argument that the relation between assessment activity and learning can be construed as assessment of learning, in which learning is taken as a product and as an object of assessment, or as assessment for learning, wherein the relevance of assessment to supporting teaching and learning is highlighted. Important differences in how assessment is approached in these orientations include who develops the assessment instruments (i.e., testing professionals or classroom teachers); the conditions under which the assessment is administered (e.g., whether it is timed or is a part of regular classroom activity and whether it is done individually or in groups); whether the assessment is administered by a specialist, designated proctor, or regular teacher; and the kinds of tasks or items that characterize the assessment and how similar these are to those employed in other classroom activities (for discussion, see Black and Wiliam 2009; Moss 2003).

There is little room to doubt the value of these conceptualizations of assessment in helping to better understand assessment as part of a process of teaching and learning and the centrality of classroom teachers in carrying out assessments that support educational objectives. Nonetheless, the extent to which these dichotomies capture all that may be considered assessment is less clear. For instance, Dixson and Worrell (2016) suggest that the primary difference between summative and formative assessment concerns what is done with the results of the assessment rather than with the specifics of the assessment tasks, instruments, or conditions per se. To this, we would add that assessment, whether it is conceived as part of teaching and learning or as simply measuring its outcomes, tends to be marked either implicitly or explicitly by the following: a “backward-looking” view of development that is concerned with learner’s knowledge and abilities as they have formed up to the present point in time; an interest in determining what learners know or can do independently, that is, without support from others or use of resources; and, finally, conviction that the resultant information provides an adequate basis for predicting a learner’s likely future.

The present chapter is concerned with dynamic assessment (DA), a framework in which teaching is integrated as part of assessment procedures for the purpose of ascertaining how responsive learners are to a short-term intervention (Haywood and Lidz 2007). The degree of improvement learners manifest, as well as the source of difficulties they experience, offers a diagnosis of development that includes learner’s current abilities and those that are in the process of emerging (Poehner 2008; Sternberg and Grigorenko 2002). DA shares with some of the above proposals a commitment to rendering assessment activity more relevant to the goals of second language (L2) teaching and learning. However, DA begins from a very different premise, namely, that assessing and teaching are not discrete activities but exist in relation to one another, together forming a coherent activity of revealing and promoting learner development. Most importantly, the future in DA is regarded not as something to predict on the basis of past development but rather it is glimpsed in the present through learner engagement and responsiveness in cooperative activity. We begin with a brief overview of the theoretical origins of DA in the writings of Russian psychologist L. S. Vygotsky (1987). After identifying important concepts and principles of DA, we turn to two recent projects that together exemplify applications of DA in the L2 field and also extend existing DA research in new directions.

Background

DA emerges from a specific feature of Vygotskian theory, the zone of proximal development (ZPD). As a dialectical thinker in the tradition of Marx (see Ratner and Silva 2017), Vygotsky approached the world as comprised of relations and processes rather than discrete objects in a state of relative stability. Among other things, this means that it is difficult to extract a single element of his broader theory and not discuss it in relation to his other proposals and discoveries. Be that as it may, full discussion of Vygotsky’s sociocultural theory and its contributions to the L2 field is well beyond the scope of the present chapter. Interested readers are referred to the work of Lantolf and Poehner (2014, 2018). For our purposes, what is most important to appreciate is that the ZPD (and therefore DA) is reflective of Vygotsky’s view of human consciousness and its development. For him, human consciousness is the transformation of biologically endowed abilities through cultural means. The use of sign systems in particular allows individuals to gain control over psychological functions such as attention, perception, and memory and to begin to think and subsequently to act in particular ways. Vygotsky referred to the cultural means through which we gain this control as mediation, and he held that historically determined forms of mediation become available to us through the societies in which we live. These include counting systems as well as language, literacy, and conceptual knowledge about the world. Such affordances, in conjunction with social interaction with caregivers and peers – and in some societies teachers – provide experiences that mediate the development of new ways of thinking.

Central to this model of human psychology is the premise that the mind is not coterminous with the brain. As Vygotsky (1987) explained, while the brain is an organ inside the skull, the human mind comprises social and cultural forms of mediation that initially exist outside physical persons but that are appropriated by individuals over the course of development. He specified intra-psychological activity as that which occurs when an individual relies upon mediation that he/she has already internalized or appropriated. Intra-psychological activity pertains to individuals completing tasks independently and which is conventionally regarded as the totality of human psychological functioning. Vygotsky (1978) argued, however, that the intra-psychological activity captures only a portion of human abilities and that it in fact develops from the inter-psychological activity , that is, psychological functioning that is shared among individuals. Here actions of remembering, perceiving, reasoning, problem-solving, and so on are carried out dialogically. Experiments Vygotsky and colleagues conducted with children, learners with special needs, and individuals who suffered brain damage, among other populations, led them to conclude that what is possible at one point in time only through cooperation with others (i.e., inter-psychologically) can later become part of a person’s intra-psychological repertoire as he/she internalizes the relevant forms of thinking (Vygotsky 1978). In distinguishing the range of abilities that have already fully developed from those still in the process of forming, Vygotsky (1978) employed the terms zone of actual development and zone of proximal development. In contemplating the implications of these insights for education, Vygotsky (1998) observed that because conventional assessments – and here he did not distinguish between those intended to contribute to teaching and learning or simply to measure their outcome – focus exclusively upon learner’s independent performance of tasks, they can only hope to capture a part of development. Vygotsky (1998, p. 200) states his position as follows:

While some processes of development have already borne fruit and concluded their cycles, other processes are only at the stage of maturation. A genuine diagnosis of development must be able to catch not only concluded cycles of development, not only the fruits, but also those processes that are in the period of maturation. Like a gardener who in appraising species for yield would proceed incorrectly if he considered only the ripe fruit in the orchard and did not know how to evaluate the condition of the trees that had not yet produced mature fruit, the psychologist who is limited to ascertaining what has matured, leaving what is maturing aside, will never be able to obtain any kind of true and complete representation of the internal state of the whole development and, consequently, will not be able to make the transition from symptomatic to clinical diagnosis.

On this basis, he concludes that “determining the actual level of development not only does not cover the whole picture of development, but very frequently encompasses only an insignificant part of it” (ibid.).

The importance of the ZPD for education, Vygotsky reasoned, is that “learning which is oriented toward developmental levels that have already been reached is ineffective from the viewpoint of a child’s overall development… [whereas] the notion of the zone of proximal development enables us to propound a new formula, namely that the only ‘good learning’ is that which is in advance of development” (Vygotsky 1978, p. 89). For assessments to take account of the ZPD, Vygotsky advocated moving beyond observation of learner independent functioning in order to engage cooperatively with learners, offering hints, prompts, leading questions, and other forms of feedback as learners experience difficulties.

In the decades since Vygotsky’s death, a number of formalized procedures have been elaborated that are collectively referred to as dynamic assessment. Sternberg and Grigorenko (2002) noted that DA procedures may be differentiated according to whether mediation occurs throughout the administration of an assessment (which they refer to as a “cake” format) or if it is delivered during a separate session that occurs between a conventional pre- and posttest (a format they dub “sandwich”). Lantolf and Poehner (2004) observed that some approaches to DA opt for open-ended dialogue in which mediation is negotiated between the assessor, or mediator, and the learner, while others advocate scripting precise mediating moves in advance of the procedure and delivering them to learners in a standardized manner as needed. They term the former approach “interactionist” and the latter “interventionist.” In both, mediation begins implicitly through behaviors such as pausing, asking whether the learner is satisfied with his/her response, and suggesting that the learner reattempt the task; mediation becomes more explicit (offering clues concerning the nature of a problem, examples, and models to guide the learner toward correcting the problem and explanations of underlying concepts and principles) until either the learner overcomes the difficulty or the mediator reveals and explains the solution. As Poehner (2008) explains, the aim of providing mediation during DA is not simply to help learners improve their score or grade but to arrive at a diagnosis of development, one that comprises what learners are able to successfully do independently, their current struggles, and the extent to which new abilities are in the process of forming.

DA has been widely implemented in cognitive education programs with young children (e.g., Tzuriel 2011) and in remedial education interventions for learners with special needs (Feuerstein et al. 2010, 2015). It has also increasingly been pursued with a more general population of learners in various academic disciplines, including language. According to Poehner (2018), much L2 DA research has involved collaborations with classroom teachers, whose primary interest is not learners’ overall language proficiency but their mastery of particular language features that have been the focus of instruction. A theory of language no doubt at least implicitly informs the curriculum followed by the teachers, but the mediational targets in such DA studies have typically included specific areas of morpho-syntax (e.g., substantive-modifier accord, verbal tense, aspect and mood, interrogative constructions, etc.). More recently, DA researchers have also begun to explore more general domains of L2 abilities. For instance, Poehner et al. (2015) report the use of a dynamic procedure to diagnose learner L2 listening and reading comprehension. In their work, multiple-choice items were written that targeted specific sub-areas within the general comprehension constructs, including lexical knowledge, morpho-syntax, discourse level grammar, and cultural knowledge. Tracking how much mediation learners required for particular test items allowed the researchers to ascertain which dimensions of listening and reading comprehension were most challenging for individuals. Levi (2012) implemented DA in the context of the English oral proficiency interview component of the Israeli national matriculation exam. Employing the rubric used by raters to mark examinee proficiency, Levi engaged learners in reflection on their own oral performances as well as those of other students, mediating their consideration of specific areas of proficiency in need of further development.

It is worth noting that both the Poehner et al. (2015) and Levi (2012) studies occurred in more formal language testing situations, which represent another important extension of L2 DA research. Earlier work had focused almost exclusively on instructional settings, occurring in either one-to-one tutoring situations or in classrooms. To be sure, each of these contexts is crucial for DA to realize its potential in L2 education. They also bring DA into contact with existing practices that might be identifiable as either formative or summative assessment (or assessment for learning or assessment of learning). As should be clear, however, the commitment to systematically integrating mediation as part of the procedure is definitive of DA regardless of the precise context in which it occurs. This is because DA holds that careful attention to the mediation learners require now provides a window into their potential independent functioning in the future. To further clarify this point, we now consider two recent L2 DA projects. The first follows the work of Poehner et al. (2015) in seeking a means to implement DA on a large scale by replacing a human mediator with support that is automatically generated during a computerized test. Embedding mediation in a computer program so that it is delivered to learners as needed is known as computerized DA or C-DA. The second project builds upon earlier classroom-based DA studies but extends it to academic writing, with an aim of implementing an individualized intervention program to identify weaknesses in learner writing and begin to improve them. The reader will also note that the former project was carried out with English-speaking learners of L2 Chinese. However, as we have attempted to clarify, DA is a framework for conceptualizing assessment that is not limited in its scope to any particular language or indeed to any specific learning domain or population. Our purpose in reporting both of the following projects is to explicate how DA was realized in two different contexts (namely, a formal testing procedure and an instructional academic writing program) and through two different approaches (a standardized, computer-based administration in the one case and an interactive, one-to-one format in the other). Together, they not only showcase the range of possibilities afforded by DA, but they also provide models that researchers and practitioners may draw upon as they consider the relevance of DA to co-constructing a future with English language learners.

Computerized Dynamic Assessment (C-DA): Diagnosing Learner L2 Pragmatic Knowledge

Qin (2018) undertook to extend the use of C-DA in the L2 field to the domain of pragmatics. She notes that an advantage of the computerized format is that it addresses the practical problem of “scalability” of DA procedures; that is, how realistic is DA in situations that require diagnosing the abilities of large numbers of learners? A computerized approach, as designed by Poehner et al. (2015), established that while computerizing mediation does not allow for the same degree of flexibility and negotiation as one-to-one interaction between a mediator and learner, it is possible to offer mediation to large numbers of learners simultaneously. While that study was concerned with listening and reading comprehension, Qin (2018) sought to explore the feasibility of diagnosing pragmatic knowledge of language use through C-DA. As she explains, knowledge of pragmatics has long been regarded as an integral component of language proficiency (e.g., Bachman and Palmer 1996), specifically underscoring learner awareness of contextual appropriateness and politeness in target language use. Following the Vygotskian premise that providing mediation is required for a full diagnosis of development, Qin’s C-DA design attempts to raise learner awareness of particular pragmatic features during the assessment in order to determine learner’s current knowledge and the extent to which their understandings were in a process of developing.

Readers are referred to Qin (2018) for full details of the project. Our discussion here is limited to the design of the C-DA procedure, with selected examples to illustrate the insights the procedure afforded into learner development. The specific pragmatic feature that is brought into focus in this discussion is learner comprehension of conversational implicature (i.e., indirect meaning) in L2 Chinese. The data come from two learners (Andrew and Jane, both pseudonyms) recruited from an elementary-level Chinese course at a private North American university.

C-DA Design

The C-DA instrument assesses learners’ abilities to comprehend implicatures of oral stimuli. The test was completed in a single session lasting approximately 1 h. Implicature comprehension was operationalized as the ability to understand the indirect, implied meaning of a speaker’s utterance in relation to its literal meaning and context. Consider the following example in English. A man asks his wife whether a shirt with a bright multicolored pattern “looks good” for their evening out. His wife responds that she really likes his blue shirt. The literal meaning is indeed that she finds the latter shirt an attractive option. The implicature – that is, the indirect, implied meaning – is that she does not like the multicolored shirt. In the C-DA test, two types of implicatures were included: indirect refusals and indirect opinions, the latter of which are generally more challenging for learners according to the L2 pragmatics research literature (Taguchi et al. 2013). The instrument applied a sandwich DA format that included four parts: (1) a pretest evaluating current abilities to understand implied meanings in Chinese, (2) an intervention session involving predesigned mediation intended to assist test-takers in improving their skills, (3) a “near-transfer” posttest that employed different indirect refusals from those on the pretest, and (4) a “far-transfer” posttest that introduced more complex items with indirect opinions. The two posttests allowed for investigation of the distance between what learners were able to do alone prior to mediation (i.e., their pretest performance) and what became possible after mediation. In Vygotskian terms, examining this distance or change brought to light learners’ ZPD for implicature comprehension. Figure 1 displays the C-DA procedure.

Fig. 1

Organization of C-DA procedure

During the pre- and posttests, test-takers were first presented with background contextual information concerning the item they were about to hear. Background information was provided in Chinese for some items but English for others, depending upon the complexity of the information to be shared. A Chinese sentence was then played containing the target utterance. After listening to the audio file only once, a statement corresponding to the target utterance was presented on the computer screen, and test-takers were asked to use a slide ruler to evaluate the extent to which they agreed or disagreed with the statement. The slide ruler is on a scale of +50 (strongly agree) to −50 (strongly disagree). Using a slide-ruler format allowed for a more nuanced tracking of changes in learner confidence in their implicature comprehension between the pre- and posttests. For instance, a significant increase toward the outer ends of the continuum/slide ruler after mediation would suggest increased certainty/confidence in the response, one indication that the learner had benefited from the intervention stage of the assessment.

Regarding the intervention session, the amount of mediation provided was determined by learner performance on the pretest. Test-takers only went through mediation for the questions they did not answer correctly in the pretest. The mediation was model-informed by pragmatics research and was intended to function as a metacognitive tool. This means that test-takers did not simply “acquire” implicature or Chinese pragmatics, but they were provided with a metacognitive tool (mediational means) to help them (1) notice and (2) pay attention to both the literal meaning and implied meaning of utterances. This mediation approach was rooted in speech act theory (Searle 1975). In this model, indirect speech acts require the hearer to interpret more than what the speaker actually says (locutionary force) by way of relying on other resources, such as shared background and linguistic and nonlinguistic cues, to understand the intended, and sometimes indirect, meaning – that is, the utterance’s illocutionary force. In this project, the intervention stage employed a multiple-choice format, presenting learners with the items they had missed on the pretest and offering a selection of five possible interpretations of what the speaker had intended to express (i.e., one correct choice that indicates the speaker’s real intention and four contextually related distractors). If a test-taker chose the correct option on the first attempt, no prompt was presented and the computerized program advanced to the next item from the pretest that he/she had missed. If the test-taker selected an incorrect option, a mediational prompt was automatically provided by the program. Mediational prompts were standardized (i.e., the same for all learners) and delivered in an order of most implicit to most explicit. A total of four prompts was available for each test item. The first prompt simply allowed test-takers to try again, the second prompt repeated the key content of the utterance (i.e., the locutionary act), the third prompt asked the intent of the utterance (i.e., illocutionary act), and the final prompt provided the correct answer.

The C-DA instrument automatically generates four numerical scores: a pretest score, near-transfer score, far-transfer score, and a learning potential score (LPS). The first three scores are implicature comprehension scores on the three tests. The pretest score represents independent performance, the near-transfer score represents mediated performance (i.e., provoked by mediation during the preceding intervention stage), and the far-transfer score represents learner success in transferring their emerging understanding and knowledge to more difficult items. These three scores were calculated based on corresponding “threshold ratings” produced by native speakers of Chinese (i.e., the rating that Chinese native speakers had selected for a given item using the slider format). Specifically, for each item, one point was awarded if the learner’s response was equal to or greater than the native speaker threshold rating, while no points were awarded if the response was below the threshold. The LPS serves to capture learner responsiveness to mediation during the C-DA procedure by taking account of the degree of change to learner performance between the pretest and near-transfer posttest, relative to the maximum possible score on the test (Kozulin and Garb 2002; Poehner and Lantolf 2013; Zhang and van Compernolle 2016). The following formula for calculating LPS, proposed by Kozulin and Garb (2002), was adopted in this project:

$$ \mathrm{LPS}=\frac{\left({S}_{\mathrm{post}}-{S}_{\mathrm{pre}}\right)}{S_{\mathrm{max}}}+\frac{S_{\mathrm{post}}}{S_{\mathrm{max}}}=\frac{2{S}_{\mathrm{post}}-{S}_{\mathrm{pre}}}{S_{\mathrm{max}}} $$

ZPD and Predicting Future L2 Development

As an illustration of how differences in learner responsiveness to mediation during C-DA can capture emerging abilities, we briefly consider Andrew and Jane as two contrasting “cases.” These learners had similar pretest scores but responded to the computerized mediation differently during the intervention stage and earned dramatically different near-transfer and far-transfer posttest scores. Their performances are summarized in Table 1. Interestingly, Andrew produced a higher score on the near-transfer posttest (score = 9) than the pretest (score = 5), and his far-transfer posttest (score = 5) was the same as his pretest score despite the increased level of difficulty of the far-transfer items. Taken together, this pattern suggests that Andrew had benefited from the intervention stage and that his understanding of implicatures in L2 Chinese was indeed emerging in a way that was not found with Jane. Specifically, Jane actually produced lower comprehension scores on the near-transfer posttest (score = 4) and far-transfer posttest (score = 2) than she had on the initial pretest (score = 5). In terms of their respective LPS, Andrew was one of the few test-takers in Qin’s study who produced high LPSs, while Jane’s near-transfer score was lower than her pretest score, so the LPS formula created by Kozulin and Garb (2002) did not apply to her.

Table 1 Andrew and Jane’s C-DA scores

In addition to C-DA scores, the two test-takers’ response processes during the procedure were analyzed for insights into their engagement with the test items and available mediation. Andrew and Jane’s interactions with C-DA were recorded by a two-way video screen recording software, ScreenFlow. ScreenFlow simultaneously captured the user screen (i.e., what Andrew and Jane saw at particular moments during the procedure) and recorded the test-takers themselves via the computer’s built-in camera and microphone. Figure 2 provides an example of ScreenFlow’s interface during Andrew’s session.

Fig. 2

Example of ScreenFlow screen capture and video

The data collected through this approach allowed for multimodal discourse analysis (O’Halloran 2011) that included the test-taker’s use of gestures, facial expressions, pauses, and other behaviors as they worked through specific C-DA items. For instance, Excerpt (1), which occurred when Andrew encountered the first item in the intervention session, documents the process through which he demonstrated comprehension of the aural text “Tomorrow is Xiaomei’s boyfriend’s birthday. Wenzhong suggests Xiaomei should take her boyfriend to the new American restaurant. Xiaomei says, ‘My boyfriend only loves Chinese food.’” The ScreenFlow data permit an analysis of learner moment-to-moment behaviors during the procedure, including the responses they select, the time that passes before they make a selection, and any verbal and nonverbal actions.

Excerpt 1

Line	Time stamp	Description
1	12:37–12:51	Audios play
2	12:51–12:53	Andrew clicks “Show the Question” on the screen
		A multiple-choice question appears on the screen
3	12:53–12:59	Andrew moves the cursor between different options
4	12:59–13:00	Andrew clicks the literal meaning option and then clicks “Next”
		The 1st prompt, “That is not right. Try again,” is shown on the screen
5	13:01	Andrew looks at the 1st prompt
6	13:02–13:04	Andrew frowns
7	13:04–13:07	Andrew moves the cursor between different options
8	13:08	Andrew clicks “OK” button in the prompt
9	13:08–13:11	Audio starts to play again
		Andrew clicks the indirect meaning option 2 s after the audio starts
		Andrew clicks “Next”
10	13:12	The prompt, “Correct,” is shown on the screen

In Excerpt (1), we see that after the audio was done, Andrew used 6 s to think about the answer while moving the cursor between different options (line 3) before selecting the literal meaning of the utterance (i.e., that Xiaomei’s boyfriend only loves Chinese food) in line 4. After the first prompt was shown on the screen (line 4), Andrew moved the cursor between different options while thinking (line 7) and clicked the “OK” button in the prompt (line 8) to replay the audio file. After listening to the recording for only 2 s, Andrew chose the indirect meaning option (i.e., that Xiaomei does not accept Wenzhong’s suggestion to go to the American restaurant) and clicked “Next.”

This excerpt offers insight into Andrew’s emerging understanding of indirectness. Indeed, he used only two attempts to figure out the correct answers to both the first and second questions in the intervention session. After this initial stage, where he seemed to develop greater awareness of indirectness, Andrew only used one attempt for the rest of the questions, meaning that he only listened to the audios again and then selected the correct interpretation from the available choices without any further mediating prompts. We interpret this as evidence that Andrew had a ZPD for implicature comprehension: he was able to make progress through the mediational prompts and transfer the emerging abilities to the rest of the questions in the C-DA procedure. In contrast, there was no identifiable development in Jane’s performance. It seems that the number of attempts she used in the intervention session was closely related to her degree of certainty in the pretest (degree of certainty was reflected via slide-ruler results): for the pretest items about which she was more certain, she required fewer attempts to select the correct interpretation during the intervention; likewise, for those items where she was less certain of her interpretation, she made more attempts. Over the course of the intervention, there was no indication that Jane’s comprehension of implicatures was changing. Moreover, Jane’s responses during an interview following the procedure revealed that she struggled even to determine the literal meaning of many of the test items.

In summary, Qin’s (2018) project evidences the diagnostic potential of C-DA with L2 learners. While Andrew and Jane earned the same score on a non-dynamic pretest, their responsiveness during the intervention stage and their performances on the posttests make it clear that it would be erroneous to conclude that their knowledge of Chinese implicature is the same. Thus, test-takers such as Andrew, who responded well and showed his preparedness to move on to more complex tasks and features of pragmatics, will have their needs met in different ways from learners such as Jane, who made no identifiable improvement during C-DA and who may require further instruction and practice to appropriately comprehend implicatures in Chinese. To be sure, the focus of Qin’s project on L2 Chinese means that attempts to extend this work to contexts involving English learners would require careful consideration of English pragmatics as a basis for defining relevant constructs that could be targeted by the procedure. As mentioned, our purpose in sharing these data is that they illustrate the potential for DA procedures to be computerized, allowing them to be more easily implemented with large numbers of learners than would be possible through individualized, one-to-one administrations. As Qin’s research reveals, by expanding the assessment procedure beyond learner history (i.e., their development up to the present moment as revealed through independent, unassisted test-taking) and taking account of their emerging abilities (according to the mediating support they required), it became possible to determine how to most appropriately meet the leaners where they are and begin to work together with them to construct a future.

DA and L2 Instructional Intervention: Promoting Learner Academic Writing

In contrast with Qin’s (2018) research involving computerized delivery of mediation during DA, Lu (in preparation) follows the DA tradition pioneered by Feuerstein (Feuerstein et al. 1979) and first adapted for use with L2 learners by Poehner (2008) of face-to-face DA sessions that are integrated with an individualized enrichment program targeting learner needs. Tzuriel (2011), discussing this approach in the context of cognitive education initiatives, characterizes this approach as first identifying learner abilities that have begun to develop and that need further intervention before learners can advance and then guiding learners through a planned sequence of activities in which the tasks and materials are designed to promote those abilities. Put simply, the mediation that occurs in DA serves to probe learner abilities, and the resultant diagnosis informs an enrichment program where the mediated activity continues across multiple sessions in order that the level of functioning learners reached through cooperation with the mediator during DA becomes a level they achieve independently (Poehner 2018). Lu’s (in preparation) project centers around a DA and enrichment program designed for use with adult English language learners in an academic writing program.

As Lu (in preparation) notes, while some L2 DA work has investigated writing (Rahimi et al. 2015; Shrestha and Coffin 2012), its application with English language learners at an advanced level, where both rhetorical and linguistic issues may become more complex, is an underexplored area. Moreover, Lu’s project follows the growing trend in writing assessment to employ integrated reading-writing tasks, which reflect a broadened understanding of the construct of writing for academic purposes (Cumming et al. 2005; Gebril 2009; Guo et al. 2013; Knoch and Sitajalabhorn 2013). According to this view, writing is inherently associated with learners’ ability in academic reading and involves producing “content responsible” (Leki and Carson 1997) texts by incorporating substantive content from source materials. Lu (in preparation) further points out that interaction and oral feedback are not entirely unfamiliar to the L2 writing domain. Indeed, among the various ways of evaluating students’ writing, responding to students’ texts through written or oral feedback has long been an important and ubiquitous classroom practice (e.g., Ferris and Hedgcock 2014; Hyland 2003; Hyland and Hyland 2006). Cumming and So (1996) consider that the intense, personalized, and goal-oriented interactions between teachers and students may embody the most powerful aspects of instruction on written composition. However, Lu also remarks that despite the value placed on providing oral feedback and interacting with multilingual writers about their texts, research has rarely explored how such one-to-one, face-to face interactions can be employed in a principled and systematic way to assess and assist students’ development in writing over time (Ferris and Hedgcock 2014; Severino and Cogie 2016; Williams 2004). Thus, the project aims to contribute to not only the L2 DA research literature but to scholarship in the area of academic writing more generally.

DA and Writing Intervention Design

Lu’s (in preparation) research was carried out in an intensive English program at a large US university. Six participants from level 4 (the highest level) of the academic literacy class in the program were recruited to participate in the study. At the time of the study, all the participants had obtained a bachelor’s degree in their home country, and, with the exception of one student, all planned to enroll in a graduate program in the university where this project occurred. None of the participants had received systematic training in English academic writing in their previous English learning experiences.

The DA and writing intervention program design included four stages: a pretest in which a conventional or non-dynamic assessment of learner writing was conducted followed by a DA of their writing, an enrichment program tailored to learner needs and abilities that were identified during DA, a posttest (again comprising a non-dynamic and DA of learner writing) that paralleled the pretest, and a delayed posttest, which was non-dynamic but that presented learners with a more complex task and that was intended to ascertain their success in transferring their abilities. A semi-structured interview was also administered with each participant at the beginning and the end of the study, respectively.

The pretest and posttest were parallel in terms of format and difficulty. Participants wrote an argumentative essay under independent and mediated conditions, respectively, in each test. As the assessment followed an integrated reading-writing design, the non-dynamic and DA procedures both required learners to first read two brief argumentative texts (approximately 300–350 words each) that take opposing perspectives on a shared topic. The participants were then asked to construct their own argumentative essay after reading the texts, taking a position on the theme of the readings and supporting their ideas with specific information from the articles as well as their own reasons or experiences. Students first completed this task in a non-dynamic manner, and the essays they produced were carefully analyzed by the researcher in order to determine potential problem areas to probe during the subsequent DA session. An analytic rubric reported in Weigle (2004) was also used to evaluate each essay according to four areas: content, organization, accuracy of language, and range and complexity of language.

During the DA sessions, this rubric was reviewed and explained to the learner, who was then instructed to read through his/her essay from the non-DA session and make any changes he/she thought were necessary without assistance from the mediator. Then mediator and learner jointly reviewed the essay. Rather than a scripted and standardized (i.e., interventionist) approach to mediation, the mediator engaged in an interactionist approach, using prompts, hints, suggestions, modeling, etc. to address the rhetorical and linguistic issues in the essay. The general DA principle of moving from implicit forms of mediation toward increasingly explicit forms was adhered to Poehner (2008). Immediately following this interaction, learners were asked to revise their essay. The initial (non-DA) version of the text and the revised version were compared by the research, along with recordings of the DA interaction, in order to determine abilities that appeared to be in each individual’s ZPD and would therefore be an appropriate target for sustained mediation in the enrichment program. Each learner then participated in a 5-week individualized enrichment program during which they met once a week with the mediator for a one-to-one tutoring activity. The rhetorical and linguistic aspects of writing that were targeted during the tutorials include unity and coherence, constructing arguments, documenting sources, pronoun reference and shifts, subject-verb agreement, etc. For example, when discussing how to use sources, learners were introduced to the concepts of summarizing, paraphrasing, and quoting, the common strategies to paraphrase, and the language that can be used to identify a source. Learners then completed exercises to identify correct and incorrect examples of using and documenting an original source, and they also drafted their own paraphrases of short texts for additional practice. The posttest was administered at the end of the enrichment program. A month after the posttest, a transfer test (delayed posttest) was given where instead of two reading passages, three short passages introducing different opinions on the same issue were provided. The transfer test aimed to assess the participants’ ability to synthesize information from more complex sources. Two raters with experience in teaching and assessing L2 writing were recruited to score all the essays produced by the participants during the tests. Through this approach, comparisons between learner pretest and posttest performances allowed for identification of development over time, while analysis of interactions revealed difficulties and emerging understandings as these became apparent through dialogue.

Full details of this project are reported by Lu (in preparation). In the following, we offer two instances of mediator-learner interactions extracted from the DA sessions in order to illustrate how mediation was dialogically negotiated and the insights this process afforded into learner abilities.

Diagnosing Learner Development Through L2 DA Interactions

Excerpt (1) is taken from the first DA session, for which learners read two passages on whether homework does more harm or good for young learners. The learner, Ryan (a pseudonym), took the view that homework increases the achievement of young learners. His essay had a clear structure with an introduction, two body paragraphs, and a conclusion. The introduction and conclusion, however, were both very brief, and the thesis statement at the beginning of the text did not relate well to the ideas developed in the body of the essay. In the excerpt, the mediator (M) brought Ryan’s (R) attention to the beginning and ending of his essay:

Excerpt 1

• 1 M: But then in the introduction and conclusion

• 2 R: It’s too short

• 3 M: Exactly. [You already know that-

• 4 R: [I know, I know (laugh)

• 5 M: Then why didn’t you (.) try to make them longer?

• 6 R: Because I want to (.) try to (.) actually I want to try to write a body paragraph first and then-

• 7 M: Umm

• 8 R: Back to correct the first- um first paragraph and then (.)

• 9 M: That’s right. That’s the correct strategy

• 10 R: Yeah, but I just figured out that I have no time. It’s my structure, so-

• 11 M: Oh, yeah, I know. Usually it’s a good idea (.) we try to develop the body first and then we come back to the introduction [and finally conclusion

• 12 R: [Yeah

• 13 M: Then your problem is that you didn’t have enough time?

• 14 R: Yeah.

In this exchange, M’s mention of the introduction and conclusion immediately prompts R to acknowledge that they are too short (turn 2). While this in itself does not reveal his understanding of the importance of a thesis and how it is developed in the remainder of the essay, it does show his awareness that neither the introduction nor the conclusion conformed to the expectations for academic writing. With regard to the ZPD, this may be interpreted as indicating an existing awareness of these writing conventions even though that was not manifested in the writing he produced.

The lines of interaction that follow do not shed further light on the depth of R’s understanding of expectations for introductions, thesis statements, and conclusions, but they do reveal the reason he was unable to use his knowledge to produce an appropriate essay. With a further mediating prompt from M (turn 5), R explains that he actually wrote the body of the essay first before trying to work more on the introduction (turns 6 and 8). In turn 10, R acknowledges that he did not have enough time at the test to develop the beginning and ending more adequately. M confirms that it is actually a good strategy to prepare the body of an essay first before refining the beginning and ending of the text (turns 9 and 11). This interaction not only reveals that R was in fact well aware of one of the problems that M was about to discuss with him, but more importantly, it brings to light R’s strategy of simultaneously composing different sections of his essay. Again, with reference to the ZPD, R’s responsiveness during the exchange leads to a very different picture of his current knowledge than would be the case if, for instance, he had been completely unaware that there were problems with his introduction and conclusion.

As the DA interaction proceeded, M and R turned their attention to each paragraph of the essay in turn. The following is the second paragraph of the essay, and it became a major focus of the DA interaction. Note that it has been unaltered, and so spelling and grammatical errors are those produced by the learner.

Second, homework probably is a big assignment, such as a team assignment that students have to finish their homework with their classmates or parents. Kind of this homework can help students to practice how to make team work succeed. In the Potter and Bullitt’ article, they showed that assignment may not have a quality of benefit for students, even it may cause children to have mental health problems. However, no studies have been founded that homework cause students weakness. On the contrary, a big assignment may help students to learn how to adapt environment and pressure during the doing homework. This skill is needed for everyone in the future. Thus, a big assignment can let students learn skills that schools have never could.

In this paragraph, several distinct ideas about the benefits of homework are included, and so in addition to linguistic errors, a major issue M raised concerned the paragraph’s lack of unity and focus. In Excerpt (2), M begins by exploring whether R had planned a main point that he wished the paragraph to convey.

Excerpt 2

• 1 M: And for the second main point (.) I feel there might be a little bit problem here. What is your main point of the second paragraph- body paragraph, I mean?

• 2 R: Um when I- when I was writing about the second point, I- I just want to cite (.) the article from the Passage B- Passage B, yeah, second article. So I just thinking, I don’t know how to::

• 3 M: How to cite?

• 4 R: Yeah, so I just think um, randomly, just like the point, and just write this

• 5 M: Okay. You- you mentioned some idea from Passage B, right? You mentioned mental health problem

• 6 R: Yeah

• 7 M: But then usually we don’t cite for the sake of citing

• 8 R: Oh really? Okay

• 9 M: We cite it to (.) to support a point, right? So what exactly is like the topic sentence in this paragraph? Or what is your main idea?

• 10 R: Um I think (2.0) I want to (.) an argument that (3.0) homework still helps students to success but- um, also they also can learn other skills like teamwork or::

• 11 M: Uh-huh

• 12 R: Actually I want to mention they work close with their friends or parents to build this relationship, yeah, because the teamwork, the big assignment

• 13 M: Umm, yeah, I know. So (.) I just feel like this body paragraph is not focused enough

• 14 R: Yeah, I know, I know

• 15 M: You talked about separate- several separate different ideas

• 16 R: Yeah

In turn 2, R did not respond directly to M’s question about the focus of the paragraph but explained that he tried to cite some counterargument from Passage B, which however was not explicitly connected to the rest of the ideas in the paragraph. That is, the authors of Passage B argued that homework brings about mental health issues among children, but R failed to logically connect this information to his own position that he was developing in the paragraph. Beginning in turn 2, it becomes clear through R’s responsiveness to M that an underlying difficulty is that R did not fully understand the purpose of referencing sources in academic writing. In turns 7 and 9, M explains that a source is generally cited in academic writing to support the point the author wishes to make citing. After M inquires again what R intended the main point of the paragraph to be, he states that homework helps children to learn the skill of teamwork (turns 10 and 12). Thus, through a series of mediating moves, a clearer understanding emerges of both R’s intended meanings and the struggles he faced that are apparent from the writing he produced.

The interaction continues as M endeavored not to revise the essay but to guide R in doing so.

• 17 M: So probably, then how can you maybe (.) modify this part (.) just this paragraph? How can you make your argument stronger and also make it more focused?

• 18 R: Um I would like to (5.0) uh, remove the counterargument, and I put some example, and (4.0)

• 19 M: Yeah

• 20 R: (5.0) And just remove the health

• 21 M: Uh-huh

• 22 R: Mental health and to connect the skill, to learn the skill by example

• 23 M: By giving some examples. Okay, I think that’s good. You can try to remove the counterargument from this paragraph, but then after that you still need to have one single clear idea in this paragraph and you need to decide (.) what the main point of this paragraph is.

In turn 17, M prompts R to modify the paragraph on his own. R then responds that he could remove the counterargument from Passage B and add some examples to support the main idea of the paragraph (turns 18, 20, and 22).

From this excerpt, we see a process of mediation revealing the learner’s current and emerging understandings of academic writing conventions. Specifically, it appears that R did not have a clear idea of what a unified paragraph should look like. He was, however, aware of the concept of counterargument, and he attempted to use some information from the reading, although was unable to effectively integrate the source material into his own argument. Through further dialoguing with M, R managed to reach a solution as to how his paragraph might be revised. At the conclusion of the DA session, R revised his essay, including the second paragraph:

Second, homework also can help students to develop other skill such as team work. Homework may as a form of project that they have to finish it with their classmates or parents. For instance, a team work that they have to breed a pet in each group, they could learn how to plan their schedule, assign each member’s tasks. This kind of work may not be easy to finish, so they can experience this part and make it better and finish it as soon as possible. Furthermore, team work is a major skill in the future. To work with other co-workers is usual thing. Thus, the benefit of homework like a big project is students can learn team-work skill

As can be seen, R’s second draft of the paragraph still included various errors, but it was better developed. It included an example and was unified around the central idea that homework can help students develop teamwork skills.

In sum, R’s responsiveness during DA – specifically, the extent to which he was able to improve his integration of details from the readings and yet employ these details in a manner that did not sacrifice the paragraph’s overall coherence – indicated that careful consideration and synthesis of arguments and counterarguments was an area of his writing that could benefit from immediate intervention. Put another way, that ability was in his ZPD – it was ripening but had not yet fully developed. Had R failed to make appropriate revisions following his interaction with the mediator, the diagnosis would have concluded that he had not yet begun to develop the understandings relevant to addressing this aspect of academic writing. Conversely, if this issue had not emerged at all during DA, it would indicate that he had already fully developed that ability. As it was, this became a focus of the instructional intervention program that M conducted with R.

Conclusion

DA is still a relatively new innovation in the field of L2 education. As such, numerous questions remain concerning, for instance, the following: factors to consider when selecting an interactionist or interventionist approach to mediation or perhaps designing yet another approach, advantages and challenges to employing DA in computerized formats, the quality of the diagnostic information obtained about learners and how this information may help teachers to best meet learner needs, and possibilities for implementing DA across varied configurations of groups (e.g., pairs of learners, groups of 3 or 4, an entire class, etc.). These are undoubtedly important issues that need to be addressed through further research. The projects reported here offer only a beginning to exploring them. Nevertheless, the inherent optimism in DA that shakes us from simply accepting a learner’s manifest repertoire of abilities as the totality of his capabilities and that insists upon the potential for development among all individuals through appropriate forms of mediation is what compels us to pursue DA as a means of co-constructing futures with English language learners.

Cross-References

• Feedback for Enhanced English Language Learning

• Mediating L2 Learning Through Classroom Interaction

• Using Assessment to Enhance Learning in English Language Education

Transcription Conventions

Symbol	Meaning
(.)	A short pause, less than 0.5 s
(2.0)	A timed pause, e.g., 2 s
[	Place where overlapping talk starts
-	Truncated word or unfinished sentence
::	Lengthening of a word or sound
( )	Researcher comments