Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Background: The Broader Context for Change

1.1 Assessment Reforms in the Region

Assessment reform has been at the heart of education reforms in major systems in the Asia-Pacific region since the turn of the century (Hogan et al. 2009; Goh and Matthews 2012, this volume; Mok et al. 2003; Ng 2010; Pitiyanuwat and Pitiyanuwat 2012, this volume; Yu 2012a, this volume). Such reform movements are driven by two motives: accountability and improvement. National systems are accountable to taxpayers, teachers and parents on public money, and there is increasing public scrutiny of government expenses in developed countries as the internet makes it easier to access information compared with the last century. In parallel, in facing challenges brought about by globalization and the rapid speed at which knowledge is created, continuous improvement is seen by many governments as the only way to stay on a par with the rest of the world (Sahlberg 2006). Research (Hogan et al. 2009; Goh and Matthews 2012, this volume; Mok et al. 2003; Ng 2010; Pitiyanuwat and Pitiyanuwat 2012, this volume; Yu 2012a, this volume) has found that reforming the education system and repositioning assessment as a means to build up capacity for continuous self-improvement is common amongst systems in the Asia-Pacific region.

In the wave of education reforms, assessment reform has often been used by systems in the region as a means to drive the changes. For instance, under the vision of ‘Thinking Schools, Learning Nation’, in 1997, the Singapore government put forward new assessment policies at school level and changed assessment from summative to formative in the implementation of the School Excellence Model and the School Awards Model (Ng 2010). In conjunction with the ‘Teach Less, Learn More’ policy of Singapore, these models of school excellence emphasized the development in students of the capacity for self-directed learning, deep conceptual understanding, sharing of knowledge and knowledge construction.

Similar to Singapore, assessment for learning is high on the agenda of education reform in Hong Kong (Berry and Adamson 2011; Lee 2012, this volume). Since 2001, the Hong Kong government has launched a large-scale curriculum reform which emphasized the shift from knowledge transmission to building the students’ capacity of learning how to learn (Cheung and Wong 2011; Curriculum Development Council 2001; Lee 2012, this volume). Governmental determination in reforming the culture and practice of assessment to promote independent learning and thinking in Hong Kong is unambiguously identified in the review of public examination system in Hong Kong report or the ROPES report (Hong Kong Baptist University and Hong Kong Examinations Authority 1998). There are clear directives from the government that ‘Assessment is an integral part of the curriculum, pedagogy and assessment cycle. It involves collecting evidence about student learning, interpreting information and making judgements about students’ performance with a view to providing feedback …’ (Curriculum Development Council 2009, Booklet 4, p. 1).

1.2 Commonalities of Assessment Reforms in the Asia-Pacific Region

Mok et al. (2003) found in their review of education reforms in eight systems in the Asia-Pacific region (Australia, Hong Kong, Japan, Korea, New Zealand, Papua New Guinea, Singapore and Taiwan) that assessment reform was not only core to the education reforms in these systems, but also that these reforms shared many commonalities across the systems:

  • Purpose of assessment: changing from selecting the best candidate for further education as the sole purpose of assessment to serving multiple purposes including the provision of feedback to support learning

  • Philosophy of assessment: changing from summative assessment to both formative and summative assessment being used, with a strong emphasis on the relationship between teaching, learning and assessment

  • Directive of assessment: changing from evaluative to learning-directed

  • Methods of assessment: changing from paper and pencil to multiple formats and methods including electronic, performance, portfolio and project-based

  • Analysis of assessment data: changing from traditional test theory to Rasch-based or item response theory

  • Drivers of assessment: changing from teacher-initiated to self- and peer-initiated assessment

  • Frame of reference: changing from norm-referenced to criteria- and standards-referenced

  • Domains of coverage: changing from assessing cognitive domains only to assessing multiple domains including cognitive, affective and social domains in whole-person development of students

1.3 Assessment as Learning Reform: Self-Directed Learning

The use of assessment reform as a means to achieve the national education aim of building sustainable capacity for self-directed learning is the most prevalent feature in the assessment reforms in the region. There is an increasing emphasis on preparing students who are ‘trainable rather than trained’ and who are capable of self-evaluation and of continuous learning throughout life (Maclean 2010, p. iv). The strong intention to build up capacity for self-directed learning through education reform in developed countries in the region is reflected in their respective aims of education. For instance, Hong Kong education aims to ‘enable everyone to develop their full and individual potential … so that each individual is ready for continuous self-learning…’ (Education Commission 2000); Japan education aims ‘to raise the ability of self-education’ and ‘the ability to shape their own lives’ (Abiko 2011, p. 359; Japan Ministry of Education 2000); Korean education aims ‘to raise a self-reliant individual equipped with a distinct sense of independence, a creative individual with a sense of originality, and an ethical individual with some sound morality and democratic ideology’ (Korean Ministry of Education 2000); Singapore education is to ‘develop self-directed learners who take responsibility for their own learning, and who question, reflect and persevere in their pursuit of learning’ (Singapore MOE 2010); for Taiwan, ‘Education and culture shall aim at the development among the citizens of the national spirit, the spirit of self-government…’ (Taiwan Ministry of Education 2001) and ‘to encourage people’s planning for self-directed learning based on theory of career development’ (Taiwan Ministry of Education 2011); and Thailand’s education system aims ‘to develop student’s learning capabilities in the areas of: self-learning, creative thinking and basic academic learning’ (Thailand Ministry of Education 2000; Pitiyanuwat and Pitiyanuwat 2012, this volume).

The capacity for self-directed learning is labelled as ‘Assessment As Learning’ by Lorna Earl (2003) and ‘Learning How To Learn’ by Paul Black and associates (Black et al. 2006, 2011). Learning to Learn is one of four pillars of education for the twenty-first century identified by UNESCO (Delors et al. 1996), along with Learning to Be, Learning to Do, and Learning to Live Together. It is the foundation for lifelong learning.

1.4 Assessment for Learning Reform

The second prevalent phenomenon in assessment reforms in the region is the move from assessment of learning to assessment for learning or the generation of feedback to inform teaching and learning. In facing challenges of increasing global competition between nations in the new century, there is strong consensus amongst governments in the region that, for their nations to succeed, they must build a capacity for know­ledge creation and knowledge transfer (Sahlberg 2006). Huge resources have been invested by these governments in making explicit policy shifts to assessment for learning. Assessment is to generate information to ‘feedforward’ for subsequent learning (Berry and Adamson 2011; Black et al. 2011; Carless 2007; Hogan and Gopinathan 2008; Lee 2012, this volume; Mok et al. 2003; Ng 2010).

Assessment as learning and assessment for learning are two new conceptions of assessment, and together, they form the foundations for assessment reforms in major education systems in the Asia-Pacific region. Assessment reform is particularly important to learners in the region because many of the education systems in the region have very strong cultures and traditions of assessment of learning. Assessment in China, Hong Kong, Japan, Korea, Macau, Singapore and Taiwan, for instance, is traditionally in the form of high-stake norm-referenced examinations that determine future prospects of education and employment of the examinees (Berry and Adamson 2011; Hogan and Gopinathan 2008).

1.5 Resolving Tensions in Assessment Reforms

In systems where assessment is mainly used for selection purposes, there are only a few winners and many losers. It is understandable that attention can easily focus on marks and grades instead of on learning in such systems. Research has shown that competitive assessment not only has pervasive debilitating effects on current learning including narrowing of learning but also induces stress and depression, deteriorates sleep quality and increases self-blaming, learned helplessness and other maladaptive beliefs as well as students’ motivation for further learning (Berry and Adamson 2011; Putwain 2009). Nevertheless, it is not easy to uproot the deeply entrenched parental and societal beliefs on the functions served by examinations. In reality, many teachers still have to carry out assessment for summative purposes in their day-to-day practice in the middle of their local assessment for learning reform, resulting in teacher stress and resistance to change (Ballet and Kelchtermans 2009; Cheung and Wong 2011; Choi and Tang 2009; Day 2008; Ho et al. 2012, this volume; Goh and Matthews 2012, this volume).

The message is clear: there is an urgent need to revise and redesign pedagogy in order to reconcile the tension between assessment as, for and of learning and to glean the benefit of each for enhanced learning and teaching. The framework of self-directed learning oriented assessment (SLOA) discussed in this book offers a feasible solution to that new pedagogy. SLOA is a coherent framework of assessment, deliberately designed to capitalize on the integrative impact of assessment of, for and as learning in the construction of assessment activities for optimal learning and for the cultivation of self-directed learning capacities in students (Mok 2010).

The overall aim of this book is to present to readers – teachers, parents, educators and education policymakers – a set of theory-driven assessment strategies, guidelines and practical examples for the successful implementation of assessment reforms in schools and classrooms. The genesis of this book is the 3-year longitudinal assessment project in Hong Kong and China (2005–2008) and the SLOA projects in Macau (2008–2011) reported in Mok (2010) and Yu (2012a, b, this volume). This book provides further elaborations on the theoretical foundations of SLOA, examines actionable assessment strategies and tools that can facilitate teachers’ work and presents practical examples where SLOA has been applied to teaching and learning in primary and secondary classes in the region. The book comprises 20 chapters and is divided into three parts: Theory of Self-Directed Learning Oriented Assessment, Tools for Implementing Self-Directed Learning Oriented Assessment and Case Studies of Self-Directed Learning Oriented Assessment in the Region. The rest of this chapter will be devoted to the theory and practice of SLOA and an introduction to the other chapters in the book.

2 Conceptions of Self-Directed Learning Oriented Assessment

Mok (2010) proposed an a priori conceptual framework to guide research on self-directed learning oriented assessment (SLOA). As the name implies, SLOA focuses attention on assessment that can support and advance learning and assessment that is self-directed by the learner. This section will expand on the meaning of these two aspects of SLOA. Furthermore, this section will explain how the SLOA framework draws from, and is being informed by, recent research in a number of domains in learning psychology (notably, self-directed learning, metacognition and feedback) and in psychometrics. Lastly, this section will explain how the three concepts of assessment of learning, assessment for learning and assessment as learning integrate and supplement each other in the SLOA framework.

2.1 Learning Oriented Assessment

The name SLOA is made up of two parts, namely, ‘LOA’ and ‘S’. ‘LOA’ comes from Carless (2007), who coined the term Learning Oriented Assessment (LOA). ‘Learning’ was deliberately placed before ‘assessment’ in order to highlight the centrality of learning in all assessment activities. LOA means that (a) assessment activities should be designed as learning tasks, (b) assessment should engage students in the evaluation of the learning progress and (c) feedback from assessment should be used as feedforward to inform current and future learning (Carless 2007). Through these three principles, LOA gets to the spirit of assessment for learning.

2.2 Self-Directed Learning

The ‘S’ in SLOA means self-directed learning (Earl 2003; Knowles 1975; Lee 2012, this volume; Paris and Paris 2001; Pintrich 2004; Schunk 2008; Shute 2008). The capacity for self-directed learning is fundamental to sustainable development in the twenty-first century, given the rapid speed at which knowledge is created. Knowledge and skills that students will need when they join the workforce have not yet been created today when they are at school. Consequently, education in the new century has to go beyond the transmission of knowledge to students. Rather, the core mission of education is to engender in students the capacity for knowledge creation, knowledge management, knowledge transfer and knowledge acquisition. In other words, education in the new century means learning how to learn (Delors et al. 1996). In the process of knowledge creation, management, transfer and acquisition, the learner must be able to set learning goals, plan the course of action, manage resources, monitor his/her learning progress, assess the level of achievement so far, generate feedback and adjust and self-regulate accordingly. The learner holds the key to success in the learning process. Unless and until the learner is capable of directing his/her own action in this process, there will be no real learning. In this regard, the SLOA framework is very much inspired by the work of Earl and associates (Earl 2003; Earl and Katz 2008, reprinted in this volume), in which she argued that assessment is actually learning and labelled the concept as ‘assessment as learning’. Engaging the learner as his/her own assessor, or assessment as learning, is the ultimate goal of assessment for learning. Earl (2003) wrote:

The student is the link. Students, as active, engaged and critical assessors can make sense of information, relate it to prior knowledge, and master the skills involved. This is the regulatory process in metacognition. It occurs when students personally monitor what they are learning and use the feedback from this monitoring to make adjustments, adaptations and even major changes in what they understand. Assessment as learning is the ultimate goal, where students are their own best assessors. (Earl 2003, p. 47)

2.3 Metacognition

Taking after Earl (2003; Earl and Katz 2008, reprinted in this volume), assessment as learning in the SLOA framework means an assessment process in which the learner actively considers and sets learning goals, deliberates upon and selects learning strategies, monitors learning, assesses learning progress, evaluates feedback information and, as a result, reaches new understanding, connects new information with existing knowledge or even revises learning goals or strategies. In other words, assessment as learning means the learner is exercising the self-regulatory process of metacognition (Brown 1987; Earl 2003; Flavell 1979; Loyens et al. 2008; Schunk 2008).

The SLOA framework incorporates a range of metacognitive tools and mechanisms, including the provision of timely feedback from assessment and explicit teaching of a range of strategies, so as to raise students’ self-awareness (metacognitive knowledge) of their own learning process and to enrich their repertoire of self-regulation skills (self-regulation of cognition). These skills include identifying key issues in the learning task, posing questions, selecting learning strategies and monitoring progress by situating these strategies in learning tasks of curriculum subjects, as well as modelling and scaffolding the strategies (Black and Wiliam 1998a; Boone et al. 2012, this volume; de la Torre 2012, this volume; Hattie and Timperley 2007; Mok et al. 2012, this volume; Hsu et al. 2012, this volume; Kalyuga 2012, this volume; Lee 2012, this volume; Choi et al. 2012, this volume; Tzuriel 2012, this volume; Yu 2012a, b, this volume).

2.4 Feedback

Assessment is formative (assessment for learning) when feedback generated from the assessment is directed towards the quality of the task or learning process, identifies misconceptions and supports the development of more effective learning strategies (Black and Wiliam 1998a; Hattie and Timperley 2007; Lee 2012, this volume; Shute 2008).

Feedback also contributes to the metacognition of the learner (assessment as learning) through generating cues for the learner to internalize three key feedback questions (Hattie and Timperley 2007, p. 86):

  1. 1.

    ‘Where am I going’: What is the desired outcome (long-, intermediate-, short-term) of my learning endeavour? What is the anticipated outcome if I approach the problem this way? How is this new learning related to my previous learning?

  2. 2.

    ‘How am I going’: What does the assessment evidence tell me about the effectiveness of my learning strategies and is there a gap between my desired goal and my current progress? If there is a gap, what are the possible causes?

  3. 3.

    ‘Where to next’: What should be my next steps? Do I have to keep going this way or should I modify my learning strategies? Should I change my goal (set higher/lower goal, change direction)? Should I seek help and, if so, from where should I get help?

2.5 SLOA: Integrating Assessment Of, For and As Learning

The SLOA framework comprises three integrative components: assessment of learning, assessment for learning and assessment as learning. They refer to the purposes of assessment or how the assessment outcomes are to be used. The relationship between them is best described as a recurrent three-component learning process (Fig. 1.1). First, assessment of learning in the SLOA framework refers to assessment activities of the teacher and their students that aim to generate evidence about current learning. In assessment of learning, the teacher and students ask, ‘Where are we in the learning?’

Fig. 1.1
figure 1

Assessment of, for and as learning

Full size image

Next, after the evidence is generated, the teacher and students ask, ‘Is there a gap between the desired learning goal and the current level of learning?’ In order to address this question, the desired learning goal has to be established and made clear to both parties. Consequently, in the SLOA framework, assessment for learning often begins with goal setting and clarification of the desired learning goal. Even though the question concerning the gap can be addressed by assessment of learning, information generated from such assessment is often inadequate to address the next question, ‘If there is a gap, how can we close the gap?’ Assessment for learning in the SLOA framework refers to assessment activities by the teacher and the students to collect evidence with an aim to feedforward to inform further learning in terms of directions and potentials. That is, assessment for learning enables the teacher to ‘modify the teaching and learning activities’ and to ‘adapt the teaching work to meet the needs [of individual students]’ (Black and Wiliam 1998b, p. 2). Third, assessment as learning in the SLOA framework means that the learner internalizes the questions of, ‘Where am I going? How am I doing? How can I learn better? How can I keep up my motivation?’ and acts upon them in a constant process of self-monitoring during learning.

2.6 Theoretical Underpinnings of SLOA

Four theories in assessment, psychometrics and learning underpin the SLOA framework. They are standards-referenced assessment, cognitive diagnostic assessment (CDA), Rasch measurement and metacognition. According to Tognolini and Davidson (2012, this volume), a standards-referenced system consists of a curriculum which clearly articulates learning outcome standards and performance standards, and assessment tasks which are set according to the expected learning outcomes for interpretation of student performance. Through checking the student’s progression against expected outcomes, the teacher can get a clear idea about the student’s growth in that area of learning, and from this, the teacher can determine subsequent actions to enhance further learning. As such, standards-referenced assessment provides a means to align curriculum, assessment and teaching and so gives meaning to assessment, enabling assessment of learning to be developmental. Instead of ranking students according to their performance, assessment is used to provide teachers with information about where their students are in their learning. In their chapter, Tognolini and Davidson (2012, this volume) explain how standards are defined and how they are used to improve classroom learning and assessment.

Assessment for learning is made possible through cognitive diagnostic assessment (CDA) which aims to generate diagnostic insights from assessment data in order to inform subsequent instructional decisions. Three chapters in this volume (Choi et al. 2012; de la Torre 2012; Kalyuga 2012) are devoted to the theory of CDA and how, under this theory, assessment can be designed to generate specific and fine-grained information about the learner’s current knowledge and skills in order to facilitate assessment for learning.

Many models are available in the literature for CDA (see Choi et al. 2012, this volume) for an overview, or DiBello et al. (2007) for a review, but as a basic first step, the test designer needs to analyse the knowledge structure to identify and define the attributes underpinning the learning. Next, assessment items are constructed with contents designed to generate diagnostically relevant information on the knowledge and skills of interest. The matrix specifying the item and target attribute relationship is called a Q-matrix (Tatsuoka 2009). Construction of the Q-matrix involves many iterative rounds of theoretical mapping of attributes by content experts and empirical testing of the items for representation of attributes. Third, a psychometric model is selected to analyse the assessment data in order to identify the learner’s strengths and weaknesses on the attributes. One family of psychometric models for CDA, namely the ‘deterministic, input, noisy “and” gate’ (DINA) model, is highlighted with illustrative examples (using a mixed fraction subtraction problem) by de la Torre (2012) in this volume. Lastly, feedback on strengths and weaknesses of individual learners is given in order to facilitate instructional decisions by the teacher and each learner.

CDA has strong potential to give diagnostic insights into learning. Nevertheless, the construction of a Q-matrix is a very demanding task, and misspecification of the Q-matrix can lead to serious misinterpretations of performance data (Rupp and Templin 2008). Furthermore, there is no easily accessible computer software for analysis of assessment data involved in DINA or other models (Choi et al. 2012, this volume; de la Torre 2012, this volume). In particular, CDA usually requires a large number of examinees to be assessed on a considerable number of items to obtain reliable estimates. Kalyuga (2012, this volume) offers a rapid diagnostic assessment approach as an alternative. Carried out either as a first-step method, wherein a learner is invited to rapidly indicate their first step to solve a problem, or a rapid verification method, wherein the learner is asked to rapidly verify the accuracy of a series of steps towards a solution, the rapid diagnostic assessment method can be used to provide diagnostic information on the learner’s current knowledge state (Kalyuga 2012, this volume).

Although CDA is gaining in popularity in education (Lee and Sawaki 2009; Leighton and Gierl 2007), its use remains limited because of its technical and psychometric complexities. Instead, the Rasch model (Boone et al. 2012, this volume) is perhaps more accessible to classroom teachers. The Rasch model is a statistical model that expresses the probability of a response (e.g. right/wrong answer) in terms of a logistic function of the difference between the ability of the person taking the test (represented by θ) and the difficulty level of an item (represented by δ). The probability of getting an item correct is given by e   (q−δ)/(1  +  e   (q−δ)),where e is the exponential function. It can be easily seen that if θ equals δ, the probability of getting an item correct is 0.5. However, if θ is greater than δ, i.e. if the person has more ability than what is demanded by the difficulty of the item, then the person has a greater than 0.5 probability of getting the item correct. And the converse is also true: if θ is smaller than δ, i.e. if the person has less ability than what is demanded by the difficulty of the item, then the person has less than 0.5 probability of getting the item correct. Graphically (see Fig. 1.2), the trait being tested can be represented by a vertical continuum, and the person ability and item difficulty on the left and right sides of the vertical continuum, respectively. It is then easy to illustrate the three situations: (a) the person has a high probability of passing the item, (b) the person has a 50/50 chance of passing the item and (c) the person has a low probability of passing the item (Fig. 1.2).

Fig. 1.2
figure 2

Person ability verses item difficulty

Full size image

Most commercially available software packages of Rasch analysis, for instance Winsteps® (Linacre 2011), produce an item-person map for all persons taking the test and all items in the test. The example given in Fig. 1.3 shows that person A17 has ability well above items 1, 5, 30 and 7; thus she/he has a high chance of answering these items correctly. However, A17’s ability is about the same as the difficult level of items 3, 9 and 18 and so she/he has only a 0.5 chance of getting these items correct. Items around this area of difficulty represent A17’s zone of proximal development (ZPD) (Vygotsky 1978). Using the item-person map, the teacher can find out the ZPD of every student and provide scaffolding accordingly.

Fig. 1.3
figure 3

Item-person map showing zone of proximal development for person A17

Full size image

Assessment as learning in the SLOA framework implies the engenderment in students of the habit of mind for self-monitoring and self-regulation in order to enhance further learning (Earl and Katz 2008, reprinted in this volume; Mok 2010; Pintrich 2004; Schunk 2008). It represents a shift in attention from assessment of subject matters to focusing on the learner’s self-awareness. Critical to assessment as learning is the learner’s metacognition. The literature distinguishes two components of metacognition: ‘knowledge of cognition’ (knowledge about oneself as learner and knowledge about strategies to learn) and ‘regulation of cognition’ (the conscientious control by the learner of various cognitive strategies for learning, including planning, regulation and evaluation) (Brown 1987).

Kleitman et al. (2012, this volume) identify self-confidence as an important aspect of metacognitive knowledge. Their programme of research in Australia and Sweden found that children as young as 9 years old can clearly articulate their own confidence judgments. They also found that the construct of self-confidence predicts school achievement even after controlling for students’ cognitive ability, age and gender. Furthermore, self-confidence is affected by classroom goal orientation (Meece et al. 1988), teacher-student relations and after-school activities. These results have significant implications for the development of assessment as learning in students.

3 Implementation Strategies of SLOA in Schools

We have gained invaluable experiences from our 3-year longitudinal assessment project (2005–2008) in Hong Kong and the self-directed learning orientated assessment (SLOA) projects in Macau and China (2008–2009) (Mok 2010; Yu 2012a, b, this volume). These experiences show that successful implementation of SLOA has to be multilevel, instigating change at system, school, classroom, teacher and student levels, and that it also requires concerted effort by all the key actors, including parents, principals, teachers, students, government officials, educators and societal leaders.

Although there is no single set of strategies that suits all situations for successful reforms, previous experience with more than 100 schools suggests three strategies that tend to predict higher chance of success:

  • Taking a whole-school approach to building a strong culture of SLOA in curriculum redesign

  • Empowering teachers through development of knowledge and skills

  • Activating students as learning partners

A whole-school approach means that all key stakeholders are involved and that clearly articulated management and implementation plans are established at all levels (Stringfield et al. 2008). Importantly, when faced with assessment data, new policies may need to be developed and curriculum may have to be redesigned. These actions are not achievable by individual teachers, learners or even the principal alone. Instead, a whole-school approach enables assessment data to be turned into actionable knowledge.

A whole-school approach creates a learning community in which teachers can experiment with new approaches in unison (Day 2008). School support strengthens teachers’ identification with the school (Henkin and Holliman 2009); increases teacher work satisfaction (Day 2008), teacher commitment (Cheung and Wong 2011; Choi and Tang 2009; Day 2008) and teachers’ willingness to implement innovations (Ballet and Kelchtermans 2009); and raises student achievement (Day 2008) (although an earlier research by Park (2005) has a different finding).

Research (Cheung and Wong 2011; Earl 2011; Fullan 2009; Pitiyanuwat and Pitiyanuwat 2012, this volume; Sahlberg 2006) found quality teachers to be a key factor to success of education reforms. Teacher professional development, especially at times of change, empowers teachers to initiate and sustain changes in their classrooms (Ballet and Kelchtermans 2009; Cheung and Wong 2011; Goh and Matthews 2012, this volume; Lieberman and Pointer Mace 2008; Pitiyanuwat and Pitiyanuwat 2012, this volume; Yu 2012a, b, this volume). The quality of professional development programmes as measured by their relevance, meaning, practical values and flexibility in choice on the format and time affects teachers’ willingness to participate (Day 2008).

Although teachers can be drivers of reform, it is the students themselves who need to commit to change (Earl 2003; Earl and Katz 2008, reprinted in this volume). Self-regulated learning is facilitated by a learning environment with a community of learners and in the context of cooperative learning. An open and autonomous classroom encourages peer students to serve as resource persons and partners in the learning process (Mok 2010; Paris and Paris 2001; Slavin 1996).

4 Tools for the Implementation of SLOA

A number of new developments in assessment and psychometrics are now available to support the implementation of SLOA. Part II of this book focuses attention on how to harness new developments in psychometrics and information technology to facilitate assessment for learning. Six tools for the implementation of SLOA are introduced: item response theory, mathematics competency vertical scales, student-problem charts, dynamic assessment, two-tier items and computerized adaptive testing. These tools have in common an emphasis on the speedy generation of valid diagnostics feedback to inform instruction. They are presented as accessible alternatives to the traditional method of using the total score as an indicator of level of achievement.

The contrast between item response theory (IRT) and classical test theory (CTT) is presented by Wu (2012, this volume) with an example data set analysed using the ConQuest programme (Wu et al. 2007) that she developed. The IRT is a mathematical model representing the relationship between an examinee and a test item. Wu (2012, this volume) discusses the concepts of item difficulty, discrimination power and plausible values in IRT in this chapter.

A critical concept in IRT is the assumption of a single latent ability (construct) underpinning an examinee’s performance on a test. Suppose the latent ability in question has meaning across several school years, then in theory, a vertical scale can be built to chart a student’s progress across year levels on the construct. Yan et al. (2012, this volume) present a new method, entitled concurrent-separate approach, using the Rasch model (Boone et al. 2012, this volume) to develop vertical scale of measurement across several school levels. The authors demonstrated how a mathematics competency vertical scale (MCVS) with reasonable psychometric properties can be developed using the new method and made feasible to track Hong Kong students’ development in mathematics from primary 2 (grade 2) to secondary 3 (grade 9) levels.

The Rasch model (Boone et al. 2012, this volume) was found by many teachers in Hong Kong and Macau to be helpful to their provision of quality feedback to students (Mok 2010). Nevertheless, to some teachers, the Rasch model can be mathematically demanding. The student-problem chart (SP chart) (Mok et al. 2012, this volume; Sato 1980, 1985) is an alternative for teachers to make sense of assessment data. The SP chart is a matrix of students’ responses to individual items of a certain assessment in which the rows and columns are rearranged such that students are arranged from high to low ability (based on their total score on the assessment), and items are arranged in ascending order of difficulty from left to right (based on the number of students who answered the item correctly). After this rearrangement, the observed pattern of responses is matched against the expected pattern, which is computed based on the assumption that each student has a higher probability of answering correctly an easier item than a more difficult item and, likewise, each item has a higher probability of being answered correctly by a more able than a less able student. By inspecting the response pattern and interpreting it against the expected pattern, the teacher is able to identify aberrant response behaviours of students. Furthermore, a modified caution index (Sato 1980) can be computed based on the SP chart for each student and each item to enable the teacher to determine if the response pattern is too different from the expected pattern and, if so, how they are different. The teacher is able to give evidence-based feedback to the students on subsequent learning. A software package SP Xpress (Mok et al. 2011) is now available for producing the modified caution index, item reliability, student performance and other psychometric indices for use by teachers.

IRT, vertical scales and SP chart are helpful tools which can be used to analyse assessment data to support student learning. Nevertheless, it is impossible to undertake high quality analysis if the assessment data itself is of substandard quality. The chapter by Tam et al. (2012, this volume) presents the method of two-tier items to provide high quality diagnostic insights. A two-tier item is conceptualized by the authors as a mini-test (testlet) comprising two parts: the first part is usually designed to assess the examinee’s ability to identify the targeted concept, and the second the extent to which the examinee can explain the rationale for the response on the first part. By design, the two parts of the testlet are related and thus violate the underlying assumption of local independence in the Rasch approach. In their chapter, Tam et al. (2012, this volume) propose a method to analyse two-tier items and illustrate with a real data set how the data can be analysed for diagnostic insights.

One important consideration in the implementation of SLOA is the speed at which assessment feedback is generated. This is especially so for classroom assessment. In Chap. 13, Tzuriel (2012, this volume) presented dynamic assessment as an attractive solution to speedy assessment feedback. Dynamic assessment is based on the author’s three decades of work in this area and is underpinned by the theory of zone of proximal development (ZPD) developed by Vygotsky (1978). In this approach, assessment and learning are tightly integrated through an iterative process of ‘assessment to ascertain the ZPD, teaching around the ZPD, learning and further assessment to ascertain new ZPD’. In a systematic presentation in six major sections, Tzuriel (2012, this volume) argues for the shift from standardized testing to dynamic assessment to support assessment for learning, and he also discusses the benefits, limitations and strategies in using this new approach.

Speedy assessment feedback can be achieved through computerized adaptive testing (CAT) as presented by Hsu et al. (2012, this volume). The CAT technology capitalizes on recent developments in psychometric theory, particularly IRT (Wu 2012, this volume) and information technology. A CAT system comprises an item bank that is constructed and calibrated according to a vertical scale about a trait (Yan et al. 2012, this volume); a set of item-selection strategies for the iterative process of ‘selection of an initial batch of test items, response by examinee, analysis on response and generation of the next batch of test items’ in order to elicit the optimal amount of information about the examinee’s level of competence on the trait; and a stopping rule which specifies criteria for the iterative process to stop. With the availability of fast speed computers, CAT can be used for large-scale assessment as well as classroom applications.

5 Examples of Implementation in the Asia-Pacific Region

Part III of the book presents six case studies of SLOA being implemented in the Asia-Pacific region in Thailand, Malaysia, China and Hong Kong. The first case is contributed by Pitiyanuwat and Pitiyanuwat (2012, this volume), who write on the history of assessment reform in Thailand and how the reform has evolved from a ‘non-formal’ form of education in the period from year 1283–1883 to the contemporary period wherein the alignment between assessment and learning is emphasized. The analysis by the authors not only shows the pathway to SLOA, the hurdles, pitfalls, rewards and achievements involved but also gives hope and direction for other Asia-Pacific education systems who are tempted to try SLOA in their own system.

One of the major areas of education reform in the Asia-Pacific region is the building up of a strong teaching force to drive the reform (Mok et al. 2003). The second and third case studies are both concerned with teacher capacity in driving assessment reforms. In the second case study, in response to the desire of the Malaysian Ministry of Education to evaluate teacher education, Goh and Matthews (2012, this volume) examined pre-service teachers’ ability for self-assessment. Through the voices of 16 pre-service teachers in Malaysia, the authors raise questions regarding the development of teacher self-metacognition – questions that are of critical importance for the successful implementation of SLOA.

The third case study also focuses on teacher capacity to implement assessment reform. This case study, undertaken by Ho et al. (2012, this volume), explores attitudes of teachers in Hong Kong towards Rasch measurement, particularly regarding the desirability and feasibility of the Rasch model as a tool for assessment for learning. Their findings suggest that although teachers recognize the Rasch model as a powerful alternative to traditional methods in generating assessment feedback, their adoption of the model for classroom applications is impeded by realistic workplace concerns including heavy workload and lack of technical support. Given that teachers’ attitudes affect their instructional decisions and willingness to adopt new approaches in their teaching (Choi and Tang 2009; Day 2008), it is important that teachers are supported in their implementation of assessment reform. Targeted professional development workshops, partnership with universities and provision of assessment item banks are proposed by Ho et al. (2012, this volume) as possible solutions to overcome the difficulties perceived by teachers.

The fourth case study, reported by Yu (2012a, this volume), involves trial implementations of SLOA with 209 primary students in three schools in China. By using metacognitive reading strategies, a specially designed reading log and a Rasch-calibrated English reading assessment system, Yu demonstrated that the SLOA approach significantly affected several aspects of teaching and learning of English reading in these schools, including changes in the physical learning environment, teacher motivation and teacher knowledge, as well as improvement in students’ English reading proficiency. Story book reading and metacognitive methods to promote reading are not entirely new strategies in the teaching of English reading, but in a country that has a long history of teacher-centred instruction, these approaches are innovative and have deep and far-reaching implications.

Encouraging results are also reported in the fifth case study (Lee 2012, this volume). Lee’s study involves an intervention designed to support pre-service sports coaches in the implementation of assessment for learning in the teaching of sports in Hong Kong. Through a series of carefully designed experimental procedures, Lee successfully instilled in his pre-service sports coaches in the experimental group the skills and strategies for using feedback to promote sports learning.

The sixth case study presented by Yu (2012b, this volume) reports on the implementation of SLOA in the subject of English at Saint Margaret’s Girls’ College in Hong Kong. Yu has provided for the readers a very detailed account on the rationale behind the SLOA project and illustrated vividly, using quotations taken from students’ and teachers’ journals, the impacts of SLOA on English instruction across several different year levels at the school. Although the results found by the study do not have statistical significance, the case report carries with it great substantive significance because through its rich description and the testimonies given by the actors, there is strong evidence of how the study has changed the school’s approach to assessment for learning.

6 Conclusion

Assessment is a concept with a long history. It has special meanings to people in the Asia-Pacific region where assessment and high-stake examination used to be synonymous. Since the turn of the century, however, systems in the region have initiated many reforms to catch up with worldwide paradigm shifts in the conception of assessment from assessment of learning to assessment for and as learning and to face the new challenges of the twenty-first century. Globalization and knowledge economies demand that we revise our vision on pedagogy to one that centres on learning how to learn. This volume presents a framework entitled self-directed learning oriented assessment (SLOA) which is strongly grounded in cognitive learning theory, powered by psychometric tools, and has been validated in several systems in the region as an enabling device for the betterment of learning and teaching in the new century.