Keywords

1.1 The German DFG-Priority Program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes”

In the past few decades, educational systems worldwide have been moving towards evidence-based policy and practice (e.g., Slavin 2002), where “evidence” often implies empirical assessment of students’ competencies as the main outcome of education at school. Thus, the assessment of competencies plays a key role in optimizing educational processes and improving the effectiveness of educational systems. However, the theoretically and empirically adequate assessment of competencies in educational settings is a challenging endeavor that is often underestimated by policy makers and practitioners.

To cope with these challenges, and to promote and coordinate scientific efforts in the field of competence assessment across disciplines in Germany, the German Research Foundation (DFG) funded the Priority Program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes”, which started in 2007 and ended in 2013. Over the six-year funding period, the Priority Program coordinated the work of a large number of research projects (see http://kompetenzmodelle.dipf.de/en/projects) with experts from psychology, educational science, and subject didactics.

The main point of reference for all projects of the DFG-Priority Program is the concept of “competencies”, which are defined as “context-specific cognitive dispositions that are acquired and needed to successfully cope with certain situations or tasks in specific domains” (Koeppen et al. 2008, p. 62; see also Hartig et al. 2008, and Shavelson 2010). According to this definition, “competencies” differ from other constructs such as “intelligence”, as competencies refer to the mastering of sets of specific challenges in specific situations in specific domains, whereas intelligence refers to mental abilities that can be used to master challenges in general. In addition, intelligence is generally not considered to be influenced by school education, whereas the development of competencies is at the core of school education. The definition of competencies as “cognitive” dispositions is in line with the way in which the term “competence” is used in international large-scale assessment studies such as PISA, TIMSS, or PIRLS (e.g., OECD 2001), as motivational and volitional aspects of competencies–in order to begin with research in this field–are excluded from being studied in those studies (Weinert 2001).

1.2 Research Areas of the DFG-Priority Program

The research addressed by the DFG-Priority Program covers different aspects of competence assessment, and is organized into four consecutive main research areas (Fig. 1.1): (1) The development and empirical testing of theoretical competence models is at the core of the research program. These theoretical models are complemented by (2) psychometric models, which in turn inform the construction of measurement procedures for the empirical assessment of competencies (3). The program is finally rounded off by (4) research on how best to use diagnostic information.

Fig. 1.1
figure 1

Research areas of the DFG-Priority Program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes”

Full size image

The following chapters of this book present the findings of 24 DFG-Priority Program projects. All projects have a primary focus on one of these four research areas; several projects moved consecutively through several areas. In the following sections, the research areas are described briefly, and an overview is given of projects within the areas.

1.2.1 Cognitive Modeling and Assessment of Competencies

Research in the area of cognitive modeling asks how competencies can be modeled adequately with regard to those situations or tasks where they are needed in specific domains. As such, models of competencies are necessarily domain-specific, and the research program covers a broad variety of domains.

A first group of domains (Part I: Modeling and assessing student competencies) concerns competencies of students at school, ranging from conceptual understanding and scientific reasoning in primary school, through geography and literary literacy, to self-regulated learning at high school (Chaps. 2, 3, 4, 5 and 6).

A second group of domains (Part II: Modeling and assessing teacher competencies) concerns the competencies of teachers, in areas such as professional vision, pedagogical content knowledge, tracking decisions, counseling, and teaching the integrative processing of text and pictures (Chaps. 7, 8, 9, 10 and 11).

A third group of domains (Part III: Modeling and assessing vocational competencies and adult learning) concerns vocational competencies and adult learning in fields such as car mechatronics, electronics, building trades, and industrial management (Chaps. 12, 13 and 14).

Modeling of change and training of competencies represents a fourth, very challenging area of research (Part IV: Competency development: Modeling of change and training of competencies). Projects are concerned with students’ physics competencies, decision making regarding sustainable development, metacognitive competencies, strategies for integrating text and picture information, problem-solving competencies, language and mathematics competencies (Chaps. 15, 16, 17, 18, 19 and 20).

1.2.2 Innovations in Psychometric Models and Computer-Based Assessment

Research in the area of psychometric models (Part V: Innovations in psychometric models and computer-based assessment) asks how theoretical models of competencies can be linked to psychometric models in order to develop assessment instruments. Innovative approaches are presented concerning multidimensional IRT models for English as a foreign language, multidimensional adaptive measurement for large-scale assessments, adaptive assessment of competencies regarding multiple representation of mathematical functions, relating product and process data from computer-based assessments, and dynamic problem solving (Chaps. 21, 22, 23, 24 and 25).

1.2.3 Reception and Usage of Assessment Results

Research in the area of assessment results (Part VI: Feedback from competency assessment: Concepts, conditions and consequences) asks what kinds of information from competence assessments can be used by practitioners in the educational system, and in which ways. A specific focus of the projects is on feedback, such as the role of feedback in formative assessment, in arguing validity and standard setting, as well as feedback effects in a dynamic test of reading competence (Chaps. 26, 27 and 28).

1.3 Conclusion

As outlined in this introduction, the assessment of competencies plays a key role in optimizing educational processes and improving the effectiveness of educational systems. However, to adequately assess competencies in educational settings is a challenging endeavor, and the German DFG-Priority Program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes” has been an attempt to move the field onto a broad national footing by funding basic scientific research on modeling competencies.

The Priority Program has had significant influence, not only in terms of scientific publications (a complete list of publications is provided at http://kompetenzmodelle.dipf.de/en/publications/km_literatur_e.html), but also in terms of stimulating additional, more-applied large-scale research programs funded by the German Federal Ministry of Education and Research (BMBF). One example of these latter is the Research Program KoKoHs: “Modeling and Measuring Competencies in Higher Education” (Zlatkin-Troitschanskaia et al. 2014, 2015, 2016). Another example is the Research Program ASCOT: “Technology-based Assessment of Skills and Competencies in Vocational Education and Training” (BMBF 2012).

As a result of both the more basic research (DFG-Priority Program) and the more-applied research (BMBF Programs), a large number of theoretical models, psychometric approaches, and assessment instruments are now available. These allow practitioners in the educational field to assess competencies in a great variety of domains. Furthermore, these models, approaches, and instruments that were developed within specific domains, can be used as a blueprint for developing models, approaches, and instruments in other domains. Thus, there are good grounds for optimizing educational processes and improving the effectiveness of the educational system in Germany through adequately assessing student competencies.