Measures of Success in Problem Based Learning: Triple Jump Assessments and E-Folios

Abstract

There are three points of summative assessment in UBC’s TELL through PBL Bachelor of Education cohort: two structured as “Triple Jumps” and one as an e-folio. All three measure and support the development of our preservice teachers’ professional competencies by simulating the professional contexts in which teachers work. This chapter describes the practices, purposes, and ongoing evolutions of our cohort’s summative assessments by articulating their relationship to MacDonald and Savin-Baden’s (2004) principles of PBL assessment. In line with these principles of good assessment, the structure of our Triple Jumps and e-folios has shifted over time toward more coherent and achievable examinations that foreground the understanding and application complex cross-curricular knowledges. This integrated, ability-based approach to assessment prioritizes reflexivity, problem-solving, and reason-giving over information retention and recitation. It also better reflects the professional competencies cultivated by problem based learning, sought by our inquiry-based cohort and requisite for good teaching.

Introduction

How best to formatively and summatively assess preservice teachers is an ongoing and evolving conversation in the Teaching English Language Learners through Problem Based Learning (TELL through PBL) cohort of UBC’s Bachelor of Education. These conversations have been driven partly by our own self-inquiries into our cohort’s purposes and practices and partly by program changes within the broader B.Ed. program. More than just idle conversation, our inquiries in the 8 years I have worked with this cohort have resulted in three distinct shifts in our summative assessment criteria. Each shift has moved us toward more coherent and achievable examinations that prioritize problem-solving abilities and reason-giving over information retention and recitation. They have also moved us toward a more integrated, ability-based approach that better reflects the professional competency sought by our cohort and the UBC B.Ed. program as a whole.

The three summative points of assessment used by our cohort are structured as “Triple Jumps,” a form of assessment common to many PBL programs (MacDonald and Savin-Baden 2004).¹ The Triple Jumps take place at the end of each of the three academic school terms ; three terms translate to three successive “jumps.” According to Macdonald and Savin-Baden (2004):

[T]he ‘Triple Jump’ exercise has three phases: hop, step and jump. In the hop phase the tutor questions the student, thus they are caught on the hop. The step phase allows the student time to research the findings and hypotheses that have emerged from the hop phase. In the jump phase they are expected to provide the tutor with a written report of their findings. (p. 11)

Our cohort’s first two Triple Jumps are most true to this original structure. They assess how well the preservice teachers have learned to ask questions of a case, draw out its complexities, conduct collaborative research, and make good individual judgments about how to proceed. The final Triple Jump differs in that the preservice teachers are required to construct a professional portfolio that includes a Statement of Educational and Teaching Philosophy. While the Triple Jump formats vary, there are strong links across their purposes and practices. Each one gives the instructors a snapshot of where the preservice teachers are on their journeys toward becoming teachers, while also providing the preservice teachers an opportunity to explore and demonstrate their professional growth over time.

In the discussion that follows, I will explain the purposes and practices of our Triple Jumps (TJ) using MacDonald and Savin-Baden’s (2004) principles of PBL assessment. These principles help clarify how our cohorts’ current framings of these exams enable meaningful summative assessments of our preservice teachers’ growth toward becoming teachers. I also discuss the evolution of the TJs, demonstrating how our cohort’s ability to be reflective and responsive has enabled us to align our TJs more closely with MacDonald and Savin-Baden’s principles.

Principle 1

“As lecturers, we need to ensure that there is alignment between our objectives and the students’ anticipated learning outcomes, the learning and teaching methods adopted, and the assessment of learning – strategies, methods, and criteria” (MacDonald and Savin-Baden 2004 p. 7).

The first two TJs mimic the structure of the biweekly case cycles that catalyze preservice teacher learning throughout the year.² All follow a case-research-response format where preservice teachers unpack problematic cases through collective inquiry that culminates in their individual responses and syntheses. These shared formats and markers of success align our methods of learning and demonstrating learning .

This alignment begins with our integrated approach to TJ planning. Tutors and instructors meet to craft our cases. For the TJ, we review the themes addressed in the preceding class meetings and research packages . We fine-tune previous TJ cases and assessment rubrics to ensure that they align with the arc of the preservice teachers’ learning and our evolving course objectives .³ We also canvass each other’s ideas for how the TJ case narratives might be altered to evoke increasingly complex and cross-curricular responses from the preservice teachers. This collaborative process continues the kind of case refinement and responsiveness that our cohort strives toward throughout the rest of the year.

The day of the TJ exam itself then compresses the typical case cycle into a single morning and afternoon. In the morning, the preservice teachers pick up a case package that includes the TJ case: a narrative comprising many ill-defined yet true-to-life themes. For roughly five hours, they work together in their tutorial groups to enact the already familiar PBL pedagogy. They identify and puzzle through the case issues. They look back at previous research packages. They access library resources for review and further inquiry. Drawing from their collective work, they then develop their own interpretations and responses.

In the afternoon, each preservice teacher sits down for a 30 minute dialogue with one of the TELL through PBL instructors. The preservice teachers offer their analyses and plans of action. The instructors continue the Socratic questioning used throughout the rest of the term to help draw out the preservice teachers’ meanings, rationales, and practical strategies. The strongest responses generally draw from a combination of the content knowledge, practical experience, and research strategies that the preservice teachers have developed and culminate in responses that align with their emerging sense of themselves as teachers.

The first two TJs assess the extent to which the preservice teachers can embody the cycle of inquiry that has grounded their learning throughout the year.⁴ How well can they trouble a school-based context, identifying possible issues and research questions? How well can they refine their questions as their academic and practical knowledge base grows? How well can they take up critical theory to ask and answer: who/what is framed as the problem; whose perspectives are represented and whose are marginalized or absent; and what questions am I asking and what questions or possibilities don’t I see? How well can the preservice teachers identify what they know, what they don’t know, what they need to know, and where to find it? How well can they synthesize a well-justified response that draws from their diverse knowledge bases and results in a specific, context-embedded plan for action?

By constructively aligning our methods of learning and assessment, the first two TJs attempt to encourage deep, as opposed to surface, approaches to learning (Biggs 2007; MacDonald 2005). They attempt to assess the growing content knowledges and competencies that the preservice teachers have been working to develop and which they will need to become effective, engaged educators.

Principle 2

“Assessment should reflect the learner’s development from a novice to an expert practitioner and so should be developmental throughout the program of studies” (MacDonald and Savin-Baden 2004, p. 7).

The TJs are structured to reflect the increasing level of sophistication and self-reflection we expect to see as the preservice teachers progress from one term to the next. While the first Triple Jump (TJ1) and the second Triple Jump (TJ2) share the same “hop-skip-jump” process described above, TJ1 attempts to recognize that the preservice teachers are still early in their development as teachers. It is structured to review and reconsider many of the case issues and broad themes of the first academic term. These themes might include building classroom community amidst diverse cultures, linguistic backgrounds and learning styles, teaching for social justice and anti-oppression education, teaching math through problem-solving, developing a balanced literacy reading program, and integrating Indigeous knowledges into classroom planning and practice. By asking the preservice teachers to synthesize and apply their prior content learning in response to familiar issues, TJ1 recognizes their still novice status.

At the same time, TJ1 is not a straight repetition or regurgitation of prior content learning . “The triple jump not only assesses what the students learned, but how they learned it” (McTiernan et al. 2007 p. 117). It is a test of process as well as product . To explore how well the preservice teachers have learned to use the PBL pedagogy, we play with the narrative contexts in which the TJ case issues play out. We might alter the grade level in which the TJ case is set and therefore the developmental stages of the learners. We might alter the class or community composition and with that the strengths and challenges for both learners and teacher. The case also includes one broad, previously unresearched theme. This addition assesses the preservice teachers’ growing abilities to problematize and inquire. It evaluates their abilities to transfer and apply their growing knowledges to new circumstances. It gauges their abilities and dispositions to respond with flexibility, collaboration, and perseverance to the unexpected challenges of teaching. Finally, it extends their learning, spurring on the development of their professional judgment and identities.

The differences between TJ1 and TJ2 are designed to account for the preservice teachers’ gradual movement from novice to expert. TJ2 follows the same case-research-response format as the first, but asks both the preservice teacher and the examiner to enter into a more formal role-play for their dialogue. As the rubric makes clear, examiners may choose to be a parent of one of the case study students or the principal of the case study school.⁵ While parents and principals may ask different types of questions of preservice teachers, both role-plays introduce even more unpredictability into the second TJ, assessing the preservice teachers’ abilities to think on their feet. The preservice teachers must loosen their expectations of being able to stick to a fully formed plan, making this experience and exam even closer to authentic professional contexts . By inviting the preservice teachers to practice using their professional voices in an extended exchange about education, TJ2 is designed to help move them toward articulating more clearly their own teaching philosophies and practices.

The rubrics for TJ1 and TJ2 also attempt to account for the developmental nature of the TJs and the preservice teachers’ journeys. While we work within a pass/fail system , we have designed the rubric to allow us to recognize gradations of performance and directions for future growth. For each criterion within the rubric, the preservice teachers receive an assessment of how well they have met the expectations of the exam: “exceeding” (**), “meeting” (√), and “not yet meeting” (−). These more nuanced gradations allow the preservice teachers to pass onto the next phase of their program, with the understanding that their continued development as professionals is required.⁶ While the TJs are summative, we do not want them to stand completely apart from the iterative, formative process of the 2-week case cycle.

Many instructors make use of the “comment section ” of the rubric to provide the preservice teachers suggestions for how to build on their developing strengths, shore up their current weaknesses, and chart possible paths forward toward continued improvement. As an example of what this feedback can look like in practice, I have included some of my own:

Example 1

Try to incorporate a richer discussion of the theories of learning and development that ground your pedagogical objectives, rationales, and practices. While you made good references to Ministry documents (e.g., Focus on Bullying), you had difficulty linking to academic theorists, even when asked to do so directly. During the next Triple Jump, try to trace the origins of the educational concepts that you employ, situating your discussion in a more explicit (CALP!) academic context. Or to borrow again from the TELL focus of TELL through PBL, and try to better match your responses to the register of the Triple Jump.

Example 2

Try to think through the concrete application of your educational goals. You clearly have a general game plan of what you want to do, but your strategies are still a little fuzzy. While you responded to my prompts for further explanation (your discussion of using a ceramic tile art project to help build classroom community is a case in point), you sometimes struggled to explain what your objectives would look like in actual elementary classrooms (e.g., how you might use literature circles and specific books to counteract bullying). Fear not, I am certain that the required assimilation to practice will come as you spend more concentrated time in the classroom this coming term.

Example 3

Try to articulate your ideas more clearly and robustly. While I know that part of your difficulty in expanding on your thinking was the result of Triple Jump nerves, as you progress through Term 2, you must develop the confidence and depth with which you relay your commitments about teaching and learning. Consider joining an organization like Toastmasters. They provide time-tested techniques for effective communication and a supportive community in which to practice. Perhaps you also can make speaking out in our large group discussions at UBC a goal for your practice in Term 2. You might try jotting down what you want to say before speaking to help you find your voice. Learn to trust your knowledge and yourself.

The third and final TJ (TJ3) has the explicit intention of inviting the preservice teachers to trace and present their movement from novice to (more) expert practitioner. How have the preservice teachers developed as teachers and people throughout the course of the program? What fundamental pedagogical commitments have come to underpin their professional priorities, planning, and practice? How do they understand their greatest, ongoing strengths and challenges? This broader reflective piece is designed to allow the preservice teachers to reflect on their growth over time. As such, I will present its format and purposes later in the discussion of Principle 3.

Principle 3

“Students should be able to engage in self-assessment, evaluation, and reflection as the basis for future continuing professional development and self-directed learning” (MacDonald and Savin-Baden 2004 p. 7).

The self-assessment practices woven into our TELL through PBL cohort pedagogy are carried through to each of the TJs.⁷ The first two TJs ask the preservice teachers to complete a self-assessment using the same TJ rubrics and criteria for success that the examiners use. This element supports them in evaluating their own strengths and challenges as learners and professionals.

Currently, we are in the process of integrating an additional reflexive element into the first two TJs. This element builds from the case syntheses generated by the preservice teachers at the end of each case cycle.⁸ In the 2013–2014 school year, we have asked the preservice teachers to review their past syntheses and come to the TJ ready to talk about how their understandings have changed from the time of their original writings. This looping back is meant to impress upon the preservice teachers that the process of “becoming teacher” is perpetual and ongoing. It also underlines the importance of the self-reflective, biweekly syntheses. So far, it is too early to assess formally how this new reflexive practice is impacting the preservice teachers’ professional development or self-directed learning. We anticipate that it will achieve the end described by Macdonald and Savin-Baden (2004): “it is through peer, self and collaborative assessment that [PBL] students are able to make judgments about how well they are learning and not just how much they have learned” (p. 5).

Building upon the opportunities for self-assessment included within TJ1 and TJ2, TJ3 encourages deeper reflection focused on retrospective and potential growth. It is built around the production and presentation of a professional e-portfolio. This e-folio includes a Statement of Educational and Teaching Philosophy, as well as the preservice teachers’ analyses of their strengths and challenges in relation to the Standards for the Education, Competence and Professional Conduct of Educators set out and enforced by the British Columbia Teachers Regulation Branch .⁹

In some contexts, professional portfolios are framed as employment-searching tools, becoming a pro forma exercise designed to highlight assets and demonstrate competencies. While preservice teachers may choose to frame their e-folios in this way, we encourage them to understand this final TJ as a tool for continued development, where they may explore the contradictions and challenges inherent in both their professional journeys and the teaching profession itself. Regardless of which framing the preservice teachers choose to pursue, the e-folios are an extension of the self-reflective work they have undertaken throughout the year, giving them a final opportunity to pause and assess who they are becoming as teachers and people.

At the center of the e-folio is the Statement of Educational and Teaching Philosophy . In it, the preservice teachers articulate the “whats, whys, and hows” of their most fundamental pedagogical commitments .

What

What should be the proper aims of schooling and education? What is “good” teaching? What do they want their students to learn, do, or know as a result of having known them? The strongest responses define the language used with exceptional clarity. They also convey a deep understanding of what the preservice teachers’ pedagogical commitments mean to them, set within a context of how others might interpret differently the purposes of schooling and education.

Why

Why are the preservice teachers’ larger pedagogical aims valuable? Why do their classroom goals serve their students, both during and after their schooling? The strongest responses offer ample, well-supported justifications for the preservice teachers’ stated educational and pedagogical objectives. They build a compelling case grounded in deep, passionate, and original commitments.

How

How will they implement their educational and pedagogical aims? How will they attempt to influence the practices, relationships, and contexts of their classrooms, schools, and communities? The strongest responses include brief yet specific examples that paint vivid portraits of what their practices actually look, sound, and feel like. Not only do they help to illuminate the statement of philosophy, but they also demonstrate that the preservice teachers’ have thought through the complexities of schooling.

Despite having worked all year with the “what, why, and how” framework, the preservice teachers are often challenged by the focus and honesty required of this concise piece. Writing the statement requires them to deeply and sincerely contemplate their professional and personal development . It requires them to identify, articulate, and synthesize the fundamental pedagogical commitments that are coming to ground their work as teachers. As such, it supports them in developing explicit and well-supported frameworks for their own decision-making. It prepares them to enter into public debates about education with multiple, varied stakeholders. Their statements of philosophy give the preservice teachers a sound foundation for ongoing reflection and dialogue, whether they are encountering contradictions in their own practices, being called upon to defend their professional choices, or advocating for their fundamental values.

The reflexivity inherent in writing their statements of philosophy also provides a strong grounding for the other written element of TJ3: their responses as critical educators to the Standards for the Education, Competence and Professional Conduct of Educators . To construct a response, the preservice teachers choose artifacts from their course and practicum work, linking each artifact to one of the eight standards.¹⁰ They then write a minimum 200-word reflection on how their chosen artifacts demonstrate personal growth, professional competency, and engagement with the standard. To help them build more coherent pictures of who they are becoming as teachers, we encourage the preservice teachers to integrate the commitments expressed in their statements of philosophy into the artifacts they choose and the reflections they write.

Preservice teachers may choose to frame their responses highlighting only their strengths, successes, and full compliance with each standard. However, we ask them to consider aiming for a more complicated conversation. How do they understand the wording and meaning of each standard? In what ways do they, or others, think each standard is important? We suggest that they apply a critical lens to the standards, considering what is at stake in and assumed by each. We ask them to imagine what the standards might look like in practice. When so located, what tensions emerge within and between the standards? How might they act in response? The strongest responses demonstrate the preservice teachers’ efforts to understand and appreciate what is at stake in the standards and their complexities, while also clearly voicing their own passions, journeys, and challenges as developing teachers. Thus, while the e-folio signals a reflexive end to their programs, it also involves looking forward toward the inspirations and tensions that will continue to face them throughout their careers.

Principle 4

“Assess what the professional does in their practice, which is largely process-based professional activity, underpinned by appropriate knowledge, skills, and attitudes” (MacDonald and Savin-Baden 2004 p. 7).

In crafting the structures, processes, and activities of the TELL through PBL cohort, we continually return to the professional knowledges, competencies, and dispositions required in the context of elementary teaching . What do we hope the preservice teachers will come to know and be able to do? Who do we hope they will be as people and teachers? Good teachers possess well-informed, well-supported, and ever-growing knowledge bases. They adapt and apply their knowledges in chaotic, ever-changing environments. They exercise good judgment supported by good reasons. They inquire into the curriculum, the world, and themselves. They see complexity in their classrooms, schools, and communities . They problem-solve collaboratively. To assess these processed-based professional activities , the TJs seek to assess the preservice teachers’ knowledges, skills, and attitudes.¹¹

Curricular knowledges form the foundations in which the good judgments and practices of professional educators are based. How best to assess these subject-specific knowledges in a cohort that uses an integrated, competency-based approach to assessment? How best to assess individual content areas when our rubrics do not include subject-specific, content-based course objectives?

To encourage a serious engagement with the curricular knowledges associated with every programmatic content area, we use a “you choose two, we choose one or two” model for exploring the TJ case issues.¹² Preservice teachers first choose two of the issues to be unpacked, working from strength and building confidence. Based on the gaps that seem to be emerging from the dialogue, instructors then choose one or two issues for continued exploration. This flexibility ensures breadth in the examination. Depth is achieved through the concession that not every issue can be formally taken up during the exam. However, as the preservice teachers don’t know in advance which of the issues will be explored, they come prepared to discuss the content of all.

The development and assessment of knowledges is only the first step. The TJs also assess the skills, abilities, or competencies that underpin what elementary teachers do in practice. At the heart of our competency-based assessment is the “what, why, and how” framework described above.¹³ For every case issue taken up during the first two TJs, the preservice teachers demonstrate their professional competency by explaining the specific meanings of the concepts they employ, the reasonings that supports their analyses, and the concrete implementations of their action plans.

To satisfy the “what,” preservice teachers must be able to explain how they understand the content knowledges they have acquired. They must be able to express the complex, academic concepts in plain language. They must be clear about what their language means when taken up in the literature and school system, as well as in their own use. Finding conceptual clarity helps the preservice teachers to imagine what these concepts look like in practice. Moreover, speaking accessibly and succinctly is an essential part of effective communication with parents and fellow educators. Fixed ideas about what concepts mean, along with assumptions about how others are using them, can lead to profound disagreements within educational debates . Finding a way through the murkiness of abstract language and oft-used buzzwords is an essential part of finding common ground with others and constructive paths forward.

To satisfy the “why,” preservice teachers must be able to articulate good reasons in support of the content knowledge they have acquired, as well as their ensuing pedagogical judgments. It is not enough for teachers to know what they want to do in a classroom ; they must also know why they want to do it. Developing good reasons that synthesize the insights of academic literatures and professional experiences helps to clarify commitments and provide frameworks for decision-making . It helps ensure greater consistency between professional priorities and practices . Moreover, for preservice teachers to be successful in the highly political and contested arena of public schooling , they must be able to convince others that their judgments are wise and their actions are well justified. These abilities are the basis for arguments in support of the professional autonomy of teachers.

To satisfy the “how,” preservice teachers must be able to apply the content knowledge they have acquired in specific, ever-changing contexts. They must move beyond a general understanding of abstract principles to the concrete details of educational practice . It is not enough for teachers to vaguely know what they want to do in classrooms; they also must work through what such a plan looks, sounds, and feels like. Knowing concretely how to implement educational and teaching objectives is the difference between a well-meaning teacher and an excellent one.

This “what, why, and how” framework is also at the heart of the assessment rubrics for the first and second TJs. As the rubrics state, for each issue discussed, preservice teachers are expected to:

• Communicate ideas and understandings in a clear, coherent, and articulate manner.

• Provide well-justified rationales for their responses/action plans, as well as for the pedagogical commitments that inform them. Rationales should demonstrate an understanding of both academic and professional knowledges.

• Articulate concrete responses or action plans to the issues. These descriptions should apply academic and professional knowledges to elaborate what the response would look like in practice.

Each element of this “what, why, and how” framework is interdependent, with the strongest TJ performances displaying a deep understanding of and consistency between them all.

In a 1-year teacher certification program , even the most effectively structured course of studies cannot equip preservice teachers with all of the knowledge they will need throughout their careers. They also must emerge from their studies with particular ways of knowing, abilities, and attitudes. The first two TJs assess how the knowledge acquired is held, applied, and transformed in response to evolving contexts: The final TJ also foregrounds understanding, reasoning, and application.¹⁴ Taken together, the three TJs test what teachers actually do in practice.

Principle 5

“Assessment should be based in a practice context in which students will find themselves in the future – whether real or simulated” (MacDonald and Savin-Baden 2004, p. 7).

Our pursuit of assessment that simulates the realities of professional practice is perhaps best demonstrated by the evolution of the TJ toward an increasingly competency-based, open-ended, and dialogical format. Elementary school teachers tend to work in complex, ever-changing contexts that require flexible responses grounded in consistent goals for purpose and practice. Over time, the TJ cases and rubric have been updated to better emulate these messy, real-life contexts, thereby better supporting the authentic and meaningful assessment toward which Principle 5 strives.

When I first came to PBL in 2006, the TJ emphasized the assessment of content knowledge . The rubric was a pastiche of subject area objectives that had been plucked straight out of the course outlines used in the rest of the B.Ed. program. For example, the content-based criteria related to just two of the nine subject areas included:

EDUC 317 Education Psychology : Special Education

• Identifies a variety of pathways to learning that take into account a variety of learners

• Addresses many challenges of, and strategies, for working with children with exceptionalities within the regular class, including working with supportive services, parents, and communities and making specific visual and sensory adaptations designed to help a student with autism be successful in learning

• Identifies challenges of involving some parents in the special educational needs of their children, highlights social factors that impact parental involvement (e.g., English language proficiency in immigrant communities, working conditions for both high- and low-income parents)

LLED 310 Language and Literacy Education

• Identifies the components of a balanced reading program, describing and discussing a wide variety of components and their implications for teaching, including guided reading and literature circles

• Identifies appropriate and diverse tools for the ongoing assessment of student reading, including running record

• Identifies how the reading program could be adapted to meet the needs of individual students

The lengthy, itemized list that resulted was unwieldy . Examiners could not reliably keep track of how well the preservice teachers had met the multiple objectives of each standalone course. Moreover, a 30-min exam simply wasn’t enough time for the preservice teachers to demonstrate both breadth and depth of content learning . Not to mention that acquiring content knowledge is only a very preliminary first step toward professional competency. In the real world of teachers, how knowledge is held, applied, and transformed in response to evolving contexts is equally essential.¹⁵

Not only was the rubric geared more toward information retention and recitation than competency-based processes like problem-solving and reason-giving, so was the framing of the TJ cases. An excerpt from TJ1 in 2008 reads:

This fall your goal is to put in place a balanced literacy program that will meet the needs of all your students. The reading levels in your class range from emergent to fluent. How would you assess your struggling readers and adapt your teaching to meet their needs? You are also looking into teaching math through problem solving as a way to engage all your students and develop their thinking. (Emphasis added)

In naming specific pedagogies, like a balanced literacy program or a problem-solving approach to math, the TJ cases unnecessarily foreclosed the methods and rationales that the preservice teachers might choose to explore in their responses.

To correct these limitations , the TJ cases were rewritten as murkier, open-ended narratives that challenge the preservice teachers to find their own paths forward. The parallel revised excerpt used in 2009 reads:

The reading levels in your class range from emergent to fluent. This fall your goal is to replace the current reading program with one that will meet the needs of all your students. You are also looking into ways to teach math to develop their thinking and get them more engaged.

Opening up the case in this way creates space for variable responses. It invites the preservice teachers to voice their own priorities, rationales, and practices. As such, the exam more closely mirrors the experiences of practicing teachers .

To further enhance the “realness” of the first two TJs, we have shifted the emphasis of the oral components from presentation to dialogue. The original rubric made no explicit mention of the form of exchange envisioned. As a result, what often resulted was more independent preservice teacher monologue than dynamic, responsive dialogue. However, in the practice context of teachers , who are public professionals accountable to diverse stakeholders, exchanges are often open-ended and unpredictable, requiring flexibility and responsiveness.

The dialogical format now specified in the TJ rubric simulates these realities. While the preservice teachers start off the exchange by briefly presenting how they understand the big ideas and issues of the case, examiners quickly enter into the mix with questions that grow out of the analyses offered. If a preservice teacher says she would use a problem-solving approach to math, I might ask how to explain to parents this approach, along with its benefits and its challenges . If describing the implementation of a guided reading program, I might ask for clarification about its specific strengths and weaknesses for Indigenous students or English language learners. As an examiner, this Socratic dialogue is very exciting; I can often see in-the-moment learning taking place as the preservice teachers integrate or extend a piece of their prior learning.

The TJs have evolved not only in response to our self-inquiries into our cohort’s purposes and practices,¹⁶ but also as a response to shifts in the realities of the broader UBC and British Columbian contexts. In 2012, UBC introduced a fully revamped B.Ed. program. Concurrent with this programmatic change, the Problem Based Learning cohort merged with Teaching English Language Learners cohort, becoming “TELL through PBL.” With this merger came the challenge of how to infuse the priorities and practices of TELL into the learning and assessment structures of PBL.¹⁷

Moreover, concurrent demographic shifts within British Columbia are bringing tangible changes to the working conditions of K-12 educators . According to the BC Ministry of Education (1999, updated 2013), “Students for whom English is a second or additional language (or dialect) are a growing segment of British Columbia’s K-12 school population. Over the past 10 years, the number of students identified as needing ELL services in BC has more than tripled” (p. 4). Similarly, the Vancouver School Board notes that within its district “25 % of K-Grade 12 students are designated ELL [and] 60 % speak a language other than English at home.”¹⁸ These demographic realties are having a profound impact on the practice contexts in which our preservice teachers find themselves now and into the future, where “many students are unfamiliar with the English alphabet or with Canadian traditions, history, culture, education systems, and lifestyles” (BC Ministry of Education 2009, updated 2013, p. 5).

These programmatic and demographic shifts gave rise to the most recent evolution of our TJ assessments. In both the first and second TJs, there is now a requirement that the preservice teachers address the opportunities and challenges inherent in school communities populated by English speakers of various proficiencies .¹⁹ The strongest TJ responses infuse the principles and practices of TELL into the discussion of every case issue, demonstrating how the cultural and linguistic resources of ELLs can assist all learners.

In sum, the TJ cases and rubrics have evolved toward increasingly competencies-based, open-ended, and dialogical formats that better represent and respond to the changing realities of professional educators in BC. The flexibility inherent in the TELL through PBL structure, where modes of learning and assessment are programmatically independent but internally integrated, has enabled us to better simulate these messy, real-life contexts, enabling us to better assess the professional competencies sought by the TELL through PBL cohort, as well as the UBC B.Ed. program as a whole.

Principle 6

‘‘Students should begin to appreciate and experience the fact that in a professional capacity they will encounter clients, users, professional bodies, peers, competitors, statutory authorities, etc. who will, in effect, be ‘assessing’ them” (MacDonald and Savin-Baden 2004, p. 7).

Assessment rooted in multiple, diverse standpoints takes place throughout the program, so that preservice teachers can learn to understand and respond to competing perspectives, purposes, and practices within educational contexts .²⁰ Likewise, the TJs require the preservice teachers to develop an appreciation of how differences in personal standpoint, institutional constraints, and professional responsibilities will impact both how others position them and how they attempt to position themselves. As the TJ1 and TJ2 rubrics state, preservice teachers are expected to “identify the key issues of the case and articulate the reasoning that frames these situations as issues. This involves considering from the perspectives of multiple stakeholders why the situation deserves contemplation. For whom do these issues matter and why” [emphasis added].

TJ2 explicitly integrates this engagement with ever-widening perspectives by adding the role-play element described above.²¹ Examiners choose to inhabit their principal or parent roles in very different ways across many different preservice teacher interviews. In the past, I have played the parent of a myriad of (imagined) children, including a highly imaginative English language learner who plays piano by ear, but has difficulty focusing and lacks organizational skills; an academically gifted but socially shy child whose aunt has a similar-sex partner; a born performer who loves to sing and dance, but struggles with written output and seat work; and a gender-nonconforming child who asks big picture questions, but has trouble with reading comprehension. Sometimes, I am helpful and cooperative. Other times, I present nothing by challenges. Every character requires different quick-witted adaptations from the preservice teachers. The role-plays of TJ2 require the preservice teachers to understand how diverse populations perceive and assess them, so that they can anticipate and respond effectively.

For TJ3, the diversity of assessment standpoints expands yet again. In constructing their e-folios, the preservice teachers must respond explicitly to the Standards for the Education, Competence and Professional Conduct of Educators that are monitored by the BC Teacher Regulation Branch.²² In demonstrating their appreciation of how these standards might be intended, interpreted, and complicated, they effectively enter into a dialogue with their profession.

During this TJ, the preservice teachers’ peers also enter into the assessment process from their unique standpoints. The final piece of the e-folio is a public interview conducted between pairs of preservice teachers. Interview questions might include the following: how has the process of selecting artifacts, creating reflections, and articulating a philosophy of education transformed you; what reflection was most difficult to write; or what do you still wonder about the standards? An open question period follows, where the entire cohort and all prior instructors are invited to enter the conversation. The feedback for this element of TJ3 rests with the preservice teachers themselves: Each interview pair receives structured peer feedback. Thus, this final celebration of the preservice teachers’ growth further extends the diversity of perspectives with which they must engage.

Not only is each TJ internally structured to vary the perspectives and priorities to which the preservice teachers must respond, but they also are so structured across the TJs. We take special care to ensure that different instructors and tutors assess each preservice teacher at each TJ. In a similar vein, tutors never examine any of the preservice teachers from their own small tutorial groups.²³ We also do not disclose the examination pairings ahead of time, so that the preservice teachers will not tailor their TJ review to their expectations of how particular examiners may focus their questions. Moreover, many instructors chose to focus their exam questions outside of their specialties, because we generally already have sufficient data about the preservice teachers’ performance in our own areas. As Principle 6 suggests, narrow preparations serve well neither the current learning of the preservice teachers nor their future careers; the TJs aim to ensure that assessment is located in a wide variety of standpoints.

Conclusion

Every TJ experience is unique. Examiners come from different areas of specialization with different understandings of good teaching. They assess different preservice teachers who come with their own distinct experiences and assumptions. These shifting positionalities provide preservice teachers opportunities to practice navigating social locations, identities, and the competing demands made of professional educators.

Every TJ is also the same, shaped by the same principles of good assessment. Macdonald and Savin-Baden (2004) provide one way of articulating these principles in relation to TELL through PBL’s current and evolving practices. Our TJ cases, rubrics, and activities strive to capture the deep professional competencies sought by problem based learning and requisite for good teaching. In the process of continuing to inquire into and improve our assessment practices, our cohort enacts the kinds of knowledges, abilities, commitments, and self-reflexive development that we encourage in preservice teachers.