Keywords

1 Introduction

According to Paulson, Paulson, and Meyer (1991), a portfolio is a selection of learner work that demonstrates the learners’ effort, progress and achievement in a range of areas. In broad terms, portfolios may include writing and speaking tasks, mini projects and learner development tasks. They are ideally compiled over a period of time in order to better represent learners’ development. As Trevitt, Stocks, and Quinlan (2011) put forward, portfolios differ from any other ‘products’ of learning in that they document ‘process’ rather than just ‘product’. They also demonstrate students’ effort and progress over the duration of a course and therefore better represent learning outcomes. Because portfolios provide us with an overall picture of students’ work, they “rescue us from the contradiction in many of the paradoxes or binary oppositions that lie at the heart of good learning and teaching” (Elbow, 1994, p. 40).

In addition, as Paulson et al. (1991) state, the portfolio acts as a bridge between teaching and assessment because it provides ample opportunities for teachers to receive feedback on the development of learners and to do remedial work. If portfolio tasks are done regularly and matched to course content and objectives, teachers can easily see their students’ strengths and areas for further development. Thus, teachers can cater for their students’ needs in a better and more individualized way. According to Huot (2002) and Klenowski (2002), portfolio assessment has become popular because it can serve different purposes at the classroom level. One of these purposes is that it aligns teaching and assessment so as to facilitate productive learning. Portfolios demonstrate students’ cognitive and linguistic abilities in depth and show the progress that they make over a period of time with regard to course objectives. Therefore, with the portfolio approach, assessment becomes more closely linked to teaching practices with the curriculum as the driving force of teaching and learning processes.

The portfolio approach to assessment makes it possible to “break out of the ‘assessment mindset’ that has so long whispered in our ear ‘You can only measure what is easily measurable’” (Elbow, 1994, p. 42). However, learning is a complex process and assessment should reflect this. In that sense, portfolios are a better representation of the complexity and individuality of the learning process because they better reflect students’ actual abilities through a wide range of tasks, multiple drafts, and other aspects focusing on self-assessment and learner development. Therefore, portfolio evaluation helps us address the real assessment issues: “What do we really want in successful students?”, “What are we trying to produce?” (Elbow, 1994). What’s more, portfolios are different from traditional assessment methods in that they “encourage a focus on the importance of discovery, experimentation” (Huot, 1994, p. 325).

Taking all these benefits into consideration and the possibility that it would have positive backwash on everyday teaching and learning, the Learning Portfolio (LP) was implemented in the SL, and has now become an essential part of our program.

2 The Teaching Context and Rationale

The medium of instruction at Sabancı University is English. All undergraduates are required to take the Sabancı University English proficiency exam or bring an equivalent internationally recognized exam score in order to begin studying their major. Otherwise, they enroll in the School of Languages (SL). The SL has an intake of about 700 students a year at different proficiency levels, from zero-beginners to upper-intermediate students. The SL aims to provide students with the necessary foundation skills and knowledge to excel in their interdisciplinary academic studies. In addition to helping students develop their language awareness, knowledge and skills in English, it also helps them develop critical and creative thinking through the provision of high quality instruction and the promotion of independent study.

The SL instructors, task groups and the director are in contact with professors teaching the faculty courses in Sabancı University with the aim of conducting research in order to understand their changing needs. As well as interviews with the professors, we also analyze their course books, attend some lectures, and analyze their assessment methods. Based on the expectations of various faculty courses, we try to align our curriculum objectives and assessment types to better prepare our learners for their future studies. To that end, recent needs analysis research revealed that more work is needed towards improving learners’ writing abilities in the faculties, and thus it required prioritization over other academic competencies.

Considering the above principles and needs, our initial aim was to include a continuous assessment method for evaluating writing in particular. Previously, our assessment practices focused mainly on more traditional, summative means of assessment. However, the feedback we received from colleagues and learners over the years revealed that there was a need in our program for a more continuous type of assessment that focused on process rather than one final product, especially for writing.

From an assessment point of view, the portfolio approach provides teachers and institutions with a tool to evaluate student performance in a more authentic way. Rarely are we required to undertake a writing task under strict time limits. Writing is most often completed in our own time, using various resources if necessary, and is edited a few times until the writer is finally happy with the product. A timed essay, on the other hand, is the most typical form of assessing writing.

Doubtless, the timed essay has benefits over alternative forms of assessment. It is standard, easy to prepare and administer, and ensures every learner takes the exam under exactly the same conditions. Moreover, marking is relatively less subjective with the use of well-written criteria, blind grading, and multiple grading. However, a portfolio better represents students’ actual abilities as it reflects students’ performance over a longer period of time under a variety of conditions. Traditional product-oriented assessment methods focus on two aspects of test usefulness, namely reliability and practicality. On the other hand, the portfolio assessment covers other areas of test usefulness, which are “construct validity, authenticity, interactiveness, and impact” (Weigle, 2002, p. 175). Taking all these into consideration, it was clear that besides the summative writing exams that we already had in our program, there was a need for a more process-based assessment method to cater for our learners’ needs in a better way.

Another priority was to encourage learners to approach their study systematically and to put more effort into their self-development as language learners. We aimed to help our learners become less dependent on their teachers, and to equip them with study habits that would prepare them for their future studies. One obvious benefit is that keeping a portfolio increases students’ learning responsibilities (Barootchi & Keshavarz, 2002). For us, this was a good starting point to break inefficient and ineffective study habits such as memorization or procrastination that were predominant among our learners due to their previous study habits, a lack of understanding of effective study strategies and methods, and cultural reasons. Our attempts to foster good learning habits such as talks, presentations and workshops helped only in a limited way in that they raised learners’ awareness of such issues and strategies, but they were not instrumental in inspiring learners to change their habits. As a result, a portfolio task group was formed to look into possible ways of incorporating the portfolio into our current program.

3 The Implementation of the LP in the School of Languages

The Learning Portfolio (LP) Project at Sabancı University School of Languages emerged as part of the curriculum and assessment renewal based on a thorough needs analysis process—in particular on teacher and learner feedback. The feedback and the needs analysis process clearly showed our learners’ need for a more systematic and more process-based approach to the teaching and testing of writing. Moreover, needs analysis research revealed that more work needed to be done towards learner development, and that the LP could address these concerns. The next section describes the implementation stages of the LP at Sabancı University School of Languages (SL).

4 Decisions Related to Content

4.1 Matching LP Content with the Curriculum

In a portfolio, tasks can be linked to specific curriculum objectives and learning outcomes. When this is the case, tasks can be “geared towards a relatively narrow target language use domain” (Weigle, 2002, p. 179). This enables the teacher to see more clearly the extent to which objectives have been achieved, and which objectives require remedial teaching, thus having a positive backwash effect on instruction.

Therefore, while making decisions regarding the content of the LP, we started by taking a detailed look at the curriculum, course objectives and course materials in order to create LP tasks that well suited the needs of our learners. After curriculum objectives and the desired learning outcomes were identified, these were matched with the tasks and materials already available in our course books. These materials were compiled in the form of a booklet to make the portfolio more organized and to enable it to be implemented and utilized regularly as part of the program. Students were given their portfolios at the beginning of the course. For each task in the LP, learning objectives and outcomes were outlined in detail for both teachers and students. A task checklist that reminded students of the specific requirements was also added for each task in the LP. These requirements and course objectives were also clearly indicated in every LP task to make the task more meaningful (see Appendix ‎1). This was done with a view to allowing teachers to easily identify areas in which their students needed further development and to aid the design of lessons and learning materials accordingly.

4.2 Variety in Task Types and Conditions

For a more comprehensive representation of learners’ abilities, our LP includes a variety of task types. We tried to incorporate tasks that had a good balance in terms of variety, style and requirements because as Weigle (2002) states “the writing ability is not a simple construct but involves numerous processes, and (that) a single writing sample written for a specific audience and purpose is extremely limited in its ability to represent the writer’s ability to write for other situations, audiences, and purposes” (p. 186).

Initially, we focused on writing. We included tasks that were personalized in nature as well as academic text-based tasks. For example, students were asked to write learning diary entries (see Appendix ‎2). In other tasks, students were asked to formulate short answer responses based on an academic text that they had studied in class (see Appendix ‎3). This ensured that students had writing practice in different styles, genres, text types and length. This also gave the chance to students who prefer freer, more personalized writing tasks over academic writing to demonstrate their abilities.

As well as task types, we also gave importance to varying task conditions. The fact that we set the tasks under a variety of conditions made the assessment more authentic. This is because the majority of writing we do in the real world is not done under strict time constraints, and thus, the assessment of writing should also not “rely solely on in-class writing as evidence of writing ability” (Weigle, 2002, p. 185). This also enabled us to give students with different learning styles and preferences the opportunity to demonstrate their learning outcomes in a more suitable way, as traditional assessment types most generally put good test-takers at an advantage. The larger variety of student samples also allow both teachers and students to see to what extent learning objectives have been achieved, and which specific objectives require more attention.

4.3 Fostering Learner Development

The portfolio approach does not merely act as an assessment tool, but also helps students to become less dependent on their teachers. Since learners’ reflection on their learning process is acknowledged as an essential component of education (Wolf & Reardon, 1996), we believed if tasks focused on various aspects of learner development such as self-reflection and goal-setting as well as developing a specific skill such as writing, they could be used as effective tools to foster better study skills and eventually begin to improve autonomy in learners. In the portfolio, we envisaged that some tasks could ask learners to reflect on, assess and evaluate their own learning processes and thus require them to make conscious choices about their own learning. For example, Appendix ‎4 illustrates a personalized task that focuses on learning styles and strategies and self-reflection.

There are several different tasks that are designed for the purpose of self-reflection in the LP. One example of these are the ‘Task Checklists’ which consists of linguistic, stylistic, organizational and content-related requirements for successful completion of a task (see Appendix ‎5). Teachers encourage learners to analyze these checklists for guidance before completing a task and upon completion of a task to reflect on their own work. These checklists also serve another unique purpose which is for teachers to refer to while giving feedback to learners and evaluating the success of a task.

Another self-evaluation method that is used in the LP aims to encourage students to self reflect on their progress in a given period of time. In our case, these tasks are done twice a semester; midway and at the end of the course. Using these tasks, students have a critical look at their own work and identify their strengths and areas for development. Based on ‘Can-Do’ statements adapted from The Common European Framework of Reference (CEFR), students are asked to set their own goals to further develop their learning (please see Appendix ‎6). Following this, teachers evaluate the extent to which the learners have been able to prioritize their goals and learner action plans. Students’ self-evaluation is also discussed in one-to-one tutorials. We believe in the importance of self-evaluation because introducing mutual responsibility between teachers and learners to carry out self-assessment results in improvements in instruction and learning, both through raising awareness regarding the quality of students’ written or spoken work and also fostering in students a more goal orientation outlook in their studies (Fulcher, 2010).

Once a predetermined number of tasks are completed, students review their work up to that point, identifying areas that they need further work on and setting learning goals for themselves. They draw up an action plan specifying areas that require most attention and ways of achieving these goals (see Appendix ‎7). Students are then invited to attend tutorials with their teacher where they go over the goals and the action plans. To encourage students to engage in more in depth evaluation of their work and progress as well as coming up with tangible future targets, teachers invite students to discuss their choices during one-to-one tutorials. Teachers guide their students with questions to train them to think more critically of their work.

It is also possible to design portfolios in a way that caters for flexibility and choice on the learners’ part, which are important aspects of learner autonomy. To this end, in our context, learners are asked to select from tasks they have completed to be evaluated on. They are required to choose and explain their performance on tasks they have chosen, based on given criteria. For example, they are asked to choose the work that best represents their abilities or the one(s) they have shown most progress on. They document their selection and justifications in their portfolio in the designated section and also discuss it with their teachers in prescheduled tutorials. The process of task selection engages self-evaluation and enhances awareness of meta-cognitive processes, thus forcing students to make conscious choices about the quality of their work (see Appendix ‎8).

For such practices to be successful in creating less dependent behavior, learners need extensive scaffolded training and guidance from their teachers. It can only then bring about favorable learning habits including how to take control and responsibility of one’s own learning.

5 Decisions Related to Assessment

Because a LP is a collection of work and thus contains many samples, it may not be realistic to double mark portfolios as we would normally do with standardized writing exams. What’s more, blind marking is favorable in such exams to make the assessment of writing more reliable. However, such a practice would contradict the nature of the portfolio since the focus is not only on evaluating one single product but on the process and development. As a consequence of this, subjectivity may increase. In addition, in order to create a positive backwash on students’ learning and development as learners in the SL, we wanted to also assess students’ meta-cognitive processes such as goal-setting and self-reflection within the LP. However, such processes are not tangible and hard to assess fairly as they are based on personal judgments.

For these reasons, we have taken some measures to increase the reliability and consistency of scoring. Most important of these measures are writing clear specifications and guidelines, supplying task previews, training the graders and conducting standardization sessions, and having a clear set of criteria.

5.1 Specifications and Guidelines

Marking should be carried out with high standards, and marking procedures need to be consistent to yield reliable scores. Clear specifications and guidelines not only help maintain marking across different levels and sections in the same institution, but also make it possible to be more consistent in developing tasks and maintaining institutional standards (Weigle, 2002). Although certain elements of the portfolio need to be flexible, some aspects need to be standardized to achieve consistency. For example, in our guidelines, we saw the need to specify the number of tasks that need to be set as in-class or outside class work, individualized versus collaborative tasks, or the number of free or academic, text-based tasks.

5.2 Task Previews

To make the assessment of the LP as reliable and consistent as possible, under the supervision of a level assessor, teachers preview tasks in order to clarify expectations for task fulfilment before a task is set. We also standardize expectations by going over task checklists (see Appendix ‎5). Teachers brainstorm possible ideas students can include in their responses as well as different ways they could organize their work so that students are evaluated fairly afterwards. This also ensures that different teachers provide similar types of guidance to students when they set the task.

5.3 Standardization

Rater-training and standardization sessions are also an indispensable part of LP evaluation to ensure reliable grading. These sessions take place before each LP evaluation is carried out. It is the level assessor’s responsibility to choose some samples that reflect various ability levels. The teachers read and discuss their views on the samples with regard to the criteria. As well as discussing the main strengths and areas for improvement, teachers also discuss and agree on the grade a specific sample would get. The level assessor supervises the process and assists with emerging issues.

5.4 Criteria

Another way we address the issue of grader subjectivity is through an easy-to-use set of criteria that balances less tangible components of the LP with concrete and evidence-based aspects. For instance, while the number of tasks a student completes or a student’s attendance in portfolio tutorials could be considered concrete and easy to measure aspects of portfolio evaluation, development of learning in response to feedback or identification of strengths and weaknesses are more subjective aspects of the criteria we use in the SL.

A clear set of criteria is indispensable for reliable and consistent grading. The teachers are required to familiarize themselves with the criteria and attend standardization sessions where we grade sample portfolios using the criteria. This is crucial to ensure consistency and inter-rater reliability. Maintaining high standards in grading procedures and ensuring reliability through the use of clear criteria is “especially important in language programs that have several proficiency levels, as it reduces the likelihood that students will be promoted or held back in error” (Weigle, 2002, p. 183). Teachers also make use of the criteria while giving written or oral feedback to their students.

In the SL portfolio, the criteria are provided to the students in the portfolio booklet, and teachers use this page in the portfolio when they are grading the portfolio. This way the scoring criteria “becomes a teaching tool as well as a testing tool” (Weigle, 2002, p. 182). Social aspects of writing can also be encouraged and can be incorporated in the criteria. For example, teachers can evaluate to what extent a student has incorporated feedback on their work (Weigle, 2002). In our criteria, this is evaluated as a separate band, and students receive a score for the degree of progress they have shown in response to their tutor’s feedback. Therefore, they are held accountable for incorporating feedback.

5.5 Avoiding Plagiarism

One risk with portfolios is related to task conditions. Since many tasks are not carried out under test conditions, there is an increased risk that some students may be tempted to get assistance from others while completing their tasks. If the portfolio tasks are recycled in time, there is also the risk that students may get the portfolios from students who were previously enrolled in the program and thus plagiarize. In our case, most of the portfolio tasks are either newly created or extensively revised to cater for the needs of the new student group. However, there are also tasks that are recycled, and the number of students is too large to spot cases of plagiarism if precautions are not taken. To avoid issues of assistance and plagiarism, we follow several guidelines.

First of all, students are provided with written guidelines in their portfolios explaining expectations. There is a statement of academic integrity which students need to sign to show they have read and understood the statement and that the work in their portfolio is their own and completed without any assistance. This helps us to make our expectations clear from the very beginning of the course. The second measure is the use of plagiarism detection software for longer pieces of work. Students upload their work onto our online learning platform, which then compares the written work against other students’, the Internet, and the original text if it is a text-based task. It then detects if and to what extent a student’s work was plagiarized. Finally, we have documented a set of guidelines which clearly indicate what teachers need to do if they suspect a student has received assistance or plagiarized.

6 Provision of Information to Students

Before the portfolio was implemented general guidelines were produced for both teachers and students. Teacher guidelines include information related to task submission. In this part, the details about task setting procedures and plagiarism detection procedures are documented. Feedback and evaluation principles and guidelines are another important part in the guidelines document. These give information about what to pay attention to while giving written oral feedback for individual tasks or after portfolio evaluation; and how to grade portfolios. There is also a separate section on tutorials, as they are an integral part of the portfolio system. Types and frequency of tutorials are also specified.

The student guidelines include information about the rationale for the portfolio, important reminders, assessment related information, and the academic integrity information. These aim to make the expectations and requirements clear and transparent from the first day. There is also a page which shows the calendar of events such as when to submit a certain task, when they will have tutorials, and the evaluation dates.

7 An Assessment of Current Practices and Procedures

The feedback cycle on the LP started even before it was implemented. A specific project group consisting of curriculum and assessment group members had done the initial planning, and designed portfolios for every level. This group then shared their work with the teachers and asked for their opinions, comments and suggestions on the design, content and grading. After revisions were made, assessors focused on their own level, and collected feedback on a regular basis.

Teachers were asked to contribute their ideas, comments and suggestions regularly during forums and meetings such as task previews before a task was set, during standardization sessions after students submitted their tasks, and before LP evaluations on specific tasks. Teachers were also sent detailed surveys in the middle and at the end of the course (see Appendix ‎9).

While surveys focused on the general views, perceptions and attitudes towards different components and design of the LP, the forums and meetings concentrated specifically on either tasks or grading procedures. This provided us with a macro perspective in that it helped us to gather feedback both on how the LP fits with the rest of the program and how it is viewed in general. Additionally, it enabled us to view the LP from a micro perspective and get feedback on specific details in a systematic way.

In the SL, learner feedback is as important as teacher feedback and is taken into serious consideration by graders. This was also the case with the LP since if the students did not appreciate and understand the value of the portfolio, then it would not have the desired effect. Therefore, in addition to teacher surveys, students were also given questionnaires twice a course. Also, every level held learner forum meetings with representatives from each class. At these forums, students’ perceptions of the portfolio were discussed in addition to other items related to course content.

The gathered feedback showed that the students appreciated the value of the LP and thought of it as one of the most useful learning tools in the system. They also believed that the weighting of the portfolio towards their overall course grade could be increased because they believed it is a good representation of their actual performance (see Fig. 1).

Fig. 1
figure 1

Extracts from teacher and student feedback

Full size image

Especially in the first year that we started to implement the portfolio, it was very important for us to collect feedback from both teachers and students to understand if the portfolio served its purpose and in what ways we could further improve it. Both teacher and student feedback from all levels indicated a positive perception of the portfolio. Most of the feedback was quite encouraging and in line with our primary goal of incorporating a continuous assessment method into our program that would bridge instruction and assessment while fostering learner development and enhancing learners’ language competencies (see Fig. 1).

One of the questions in the student survey focused on writing: “To what extent do you agree with the following statement: “The Portfolio has helped me in improving my writing skills”. Of the 92 students who responded to the survey, 94.15 % agreed with the above statement. This proved that learners believed that keeping a portfolio was an essential tool in their learning process.

8 Future Goals

The feedback we received from teachers and students also provided us with ideas and suggestions on how to further improve the LP and what amendments can possibly be made to the current design and content.

9 Continuous Speaking Assessment

One of the most commonly raised suggestions from both teachers and learners was the need to make the assessment of speaking continuous and more process-oriented when compared to the traditional oral exam. Students suggested that such an approach would provide much needed relief for the exam-anxiety they have been experiencing and encourage them to place more importance on speaking on a daily basis rather than studying towards it before the exam. As for the teachers, it was suggested that making speaking part of the portfolio would give them the chance to evaluate their learners’ speaking competencies through a variety of tasks, enabling them to focus on a range of speaking sub-skills.

10 Allowing for Peer Assessment

In our current system we get the students to self-assess, and set their own learning goals as well as draw an action plan. However, teachers also suggested incorporating peer assessment. Such practices are considered highly effective and informative among formative assessment methods. As Brown states (2004, p. 276), “self-peer-assessment are among the best possible formative types of assessment and possibly the most rewarding”. Possible tasks that we are considering for peer assessment are filling out peer checklists and questionnaires, rating someone’s oral presentation holistically, peer editing and peer proof reading. Peer assessment is also important for acquiring meta-cognitive awareness in learning. This is because being able to judge to what extent given criteria have been achieved is the initial step towards becoming able to produce high quality output.

Based on the feedback we received, we are also considering expanding our task variety to include more styles and genres—specifically creative writing. For instance, we are thinking of getting students to write response papers, short stories at lower levels, and answers to document-based questions. In addition, we have started working on creating mini-projects for our learners in order to incorporate elements of project-based learning in our portfolio. This, we believe, will enhance task variety as well as increase the amount of collaborative work. Such additions and changes to the Learning Portfolio will truly supply evidence of students’ learning progress rather than only specific learning outcomes.

11 Conclusion

Having implemented the Learning Portfolio for over a year, we were able to observe several desired outcomes. For instance, there has been an improvement in learners’ commitment to and enthusiasm for developing their skills. We have also witnessed that learner responsibility and awareness towards their language learning in terms of the attendance rates for tutorials, response to feedback, their ability to self-reflect and set goals for their own learning have improved significantly. As the research results also indicate, students appreciated the value and positive effect of keeping a learning portfolio on their learning in general.

Although at times it was challenging for teachers to keep up with the demands of the portfolio in terms of arranging time to give written and oral feedback, the teachers also embraced the Learning Portfolio as an indispensable component of our assessment scheme. As also stated in the research results and the future goals section of our chapter, they would like to make the learning portfolio an even more inclusive tool that could best represent students’ performance.