Alternative Assessment: Student Designed Test Evidence in an Iranian EFL Context

Abstract

In the past decade, educators have realized that alternative assessments are an important means of gaining a dynamic picture of students’ academic and linguistic development (Tannenbaum, 1996). Alternative assessment consists of portfolio assessment, self-assessment, projects, observations, presentations, journal keeping, interviews, and student designed tests (Coombe, Folse, & Hubley, 2007). The purpose of this study is to find out how the process of having students design tests would help them and their classroom teachers in assessing students’ progress and learning. To this end, 120 junior high school male third graders studying at Imam Khomeini School located in Tehran, Iran were selected, grouped, and asked to design tests based on their English course book. Each group was given two chapters of their course book to design tests. Students were free to write as many test items in any format for any skills or sub-skills they would like. The data collection procedure was done in the classroom so they could not use sample test items nor could they use their workbooks in which students can find some sample tests. The collected questions were analyzed in terms of test format as well as skills and sub-skills. Furthermore, in order to have a better understanding of the probable reasons behind designing such tests by the students, both students and their teacher were interviewed. The results of the study revealed that students benefited from their tests, and the study helped them to review the book content in detail. However, the results revealed that students did not pay enough attention to certain key parts of each chapter, and their test-item formats suffered from a lack of variety.

1 Introduction

The importance of testing and assessment is not lost to scholars, teachers, and students in the EFL/ESL literature. Testing and teaching are so closely interrelated that it seems it is impossible to work in one field without being concerned with the other. Among all testing methods, alternative assessment has become a vital tool for educators wanting to gain an understanding of learners’ academic and linguistic developments. Hancock (1994) defines alternative assessment as an ongoing process involving the student and the teacher in making judgments about the student’s progress in non-conventional language strategies.

Student-designed tests are one of the many types of alternative assessment; however, there is a dearth of research on this subject in the literature. Educators believe that student-made tests enable the teachers to see where the gaps in their students’ understanding are and at the same time it provides the students with opportunities to review for an upcoming test and to relate the course to their own interests. Furthermore, student-designed tests are good practice and review activities that encourage students to take responsibility for their own learning (Coombe, Folse, & Hubley, 2007).

Among all the alternative assessment methods, the focus of this chapter is on student-designed tests. We investigated the effectiveness of students’ involvement in testing and assessment to see whether it could help them to better learn the course content and have a better understanding of their own progress. In addition, we set out to know if this type of task could help the teacher diagnose their students’ weak points so as to enable them to find reminders for their area of weakness.

2 Review of the Literature

There is a mutual relationship between language tests and language learning and teaching. Tests assist teachers in a number of different ways. They help teachers diagnose their students’ strengths and weaknesses. They often give teachers insights into the process of teaching and feedback on learning (Bachman, 1990; Brown, H.D. 2004). Thus, testing and assessment have gained great importance in the literature.

Lambert and Lines (2000) define assessment as “the process of gathering, interpreting, recording and using information about pupils’ responses to educational tasks”. Likewise, Coombe, Folse, & Hubley (2007) state that assessment refers to varied ways of gathering information about learners’ language abilities as well as their achievements. Assessment can be either formative or summative. Formative assessment examines the way students develop, but summative assessment looks at what students have achieved during a course of study (Lambert & Lines, 2000).

Regarding assessing students’ development during the course, Hamayan (1995) suggests alternative assessment and defines it as the procedures and techniques which can be used in the context of instruction in everyday school or classroom activities. Alternative assessment might also happen out of a classroom and the subjects being tested might be asked to demonstrate their knowledge in different ways (Smith, 1999; Tannenbaum, 1996).

Tsagari (2004) lists the following advantages of alternative assessment: (a) aids in the evaluation of the process and product of learning as well as other important learning behaviors, (b) enables the evaluation and monitoring of instruction, (c) supplies meaningful results to a variety of stakeholders (d) relates learning to cognitive psychology and related fields, (e) represents a collaborative approach to assessment, (f) supports students’ psychologically, (g) promotes autonomous and self-directed learning, and (h) provides new roles for teachers. Furthermore, she has listed the most common methods of alternative assessment in her article as: conferences, debates, demonstrations, diaries/journals, dramatizations, exhibitions, games, observations, peer-assessment, portfolios, projects, self-assessment, think aloud, story retelling, and student-designed tests.

An approach within alternative assessment is to have students write tests on course material. This process results in greater learner awareness of course content, test formats, and test strategies. As mentioned previously student-designed tests are good practice and review activities that encourage students to take responsibility for their own learning (Coombe et al., 2007; Brown, J.D. 1999), and enable the teachers to see where the gaps in their understanding are and at the same time provide the students with opportunities to review for an upcoming test and make the course more relevant to their own interests. Baron (2004) also argues that if students are to take more responsibility for their own learning, they should be involved in the assessment process, and that students should not merely be passive recipients of results, but have a voice in the designing of some aspects. She believes this procedure (a) involves the students in the assessment process (b) helps students to realize what materials and what types of issues are the most important, and (c) provides a fresh pool of questions and promotes partnership between teacher and the students.

3 Method

The purposes of this study are multi-faceted. The first aim is to determine what students chose to include in terms of (a) test content such as listening, speaking, reading, writing, vocabulary, grammar, spelling, and pronunciation in terms of the number of test items and (b) test format (multiple choice, true/false, short answer, matching, etc.).

Moreover, the researchers sought to find out whether students focus on all aspects of a unit to design their tests or whether certain parts of a unit are considered important for students in designing their tests.

To these ends, the following research questions were considered:

• 1. Are there any specific areas students focus on regarding test content and test format in student-designed tests?

• 2. Will students be creative in their tests in general or will they follow a certain test format?

• 3. Will students consider all sections of a unit worthy of inclusion when writing their tests?

In the following section, the researchers describe the process of data collection and give information about the participants and the instrument for data collection.

4 Participants

One hundred and twenty 14-year old male junior high school third grade students participated in the trial. All of them were studying English as a foreign language and were chosen, in part, because their English teacher agreed to allocate a session for this study.

The school is located in district 15 in the South East of the Tehran province. Students took part in English classes over three sessions a week, and each session lasted 90 min. It was the last month of the educational year in Iran when the researchers had students design questions from their English books. Their teacher, 33, held a BA in TEFL and had 12 years of teaching experience.

5 Instruments

The main instrument in this study is student-designed tests. In addition, we conducted a semi-structured interview with the teacher of the class at the end of the study to gauge his opinion on the probable reasons behind certain aspects of the student-designed tests. The investigators also wanted to know the teacher’s opinion about the merits and demerits of the present study and whether or not motivating students to design tests would foster their learning. In addition, students participated in a focus-group interview to see if they liked the experience.

6 Procedure

Based on the research aims, the students in each class were put into five groups consisting of four students each. All groups were heterogeneous—containing a top, two mediocre, and one weak student. The categorization of the groups was based on each student’s mean score from eight previous formative tests they had taken. Finally, after collecting and analyzing the data, the results were shown to the teacher. The classroom teacher was asked to share his opinions of the student-designed tests and the merits of the research project in a semi-structured interview with the researchers, which took 30 min (see Appendix ‎1 for the interview questions). The interview was recorded, with the interviewee’s consent, then transcribed and analyzed.

Following that, based on the second and third research questions, students were put into groups of 15 to take part in a focus group interview. The interviews which took nearly 40 min were recorded, transcribed, and analyzed. In the interview with students, the researchers were able to find students’ rationale behind the tests they designed (see Appendix ‎2 for the interview questions).

7 Data Analysis

The collected data from the student-designed tests were carefully studied by the researchers. The data was labeled, categorized, and then counted. The interviews were recorded, transcribed, and analyzed. Below is a report on the analysis of the collected data.

8 Student-Designed Tests Results

The test items developed by the students were carefully labeled, categorized, and then counted. The outcome was organised into two main categories: (a) skills and sub-skills, and (b) test format, which will be elaborated upon in turn.

9 Skills and Sub-skills

The first category analyzed is skills and sub-skills. It was important for the researchers as well as the classroom teacher to see which skills and sub-skills were more popular in SDTs. The analyzed data revealed that grammar tests topped the list. As Table 1 illustrates, 62 % of the student-designed test items were grammar related. Next behind grammar were vocabulary items, at 21 %, which is a substantial drop. Pronunciation, at 10 %, was the third most common test item. The last two test item categories were reading and writing with 5 and 2 % respectively. As the table depicts, none of the designed tests contained items related to listening and speaking.

Table ‎1 Skills and sub-skills covered by items in student-designed tests

10 Test Format

The second categorization of the data is test item format. The format of test questions included in the SDTs is important for the researchers to know since they wanted to determine if students see the language as discrete items or integrated.

As shown in Table ‎2, the most common question format in the tests was multiple choice. They top the list at 50 %. The next most prevalent test item format is considerably below MCQ and stands in second place with only 16 %. After short answers, unscrambled sentences were the third most popular item format with 13 %. The last four item formats are transformation, spelling, T/F, and matching with 9, 6, 4, and 2 % respectively.

Table ‎2 Test format in student-designed test

11 Teacher Interview After Data Analysis

The teacher of the class was interviewed to garner his opinion regarding the study. Following this, the recording of the interview was transcribed for further analysis. According to the teacher of the class, the procedure had its advantages and disadvantages; however, the positive points outnumbered the negative ones. He believed that student cooperation in such tasks would result in peer learning and that the resulting student discussions over the right answer will no doubt help students in general and weak students in particular to master the lesson. He stated: “Students learn from each other. They ask questions and discuss the answers and this ends in learning. They also review what I had taught them.”

Also, he believed that the experience of designing their own tests would give students self-confidence in their exams. In addition, there was competition among different groups to write better questions and the interaction among each group member was impressive. Another benefit noted was that this adds variety to in-class activities, which stops the class from being monotonous and boring. Finally, according to the teacher, the most important outcome of the study is that students study the contents of the course book.

However, we recorded a number of perceived disadvantages of student-designed tests. First of all, they were considered time consuming: “It is really time-consuming to spend a session on this task, particularly if this is going to be done every now and then”. Second, a problem that is thought to exist in this activity is task deviation. That is, those groups who finished the task sooner than the rest may distract others in the class because they had nothing else to do. His recommendation for this problem was to keep the students busy with other tasks.

The teacher also stated that based on what students had designed he could find out which part of each lesson was not of interest to students or might not have been learned. This can be determined since there is likely to be no test item from those parts or the items would be very poor. He defines poor items as the ones for which it is easy to find the answers or the test format is not selected appropriately.

12 Student Interview After Data Analysis

Students took part in a focus group interview and the list below summarizes what they think of as advantages of student-designed tests (Table 3).

Table ‎3 Advantages of student-designed tests from students’ perspectives

What students mostly commented on in their responses to interview questions was the revision over the content of the course book. Students believe that the experience helped them to read the course book in depth and find answers to their questions. They also liked the group work and the cooperation involved in writing tests. The discussions they had when coming up with distractors and the right answer helped them to learn better and recognize their weak points. What also interested them was the increased familiarity with different test types, which helps prepare them for future exams.

Students were also asked to name some disadvantages associated with the task. Table ‎4 summarizes their views:

Table ‎4 Disadvantages of student-designed tests from students’ perspectives

As Table 4 depicts, the majority of students (70 %) believe that there are no disadvantages to this activity. However, 15 % of the students stated that some group members were uncooperative, and other group members had to shoulder their responsibilities. Ten percent of the students felt they needed more time (more than an hour) to design better quality test items. In addition, 2 % of the students believed that some group members simply copied test items from test books and used them. Finally, 1 % of the students stated that they were unable to design good tests and needed some sort of training on how to develop a good test.

13 Findings and Discussion

This study has some very interesting positives which stunned the classroom teacher and the researchers. The collected data from the SDTs are a reflection of the Iranian educational testing system context these students are in. In the Iranian context, students have to take tests which are mostly multiple choice questions replete with grammar, especially in special high school entrance exams or university entrance exams. This is in-line with one of the most important findings of this study regarding item types (Grammar: 62 %; Multiple-choice questions: 50 %).

While SDTs contained many vocabulary and grammar questions their formation was found to help students to review course content, particularly if it is carried out in a group. Individuals might consider some specific sections of a unit to be more important than other parts which may lead to an imbalance in course material coverage, whereas in groups, everybody is involved in making decisions on what to include in the design. This group task can therefore provide students the opportunity to read, review, and learn; and consequently have a new look at what they considered unimportant or might not have learned in the first place. SDTs guide the students toward what to study. Therefore, we consider SDT an effective tool in students’ learning.

In addition, SDTs can be a tool for the teacher to provide an engaging opportunity for students to review the content of the course book. Teachers usually design tasks and activities such as role-plays, games, information gap activities, etc., in order to review what they have covered during a month or two. SDT could be used as a fresh alternative for students to review the content.

An interesting occurrence in this study was the reduction of test anxiety mentioned by students. Since students are involved in the process of designing tests, and the final product is shared amongst the groups, students became familiar with a wide range of test items and formats. Since the teacher will use student-made tests as the classroom quizzes, this gives students more self confidence in answering exam questions because students have already designed or studied similar test items in advance. Moreover, they have studied more of the content and have discussed the right answers in groups which particularly helps weak students to learn the content. Thus, students will have less stress in exams since they have had ample preparation and are familiar with exam format and content.

As the collected data supports, there is not much focus on the skills of listening, speaking, and writing in the Ministry of Education books that are being taught in schools in Iran. On the contrary, the books focus merely on reading, vocabulary, grammar, and pronunciation. It seems that the reason why the student-designed tests had a greater focus on grammar, vocabulary, and reading, as Table 1 shows, is the contents of their books.

Considering the format of the questions; students mostly designed tests using MCQs. We believe that the high proportion of MCQs is due to the fact that students have to take an English MCQ test once a month. The purpose of this test is for the school and school teachers to check students’ progress. These tests have influenced students’ point of view regarding test format and test content.

14 Some Guidelines

Based on the results of the investigation, as well as the researchers and the classroom teacher’s experience in this study the following guidelines are humbly suggested for those who are interested in utilising SDTs in their own specific context.

• 1. Allocate some time in your course.

It is crucial to allocate some specific time in our course plans in general and lesson plans in particular. When teachers are thinking and writing a plan for a course of study, they should allow some time for SDTs. Following are some aspects of the plan for a SDT.

• 2. Brief your students on the process.

When students know why they are designing tests and are told about the probable benefits, they will be more willing to design tests. Also, teachers should talk about the process. This awareness reduces students’ stress and puts them at ease when generating tests.

• 3. Group your students based on their abilities (homogeneous/mixed-ability).

Teachers have two options in grouping students: (a) homogeneous groups, (b) heterogeneous groups. Each way of grouping might have its own advantages and disadvantages. However, what worked well in this study was groups with mixed-ability students in which the good students helped the weaker students. In homogenous groups, the weak-student groups might be left on their own and consequently, less learning might occur and the final product (SDT) might not be very effective.

• 4. SDTs should be done in the classroom.

There are many test books, workbooks, and on-line sample tests which are at students’ reach outside the classroom. Moreover, some students might miss the homework and come to class without doing their homework. Due to one of the goals of SDT (group activity, learning, reviewing the content, etc.), it is highly recommended that the task be done in the classroom. This also allows time for feedback during the process. While students are doing the task and writing their tests, teachers can monitor group dynamics and provide feedback where necessary.

• 5. Use students’ final products.

A very important suggestion here is to use students’ tests in one way or another. This will give students a sense of achievement and ownership as well as willingness for further group activities and more classroom tasks. Students’ final product can be posted on the school website, classroom boards, or even used as classroom quizzes.

15 Conclusion

Student-designed tests can assist students to review the course material. Moreover, they foster learning and motivate students to be actively involved in the classroom dynamics. In addition, student-generated tests can broaden the teacher’s view on what students consider important or trivial. Likewise, the student-made questions can mirror the sections that students consider unimportant. By engaging students in designing tests, the teacher can reduce students’ test anxiety and make students like exams and tests. When the classroom quizzes are selected from students’ productions, they can make sure they will get a good mark which helps them to be more confident at the exam session.

Appendices

Appendix ‎1: Interview Questions with the Classroom Teacher

• 1. How did you find the experience?

• 2. Do you think students are capable of designing tests? Why? Why not?

• 3. How do you think this experience will help you in your teaching?

• 4. How will this experience help students in their learning process?

Appendix 2: Interview Questions with the Students

• 1. Did you like designing tests? Why? Why not?

• 2. How did the group work help you in designing tests?

• 3. What did you learn from this experience?

• 4. What are the advantages and disadvantages of student-designed tests?