1 Introduction: Purpose and Testing Context

Intensive English Programs (IEPs) inside universities face different assessment challenges than those of their academic and administrative colleagues from other disciplines. IEP assessments differ from other disciplines in multiple ways. First, students take intensive English either as a form of skills training or a pathway into degree programs. Students do not graduate with a major or minor specialization, and they often study English full-time and exclusively. In addition, IEP programs in postsecondary institutions exist, at least in part, to generate revenue. As such, they often have separate tuition structures, admissions policies, and budgets (Norris 2016; Richards 2017), meaning that assessment approaches within IEP programs are evaluated from a different set of standards than others on a postsecondary campus.

In this specific assessment context, there are three main functional programming units to consider: (1) The IEP provides coursework for international students to attain a level of English proficiency in lieu of having an official TOEFL or IELTS score upon admission. For this context, the IEP is known as the American Language and Cultures Institute (ALCI). (2) The outside pathway provider recruits and places international students into the IEP and other university programs. Providers typically contract with a university via a corporate model to recruit and admit students to share revenue. The outside pathway provider has the purview to place students into academic courses without clear student English language proficiency levels assessed and/or vetted directly by the ALCI. (3) University coursework is attained when a student has reached level 5 (high-advanced) in the IEP. Upon achieving this level of academic English proficiency, students will begin their university career by enrolling in university courses at the 100 level, for example, History 101, Math 101.

Many institutions like the specific university context described above, unknowingly have adopted an English language proficiency policy that reifies languages as static, bounded, and evaluated according to a narrow canon of rules, and it also reifies social identities in terms not of language use but of nationality (Banjong 2015; Schlaman 2019). This ideology supports a limited view of linguistic and social communication in which the ideal speaker is thought to be a monolingual native speaker of a social variety of English. Many times over, Basic Interpersonal Communicative Skills (BICS) (Cummins 1981) are evaluated by English language assessment for higher education decisions. BICS and CALPS are the names given to two broad registers of language, Basic Interpersonal Communicative Skills (BICS) and Cognitive Academic Language Proficiency (CALP), in the 1970s by Canadian educator Jim Cummins (1981). BICS language is sometimes called social language or even survival language, takes only three to five years to develop (Cummins 1981). BICS can be acquired informally, at least in part, through social interactions or in social media. Before educators understood the difference between BICS and CALP language, many students were exited from language programs before they were ready.

Social language discourse practices (BICS) in relation to university course readiness had a firm hold on the evaluation of English proficiency readiness from an international recruiting and marketing perspective led by the university context described. This specific campus context contributed to the IEP having little control regarding the students being recruited and placed into English and academic courses where (CALPS), academic discourse is prevalent. The English proficiency assessment issues could be resolved, if the outside pathway provider was thoroughly probed by campus leaders for a better understanding of the provider’s English proficiency testing practices and academic quality standards of students recruited. For their part, in order to rectify the problem of misplaced students, the IEP faculty needed to innovate assessment and testing measures to ensure academic readiness for the international students learning on campus that were placed by the outside provider into ALCI coursework and other academic courses without proper language testing. The IEP faculty grounded their innovations for language assessment in Assessment for Learning Principles and Design (Assessment Reform Group 2002) to aid in supporting the IEP’s mission to support language learning through best practices in second language acquisition and reach all English levels of the international student population.

An example of a BICS assessment (without CALPS) was provided from our University International Recruitment Office. The office liaison had described an official campus correspondence where many of the students in the new cohort of international students were described to be English proficient due to the fact that the liaison had spoken to them personally on the phone. This anecdote stresses the necessity for a clearer and more comprehensive understanding of BICS campus wide. Here is the assessment challenge: Students were considered academically prepared for the study of university-level content on the basis of BICS-types of assessments sanctioned by the university and the outside pathway provider. Examples of student assessments reviewed consist of phone calls, email correspondences, and the ability to complete an application for university admission. The assessments occurred without language testing expertise or collaboration with English proficiency experts and researchers on campus. Thus, many students began to flounder, for their social English skills could not support their academic endeavors.

Developing learners’ communicative competence is a large part of the ALCI program. The five-level program is based on aspects of communicative competence, including linguistic, strategic, discourse, and sociolinguistic areas. The program and placement system for coursework demonstrates the abilities of students to master all four aspects of communicative competence to create a skillful language user. The program is comprised of five levels from beginner to high advanced. Once a student places into level 5 in the IEP/ALCI, a student is considered to have reached the level of English proficiency required for university coursework. Students take 20 hours of face-to-face coursework, and students are sequenced into courses that specialize in specific content areas, cultural events, reading, writing, listening, speaking, and grammar.

Recruited international students should be placed into the IEP based on grade point average (GPA), high school coursework, and English proficiency assessments (among other factors) set by the university. However, the students on campus are placed via admissions and the outside pathway provider based on provider business practices, GPA, and individual student English capabilities set by the provider, typically outside the realm of academic rigor.

2 Testing Problem Encountered

The BICS’s effect on the assessment of multilingual students in university IEPs, particularly creates a financial burden for students and risk of academic probation that results from a student’s delaying fulfillment of the first-year English requirement for three semesters. However, a financial gain for the international pathway provider and the university is created. On one side, the university would gain from the assessed placement into the lowest level of the three-course ESL sequence, for it requires a university-sponsored pathway for the English proficiency program to garner more revenue and support students’ English proficiency due to the longer duration of time spent in the program by students. On the other side, the outside pathway provider markets and promises quicker entry into the university course work not based on English performance. Thus, English assessment and subsequent placement are often overlooked or considered not a priority. The course sequence consists of a basic introduction to writing for international students, a course to support academic presentations, and a Freshmen Year English writing course designed for an international population.

The responsibility of the IEP for making decisions regarding international student language skills has proved to be complex, especially when multilingual learners are seen to need support services from the university like tutoring or extended class times. For example, the IEP can offer individual assessments for students, occasionally circumventing the ineffective placement process developed via staff on campus. Such forms of language support structures, helpful though they may be and easy to accomplish given this autonomy, allow non-native English-speaking students to be placed outside of the normal university curriculum, rather than to be supported across the curriculum by means of an inclusive placement structure for English assessment. Thus, unintentionally, the IEP constructed its position as an unofficial campus gatekeeper that may help English language learners navigate outside the structure of the university. A greater limitation for the university develops where students work around the curriculum and shelter their linguistic skills from view of the academic community.

Unfortunately, the IEP program is seen as peripheral, non-academically dense, and expendable. This view of the IEP and English proficiency was constructed in part due to the strong influence held on the campus by the outside pathway provider contracted by the university. The outside provider’s business-minded goals do not coincide with student support for English because students are promised a quick entry into university programming based on GPA, educational background, and high school courses. English skill is not stressed or seen as essential for student success by the provider. The outside provider was established on campus before the IEP, and the provider molded many of the academic and English policies and beliefs in existence.

2.1 Pathway Programming Challenge

Outside pathway programming or the “bridge program” for recruitment of international students has had a profound influence on the university and its ability to assess and support an international student population. The suggestion here is that the social language ideology for university readiness (BICS) has such a firm hold on the practices of the campus community that the IEP has little control over the students recruited and placed into English and other academic courses. This situation might be resolved if the outside pathway provider were vetted for English proficiency practices and academic quality of students recruited for the campus. Both the IEP and the pathway provider work with faculty across disciplines to effect changes regarding international populations and how the relationships between campus and international community are viewed. However, the pathway provider position is housed outside of the university structure and chain of command, while the IEP is housed in the English department. The pathway provider is for profit and earns funds from the students; the IEP program is part of the larger non-profit side of the university. The rest of the university, then, is symbolically absolved of responsibility for educating multilingual populations. The university has unintentionally constructed the IEP’s position as the place where the English of international students is policed and debated on campus. Many “problem English students” are sent to the IEP, and the IEP is provided with limited knowledge of pathway recruited students’ English level or academic background.

2.2 Assessment NEED

This assessment challenge developed out of a necessity to support students that were evaluated outside of the expertise of ESL professionals and the utilization of BICS assessments to support the stance of academic readiness. Currently, all of the ALCI student population share the same schedule of courses. However, each individual student is provided with an individualized path of language study due to the varied levels of English proficiency of recruited students admitted. A shared multilevel classroom creates a language learning environment equivalent to a “one-room school house.” Inside the walls of this “contemporary one-room school house,” the instructors and pioneers of inclusive placement-intensive English instruction found a need for a single multilevel assessment using the same source and/or material (e.g., TV episode) to support all of the students within the four walls. The assessments support the varied linguistic levels of the students in the program. The assessments include beginning and end-of-term assessments, and other formative/summative assessments to enhance instructor knowledge of students’ language skills. The IEP has innovated a new means of testing to alleviate some of the stress of multiple levels of proficiency in one class, for both students, faculty, and the campus.

The student population hails from all parts of the globe. In the past three years, the IEP has hosted students from India, Pakistan, Jordan, Iraq, China, Japan, Colombia, and Vietnam. The international student population age range is from 18 to 25. The initial English proficiency for students recruited for the IEP is high-beginner/novice based on ACTFL proficiency levels. The program is designed for students to spend no more than three semesters in the intensive English path of study.

3 Solution to the Problem

3.1 Adoption of AFL

AFL assessments were implemented by the IEP as a multilevel solution to the problem of students being misplaced into academic course by the outside pathway provider. Designing the multilevel assessments for English proficiency is founded in Assessment for Learning Principles (AFL) (Lee 2011; Lee 2017; Lee and Coniam 2013). AFL is an educational framework built around 10 principles seeking to assess students in a way that creates awareness of their current skills and knowledge gaps, that provides the ability to map future learning and goals. In 2002, the Assessment Reform Group released 10 principles to consider when incorporating AFL in the classroom:

  • Is part of effective planning

  • Focuses on how students learn

  • Is central to classroom practice

  • Is a key professional skill

  • Is sensitive and constructive

  • Fosters motivation

  • Promotes understanding of goals and criteria

  • Helps learners know how to improve

  • Develops the capacity for self-assessment

  • Recognizes all educational achievement

The IEP’s implementation of this assessment idea is founded upon authentic language use and extending the concept of BICS to CALPS. Walking around the campus, IEP instructors frequently heard students discussing events from their favorite TV shows or movies. This observation led the program to employ TV shows and movies as frequent topics of conversations due to the fact that in all cultures, television creates an authentic language learning medium. Therefore, one show/genre was selected for the entire term, and an episode was shown each class.

AFL enhances learning in the classroom by treating assessments as a process where learners display their knowledge and skills and then analyze their responses to map out future learning (William 2011). Therefore, it is not just the students participating in the assessment, but also the instructors. Instructors, in tandem with their students, analyze the assessment results and decide where learners are in their learning, where they need to go, and how best to get there (Assessment Reform Group 2002). As shown above, AFL design principles are complex and cannot be realized in isolation; instructor/student collaboration is key to identify and account for interrelationships between teaching, learning, and evaluation. This process, when applied appropriately, is crucial in developing students’ confidence and motivation for language and culture acquisition, for both summative and formative testing situations. In the end, AFL, for the purposes of this testing selection, illustrates the pivotal role assessment plays in reinforcing and extending learning and learner autonomy in language learning settings (Dann 2014; Lee 2017).

These AFL-backgrounded assessments serve as placement tests for the beginning/end of term. However, to prevent students from being stressed about their performance (key to AFL), students are evaluated according to AFL principles on the entire learning process. Therefore, if the students perform to their best ability and complete all testing sections, test performance will not negatively impact grades. AFL emphasizes motivation without negatively impacting students who progress at a slower rate. For example, students are graded on their classroom participation, two-three smaller assessments, and their classwork as a whole. These newly introduced AFL-based tests function as a final assessment that highlights the students’ skills and serves as a placement test for the following term. The tests only have a negative impact on the students’ grade if they put in little to no effort (scoring lower than their current level). The tests place significant emphasis on the writing process: students’ knowledge of the ability to revise work using resources. AFL stresses the importance of a continuous feedback loop between instructor/student to foster oral and written academic work. The students must demonstrate that they are capable of both skills before being placed at the university-level coursework and performing with native-English-speaking peers. Students are given the level of their performance following the test while the information and process are still fresh in their minds. Immediate feedback allows the students to ask more specific questions about their performance and plan ways to move forward effectively.

3.2 Assessment Descriptors

Using AFL tests for varied proficiency levels, instructors show episodes of an American sitcom (a situation-comedy show from television) for students to review throughout the semester/term. The sitcom functions as a focal point in and out of the classroom for activities, content area focus, and assessment. A thirty-minute episode of a sitcom serves as a basis for student-generated and accessible knowledge during a class for evaluation. The consistent and familiar scaffolded content and contact with specific characters, social situations, accents, cultural phenomena, etc., provide more equity and balance in the classroom for introduction to knowledge and skill sets.

In regard to content or material, the IEP faculty chose American sitcoms because they are generally 30 min and provide 2–4 storylines each episode. This structure allows for multiple examples and activities to be taken from the show based on each storyline. Additionally, as students watch more of the show, students complete language-specific assignments focusing on season-long plot lines (especially the more advanced students). Students are asked to perform lesson or test tasks immediately following the episode, to practice their ability to intake new information and material and then reflect on it in speech or writing, as they would in an academic course.

Assessment focuses on four key aspects:

  • A familiar TV show and set of characters.

  • Parts 1–2: closed-book answers on the material/plot. The length and complexity of these parts are based on the level of the student. For example, the high beginners are given content-specific questions and only required to answer in complete sentences. The low-intermediate students are asked to perform the task of writing a summary, using the writing process.

  • Part 3: open-book revision. Students are given the chance to check over their work and make corrections in colored pens.

  • Reflection. Students are asked to identify what they did well, what they struggled with, and what sources they used (ranking the sources for helpfulness on a scale of 1 to 10).

Test Protocol:

  • Explain vocabulary words for the chosen episode (included on test)—10 min;

  • Watch episode (or short clip if needed for time constraints)—30 min;

  • Complete Parts 1 and 2 with closed books and notes—1–1.5 h;

  • Using colored pens, complete Part 3: revision with open resources—30 min–1 h;

  • Complete a self-reflection questionnaire and turn in—5 min.

4 Insights Gained

The multilevel AFL assessments were designed to enhance English proficiency evaluation, discover curriculum improvements, and find the knowledge gaps of the international student population, in addition to assigning accurate level placement. Traditionally structured tests (using test item formats such as cloze or fill-in-the-blank) were not giving accurate representations of students’ abilities to produce and understand English in a university setting, which led the program to introduce more open-ended and performance-based test items in placement exams. The tests presented here were adapted from a series of classroom activities that received high levels of interest from students. The assessments produced increased student participation and production of spontaneous English (both written and spoken). The tests led to increased peer dialogue, in-class discussion, and analysis of the TV show. Furthermore, instructors could summarize how well students understood what they were watching and hearing, and synthesizing information from recent episodes, when speaking or writing in class. Language learners deal with multiple complexities during the assessment process of coding and decoding messages from the classroom to the sitcom. Even for native speakers, the process of forming thoughts and ideas and expressing them coherently through language is not a simple endeavor. The assessment presented supports students’ “strategic competence” to employ a number of strategies to communicate in and out of the classroom. Moreover, this assessment process focuses on competence strategies that have traditionally received little attention in language learning settings, and serve a more pervasive role in and out of the classroom. Students acquire strategic competency strategies through AFL testing that include: confirmation checks, avoidance, and commands. The strategies are meant to be thought of as fluid and spontaneous parts of a student’s language acquisition capabilities and use (Ellis 1997; Lee 2017).

As a final insight, incorporating the entire writing process (outline, write, and revise) into the three parts of the test helped the students realize the effectiveness of the AFL on the quality of their written work. Instructors review each assessment designed for content, using our innovative guide that is focused on three main categories: Material Comprehension, Unrevised English Skills, and Revision Skills. See Table 19.1 for a breakdown of each category:

Table 19.1 Assessment content review guide
Full size table

The AFL multilevel assessments follow the same procedures used in class activities to maintain a comfortable situation for the students, and emphasize key understanding of the requirements of the language tasks. The assessment ranks each category on a 5-level basis to reflect the ALCI/IEP structure of courses. A passing level would be considered the current level or above. If a student scores below their current level, then a one-on-one meeting will address whether the low score is due to misunderstanding or lack of attention on the student’s part. This process fosters student motivation (per AFL) by allowing the student to focus on content and production rather than grades (Lee 2017). By comparing respective student performance to the level expectations (both their current level and the exit level), students are able to see improvements and gaps of these particular skills in a meaningful and constructive manner (Lee 2011). Additionally, by scoring the tests according to level placement rather than a fixed score, instructors help students remain focused on the overall goal of graduating from the IEP and building their English skills.

5 Conclusion: Implications for Test Users

The IEP and AFL assessment model shown by the assessments presented here, provides a clearer examination of students’ English proficiency to perform BICS and CALPS successfully by the students enrolled in our campus IEP. This approach creates much-needed transparency for the process of evaluating students’ language levels and readiness for university coursework evidenced by the student context studied. Most importantly, the AFL assessment created supports authentic language use, mimicking the real-life study skills that students need for academic achievement in university-level courses (Lee 2011; Lee 2017). The possibility of further research and inquiry exists to investigate the AFL assessment model with a myriad of different contexts, student populations, language proficiencies, and instructional practices.

AFL is a holistic process and is not achieved by individual educators, university staff, outside programs, and a campus working in isolation. Instead, it is paramount that everyone involved in this AFL process, and international programming, work collaboratively to review curriculum and plan a comprehensive program that takes into account the interrelationships between teaching, learning, and assessment for international student language support. The campus can then develop strategies to support BICS and CALPS with all stakeholders involved. To implement AFL, a campus needs to define and communicate goals and expectations clearly to international students, provide them with opportunities to engage in language learning rather than reduce them to passive examinees, and prompt them to take responsibility for learning. AFL should be considered a key professional skill for instructors in Intensive English Programs, and a consideration for continuing professional development for internationalizing a campus through language.