Keywords

1 Introduction

Recent conflicts in Iraq and Afghanistan have highlighted the importance of language, regional expertise, and culture (LREC) capabilities to mission success [1–3]. For DoD purposes, general 3C can be defined as the knowledge, skills, abilities, and attitudes (KSAAs) that enable Warfighters to successfully operate in foreign, often unfamiliar, cultural environments. General 3C focuses specifically on the ‘C’ of LREC capabilities. In the last decade, the Department of Defense (DoD) has invested a substantial number of resources to better understand what general cross-cultural competence (3C) entails in operational settings, how to train Warfighters to develop these capabilities, and how to assess the extent to which Warfighters possess them (see [4, 5] for reviews). Recently, the DoD guidance [6] has also urged all branches of the U.S. Armed Forces to take the necessary measures to enhance the LREC capabilities of its Warfighters.

As part of this DoD effort to grow LREC capabilities (and specifically 3C) across the force, Naval Special Warfare Advanced Training Command designed, developed, and is administering a general 3C course as part of the pipeline training of Naval Special Warfare (NSW) Operators. More specifically, an integrated DoD-industry team developed the 40-h training course to enhance the general 3C skills of NSW Operators, with a particular emphasis on supporting the cognitive demands inherent in NSW missions. The course combines interactive classroom exercises and guided discussions, computer-based exercises, homework assignments, and immersive practical exercises (PEs) to learn, apply, and practice 3C skills. This paper focuses specifically on the design and development of immersive, mission-centric PEs that students complete in a mock village while interacting with international role players.

2 The Development Process

2.1 Development of Learning Goals and Instructional Cadre Composition

Because the development of the immersive, scenario-based PEs took place within the context of the full general 3C course, some details about the development of the course are relevant to provide context. Prior to the development of PEs, we developed the learning goals for the course based on the review of a diverse set of resources:

  1. a.

    Training guidance provided by the DoD [6] and U.S. Special Operations Command [7].

  2. b.

    Models of 3C in operational settings (e.g., [8–10]).

  3. c.

    Existing curricula to train 3C to Operators.

  4. d.

    A needs assessment conducted on NSW Operators for a previous effort.

The resulting learning goals included, for example, ‘understand how culture impacts one’s and other’s cognition and behavior,’ ‘make sense of and learn from novel cultural situations,’ and ‘think about the world from the perspectives of individuals from diverse cultural backgrounds.’ The course learning goals guided the development of classroom sessions, activities, and PEs, as well as assessment criteria and rubrics used during culmination exercises.

We used a task-based, student-centered, pedagogical approach during the development of the 3C course, which is in alignment with the approach used to develop the language curriculum taught at the NSW LREC Program. We also decided that the instructional cadre for the 3C course should include an instructor with a background in anthropology and multiple facilitators with operational experience. Typically, there is at least one facilitator for every five students. This arrangement allows the course to balance theoretical rigor and operational relevance.

2.2 Initial Development of PEs to Train and Assess General 3C Skills

A key component of the course is a set of cognitively authentic, mission-centric, increasingly complex, scenario-based PEs that students complete in an immersive, interactive environment. These PEs are integrated within the 3C course and provide a platform to train and assess students’ 3C knowledge and skills.

While the scenario-based PEs completed in the village support all of the learning goals of the course, the following two goals are their primary focus:

  • Use general 3C knowledge and skills during the preparation, planning, and execution of NSW tasks to increase mission success in cross-cultural situations.

  • Reflect on one’s own and team’s cognition and performance in cross-cultural situations.

The structure of PEs is consistent with several cultural frameworks: Byram and Nichols’ cultural knowings [11], Kolb’s experiential learning theory [12], Crawford-Lange and Lange’s content and process [13], and Lussier’s saviors [14]. Students learn from in-class activities and PEs through reflecting on their own experience and performance. The primary role of facilitators during PEs is to promote students’ learning through questioning and promoting self-reflection. Importantly, the PEs completed in the village also provide an ideal platform for instructor and facilitators to assess students’ 3C knowledge and skills. The cognitive authenticity of the scenarios places students in situations in which they display behaviors akin to those they would display during real-world cross-cultural encounters.

In order to ensure the operational realism and relevance of the scenarios, we reviewed documented mission sets of NSW Operators and interviewed several NSW Operators. These interviews focused on the types and range of missions performed by NSW Operators, the extent to which 3C skills were needed during operations, and the identification of critical incidents to guide the development of village scenarios. In addition, the facilitators in the instructional cadre, who are current or former NSW Operators themselves, provided invaluable input to improve the scenarios.

In terms of the mock-village design, we leveraged existing capabilities at the NSW LREC Program and tailored aspects of the village to fit better the learning needs of the course. The physical buildings within the village already existed for other training purposes. We designed the mock village stations to expose students to a variety of plausible situations (e.g., village leader house, market, café) and cultural backgrounds. Language teachers and other foreign nationals served as role players in the mock village (e.g., one station had role players from Philippines, another station from South Korea). A different foreign language was spoken at each station. Students were encouraged to leverage any foreign language skills they might have, but in most stations, they had to communicate through interpreters. Even though the culture and language at each station was different, for the purposes of the scenarios all the villagers belonged to the same community and had a common backstory. While not realistic, we chose the extreme multicultural makeup of the village because it provided pedagogical advantages over a village with a single culture. That is, it provided an accelerated learning environment to develop general 3C skills, as well as an ideal platform to assess the extent to which students displayed these skills across cultures.

2.3 Iterative Testing and Revision of the PEs

We developed the scenario-based PEs in multiple phases. In February 2015, we tested an initial, one-day version of the PE immediately after the last day of an existing 3C course. The primary goal was to explore its feasibility as a culminating training exercise and a testbed for the development of authentic 3C assessment. This one-day version of the PE showed great promise and was the foundation of the culminating PEs that students will complete on day 5 of the NSW 3C course.

Within a month of completing the initial one-day pilot, we completed the first iteration of the full 40-h 3C course. In this first full version, students visited the village twice (days 3 and 5) to gather atmospherics and assess the overall situation within the village, and conducted two negotiations with local villagers (days 4 and 5). These two visits to the village and two negotiations remain a central part of the village PEs in the current version of the course. Over the next several iterations, however, we have continued to improve the village scenarios to better represent the mission demands of NSW Operators, to better serve the pedagogical goals of the course, and to provide a more consistent village narrative. Role players have received extensive training since the first full course, which has resulted in more elaborate and consistent characters. Additionally, the instructional cadre has further elaborated the village story, resulting in a more realistic village feel. Throughout this process, our team has continued to work with the instructor and facilitators to revise the assessment rubrics used by facilitators during the exercises.

In addition to further developing the PEs tested in the initial full course, we added two more PEs to the course based on feedback received from students. First, we added a second, shorter negotiation to day 4. This second negotiation gives students an additional opportunity to apply their negotiation skills while interacting with foreign nationals, but also contributes to the backstory for day 5. Second, and more importantly, we added an additional visit to the village at the beginning of day 1 (before formal instruction begins). Prior to the addition of the day 1 village visit, students often reported that it was difficult to relate day 1 and 2 classroom content to operationally relevant situations and often questioned the relevance of classroom content to their missions. The day 1 visit addresses many of these concerns and it also helps students realize what they do not know, appreciate the challenges of cross-cultural environments, and appreciate the value of having interpreters (which are not available in this initial visit). The final design, which has three visits to the village (days 1, 3, and 5) and three negotiations (two on day 4, one on day 5), is described in more detail below.

3 Key Design Features

This is not the first course that uses scenario-based PEs or that implements immersive, interactive PEs in a mock village with role players. However, the design of our immersive, scenario-based PEs provides high levels of cognitive authenticity, unique pedagogical opportunities, and unique opportunities to assess general 3C skills during actual cross-cultural interactions. In this section, we summarize some of the features that we think make our scenario-based PEs unique and ideally suited to train and assess general 3C skills.

3.1 Village and Scenarios

Single, Evolving Mission (From Day 1 to Day 5). We designed the village and negotiation PEs that take place throughout the week as stages of a single, evolving mission narrative. For example, each time they visit the village, there is an evolving backstory justifying the visit, but all of the visits are to the same village and fit within the larger context of the ongoing mission. Furthermore, each visit also targets different learning goals of the course. For example, the primary goal of the first visit on the morning of day 1 is to make students aware of their own cultural biases and challenges, expose them to the importance of 3C skills to their missions, and help them realize that 3C is a challenging topic.

Students go back to the village on day 3, after the instructor has introduced 3C concepts and skills in a classroom setting. During this visit, students need to build rapport with the villagers, and gather intelligence and atmospherics about the village. This second visit to the village also provides repeated opportunities to learn how to interact with, and manage, interpreters in operational settings. Students can apply what they learned in subsequent iterations across the stations in a safe but realistic environment. By design, students visit all six stations, providing repeated opportunities to practice 3C skills in different cultural settings, make operational mistakes, and receive feedback from their peers, facilitators, and role players on their performance.

On day 4, students complete two negotiations, one with the village leader and another with a government official from the host nation. These negotiations provide a unique opportunity to apply lessons about persuasion and influence, as well as negotiation basics, in a cross-cultural context. They also set the stage for the culminating PE on day 5.

The PE on day 5 serves as a culminating exercise for the course. At this stage, students apply what they have learned throughout the course and it is an opportunity for facilitators to assess the extent to which students show improvement. Students are tasked with gathering additional intelligence and atmospherics (e.g., leverage points, power players) throughout the village. While the day 5 visit is an opportunity to apply the skills they have learned, it also offers new and unexpected challenges for students to test those skills. Following the visit, students report what they learned, receive guidance, and conduct one more negotiation with local villagers to accomplish a specific mission objective. In day 5, students lead their own debrief in which they reflect on their own performance and receive feedback from their group members. The facilitators and role players provide relatively less feedback to promote this self-reflection and group discussion.

Multicultural Village with a Common Backstory. In the early stages of the development process, we had considered designing the village with a single culture. In fact, and in spite of its obvious pedagogical advantages to teach general 3C skills, some of us were skeptical about the feasibility of a multicultural village. Ultimately, students benefited immensely from getting exposure to multiple, unique cultures in quick succession. Especially for students who had never traveled outside of the U.S., the multicultural village allowed them to experience first-hand differences in customs, communication styles and norms, deeper cultural differences, and the impact of these differences. The multicultural makeup of the village allowed us to create an accelerated learning environment for 3C skills that would have been impossible otherwise.

While the culture at each station is different, all stations are part of the same village, which has a common history and organization. When they visit the village, students often gather intelligence about the history, current events, and organization of the village. Keeping the village story straight across stations was initially challenging, but as role players become more experienced, the common backstory also becomes more consistent.

Working with Interpreters. Since the role players at each station speak a different language, we assigned a separate interpreter to each station. At every station, students have to meet a new interpreter, introduce themselves, set their relationship, and work with him or her within that station. While having to meet six different interpreters within the span of 2 h and having only a few minutes to set the relationship with them was unrealistic, it provided an accelerated learning environment for students to realize what was important when working with interpreters. Actually, during the visit to the village on day 3, students typically go from having no previous exposure to interpreters to being able to list the main topics they should cover when meeting a new interpreter over the course of the visit. Over the course of the visit, students typically integrate what they learn about working with interpreters when planning the following station. Good facilitation is critical for this to happen, since the key here is to pose the right types of questions rather than providing the answers.

3.2 Practical Exercises

Multiple Opportunities to Practice. The only way to develop cognitive skills is to practice repeatedly [15]. The PEs provide multiple opportunities for students to practice their observation, sensemaking, and communication skills while interacting with individuals from a variety of cultures. For example, during each of the last two visits to the village, students engaged with foreign role players at six different stations (e.g., tea shop, market, village leader house). The ability to complete six of these rotations in a 2-h period allowed students to engage, make mistakes, learn from those mistakes, and integrate those lessons into their next interaction. In addition to the village visits, students also completed three separate negotiation PEs in day 4 and 5. The numerous opportunities to engage in hands-on, mission-centric, interactive PEs with foreign nationals created an ideal setup for 3C skill development.

Self-Reflection and Multifaceted Feedback. After each interaction (i.e., rotation at the village or negotiation), students complete a debrief session with their team. This debrief aims to promote reflection on one’s own and team performance. During the first few rotations of day 3, each student also completes one rotation as an observer to encourage them to observe the behavior of their group members during the interaction. In these rotations, the student observer leads the debrief session by describing what he observed about his group’s performance while standing on the sidelines, focusing primarily on cultural elements. Immediately after each of these debrief sessions, the team has a few minutes to prepare for the next station. Facilitators encourage students to integrate the lessons learned from reflection and feedback into their plan for the next station.

Once all students have been a student observer once, the student who led the interaction begins the debrief session by assessing his own (and his team’s) performance, followed by the other group members discussing what they observed. The facilitator ensures that the debrief stays on track and provides additional feedback at the end. The facilitator also invites the role players for that station to provide their feedback to students. Often, the discrepancy between how the students and the role players viewed the students’ performance resulted in invaluable learning opportunities. It is important to note that the majority of the debrief period is led by the students. This encourages them to reflect on their performance and learning from their own mistakes, rather than waiting to be told what they did wrong. Facilitators only provide feedback when the students miss key points.

3.3 Assessment of Cross-Cultural Competence

Authentic Assessment and Rubrics. We designed the cognitively authentic, mission-centric scenarios to recreate many of the demands of operational environments. As a result, students facing these scenarios display observable reactions and behaviors that lend themselves to authentic assessment. We also developed an assessment rubric that describes observable behaviors and links them to individual course learning goals and objectives. During day 5 PEs, facilitators use this rubric to assess the general 3C knowledge and skills displayed by students. Assessing student performance within the context of the immersive village exercises makes our assessment more difficult for students to fake than paper-and-pencil assessments. It also allows facilitators to assess the practical application of acquired 3C knowledge, skills, and awareness during both planning and execution of a mission in a cognitively authentic environment. We are continuing to refine our assessment rubric to enhance its effectiveness and ensure that different facilitators can administer it consistently.

Individual Baseline. Students can vary widely regarding their prior international experience and actual 3C skills at the beginning of the course. The assessment rubrics used in village visits are different from those used in negotiations, but the rubrics used on days 3 and 4 are the same as those used during the culminating PEs on day 5. Therefore, facilitators can use village and negotiation assessment rubrics during days 3 and 4 to develop a baseline for individual students. This is an additional benefit of providing multiple opportunities to practice.

The initial baseline can help facilitators identify specific knowledge and skills to target for individual students. It also allows facilitators to assess not only the student’s 3C knowledge and skills at the end of the course, but also the extent to which each student improves as he gains more experience. The ability to identify students who showed dramatic improvement when provided with opportunities to learn and develop 3C skills is important. These students are fast learners when given opportunities to learn and apply 3C skills in context and are likely to improve even further as they gain more experience.

Final Learner Profiles. During day 5 PEs, facilitators assess students on all the learning goals and objectives of the course. As a result, facilitators create a learner profile that specifies the strengths and weaknesses of each student. We have encountered some challenges developing an assessment rubric that facilitators can use to assess students consistently. However, we continue to revise this rubric and work with facilitators to get it implemented. Ultimately, we hope to provide this profile to each student so that he can better understand his strengths and weaknesses, and take the appropriate actions to leverage or improve them during his career.

4 Lessons Learned

The lessons learned from developing and implementing this innovative course are significant, and they continue to expand as we refine the structure and presentation of the course. Even when only considering the village PEs, we have identified several important lessons learned. We hope that these lessons can help others developing similar scenario-based PEs for training and assessment purposes.

Cognitive Authenticity versus Physical Realism. Developing a mock village to simulate a foreign environment may seem like a daunting task at first. However, the physical realism of the village does not need to be high for the PEs to be effective and elicit relevant, observable (and assessable) behaviors. Rather, the emphasis should be on the creation of cognitively authentic, scenario-based PEs that reflect the relevant cognitive challenges that Operators face in the field. For the PEs to work, it is also critical that (a) role players behave and react in realistic ways and (b) how students perform affects both the role players’ behavior and the students’ ability to complete the mission successfully. For example, if a student treats the interpreter poorly when they first meet, the interpreter will act less friendly and be less helpful during interactions than if the student builds good rapport. Similarly, villagers will share more information with students who successfully build rapport during the interaction, which will directly affect their ability to complete their mission (i.e., gather intelligence and atmospherics on the village) successfully.

Pedagogical Value versus Operational Realism. If we created scenarios that reflect operational realities as closely as possible, we would have never developed the village scenarios the way we did. For example, rather than having students engage in a longer interaction with a single culture (which would more closely resemble operational situations), we decided to have students complete six quick rotations through six different cultural settings during each visit to the village. Experiencing six different cultures in such rapid succession within a single village may not be realistic, but the accelerated learning opportunities it provides are beneficial to skill development. Furthermore, this design requires rapid adaptation and is relevant to the types of challenges Operators face in the field.

The PEs provide repeated opportunities for students to practice critical cognitive skills, reflect on their performance, receive feedback, and apply what they learn throughout the process. It also gives students the opportunity to work with interpreters in six different cultural settings, while concurrently exposing students to cultural and customary elements across these cultures. The advantages of this approach go beyond benefits to the student’s learning. It also gives the instructional cadre repeated opportunities to observe students’ behavior, assess their performance, and provide feedback to enhance their learning.

In addition, student reviews of the scenario-based PEs have been overwhelmingly positive in terms of perceived value and lessons learned. Students reported that the PEs were fun and allowed for a unique opportunity to apply the lessons learned in the classroom. Several students even expressed a desire to spend additional time in the village scenarios.

Versatility of Training Scenarios. The village scenarios provided a versatile platform to create different types of PEs. The primary goal of the PE on day 1 is to raise awareness of the importance of 3C to mission success among students, while the PEs on days 3 and 4 provide repeated opportunities to learn experientially. For example, during the visit to the village on day 3 students learn about working with interpreters in an inductive manner. That is, when they meet their first interpreter, students have little guidance about what to do. During the first rotations, they struggle and make mistakes. However, with a little help from the facilitators, they reflect on their mistakes, learn from them, and apply their learning in subsequent rotations. At the end of the day, students improve their ability to develop a relationship and work with interpreters, and have concrete experiences to link to these lessons learned. While experiential learning continues to take place throughout the week, the primary goals of the PEs on day 5 are different. These PEs provide a setting for students to apply what they learned during the class and for facilitators to assess students’ success at applying 3C knowledge and skills while they engage in cognitively authentic situations.

The Importance of Reflection. Reflection is critical for learning to occur and to continue beyond the duration of the course. While the exercises create valuable experiences and promote significant learning by themselves, effective reflection opportunities are a force multiplier for the training effectiveness. Facilitator guidance and observations, role-player feedback and, most importantly, guided self-reflection greatly enhance student learning during PEs. Students often come to their own “aha” moments as they discuss recently completed rotations with their teams, and realize their own miscues or mistakes in interacting with role players. Throughout the PEs, we are promoting the concept of the reflective practitioner [16] by asking students to critique their own and their team’s performance before facilitators or role players provide any feedback. That is, students can work on developing the broader abilities to develop and test hypotheses in real-time, recognize their own affective reactions to events as they occur, and more quickly develop understandings of unique operational environments. Our hope is that students leave the course and apply this reflective thinking across their careers.

Multi-Disciplinary and Resource-Intensive. The resources required to design, develop, conduct, and maintain the PEs described in this paper are significant. A team of educators, cognitive psychologists, anthropologists, curriculum developers, and NSW Operators all worked together during the design, development, and implementation of this course and PEs. While such a multidisciplinary team inevitably results in differences in opinion and occasional disagreements on approaches, it notably improved the quality of the final product.

The instructional cadre is also multidisciplinary: one lead instructor with anthropology experience and a set of highly involved facilitators with operational experience. We have found this to be a synergistic combination. On the one hand, the lead instructor ensures that the course and PEs meet the learning objectives and the students are engaged and grasp key cultural concepts. On the other hand, facilitators ensure that the content and PEs remain operationally relevant, and that the students make the connections to their future missions.

In addition to the development team and the instructional cadre, a group of approximately 20 foreign role players participates in the delivery of every course and set of scenario-based PEs. Given the large numbers of people involved, there are occasional changes that require new people filling positions. While extremely challenging and resource intensive, the value added by merging all of those perspectives greatly outweighs the inherent challenge in managing such a diverse team. We strongly believe this arrangement provides unique benefits that cannot be achieved with a simpler structure.

5 Summary and Conclusions

Our multidisciplinary team designed and developed cognitively authentic, mission-centric, increasingly complex PEs to train and assess general 3C skills. Over the past year, students have completed these training scenarios in a mock village with foreign role players from a variety of regions. We have also developed and used assessment rubrics to assess students’ general 3C skills in this context. Over a dozen classes of NSW Operators have completed this training and the reviews have been vastly positive. Throughout the process, we have incorporated learner feedback to improve the training scenarios and better serve students’ needs.

Developing a hands-on, immersive course to teach general 3C skills is not easy and doing it right can be quite challenging and resource intensive. Multidisciplinary collaboration adds challenges, but it is necessary. We found the DoD-funded research that preceded this effort to be instrumental and helpful in developing the learning objectives and guiding the direction of the training. We also think that the lessons learned from our experience developing this course can help others who face similar challenges.