A Cross-Cultural Instructional Design Case Situated in a Global Workplace Learning Context

Abstract

We conducted a needs assessment and instructional design project to deliver training to international nongovernmental organization humanitarian logistics staff in a multinational context. We describe the context, artifacts, and training tools developed through application of instructional strategies including modeling, spaced learning, scenario-based learning, simulations, and beyond blended learning modalities, of this design case. We also provide background on our critical design decisions, our application of Edmundson’s (The cultural adaptation process (CAP) model: designing e-learning for another culture. In: Edmundson A (ed) Globalized e-learning cultural challenges. Information Science Publishing, Hershey, 2007) cultural adaptation process model, and provide links to all materials, which were published under the Creative Commons license.

In this chapter, we discuss the design of training materials for a workplace learning project aiming to build the global supply and logistics capacity of local partners of international nongovernmental organizations (INGOs) in the humanitarian sector. Instead of providing training from a central place, such as at organizations’ headquarters or regional hubs, a project team set out to design performance improvement materials, tools, and techniques for local staff of the INGOs in the beneficiary countries to use themselves. The project team consisted of a project coordinator, instructional designer, and subject matter expert trainer, who were managed by one of the INGO consortium member organization’s managers.

What does it take to develop a performance improvement solution for distinct target audience groups that are located across the world, in some of the countries where humanitarian organizations operate? We present the context, describe the artifacts and critical design decisions. Also, we discuss the distinctive aspects of the design, development, piloting, and implementation of this performance support and training program. This program includes job aids, operational systems assessment tools, online learning, train-the-trainer, and participant workshops to support the development of knowledge and skills in the field of supply and logistics of local partner organizations’ staff.

Context

INGOs operate independently from governments and receive funding from private donations, charitable foundations, and/or church organizations. An INGO is like a nongovernmental organization (NGO), but it operates internationally and often has a presence in multiple countries. According to the World Bank (1989), INGOs are “characterized primarily by humanitarian or cooperative, rather than commercial, objectives,” and they “pursue activities to relieve suffering, promote the interests of the poor, protect the environment, provide basic social services, or undertake community development in developing countries” (para 2). Thus, to enhance logistical efficiency, in the past 10 years, international humanitarian aid organizations have begun to change how they deliver aid in crisis situations. Instead of staff from their headquarters or regional offices directly delivering aid, they have begun partnering with local community-based organizations whose staff deliver aid themselves. These local organizations are more adept at facilitating advocacy, policy research, civil society building, etc. They do not necessarily have the talent and capacity to deliver aid during crisis situations. Hence more recently, INGOs have begun to provide training to the staff of these organizations so that they can independently prepare for and effectively respond to crisis situations, as they arise. An advantage of working with local partner organizations is that they have better knowledge of the local markets than aid workers who are flown in from other countries. Building the capacity of local partner organizations in supply and logistics knowledge and skills can improve the efficiency and effectiveness of delivery of humanitarian aid. Capacity building encompasses a very diverse and wide range of activities, processes, knowledge, and skills that organizations undertake to focus on performance improvement by managing organizational culture, relationships, and limited resources allocated across different stakeholders, within and across systems, to effectively meet strategic goals and deliver to a mission. Capacity building may focus on rural economic development, environmental development or protection, schools, and the social sector and at various levels (e.g., community, organization, or individual) (Aref, 2011; Bain, Walker, & Chan, 2011; Barker, 2005; Hinrichs & Richardson, 2015; Wing, 2004).

Historically, members of the supply and logistics department in the humanitarian organization Oxfam Great Britain (Oxfam GB) designed performance improvement solutions and training that headquarters staff delivered to their local partner organization staff in priority countries. As time went on, this delivery model became unsustainable for a variety of reasons. These reasons included high staff turnover rates in INGOs and local partner NGOs, as well as changes in the selection of local partner organizations, and more scrutiny of operational functions from donor agencies such as the European Commission Office for Humanitarian Aid and Civil Protection (ECHO) and the US Agency for International Development (USAID). There was a constant need for retraining, which became impossible to meet. Hence, the Oxfam Great Britain (Oxfam GB) supply and logistics performance improvement team, which consisted of the deputy department head and one of the authors of this chapter who was the department learning and development project manager, came up with a new approach to design performance improvement and training. This new approach shifted the responsibility for implementation from a centralized headquarters to local INGO staff in country who train the local partner organization staff. ECHO showed interest in co-funding a pilot project, if Oxfam GB would collaborate with other international humanitarian actors.

In 2013, ten humanitarian INGOs formed a consortium to develop a capacity building program aimed at the supply and logistics function. The project was called Partner Capacity Enhancement in Logistics (PARCEL) project. Instead of the historic centralized implementation approach, the consortium aimed at decentralizing the training. The goal of this decentralization was to create a more sustainable relationship between the INGO office supply and logistics staff located outside of the UK, near to local partner NGOs, and the local partner organizations staff responsible for supply and logistics activities. The participating organizations were World Vision International, the Save the Children International, Concern Worldwide, Mercy Corps, Tearfund, Oxfam America, Oxfam Australia, Oxfam GB, Oxfam Netherlands, and Oxfam Spain.

The program costs approximately €400,000, and ECHO only allowed 18 months to complete the project. ECHO provided 80% of the funding for this 2-year project and the INGOs 20% or 2% each. None of the ten consortium members would have been able to afford the full cost of the project alone. By working together, the INGOs kept their share of the costs down.

At their headquarters in Oxford, UK, Oxfam GB hosted the project team consisting of one project coordinator, an instructional designer who is also an author of this chapter, and a subject matter expert trainer with experience in humanitarian logistics, under the management of the deputy head of supply and logistics. When this deputy head left the organization, the learning and development project manager of Oxfam GB’s supply and logics department, who is also an author of this chapter, managed the project team under the supervision of the head of supply and logistics. Oxfam GB also provided administrative assistance. Additionally, several interns supported the project team, and staff from the consortium agencies sometimes volunteered their time.

Artifact

The PARCEL project team set out to develop training materials for international agencies’ staff to adapt and use when building the logistics capacity of their local partner staff. Based on a set of the agreed standards created by a consortium, the team developed a large package of performance improvement and training materials, including:

• An assessment tool to assess local partner organizations’ adherence with the standards and identify potential capacity gaps.

• A logistics toolkit with forms and templates for partners to use in implementing the PARCEL standards systematically throughout their organizations.

• Pick-up-and-go training materials to address capacity performance gaps, including a train-the-trainer package and a partner training package.

The project team tested the materials in five countries: Pakistan, Ethiopia, Jordan, Mozambique, and Haiti.

Standards

At the start of the project, the project coordinator and instructional designer, along with the consortium representatives of the supply and logistics function from each of the ten INGOs, came together in a 2-day meeting to agree on the minimum requirements that logistics processes and procedures must meet. Some of these requirements are given by rules and regulations of donor agencies. For example, donors will require that INGOs use a competitive bidding process for purchases over a certain amount. In the meeting, everyone shared their best practices in several areas of logistics. The participants grouped standards into asset management, procurement, warehousing, fleet management, distribution, and cross-cutting issues such as health and safety.

Based on the interests of individual participants and existing best practices in their organizations, the topics were divided up at the end of the meeting to work out details. Several participants volunteered to produce draft standards in one or more of the areas, sometimes collaborating with other participants. Drafts were then shared electronically and refined in several feedback rounds using email, cloud-based file sharing, and phone-conferences. Once agreed upon, the standards formed the basis for all materials that the project team developed.

Performance Support Materials

The performance support materials prevent unnecessary training and support the target audience to build capacity for each of the standards. The performance support materials designed for this project included both an assessment tool and a logistics toolkit. All source files are available on ParcelProject.org in the Resources page links. These performance support materials are designed for use by both the partner NGO staff and their sponsoring INGOs.

The PARCEL Partner Assessment Tool helps partner organizations assess their logistics systems capacity against the PARCEL standards. Specifically, the tool helps identify the resources, internal procedures, and policies to comply with the PARCEL standards and reveals areas for improvement. The assessment focuses on the five areas of the standards mentioned above. The PARCEL Partner Assessment Tool is for use prior to selection of individual training modules, which may be desired to support partner organizational capacity building goals. Together, a sponsoring INGO and partner NGO staff can assess the strengths and opportunities for organizational capacity building in humanitarian supply and logistics through the systematic analysis of current operational practices.

The logistics toolkit is a collection of forms used in five of the six logistics areas to support the organizational processes and systems required to meet the PARCEL standards. Each area includes a guidance document, which provides explanations on how and when to use the tools. The tools are recommended forms from consortium agencies of the PARCEL project. They have been designed to be as generic as possible and may be modified to suit the needs of the organization. No tools were available for the cross-cutting standard. The logistics toolkit is for use after the implementation of select training modules, based on organizational capacity development goals. NGO staff can select and modify specific form examples for systemic implementation of the PARCEL standards in operational practices, which would potentially be supported through coaching from the INGO.

Training Materials

The training materials are organized for the two different employee groups—NGO partner staff and INGO trainers. The materials for both audiences support instruction and learning needs with facilitator guides, participant manuals, and the ancillary tools mentioned previously. The materials cover all areas of logistics described in the PARCEL logistics standards. The elearning modules can be hosted on an organization’s own learning management system. The Articulate Storyline source files and SCORM output files are available on ParcelProject.org in the Resources page links. For those without an LMS, the elearning materials are available on DisasterReady.org.

NGO Partner Staff

The partner training program was designed around a realistic scenario to introduce local NGO staff members to the logistics standards and the processes necessary to implement those standards in their own organizations. The training materials for this audience include elearning and face-to-face workshops.

The elearning materials are divided into six 15-minute modules that introduce the standards in a scenario-based format, as shown in Table 1. In scenario-based learning, a series of decision points leads the learner through different actions prescribed by the standards (Clark & Mayer, 2012). Learners make decisions in mini scenarios presented to them, with the logistics standards as guide. For example, in the asset management module, the learner gets the role of the logistician responsible for asset management and needs to decide what to do when a new laptop gets delivered to the office (Fig. 1). The scenarios represent authentic workplace cases where employees would make contextualized decisions aligned to desired logistics systems performance.

Table 1 Scenario-based elearning modules and descriptions

Fig. 1

Example screenshot of elearning module on asset management

The face-to-face workshops materials took the form of what we called pick-up-and-go packages (available on ParcelProject.org in the Resources page links). For each standard, several distinct sessions were planned with an associated package. The packages are such that anyone with some background knowledge of the subject matter and in training should be able to pick up such a package and deliver the session. Each package includes at least facilitator’s notes, containing a detailed script of the session, including timings, presentation notes, and descriptions of learning activities. The guide also describes the best setting for the space and materials needed. If applicable the package also includes instructional training materials such as slides, handouts, and descriptions of activities. Examples of these instructional training materials are shown in Fig. 2, 3, 4.

Fig. 2

Examples slides

Fig. 3

Examples handouts

Fig. 4

Examples description of activity

INGO Trainers

The INGO train-the-trainer program was designed to prepare INGO staff to do the capacity building in country. The intended sequence of the training materials includes introductory information provided in elearning courses, which preface instructor-led training events. Lastly, follow-on activities are included; they are designed to sustain learning on the job, after the training concludes.

The six elearning scenarios previously mentioned are also made available to INGO trainers to support a shared mental model and framework for supply and logistics capacity building among collaborating organizations. We added a seventh scenario-based module for the train-the-trainer materials to introduce the capacity building model. This module is available in the same modalities and systems as the partner elearning materials. It is about sustainable learning—the process of building capacity and applying learning to real-world situations and then reflecting and adjusting one’s approach.

During the face-to-face workshop, the INGO trainers learn how to support NGO partner staff learning about how to put the standards into practice in their organizations. During the workshop, they learn to answer common questions that arise about the standards during partner training. The workshop also simulates the partner training environment by providing participants an opportunity to practice training skills by teaching one partner session to other INGO participants.

After participants complete the elearning, at the end of the face-to-face workshop, they develop an action plan to describe what they want and need to properly implement the standards in their organizations. The action plan is a contract between the organization and the INGO that they partner with. The INGO supports the implementation of the action plan after the workshop conclusion through continued meetings and group web conferencing events.

Additionally, sustainment activities are provided to facilitate an ongoing community of practice, to support learning and model the desired behavior with training-of-trainer’s participants. An example activity is a webinar for participants who use the assessment tool. The instructional designer developed a second example webinar, where the subject matter expert facilitates an interactive webinar session. In this session, participants shared experiences and results of their capacity building efforts with partners and practice coaching skills using Skype.

Critical Design Decisions

Needs Assessment

Although the INGO consortium members involved with the project felt confident in their understanding of the performance improvement needs the project team aimed to address, the instructional designer wanted to survey representative partners. The purpose of the survey was to find out about their experiences of capacity building, their access to technology, and how confident they felt about their organization’s logistics processes and systems readiness to respond in a humanitarian crisis. Persistence paid off, as the needs analysis proved to be a key source of information for the instructional design. In this section, we describe the actions taken to assess the performance and learning needs (Breman, Giacumo, & Griffith-Boyes 2019).

The instructional designer sent a survey to 142 partner organizations in 21 countries (Breman et al., 2019). The 106 responses gave the project team information about the access to technology, willingness to use these technologies for learning activities, and experiences with and preferences for training methods (Breman et al., 2019). The responses debunked some of the misconceptions that the consortium members had about the target audience. For example, they assumed the partners did not have access to technology and would not like to use technology for learning (Breman et al., 2019). The survey showed that organizations had much more access to technology and experience with technology-supported learning than initially thought, which opened the way for a more blended approach (Griffith-Boyes, 2014).

The instructional designer did some research into the use of text messages to deliver micro-learning and/or motivational messages. There are examples where text messaging was implemented in projects in developing nations (see, e.g., Isaacs, 2012). These projects are generally small in scale in a specific country context. Often, they are sponsored by a phone manufacturer and/or local mobile service providers; facilitators send out text messages in the same country. However, when the instructional designer investigated the possibilities to run such projects on an international basis, the service charges were enormous and unaffordable.

In terms of preferences with regard to capacity building options, the respondents asked for a system of performance support. They requested access to experts, support from peer networks, and context-appropriate training tailored to their specific needs. And, they wanted to receive training in a variety of training modalities, including face-to-face and online.

Based on the results from the survey, the instructional designer chose a blended learning curriculum, including a combination of online, face-to-face, and more flexible sustainment activities (Breman et al., 2019). This strategy was chosen because the needs analysis showed that partner staff were willing and able to participate in a variety of training delivery modalities (Breman et al., 2019). The instructional designer determined that the participants’ use of elearning prior to workshop attendance would afford everyone with a common framework and language upon which to build new skills. The project coordinator and instructional designer created a systematic communications approach to take participants first through elearning, then face-to-face, and finally sustainment activities, by distributing packaged email scripts, facilitation guides, and communications materials. They created a systemic approach by involving INGOs and partner NGOs in operational systems performance assessments, workshop selections, and two different levels of workshops, which are each built to integrate deliberate feedback loops, tools to support organizational performance, and ongoing learning partnerships.

Materials

The project team set out to develop materials that anyone could use and adapt for any location in the world to support the development of capacity in INGOs logistics coaches and trainers as well as the logistics functions of local partner NGOs. These materials include both performance support and instructional materials. We detail the most critical design decisions made during the process of creating the artifacts previously described in this section.

Designing for a Global Audience—Is that Even Possible?

International humanitarian organizations respond to emergencies when they occur in so-called failed states (Oxfam, 2015) or at a scale that overwhelms the local government such that they ask for assistance. Where these crises happen often, international humanitarian organizations work with local partners to develop the capacity to respond in country. Regulations governing humanitarian organizations do require many common performance standards across the globe, and there are also some commonly agreed-upon ethical practices across cultures. In this case, we focused on designing performance support and instructional content that related directly to these agreed-upon performance standards and ethics because many partner NGOs cannot access available funding or lose access due to a lack of human resources and operations capacity. While we succeeded in gaining representation from consortium members in critical program design and development decisions, we fell short on gaining representation from partner NGO staff in the form of reviews to support materials design and development decisions prior to pilot testing.

The instructional designer built the overall curriculum design with the principle of spaced learning in mind (e.g., Thalheimer, 2006). This strategy was chosen because the instructional designer was reading about it and the learning and development project manager had used it previously with some success in the organization. The instructional designer used spaced learning by repeating the same points from the elearning in the face-to-face workshops and sometimes in the sustainment activities, which are each implemented after a break and are interwoven with other related content. In other words, we designed a clear learning pathway from building the foundations, enabling more complex and active learning, and then putting new skills into practice through sustainment activities.

We developed the elearning using the principle of scenario-based learning. We chose this approachbecause the more successful learning activities from previous projects in the organization linked new concepts with realistic workplace situations. We created these scenarios in dialogue with the consortium members, the subject matter expert trainer, and the instructional designer.

Simulation learning is a form of experiential learning with a high degree of authenticity in a safe learning environment (Breckwoldt, Gruber, & Wittmann, 2014). We chose this strategy because the learning and development project manager had success with it in previous organizational performance improvement projects. Also, the nature of humanitarian emergencies creates high-stress, fast-paced, and sometimes dangerous environments, where novice learner performance is unacceptable. We implemented this strategy in the partner workshop with a realistic emergency scenario, such as a drought or earthquake as a simulation. Learners work through a series of activities related to the PARCEL logistics standards, learning how to apply the standards in such a scenario. The activities simulate the work they would do in a real emergency. By the end of the workshop, participants complete all elements of a logistics plan for the specific scenario.

In this design project, the consortium members’ goal was to formalize a support system built on a community of practice framework in which individuals support each other’s learning well after the last day of training. This community of practice is thus “organized around professionals who perform similar activities and use their strong social bonds and high levels of intentionality to extend and improve their practices by building a base of shared knowledge” (Smith, Hayes, & Shea, 2017, p. 220). We chose this strategy because of the high staff turnover rate in INGOs and partner NGOs, with the intent to build more connections between new employees and existing employees for the purpose of sharing knowledge. Further, we hoped to move from hierarchal connections between one INGO’s staff and its partner NGO staff to connecting different partner NGOs’ staff to each other. We did this by putting different partner NGOs in the same workshop with networking activities and through a discussion tool on the project website.

Pilot Testing

The consortium members selected five countries for pilot testing: Pakistan, Ethiopia, Jordan, Mozambique, and Haiti. Experienced trainers delivered the train-the-trainer workshops in Pakistan, Ethiopia, Jordan, and Mozambique. The train-the-trainer participants all delivered one or more sessions in the subsequent partner training workshop, under the guidance of the experienced trainers. In Haiti, the team focused on testing the assessment tool and linking the results to targeted partner training. In this section, we discuss what the project team considered in the pilot test planning, the pilot testing, and what can be learned from the pilot tests.

Representation

Between input from the consortium members, the NGO partner staff needs assessment responses, the project team, and the authors’ combined experience supporting performance improvement and workplace learning in diverse locations across the globe, we started the project believing we had adequate representation to build an efficient and effective solution for delivery across the globe. In terms of getting diverse representation, the program and materials design, development, and implementation across multiple organizations and geographical cultures were successful. Representatives from each INGO bought in, contributed feedback during review cycles, and implemented assessments and training for their staff. The pilot testing of the partner training materials across multiple organizations and geographical locations resulted in feedback that also contributed to revisions and polished materials for distribution. In short, we are confident that the extent of representation we solicited resulted in materials that are ready for adaptation for any location in the world.

Cross-Cultural Factors

Edmundson (2007) describes strategies of cultural adaptation of elearning materials for four levels of complexity. The four levels include (1) simple information, (2) hard skills and core concepts, (3) soft skills and complex knowledge, and (4) attitudes and beliefs. From simple to complex, the suggested adaptations are translation, localization, modularization, and origination, where higher-level strategies also require the underlying lower-level strategies. We found this to be a useful framework to reflect on our experiences of the PARCEL project. The content levels for this project primarily focus on lower-level knowledge and skills. This supports our focus on translation and localization.

Translations

Logistics standards and processes were universally applicable, simple knowledge. Thus, we decided translation of the instructional materials would suffice when the target audience did not speak English proficiently enough for workplace performance. Logistics staff from INGOs deemed English to be acceptable for the partner NGO materials delivered in the first two countries, as well as for the train-the-trainer materials for all of the pilot countries. Partner training materials were developed in English and translated to Arabic, Portuguese, and French.

The consortium members overestimated the English language skills of their staff and partner staff in all cases. For example, in Mozambique, six of twelve train-the-trainer participants did not speak English at all. And while the Pakistani trainers did speak English, only a small percentage of the local partners did. In Pakistan, the trainers all volunteered to translate. Thus, the early pilot testing resulted in unexpected time needed to deliver the training, as well as learner frustration, taking attention away from learning. We encumbered unexpected costs in subsequent pilot events when we used professional translators.

When the project team solicited feedback from participants in the workshops, we encountered problems with the quality of the translations in Arabic and Portuguese. The participants in Jordan questioned the quality of the Arabic translation in general. In Portuguese, the problems were related to profession-specific issues. A literal translation does not always reflect the jargon used in a specific field. The project did not have the resources in house, or the time and money, to check the quality of translations in Arabic and Portuguese. All final materials were published in English, as well as partner training materials in French. And even before the project was completed, a volunteer translated the standards, assessment tool, and elearning text into Spanish. After completion of the project, a group of INGOs that work in Spanish-speaking countries funded the translation of the remaining materials.

Localization

Designers can adapt materials to local audiences in several ways. In the Parcel project, the team considered imagery. The adapted scenario materials are based on common geographical crises occurring in different locations.

Images

Given that the project team developed materials for a global audience, they decided to select imagery from the collections of different INGOs showing a mix of people and places. The team assumed that no one would feel excluded if participants of trainings recognized themselves in part of the pictures. However, one experience in one of the pilot workshops might indicate that this was not the case.

The train-the-trainer workshop includes a section about cultural awareness. Perhaps fueled by this subject, teaching participants to look critically at their training content and materials to make sure it does not conflict with the culture of the target audience, a participant in Pakistan gave disapproving feedback about the use of pictures showing women sitting next to men and images of sub-Saharan African people. While this might be understandable from a cultural perspective, it was surprising to us to get such comments from people who work for organizations that fight gender discrimination and poverty.

As previously noted, when we first prepared the materials for cross-cultural implementation, we decided that the training dealt only with simple knowledge and skills. In post hoc reflection, we realize that we also briefly touched on origination beliefs in the cultural awareness portion of the train-the-trainer instructional materials. Our misconception led to this participant’s reaction. We continue struggle with this point. What culture should one adapt materials to in humanitarian and international development project contexts? The sponsoring organization’s culture? The local group’s culture? This is something we have not found the answer to yet.

Scenarios

The partner training materials are scenario-based simulations. Given a realistic scenario of an emergency, groups of participants create a logistics plan for the response. Different regions and countries will be prone to different types of emergencies. So, the training materials are meant to be adapted to include a realistic emergency scenario. The project produced three different scenarios for the five pilot workshops: an earthquake, a drought, and a scenario related to internally displaced people due to flooding. The logistical plan for different scenarios and the challenges around it do not differ a great deal. In the end one needs to procure supplies and get them to beneficiaries in an efficient and cost-effective manner. The final materials that are shared on the project website have placeholders where training organizers can enter a relevant disaster, including locations, dates, and specific challenges related to the disaster. During pilot testing, the adapted and localized scenarios were very well received by workshop participants.

Project Management

Projects of this scale and complexity require adequate planning of time and resources, both human and financial. By working together, the INGOs kept their share of the costs down and convinced the institutional donor of the widespread need for the proposed products. But this particular donor also had very strict regulations for the delivery schedule, which put a lot of pressure on the project timelines, with a grant period of 2 years. By the time the proposal was accepted and funding was available, only 16 months were left; this meant we needed to adapt the project plans.

When we first planned the project, we planned to conduct assessments with several partner NGOs to determine training needs and customize workshop content. Yet, the input and review cycles for the standards took longer than expected. However, the pilots were already planned and so we chose to delay completion of the assessment tool in favor of completing the workshop materials. We anticipated delivering a pilot implementation and then making necessary changes to the materials before delivering the next pilot implementation. In the end, we ran out of time to complete major revisions after each pilot and were only able to complete them after every other pilot implementation. We had to skip the full train-the-trainer workshop in the last pilot implementation and instead piloted the assessment tool for the first time, because it was only just completed at the end of the pilot phase and we did not have enough time for implementation.

Conclusion

On December 29, 2014, the project team made all resources available through the project website. The project team delivered on time. There is little information about how and how much the materials are used. The authors and many others who were involved in the project have moved on to different organizations and projects. However, some indications of use exist.

Four years later, statistics from the project website show that the standards document has been downloaded almost 1000 times in English and over 175 times in French, a set of posters of the standards have been downloaded almost 1200 times, and the logistics assessment tool has been downloaded more than 500 times in English and almost 150 times in French. More than 1000 users registered to access the elearning modules on the project site, and about 150 completed all modules. Some of these registered users are from original consortium members, and many are from different agencies, judging from their email addresses. The largest group has private email addresses from providers such as Gmail or Yahoo!.

Given the open access to source materials, we don’t know in what other ways the materials are available. For example, the elearning modules are also hosted on disasterready.org where several users have entered reviews. Oxfam GB also hosts them on their own learning management system. Given the open license, anyone could be hosting and sharing materials from the project.

From time to time, when agencies involved in the consortium organize an event around the standards, they reach out to others and see if they want to join. For example, in June 2015, Tearfund and Oxfam co-hosted a train-the-trainer event in London. Next to consortium members, staff from two other agencies joined the workshop. The project website mentions partner training in El Salvador and a train-the-trainer workshop in Bangkok, Thailand, with participants from ten countries representing seven agencies. Finally, a Facebook page exists titled Humanitarian Logistics Nepal, showing pictures with project materials in them and mentioning logistics training organized by an unknown agency. The page has since been used to share logistics information, training announcements, and vacancies.

The primary design constraints we navigated in this project included the limitations imposed by grant deadlines, a universal design for performance improvement in different organizational workplaces, geographical and cultural settings, and INGO-NGO partnerships. All design projects come with deadline limitations. The unique nature of grant deadlines can be especially challenging because they happen outside of the context of normal intra-organizational operations. Normally, as intra-organizational priorities and needs shift, so do project resources and deadlines, such that a natural balance occurs between the available resources and a project team’s abilities to get things done. When working on a grant schedule across multiple separate organizations, shifts in each organization’s priorities can disrupt the balance between available resources and the need to meet externally set, rigid deadlines. While we acknowledge that externally set, rigid deadlines can also keep an inter-organization collaboration progress from moving forward, the lack of balance between and fluidity of resources and deadlines means that project teams must be prepared to make more concessions in design work.

We also communicate caution to teams setting out to create a universal design for different organizational, geographical, and cultural settings. While it is possible to create a base of materials for cross-cultural implementation, these materials will most certainly need significant pilot testing along with additional adaptations prior to future implementation in each different setting. We recommend a thorough analysis of the desired learning and performance outcomes with a systematic approach to adaptation for each culture and adequate representation, which matches the target audience’s needs with design choices (Asino & Giacumo, 2019).

Lastly, the nature of INGO-NGO partnerships is complex. Unlike performance improvement and training initiatives in most private organizations, sophisticated design work in this context cannot result in performance improvement and training that is prescriptive, directive, and outputs standardized performance operations. While we did develop performance support systems examples and training materials that speak to industry-wide accepted standards, we did not attempt to install specific operations, organizational protocols, or one-size-fits-all implementation strategies in partner NGOs. Instead, we designed for an INGO coaching approach to facilitate organizational partnerships, an operational assessment tool to support partner NGO selection of targeted training, example toolkit resources which can be adapted for specific contexts, and support for local implementations.

As the project unfolded, we periodically looked in the literature for examples of successful projects similar in nature, especially when we ran into pilot implementation adaptation challenges. We didn’t find any examples of authentic design cases that would help us decide how to proceed. We hope that this design case helps emerging designers and project managers make informed decisions, avoid what doesn’t work, and build upon what does work.