Introduction

With the challenges of a global pandemic, political and social unrest, and the consequences these issues bring, there is a universal call for empathy, for each other and of one’s self as we attempt to maneuver through this tumultuous time. For instructional designers, this includes employing empathy and empathic design as they grapple with how to design instructional interventions for end-learners.

In 1978, Stanford University’s focus on Human Center Design resulted in coining the term design thinking. This methodology has since become commercialized in business, education, and healthcare. Empathy is the first stage in the design thinking process, now a popular buzz word in design research and practice. It suggests that empathy results in a design that meets the audience needs. But how do we know if this is true? As professors of instructional design, we study how designers produce meaningful design deliverables. We define a meaningful design deliverable as one that includes: (1) a presentation of activities specifically for the learners’ world/context, and an opportunity for learners to apply these activities, and (2) two important aspects of empathy for action, (a) presents activities and experiences where the learning and the performance context meet, and (b) focuses on the learners’ localized context of use.

We study how graduate design students embrace empathy for others (the end-learners, client, stakeholders), for context, and for one’s self while designing a product or experience that is meaningful for the learners and their context of use (Baaki & Tracey, 2021). As such, we do not merely teach students to define empathy or the generic purpose of empathy; rather we teach graduate students to employ empathy and empathic design in their design activities. In our context, we teach empathy to drive design action thus empathy becomes an enabler of the design process. We wanted to see the results of teaching empathy and empathic design on the design products produced. In this article, we describe how we designed our course to include instruction on empathy and empathic design, how our student’s employed empathy while engaging in empathic design, and the results of their efforts. The focus of this research was on studying the connection of empathy and empathic design, with the resulting design deliverable. This question guided our research:

How did designers use empathic design to produce meaningful design deliverables?

Background

Instructional design and empathic design

The instructional design approach we teach our students includes identifying the design problem/opportunity, expressing sensitivity to the end-learner, identifying learning outcomes, verifying content, designing an intervention including instructional strategies and activities, and determining an evaluation plan to measure the success of the design approach. Empathic design does not replace this approach, rather it enhances it as it seeks to get closer to the lives and experiences of the end-learners, the learning experience, the application context of the learned material and of the designer/design team designing the intervention.

Previous studies

Haag & Marsden (2018) had information technology students in a design workshop use personas as proxies of similar and dissimilar users. They concluded that student designers found it difficult to empathize without the use of the personas to conjure empathy for users. Haag and Marsden concluded that when students confirmed whether a persona was similar or dissimilar to themselves, the students discovered empathy gaps regarding users that were considerable older and less technically savvy.

Through a 5-year industrial design school workshop, Gagnon and Côté (2014) used personas to help designers question stereotypes like a businessman in a navy suit or an elderly woman using a cane. The authors contended when young designers have an empathic attitude and the experience allows designers to open their minds, designers may find new ways in approaching people.

Although design literature points to cultivating empathy among student design teams (Haag & Marsden, 2018), teaching empathy in design courses (Levy, 2018) and developing course environments where students think empathically (Woodcock et al., 2019), the above studies focused on classroom projects where the actual deliverable was the product of the classroom experience. Looking at meaningful design deliverables that are put into practice, Tracey & Hutchinson (2019) examined how instructional designers imagined both the emotional and cognitive learner experience by designing an interactive case study to promote innovation and collaboration among physicians, radiation physicists, and radiobiologists. For the instructional designers, findings indicated that designers visualized learner interaction and opened themselves to the physicians, physicists, and radiobiologists as the instructional designers designed a Virtual Hospital. For the audience of focus, the Virtual Hospital aligned with the physicians’, physicists’, and radiobiologists’ perceptions of their day-to-day activities.

Over an 8-week project-driven initiative to teach graduate students how to become sensitive to the feelings and experiences of adult learners with literacy-related knowledge skill gaps, we followed 11 student design teams as they constructed personas that are used in courses and professional development projects for a nonprofit organization teaching literacy to adult learners. Measuring instances of empathy as student design teams collaborated in the persona construction, we witnessed designers opening themselves in a responsive way to the adult learners’ experiences and feelings (Baaki & Tracey, 2022b). Student designers showed empathy for the adult learner, someone directly involved with the adult learner persona, and for fellow design group members.

It is possible that empathic design naturally leads to more discussion and collaboration during the design process which can result in more engaging and meaningful products (Keahey, 2020). We believed the natural progression in our research was looking at if designer empathy and employing empathic design would result in meaningful deliverables.

Conceptual frameworks

We used two conceptual frameworks to guide this study, Kouprie & Visser’s 4-phrase framework (2009) and Batson’s eight distinct empathy measure (2009).

Kouprie & Visser 4-phrase framework

Since our students are new to an empathic design approach, we teach a 4-phase framework for empathy for design (Kouprie & Visser, 2009) to support empathic design practice. Based on the principle that a designer steps into the life of a learner, the four-phase framework begins with (1) discovery, where the designer enters the learner’s world. Here, the designer makes contact either in person, through research, through the review of descriptive materials, and/or through observations of the learners. During the second phase, (2) immersion, the designer takes an active role by mentally living in the learner’s world, trying to look through the lens of the learner’s perspective. The third phase, (3) connection, is where the designer connects on an emotional level with the learner, to achieve emotional or shared understanding. We maintain that both designer affective and cognitive components are critical during this phase as it is important for the designer to understand the learner’s feelings (affective) and meanings (cognitive). Xie (2020), has extended this view by recommending designers of digital instruction consider four types of engagement to support learners’ experience, including behavioral, cognitive, affective, and social engagement. In the final phase, (4) detachment, the designer leaves the learner’s world, detaching the emotional connection with the learner, stepping back into the role of designer (Kouprie & Visser, 2009; Tracey & Hutchinson, 2019). Although we describe empathic design in a linear fashion, neither it nor the instructional design process is linear; both are iterative approaches to designing instructional interventions. While our students used the Kouprie and Visser (2009) 4-phrase framework to engage in empathic design, as the researchers, we used Batson’s (2009) eight distinct empathy measure to gauge their efforts.

Batson’s eight distinct empathy measure

How do we recognize and label that empathy is happening in design? We were guided by Batson (2009) who indicates that “The best one can do is recognize the different phenomena, make clear the labeling schema one is adopting, and use the schema consistently (p. 8).” Working with his students in Psychology, who could not understand the nuances of empathy, Batson attempted to illustrate the different types of empathy. He identified eight distinct phenomena that have been called empathy. (Table 1). Batson notes that the distinctions among the concepts are subtle making them interrelated. Guided by Batson’s explanation of each concept, we identified five of the concepts as relevant to our study: Concept 1, Concept 2, Concept 3, Concept 5, and Concept 6.

Table 1 Batson’s (2009) eight distinct empathy concepts
Full size table

For Concept 1, a person may know another person’s internal state from what that person does or says. In Concept 2, perceiving another in each situation leads one to match the other’s neural state where one comes to feel something of what the other feels. For example, when watching someone on a tightrope, one may find them self-tensing and twisting. When feeling as another feels (Concept 3), an empathizer need not feel the same emotion, only a similar emotion. Batson refers to Concept 5 as imagine-other where the empathizer takes the perspective of the other person. What is important is not so much what the empathizer knows about the feelings and thoughts of the other person but rather the empathizer’s sensitivity to the way the other person is affected by their situation. Finally, Batson distinguishes Concept 5 from Concept 6 by calling Concept 6 imagine-self where the empathizer changes places or takes on the role of the other person. In imagine-self, the empathizer imagines how they would think in the other person’s place. For example, if a friend confided their fears that they were going to lose their job, in imagine-self, you would imagine how it would feel to be your friend. In imagine-other, you would imagine how your friend is thinking and feeling about losing their job. In imagine-other, your imagining is based on what your friend says and does and on what you know about your friend’s character, values, and desires (Batson, 2009).

We did not identify Concept 4, Concept 7, and Concept 8 as applicable to our study. Batson explains that Concept 4 has an aesthetic element, for example if you see a painting of a waterfall and picture yourself in the waterfall. Concept 7 and Concept 8 better describe clinicians, counselors, and physicians committed to helping people who are suffering (Baaki & Tracey, 2021). Batson’s empathy framework was essential in defining instances of empathy in our students when we analyzed their design meeting transcripts.

Study context

Historically, students of instructional design were taught that design could be accomplished using a step-by-step model. The reality, however is that instructional design is a complex messy iterative difficult process. We guide our students through a process that focuses on the end-learner, their localized context of use, or the moment they need to use what they have learned, the content, and the stakeholders. Our students understood that they had to produce something for their client. This was not a hypothetical design assignment, this was an actual client with a design problem that had to be solved. We taught design as an iterative process and incorporated the 4-phrase framework to help guide them to employ empathy and empathic design. We used Batson’s empathy measure to identify their instances of empathy in our data analysis. Below we present how we incorporate empathy and empathic design in our course. It is important to note that the 4-phrase framework is relevant to all human design as empathic design is applicable for all types of design. Whether designing for basic arithmetic, or teaching at risk youth sleep skills, empathy for the end-learner is needed.

Designing with empathy

Design has a beginning, middle and end with numerous decision-making points that move the design forward. Incorporating the Visser & Kouprie 4-phrase framework, designers make design decisions keeping in mind empathy for the end-learner, for the learning and performance context and for one’s self regarding one’s biases and designer precedents. In this context, we teach empathy as a means to an end, the creation of meaningful and effective design deliverables, defined as those that provide a learner with what they need in a specific situation or moment, or in the localized context of use. Here we illustrate specific examples when designers must employ empathy during design to create meaningful deliverables.

Empathy for end-learner

In design, we define empathy as an intuitive ability to relate to other people’s thoughts and feelings while maintaining one’s self. In other words, it is trying to stand in another’s shoes while staying in your own. Kouprie and Visser’s (2009) 4-phrase framework drives empathy for others. When designers engage in empathic design, they open themselves in a responsive way to the lives, feelings, and experiences of others, in this case the end-learner. In empathic design, designers must be willing to engage with the end-learner through research, interviews, design team discussions and/or observations, with the goal of generating insights that will support effective design decisions.

Empathy for the end-learners’ learning and performance context

When embracing empathy and empathic design, we mentor our design student teams to imagine others (Batson, 2009), in this case the end-learners, as it allows them to remain designers while opening themselves in a responsive way to the feelings and experiences of the learner. These feelings and experiences include not only what the learners bring to the instructional experience, but how they interact with the instruction during the experience and their performance because of the instruction. Empathic design can include empathic forecasting, where the designer aims to predict the learner’s experience while engaged in the learning process (Tracey & Hutchinson, 2019). Here designers combine learner analysis information gathered through interviews, research, observations, and discussions, identifying specific moments of use where context is scaled back to what is needed in a situation or moment, what we define as a localized context of use (Baaki & Tracey, 2019). In other words, what is needed in this situation with these learners, rather than what designers believe should be taught, or an idealized version of the learning and performance context. A localized context of use is a marker of relevant relationships where what is important is what information does in a situation, not necessarily about what information is (MacPhail, 2014). For example, for the United States Coast Guard, a hurricane response mission or a drug interdiction is a complex information network of people, technologies; and cultural, social, legal, and political processes. Within this information complexity, in the moment when a Coast Guard member climbs down a helicopter ladder to attend to a person stranded on a rooftop, a localized context of use in what the Coast Guard member does directly impacts the member’s and stranded person’s wellbeing. Although some question the rationale of designers predicting how learners may interact with and experience the instruction (Mehta, 2020) we maintain that as professional designers, it is our responsibility to discover as much as possible about our learners while exercising our professional knowledge and expertise in designing the best intervention for their localized context of use. Additionally, there are specific contextual features that can be determined in a localized context of use, including the physical, technological, and social environments, all suggested to best support learner experiences specifically in the shift to digital remote learning (Xie, 2020).

Empathy for one’s self

Designers are the dynamic drivers of the design process who use their knowledge, experience, and intuition to navigate the design space constructing design decisions until an innovative outcome is reached (Tracey, 2015). As such, the designer is an integral part of the design process who comes to the design activity with biases, inspirations, and uncertainty (Tracey & Hutchinson, 2016). In teaching designer’s empathy for one’s self, we do so through the lens of what designers bring to the design activity, since perception and understanding are influenced by the designers’ experiences (Fraquelli, 2015). We teach our graduate design students to reflect on their repertoire of interests, biases, and understandings (Cross, 2011), as they analyze the design problem and make design decisions. Empathy for one’s self, others and context contribute to design decision-making, what we call empathy for action, resulting in the design of a meaningful design deliverable.

Understanding empathy was important for our instructional design students, but it was also critical for us as their instructors. While teaching our students to have empathy for the end-learner, the context and for one’s self, as their instructors, we needed to have empathy for them as they struggled while designing.

Empathy for graduate student learning context

We charged our graduate students to apply empathy and empathic design while engaging in authentic design projects with clients and for specific end-learners. As instructors, this design challenge required our continual support to each design team. Not only were we teaching student teams to apply instructional design principles, but we were also constantly pushing them to explore an empathic design process. We understand that too often when confronted with creating a design brief, design students tend to do what they know and that in design practice designers who are unable to identify with the complex needs of learners and of a design brief are at a disadvantage (Woodcock et al., 2019). Therefore, we challenged our student design teams to stretch themselves beyond their comfort level while designing. As design mentors we introduced the emotional value of the design by constantly inspiring the design teams with questions such as: What does this intervention mean? How does the intervention feel? How will the learners experience this intervention? How will the learners’ lives change after this intervention? This approach helps designers step away from a pre-occupation with just the block and tackle of design methods (Woodcock et al., 2019), toward designing with the learners in the forefront of every design decision. We also realize that empathy in design requires deliberate practice, and an empathic attitude needs to be championed, nurtured, and practiced regularly (Battarbee et al., 2014). As the instructors of the student design teams, we are enthusiastic champions of the empathic attitude and tell and retell stories to our student design teams that keep empathy and the empathic attitude alive (Battarbee, et al., 2014).

In the methods section we describe how we designed the class teaching empathy and empathic design. It is important to note however, that the goal of this research was to look at if empathic design resulted in a meaningful design deliverable. Defining and measuring a meaningful design deliverable is difficult and illusive. Research in design tends to focus on controlled lab studies with the emphasis on the design process rather than the deliverable. We wanted to go beyond that to move the research in design practice to the next level (Haag & Marsden, 2018). When we discuss producing meaningful design deliverables below, we examine the deliverable the students produced for their client. Although complex, we maintain as researchers these steps in design processes and preparing designers are critical to the field.

Method

Research design

This qualitative research study was part of a 15-week project-driven initiative teaching empathy and empathic design as a means to an end while design students worked with an authentic client producing a meaningful design deliverable. We were interested in exploring if empathy and empathic design resulted in the design of a meaningful deliverable. This question guided this research.

How did designers use empathic design to produce meaningful design deliverables?

Participants and instructional context

We conducted our study at two different universities, one in the Midwest and one on the East coast where we teach advanced instructional design courses to graduate students. The Midwest University’s Advanced Learning Design course had 13 students (four males and nine females) divided into three teams of three students and one team of four students. This 4-credit course was taught using the Learning Management System Canvas and synchronously via Zoom. The East coast University’s Advanced Instructional Design Techniques had 18 students (eight males and 10 females) divided into two teams of three and three teams of four. This 3-credit course was taught using the Learning Management System Blackboard and synchronously via WebEx. The students ranged in age from 22 to 65 years, all in a masters or doctoral program, familiar with online courses and group work as these are requirements in each program. This course is required for all of the students at the Midwest University and required for the doctoral students and an elective for the masters’ students at the East coast university.

Graduate design students

Our graduate design student teams worked with two clients from two nonprofit organizations. Organization “A” was committed to improving children’s health, well-being, and academic performance by providing sleep education and bedtime essentials to economically disadvantaged students and their families (4 design projects). Organization B focused on designing educational solutions that are customized to the needs of adults with literacy-related knowledge skill gaps (5 design projects). Table 2 provides the specific projects for each organization. The purpose of the study was described in the consent form, so our graduate design students were aware that the study focused on empathy and empathic design. Since our research focused on if empathic design resulted in meaningful design deliverables, student increased level of being empathic as a result of knowing they were part of the study was irrelevant. It did not matter if they displayed increased empathy, the key was if that increased empathy produced a meaningful product. Our goal was to see if increased empathy resulted in meaningful deliverables.

Table 2 Projects for organizations A and B
Full size table

Instructional context

Prior to the beginning of the semester, the authors met to design the learning context and our instructional approach to teach and apply empathy and empathic design to the graduate design student teams, then guide them as they applied empathy and the empathic design approach in their design. During Week 1 (Table 3) we introduced the 4-phase empathic design approach (Kouprie & Visser, 2009) and taught what it means to have empathy for others. Students applied this immediately in an assignment titled The Power of Observation. The assignment asked students to pick an activity familiar to them, then observe someone else doing that activity for 60 min documenting their observations and reflecting on specific written prompts. They presented their findings to the class the following week. During Week 2 (Table 3) we taught empathy for context, where we had students create empathy maps and learner personas of their end-learner, their end-learners. During Week 3 (Table 3) we taught empathy for one’s self providing students with reflective questions and discussions focused on their designer precedents. During this time, the design teams worked with the client to develop a team milestone document where both parties agreed to due dates for deliverables. The following deliverables were shared with the instructors and clients: Design problem/opportunity, end-learner description and outcomes and assessments, content outline, design plan: instructional strategy/activities, and evaluation plan.

Table 3 Key weekly activities for the organization A and B design projects
Full size table

The students met with the client, researched, interviewed and/or observed the end-learner group when possible and gathered instructional content. The teams, often meeting several times a week outside of class, were charged with providing us an audio file of one team design collaboration during Week 7 and again in Week 14 (Table 3). We chose Week 7 because the teams would be immersed in the design plan at this point in the semester, and Week 14 because final design decisions would be made at this point. The teams submitted the final design deliverable to the client and to us for a final grade. Each student individually wrote a designer reflection answering prompts on empathy, empathic design, and the overall design experience.

Data sources, collection, and analysis

In an effort to answer our research question: “How did designers use empathic design to produce meaningful design deliverables?” we gathered and analyzed numerous data sources. Here we identify the data sources, and the analysis methods.

Analysis of team collaborations: instances of empathy

As previously stated, student design teams consistently met to collaborate on the design. We asked each team to record one team collaboration by Week 7 and again by Week 14 of the semester. These audio recordings were transcribed by our two advanced doctoral graduate research assistants who had taken these courses as part of their coursework during their program. Both research assistants had concluded their coursework and were in the research dissertation stages of their program, well versed in data analysis. We noted the total number of words transcribed and identified instances of a distinct empathy category (Batson, 2009) during four rounds of analysis as described in detail in the Results Section. In Round 1 we reviewed the transcripts from our own design teams identifying instances of empathy. In Round 2 we our two graduate research assistants review our analysis and identify discrepancies. We confirmed that our graduate research assistants had taken the required qualitative statistics courses then trained both in the procedures needed to analyze the data for this specific study. During Round 3, we exchanged our analyses and resolved discrepancies in what category was the fit. In our final round of analysis, Round 4, we finalized any remaining discrepancies and then finalized all instances of a distinct empathy category. We discuss the results and the percentage of agreement for each analysis in the results section below.

Analysis of team collaboration: instances of empathy for others, context, and one’s self

Our goal in these rounds was to further analyze the empathy instances identified in the first rounds by classifying them in terms of the focus of empathy demonstrated in each instance: empathy for the end-learners, empathy for the context, or empathy for the designers (self), described in detail in the Results Section. In Round 1 we reviewed the instances of a distinct empathy category from our own courses. Once again, we had our graduate research assistants review our analysis in Round 2 documenting (a) agreement/disagreement that students showed empathy for others, one’s self, or context and (b) provide insight on what we may have missed. We then exchanged our analyses with each other and resolved any discrepancies identified in the first two rounds or identifying instances of empathy that may have been missed.

Review of final design deliverable

Each graduate student design team produced and submitted a final design deliverable. Upon completion of the course and final grade submission, we reviewed each design deliverable focusing on the learning and performance contexts, the localized context of use. Our review focused on the presentation of activities, specifically the visuals, language, etc., (learning context), the nature of the activities based on the end-learners’ world/context (learning and performance contexts), and the application of the activity (can they use it) (performance context). We developed an analysis table to document instances and write notes and explanations detailed in the Results Section. In Round 1 we reviewed the final design deliverables from the student design teams in our own courses. In Round 2 we conducted a blind review of the final design deliverables from the student teams in the other course, not reviewing the notes of the Round 1 review. We then met in Round 3 to discuss our analysis providing context to the projects from our experiences working with our teams. This turned out to be a critical step in our analysis, as we could ‘fill in the blanks’ of the challenges the student designers had while attempting to design a meaningful deliverable. For each project we then came to final consensus if the graduate student design team (a) designed a meaningful design deliverable, (b) included some elements of a meaningful design deliverable, or (c) the team did not design a meaningful design deliverable.

Trustworthiness

To ensure trustworthiness, we confirmed that each data source was analyzed by four raters. The two instructors hold doctoral degrees in Instructional Design and Technology, along with a combined research experience of over 30 years. Both graduate assistants were advanced doctoral students with extensive coursework and training in research methods with specific training in qualitative analysis. The instructors supplied appropriate definitions, research articles, and specific analysis directions to aid the graduate students analyze these specific data sources. We also ensured trustworthiness by having multiple data sources, including team design recordings and final design deliverables, with all four raters analyzing each source, blind reviews, followed by discussions to resolve discrepancies.

Results

We used a protocol analysis methodology to document verbal exchanges during the teams’ collaborations. For eight of the nine teams, we reviewed two recorded collaborations. For the other team, we reviewed one recorded collaboration. We transcribed 106,750 words from 17 collaborations. We begin by presenting the results of our analysis of team collaborations focusing on instances of empathy and instances of empathy for others, context, and one’s self. We then provide our review of the final design deliverables. Finally, to answer our research question, we provide a comparison of the instances of empathy in each team and the determination if their design deliverable is rated as meaningful for their end-learner and their context.

Team collaborations: instances of empathy

We participated in four rounds to assess the team collaboration transcriptions. In Round 1, the instructors reviewed their own students’ collaborations and identified instances of a distinct empathy category as outlined by Batson (Table 1). In Round 2, our advanced graduate research assistants each reviewed one of our assessments with the direction that Batson’s concepts 1, 2, 3, 5, and 6 were relevant to the study. The instructors explained that they had gone through each collaboration transcript, to identify instances of empathy to fit a Batson concept. The graduate research assistants provided agreement/disagreement on what the instructors had categorized and provided insight on what the instructors may have missed. In Rounds 1 and 2, for 317 total instances of a distinct empathy category, there was 74% agreement on the identified instances of a distinct empathy category. In Round 3, the instructors exchanged assessments and provided agreement/disagreement on instances that the instructor and graduate research assistant had agreed on and attempted to resolve disagreements on instances that the instructor and graduate research assistant had disagreed on. After Round 3, there was 94% agreement on the identified instances of a distinct empathy category. In Round 4, the instructors finalized any remaining discrepancies and then finalized all instances of a distinct empathy category.

For the 17 recorded collaborations, Table 4 details the 317 instances of a distinct empathy concept. We identified 74.5% of the instances of a distinct empathy concept as imagining how another is thinking and feeling. Batson (2009) describes this concept as imagine-other where one person opens themselves in a responsive way to another person’s feelings and experiences without losing awareness that the other person is a distinct person.

Table 4 Details of instances of a distinct empathy concept
Full size table

Imagine-other afforded team members to remain in the designer role and open themselves to the situation of more than one end-learner. For Organization “A”, which was committed to improving children’s health, well-being, and academic performance by providing sleep education and bedtime essentials to economically disadvantaged students and their families, a team was tasked to conduct a cognitive task analysis to detail the process and procedures of a fundraising event and generate training recommendations for the lead volunteer position. One team member expressed imagine-other as they opened them self in a responsive way to the audiences of focus which were not only the lead volunteer position, but also the volunteers. The team member is sensitive to what the training would entail and how a focus on a lead volunteer, volunteers, or both would affect the instructional strategies and activities.

Let me go back a step here. In terms of this training piece. This training would be appropriate for all the volunteers as well as the lead volunteer, right? Are we designing this for just the lead, or looking at it as the volunteers as a whole? Which would affect what activities we create. Possibly.

Later in the collaboration, remaining sensitive that the lead volunteer and the volunteers were just volunteers and their training time was at a premium, the team member expressed imagine-other by showing empathy for the volunteers’ learning context. The team member reflected, “Would it be easier (for the volunteers) to just design this as a one to 2-h onsite training?” The team member is sensitive to the volunteers’ feelings and experiences to how the training is delivered. The team member suggests that how the training is delivered will impact the instructional strategies and activities.

Team collaborations: instances of empathy for others, context, and one’s self

We identified the 317 instances of empathy as empathy for others, empathy for context, or empathy for one’s self. When design teams showed empathy for others, students intuitively identified with other people’s thoughts and feelings (Baaki et. al, 2021) as illustrated with this student’s reflection of their audience of focus, “One thing that stood out to me is the difficulties people face and how it affects their ability to make decisions in their life.” Showing empathy for context meant that students identified specific moments of use where context (performance or learning context) was scaled back to what is needed in a situation or moment (Baaki & Tracey, 2019) illustrated with this student’s insight regarding what might be their learner’s unique motivation to complete the GED test prep course,

I guess for this part it would be what is... I think the task might be connected with the overall goal, what tasks are the users trying to complete? In this particular case it would be finishing the... because he's in that residential recovery program it requires him to take the GED test preparation course and so that's what he needs to do that's what he is trying to accomplish right now. That's the immediate goal I guess

Having empathy for one’s self occurred when students identified that perception and understanding are influenced by their experiences (Fraquelli, 2015) illustrated with this student’s insight resulting from their team design experience,

I tend to enjoy working on my own because I like moving at my own pace, in my own direction, and without resistance from others. Both of my partners gave me so much valuable insight that I started to rethink working in isolation. They (my partners) were able to look at our tasks and persona in ways that I never could. The ideas they shared made our project so much more well-rounded than it would have been if completed by just me. Moving forward I want to continue to encourage myself to be more comfortable with this type of collaboration. I think it will be crucial in my development as a designer.

Prior to reviewing the 317 instances of empathy we ensured that all reviewers understood our guiding definitions for empathy for others, context, and one’s self.

We participated in four rounds to assess the instances of empathy. In Round 1, the instructors reviewed their own students’ collaborations and identified instances of empathy for others, context, or one’s self. In Round 2, two graduate research assistants each reviewed one of our assessments with the direction to follow our guiding definitions, provide agreement/disagreement on what the instructors had categorized and provide insight on what the instructors may have missed. In Rounds 1 and 2, there was 90.5% agreement on the identified instances of empathy for others, context, or one’s self. In Round 3, the instructors exchanged assessments and provided agreement/disagreement on instances that the instructor and graduate research assistant had agreed on and attempted to resolve disagreements on instances that the instructor and graduate research assistant had disagreed on. After Round 3, there was 90.9% agreement on the identified instances of a distinct empathy category. In Round 4, the instructors finalized the remaining 29 discrepancies and then finalized all instances of empathy for others, context, or one’s self.

For Table 4, we followed Batson’s (2009) position that when interacting with the end-learner, students may experience different states that correspond to a direct instance of empathy. In Table 5, finding 317 instances of empathy, we were interested if the instance of empathy was directed toward others, the context or one’s self. Referring to the example above, the team member showed imagine-other for the volunteer leader and volunteer and the volunteers’ learning context (2-h onsite training). Though both instances were imagine-other, the former instance was empathy for other while the latter instance was empathy for context. For the 17 recorded collaborations, Table 5 details the instances of empathy for others, context, or one’s self. We identified 52.1% of the instances as empathy for others. At both universities, students first experienced empathic design in the prerequisite introductory instructional design course. In the introductory course, the instructors emphasized empathy for others while in the two advanced instructional design courses, the instructors first introduced empathy for context and empathy for one’s self.

Table 5 Details of instances of empathy for others, context, or oneself
Full size table

Team members interacted directly with stakeholders at Organization A and B. Using an empathic design approach, the instructors guided teams to have empathy for the end-learner, which for each design deliverable was ultimately learners. In reviewing instances of empathy for others, in addition to expressing empathy for learners, students showed empathy for teammates and organization stakeholders. For example, a team working with Organization B redesigned an interview skill learning module for adults with literacy-related knowledge skill gaps. As the team designed module assessments, one team member expressed empathy for the adult basic educators who are responsible for teaching the interview skills module and evaluating each adult learners’ progress. Even though designing assessments was an appropriate means to evaluate the adult learners, the team member understood that adult basic educators can be volunteers with little to no formal teaching education and experience. The team member posed a question to the team, “So, let’s think about our volunteers (adult basic educators). Are any of our assessments going to be too difficult for them to grade?”.

When expressing empathy for context, teams showed empathy for the learning and performance contexts. A team working with Organization B redesigned a money management module for adult learners with literacy-related knowledge skill gaps. In the team’s two reviewed collaborations, the team had 13 instances of empathy for context. In seven of the instances, team members expressed empathy for the performance context while in six instances team members showed empathy for the learning context. Regarding the learning context, team members discussed designing a problem-based learning scenario in which adult learners would develop a budget plan. Team members struggled with how an adult learner’s performance context aligned with the budget plan activity. Team members questioned the amount of money an adult learner could place in a savings account each month. One team member brought up if an adult learner can only put $5 in a savings account each month is it worth the effort as the bank may charge fees for accounts below a minimum amount.

During the 15-week project-driven initiative, the instructors met regularly to discuss how the design projects were moving forward. Instructors discussed how students struggled with empathy for one’s self. In class discussions, group discussions, and email exchanges, students expressed that empathy for one’s self would result in students’ biases and lack of objectivity in designing the deliverables. For example, the second instructor received an email from a student who had family members who had literacy-related knowledge skill gaps. The student felt that their sensitivity toward family members prevented them from being objective is designing an effective lesson. Although both instructors contended that a designer’s perceptions and understanding of an end-learner are influenced by a designer’s own experiences, we identified only 27 instances of empathy for one’s self. Only four of the nine teams showed any instances of empathy for one’s self, and one team had 16 of the 27 (60%) of the total instances of empathy for one’s self.

The team that made up 60% of the instances of empathy for one’s self worked with Organization A to design a resource that organized and documented the process which staff ensure the proper planning and delivery of the sleep education program to kindergarten through fifth grade classrooms. The team spent time with organization stakeholders to understand the current process to ensure proper delivery of the sleep education program. The team found the current process confusing and lacking organization and direction regarding how resources were to be used. In developing an initial design plan, one team member discussed confusion around a bingo sheet activity:

…So, I intentionally left it (bingo sheet activity) out of the design plan…It just appeared as here’s a bingo sheet…But they (Organization A stakeholders) called out that the bingo sheet needs to be in there (plan and delivery of the sleep education program), and I was like, well, when do you play bingo?

Another member of the team expressed empathy for one’s self when discussing the number of folders used to hold the sleep education program resources. With multiple folders of resources, the team member commented, “And then I am like, okay, what do I do with this one (folder)?” The team member concluded that in their experience one folder with all necessary information alleviates confusion of what resources to use.

Final meaningful design deliverable

Each team’s goal was to produce a meaning design deliverable which the organization would be able to implement. Once again, we define a meaningful design deliverable as one that includes: (1) a presentation of activities specifically for the learners’ world/context, and an opportunity for learners to apply these activities, and (2) two important aspects of empathy for action, (a) presents activities and experiences where the learning and the performance context meet, and (b) focuses on the learners’ localized context of use. In Round 1 the instructors reviewed the final design deliverables from the student design teams in our own courses. The instructors determined the elements of a meaningful design deliverable and created a table documenting these elements. Using the created analysis table to comment on instances of a meaningful design, the instructor’s individual review focused on the presentation of activities, specifically the visuals and language used, (learning context), the activities based on the end-learners’ world/context (learning and performance context), and the application of the activity (can they use it) (performance context). In Round 2 the instructors conducted a blind review of the final design deliverables from the student teams in the other course, not reviewing the notes of the Round 1 review. For each design deliverable, the instructors concluded if the project was (a) a meaningful design deliverable, (b) showed elements of a meaningful design deliverable, or (c) was not a meaningful design deliverable. The instructors then met in Round 3 to discuss the analyses providing context to the projects from our experiences working with teams. This was an important phase of the analysis as a team’s design journey influenced the final design deliverable. For example, working with Organization B, a team redesigned a money management learning module for adult learners with literacy-based knowledge skill gaps. Team members had a difficult time understanding that adults who live paycheck-to-paycheck may not have the ability to hold a savings account. In fact, at one point, the team had discussed including a 401 K (financial retirement plan) element to the money management learning module. The team’s instructor had experience working with adults who live paycheck-to-paycheck and provided feedback to the team when the team submitted design elements to the instructor prior to submitting to the client for feedback. Sharing the team’s struggles helped the other instructor better understand why the final design deliverables showed elements of a meaningful design deliverables but fell short of a meaningful design deliverable. Table 6 summarizes our review of the final design deliverables.

Table 6 Summary of Team Final Design Deliverables
Full size table

Three of the nine projects were identified as meaningful design deliverables. Working with Organization B, a team redesigned an understanding mathematical ratio learning module for adults with literacy-related knowledge skill gaps (Fig. 1). In the lesson overview, the team noted, “The goal for this unit is to provide learners with a working knowledge of ratios that they can apply to their everyday lives, education, or occupation.” Regarding the presentation on activities, the team designed an engaging PowerPoint that allowed the adult learners to complete the lesson either by using a printout or follow along with the PowerPoint. About module activities that fit adult learners’ world and context, learners had options to choose an activity that aligns with a potential job position. For example, learners could take on the role of a coffee shop worker where the ratio activities would apply to the ratio of coffee to milk for a specialty coffee beverage. For the application of learning activities (can adult learners use it) understanding rations begins as an abstract concept. Adult learners; playing the role of a gardener, restaurant worker, or gardener; quickly realize that they negotiate ratios in context daily.

Fig. 1
figure 1

Mathematical Ratio Participant Guide

Full size image

Four of the nine projects showed elements of a meaningful design. As noted above, one team redesigned a money management learning module for adult learners with literacy-related knowledge skills gaps. The team’s presentation of activities included some easy-to-follow videos and helpful printed materials that adult learners could reference after completing the lesson. The team presented Michael whose money management scenario was relatable to adult learners’ world. However, the team chose a video that explained how a certain percentage of a monthly budget should go towards needs (e.g., utility bills, rent, medical expenses and groceries), a set percentage to nice-to-have services and items, ana a set percentage to savings. As noted earlier, this is an example of the team not fully understanding that the adult learners may not be able to regularly budget money is set percentages. Although the team provided worksheets and direction on how to apply budgeting in adult learners’ daily life, the module never addressed an adult learner’s challenge when the take home pay is less than the expenses that must be paid.

Two projects did not deliver a meaningful design deliverable. A team redesigned a learning module focused on identifying main ideas and details in a written narrative. Identifying main ideas and details is a skill tested on high school equivalency tests which many adult learners with literacy-related knowledge skills gaps need to successfully pass to meet a community college or job requirement. The team’s presentation of activities was hindered by text that was both wordy and difficult to follow. The team’s main activity could only be completed with team of adult learners, impossible for adult learners working individually to prepare for a high school equivalency exam. To meet the meaningful design deliverable element of application of the learning activities, the team failed to show how an adult learner would apply the content. For example, the team never designed a connection between main ideas and details to writing an email or writing a note to a teacher or employer. We concluded that if the learning module’s goal was to strictly prepare an adult learner for a high school equivalency exam, the team should have made this very clear at the beginning of the module.

The two teams with the most instances of empathy produced a deliverable that only had some identified elements of a meaningful design deliverable. On the other hand, a team that had 27 instances of empathy did not produce a meaningful design deliverable while a team that had 28 instances of empathy did produce a meaningful design deliverable. This indicates an inconsistency in instances of empathy equaling a meaningful design deliverable. For two teams that produced a meaningful design deliverable (develop training for onboarding employees and redesign a learning module focused on using the internet to search a career path), one team only submitted one collaboration and the other team partially recorded its second collaboration. Once again, our findings show that instances of empathy in a team collaboration do not necessarily result in empathy for action in the design of a meaningful deliverable. This is an important finding in that we understand that there is something missing. Design teams who employ empathy and empathic design does not necessarily create a meaningful design deliverable. In the discussion section we address this missing piece.

Discussion

In 1978, Stanford University’s focus on Human Center Design resulted in coining the term design thinking. This methodology has since become commercialized in business, education, and healthcare. Empathy is the first stage in the design thinking process, now a popular buzz word in design research and practice. It suggests that empathy results in a design that meets the audience needs. But how do we know if this is true? We set out to determine if empathy and consciously engaging in empathic design results in a meaningful deliverable. Our research results indicate that instances of empathy did not guarantee a meaningful design deliverable in our study. Does this mean our study is a failure? No, it means that there is a missing link in the equation: empathy & empathic design = meaningful deliverables. Our research suggests that embracing empathy and participating in empathic design increases empathy toward one’s self, the team and the end-learner, but does not necessarily result in a final deliverable that is meaningful for the learner and their localized context. We believe further investigation is needed; therefore, we begin our discussion reflecting on our findings, then turn to the area that warrants deeper exploration, the design and review of meaningful design deliverables.

Reflecting on our findings

As instructors and instructional design practitioners, imagine-other aligns with our approach to empathic design. We were not surprised to find instructional design students’ empathy evolving by imagining how the end-learner might be thinking and feeling. In a previous study (Baaki et al., 2021) where students participated in an 8-week project in our introductory instructional design course, in 24 collaborations, we identified 73% of instances of a distinct empathy concept as imagine-other. The similarity between the two studies may point to the influence of our stance that imagine-other aligns with an empathy for action approach to instructional design. Upon reflection however, we realize we set high expectations for our students. Although students at both universities had experienced empathic design in the prerequisite introductory instructional design course, the students were still novices in practicing an empathic design approach. We saw empathy in our students imagining of others in our previous studies, and believed our students were ready to take the leap to producing meaningful deliverables. Our expectation was, “you see empathy for the learner? Now design a meaningful deliverable” We assumed if our design students have empathy, they should be able to design, but we have discovered that there is a missing piece, an identified framework. We discuss this in the next section.

Design and review of meaningful design deliverables

If empathy is a means to an end, then the end is a meaningful design deliverable. Three teams designed a meaningful design deliverable, four teams designed a deliverable that had elements of a meaningful design deliverable, and two teams did not deliver a meaningful design deliverable. Although we considered our lens to evaluate the design deliverables as demanding, delivering a meaningful design deliverable or not delivering a meaningful design deliverable did not directly affect students’ final grades. In other words, the fact that only three teams delivered a meaningful design deliverable based on our lens, does not mean that designers from the other six teams failed the course.

Our criteria for a meaningful design deliverable (presentation of activities, activities include the learners’ world/context, and learners can apply the activities) focuses on two important aspects of empathy for action in design. First, a meaningful design deliverable presents activities and experiences where the learning and the performance context meet. Second, a meaningful design deliverable focuses on the learners’ localized context of use. To gain a deeper understanding of this phenomenon, we now focus on two teams who displayed instances of empathy for others, one producing a meaningful deliverable while the other did not. Both teams worked for Organization B who needed these courses for adult learners with literacy-related knowledge skill gaps to prepare them to pass their high school equivalency exam. These topics therefore were required for the non-profit to ensure their adult learners’ success.

Understanding ratios

One team working with Organization B delivered a meaningful design deliverable that focused on teaching math ratios. The team interviewed math tutors and volunteer workers who worked with adults studying for the high school equivalency exam, and a non-profit executive director with expertise in instructional design and the adult learners. Team members analyzed math content looking at learner challenges working with ratios. The goal for the course is to provide learners with a working knowledge of ratios that they can apply to their everyday lives, where the learning and performance context meet. Course content is simple, ratios are used all the time. The course design enabled adult learners to choose the context they wanted to learn ratios in; Fourbucks Coffee, Rita’s Café, Joe’s Auto shop or Cindy’s Home Improvement Garden, their localized context of use. The activities in the course provided options for learners to choose activities that fit their world [e.g., designing a garden, filling a coffee order, working with gears]. For example, if they took on the role of a coffee shop worker, the ratios apply to what they could see on the job [milk-to-coffee ratio] (Baaki & Tracey, 2022b). These activities resonated with the adult learners as many worked in these or similar jobs. When interacting with this content, adult learners realize they deal with ratios in different contexts every day. Each activity is grounded in the localized context of the adult learner’s life and work.

Basic money management

The second team working with Organization B did not deliver a meaningful design deliverable focused on teaching basic money management. Although this team did not talk directly with end learners with money management problems, their instructor had extensive experience with this learner group and attempted to guide them during numerous meetings. They also met with the executive director of the non-profit on several occasions who had a deep understanding of the learners and the learning and performance context. Rather than deeply learning about their end learners needs, this team chose to design their course based on generic money management information. For example, the team chose a money savings percentage system identifying a percentage of money for savings, bills, and spending. Knowing the end learner, the instructor advised them to not teach a percentage system, but instead teach an envelope system, where the learner puts a small amount of money in each envelope, one labeled bills, one labeled savings and one labeled fun spending. In this way, the learner would understand they need to put something tangible in each envelope, rather than trying to determine a percentage they might never be able to meet given their minimal earnings. The team chose to not implement this alternative, instead continuing to focus on percentages. Although this team did display empathy for the end learner, it did not translate to designing for the learners’ localized context, they chose to focus on content.

Teams were consistently prompted to use empathy for others, self, and context to design a meaningful design deliverable. Be it interaction with the client, team collaborations, creating empathy maps, or direct written or verbal feedback from the instructors, we designed the 15-week project with an ever-present empathic design environment. Batson (2009) argued that teaching empathy is not about explaining a form of knowledge but to explain a form of action. We maintain that empathy and empathic design for instructional designers drive motivation to do something, to design something to deliver a meaningful design deliverable. But there is a missing piece.

The novel, effect, whole framework

While writing these research results and their impact on empathy and empathic design, we realized a bridge connecting empathy, empathic design and measurable design deliverables is missing. We turned to the design literature and discovered a framework that may better guide the empathic design efforts in our designers while supporting our measurement of meaningful design deliverables. The Novel, Effect, Whole framework for evaluating creative products (Henriksen et al., 2015) may serve as this bridge. This framework focuses on creativity driving the production of useful solutions to problems, in other words, meaningful design deliverables. As applied to instructional designers designing interventions, the authors maintain that the goal of creative performance is to solve problems and create innovative ways of thinking or doing. Their framework includes three measures, Novel, Effect and Whole. Creative work is Novel when it produces something into the world that did not exist before at least in the specific context. For instructional designers, it is a new instructional intervention to solve a problem or meet a need. Novelty alone does not offer creativity; however, it requires purpose or usefulness. A creative product must add value or be Effective towards a purpose, designing a deliverable where the learning and performance context meet. Creative products (ideas, artifacts, etc.) are sensitive to context, and must be valued within the context in which the products were created. Beyond being novel and effective, creative products have a certain aesthetic quality—the Whole—which is connected to and evaluated within a context (Henriksen et al., 2015). Creative products and solutions are deeply bound to the context within they occur, designed for a localized context of use (Henriksen et al., 2015). Henriksen, Mishra and Mehta created a rubric to evaluate the Novel, Effect and Whole of designed artifacts. They validated this rubric by analyzing 350 different student-generated artifacts and performed an inter-rater reliability test by having two coders independently code 10% of the projects. There was an 87% agreement between the coders. Their goal for this rubric is to fit with the very nature of creativity, flexible in interpretation and sensitive to context. In other words, when applying the rubric to different projects, the definitions of Novel, Effect, and Whole stay in place, while the project directors then determine the contextual elements.

Unbeknownst to us at the time, the rubric we used to measure the deliverables for this study mirrors the Novel, Effect and Whole Framework (Henriksen et al., 2015). When measuring the presentation of activities, we looked for activities that focused on the meeting of the learning and performance context or Novel and Effective activities for the end learners. We focused on the visual presentation of activities and experiences for the learners in their localized context of use, the Whole in the Henriksen et al. (2015) framework. During the next phase of our research, we will provide our design students with the Novel, Effect, Whole framework along with the empathic design approach. We will then study the design deliverables produced to determine if this is the bridge connecting empathy, empathic design, and the design of meaningful deliverables.

Conclusion

The need for empathy and empathic design will not subside. The design thinking methodology, with empathy as its first stage is ingrained in numerous design practices. Empathy and empathic design resulting in a meaningful design deliverable that meets the audience needs however, is an assumption that may result in designs that are not meaningful for the end learner. Our research sought to determine how graduate student instructional designers use empathy and empathic design to produce meaningful design deliverables. Seven of the nine student teams produced deliverables that the clients implemented at the end of the semester while two products needed additional design work before being turned over to the client. Although seven deliverables met the client need, our research indicated that the instances of empathy did not necessarily result in a meaningful design. This doesn’t mean that the instances of empathy were not present, but there is a missing link in the empathy & empathic design = meaningful deliverables equation. The Novel, Effect, Whole framework may be the bridge connecting empathy, empathic design, and meaningful design deliverables. During the next phase of our research, we will provide our design students with the Novel, Effect, Whole framework along with the empathic design approach. We will then study the design deliverables produced to determine if this is the bridge connecting empathy, empathic design, and the design of meaningful deliverables.

A meaningful design deliverable presents activities and experiences where the learning and the performance context meet. A meaningful design deliverable focuses on the learners’ localized context of use. As designers, we respond to the learner’s feelings, experiences, and situation. Our response results in meaningful design deliverables. We embrace Batson (2009) in that we believe teaching empathy is not about explaining a form of knowledge but to explain a form of action. Empathy and empathic design for us as instructional designers, drives motivation to do something, to design something, to deliver a meaningful design deliverable.