Keywords

1 Introduction

Preparing undergraduate students for their next career steps, including graduate school, professional school, and industry, requires the attainment of both technical skills and soft skills (Scheufele & Krause, 2019). In the biological sciences, necessary skills include the ACE-Bio Basic Competencies of Biological Experimentation (Identify, Question, Plan, Conduct, Analyze, Conclude, and Communicate; (Pelaez et al., 2017; Chap. 1 in this volume)) along with leadership skills, creativity, critical thinking skills, teamwork, and innovation . The students involved in this described summer innovation program utilized all seven of the ACE-Bio competencies during their design and prototyping process. We explicitly assessed growth in the ACE-Bio competency, Communicate, as well as additional skills that are important to successful biological experimentation and innovation . Individuals in these various situations must be creative and committed to their work. They must also have strong critical thinking skills and be able to communicate their ideas effectively. Importantly, no big project is completed alone; therefore, it is important for students to have good collaboration skills (Jarmai & Vogel-Pöschl, 2020). These innovation skills are important for all career paths and are highly sought-after (Li, 2017). Teamwork is necessary for almost all aspects of life (Salas et al., 2018). These skills are also useful in biological experimentation where teams are often essential to project success. Without diligent teamwork, projects are often delayed or abandoned. In research careers in particular, projects require broad skill sets and often-times no single person is an expert in all of these areas. Collaboration skills are required in order to design better experiments, ask more refined questions, and conduct more thorough analyses (Bennett & Gadlin, 2012; Cheruvelil et al., 2020). Creativity and critical thinking are important in experimentation, particularly during troubleshooting and experimental design (DeHaan, 2009). These skills are used to help develop solutions after identifying a gap in knowledge or a limitation in previous research. Creativity and critical thinking are also needed to draw inferences and conclusions (DeHaan, 2009). In biological experimentation, having a strong commitment to one’s project is critical for success, rather than simply going through the motions. Finally, the ability to communicate one’s findings is perhaps the most important piece of experimentation. Scientists present findings in both written and verbal formats at poster sessions, oral presentations, lab meetings, and popular media. Effective communicators can communicate findings to a layperson audience (Brownell et al., 2013; Scheufele & Krause, 2019). In a world of “fake news” and pseudoscience, the ability to communicate findings to diverse audiences is an imperative skill (Scheufele & Krause, 2019). Measuring these innovation and teamwork skills will assist educators to assess student growth in soft skills that lead to success for scientists and entrepreneurs.

Teaching soft skills in classroom settings can be challenging. When done effectively, it requires extensive feedback from both the instructor and from peers. While peer review is implemented in many classroom settings, feedback may not be given in a manner that promotes growth and change (Bailey & Garner, 2010; Poulos & Mahony, 2008). Therefore, improving feedback mechanisms to allow for conversation can also support growth. It is known that students benefit from both giving and receiving feedback (Nicol et al., 2014) so it is important that they are given opportunities to practice this. Understandably, individuals often find feedback uncomfortable (Hattie & Timperley, 2007), but practice in giving and receiving constructive feedback may demonstrate its value.

In higher education settings, teamwork and communication soft skills are often “taught” by using planned activities such as group work and oral presentations. However, methods for assessment of the impact of such activities are limited. One 17-item instrument proposed by Corwin et al. (2015) has been used to assess skills in collaboration , discovery and relevance, and iteration in laboratory courses. In particular, the authors assessed the impact of course-based undergraduate research experiences as a way of growing these skills (Corwin et al., 2015). A recently published rubric was proposed for assessing student critical thinking skills in written assignments, which could be used in evaluating laboratory reports, for example (Reynders et al., 2020). Interdisciplinary or interprofessional teamwork can also be used to expose students to collaboration challenges (Havyer et al., 2014; Morphet et al., 2014). While innovation challenges have been used successfully in many business and entrepreneurship programs (Harkema & Schout, 2008), innovation challenges in health care or other contexts are less frequent and constitute an especially promising way to engage STEM students in collaboration and teamwork (Pellegrini & Jansen, 2013). Systematic assessment of the impact of these experiences is possible, but not the usual practice. Here, we present a template for implementing learning activities and assessing the impact of innovation , problem-based learning, or biological experimentation on the students’ collaborative and teamwork skills.

2 Assessment Tools

Our proposed toolkit assesses student growth in skills of communication, creativity, critical thinking, and collaboration . This toolkit includes the Qualities of an Innovator survey, behavioral assessment surveys, reflection questions, and semi-structured interview questions. This toolkit was developed to understand student growth during participation in innovation and experimentation projects across disciplines. In our pilot use of the toolkit, we utilized each component at the beginning of the experience and following the experience. Behavioral assessments were used weekly to gauge student perceptions of their own behaviors and those of their teammates. The Qualities of an Innovator survey (Table 20.1) was built to assess a students’ inclination for innovation and their confidence in their skills in collaborating, communicating ideas, creating, and thinking critically. This survey is a self-reported indication of confidence in their innovation skill set, rather than a measure of skill attainment. For each skill, a percentage of possible points was computed since not all skills were assessed by the same number of items.

Table 20.1 Qualities of an innovator survey. Items A-I were used to assess inclination for in-novation. Items J-L were used to assess communication, items M-P were used to assess creativity, items Q-T were used to assess critical thinking, and items U-X were used to assess collaboration . Item E was reverse scored
Full size table

Behavioral assessments were developed to evaluate perceptions of ability to collaborate, create, communicate, and think critically. It also measures commitment to the project. Behavioral assessments can be modified for self-assessment, peer/teammate assessment , and evaluator or instructor assessment (Table 20.2). For modification to a peer or teammate assessment , all “I” statements become “my teammate” statements. To modify to an instructor’s assessment , the statements lead with “the student.” Behavioral assessments were designed to evaluate these skill sets for an individual and compare self-perceptions with those of others. As with the Qualities of an Innovator survey, scores were expressed as a percentage of possible points for each skill. At the end of the survey, participants were asked to identify their greatest strengths and weaknesses.

Table 20.2 Behavioral assessment , in the form of a self-assessment. These assessments can be modified to be used by leadership or by teammates or peers. Items A-C were used to assess communication, items D-G were used to assess collaboration , items H-K were used to assess critical thinking, items L-N were used to assess creativity, and items O-Q were used to assess commitment. Items D and J were reverse scored
Full size table

Semi-structured interview questions (Tables 20.3, 20.4, and 20.5) explore key constructs in more detail. These interviews help evaluators to assess growth in the innovation skillset, including communication, collaboration , critical thinking, and creativity. The interviews also provide additional insight into challenges students may face as they growing in these skills. A suggested timeline for use of these interview questions is pre- and post- experience, with a more long-term post-experience time point to determine the long-term impact of the program or experience. Examples of student responses to the semi-structured interview questions, collected in a Pilot Study of Clinical Innovation Presidential Fellowship Program participants at the University of Alabama at Birmingham are provided with the interview questions from each time point: pre-innovation challenge (Table 20.3), post-innovation challenge (Table 20.4), and at a 3-month follow up (Table 20.5). NVivo software was used to help identify common themes in the student responses.

Table 20.3 Semi-structured interview questions suggested for an early time point in the project, perhaps pre-innovation or experimentation experience. Questions are organized by theme and common responses from the pilot program are recorded. This interview was conducted in the second week of the pilot program
Full size table
Table 20.4 Semi-structured interview questions for late-stage participation in innovation or experimentation programs with summarized student responses from the Clinical Innovation Presidential Fellowship Program following week 10
Full size table
Table 20.5 Semi-structured interview questions regarding the long-term impacts of participation in innovation or experimentation programs and summarized student responses from the Clinical Innovation Presidential Fellowship Program 3 months post- program conclusion
Full size table

3 Feedback and Guided Reflection

Providing structured feedback based upon the behavioral assessments and the Qualities of an Innovator survey can help students to understand their self-perception versus perception of others for effort in these areas. Structured feedback was provided to each of the Clinical Innovation Presidential Fellowship Pro-gram students at weeks 6, 8, and 10 (Beno et al., 2020). This feedback included a compilation of behavioral assessment data from their teammates compared to their self-evaluations. Assessment data was broken down by skill area and provided to students as a percentage of possible points. All strengths and weaknesses comments were shared with the students, as well. Following these feedback sessions, the use of guided reflection questions (Table 20.6) helps encourage student understanding and promote discussion of the feedback that they received. One of the biggest potential benefits of feedback comes from this reflection piece, as individuals gain a stronger sense of understanding, growth, and purpose. Examples of the themes in student responses from the Clinical Innovation Presidential Fellowship Program are provided with the guided reflection questions (Table 20.6).

Table 20.6 Guided reflection questions and summarized responses from students participating in the Clinical Innovation Presidential Fellowship Program
Full size table

4 The Innovation Toolkit at Work

The University of Alabama at Birmingham Clinical Innovation Presidential Fellowship Program offered 10 interdisciplinary students (traditional STEM and nursing) the opportunity to develop solutions for real-world problems in the hospital setting over the course of 10 weeks in 2019 (Beno et al., 2020). Fellows were introduced to the design thinking process, which has been presented as a promising method for teaching innovation (Altman et al., 2018). In the specific cases discussed here, teams were tasked with developing solutions to prevalent issues surrounding sepsis. Sepsis is a leading cause of death in hospitals worldwide (Rudd et al., 2020), and these teams worked to develop new methods for detection and training to alert for sepsis. While these projects were not traditional biology experiments, they required intensive research analogous to course-based undergraduate research experience or mentored independent research.

4.1 Timeline and Methodology of Assessment

For purposes of the fellowship, students were divided into two interdisciplinary teams. These students served as pilot testers for the Innovation Toolkit. Through regular feedback and guided reflection , students identified areas for improvement and discussed important steps for implementing changes that helped them to reach their goals. A timeline of the activities in this fellowship program and assessments used are outlined in Fig. 20.1. In the Clinical Innovation Presidential Fellowship Program, participants spent the first week in a clinical immersion, getting to explore issues related to sepsis in the clinical setting. In week 2, the participants were split into interdisciplinary teams and began to work on problem definition. Ideation and exploration began around week 4 and by week 6 both teams were working on prototyping their innovative solutions prior to a final presentation in week 10. Throughout the innovation experience, participants worked on building empathy and had weekly structured practices for the final presentation. At week 6, all participants partook in a team reflection session. During these sessions, an evaluator asked students to discuss their structured feedback with their teammates and guided participants through determining action items, changes individuals could make to become more effective teammates. Week 6 was therefore an important evaluation point for the Qualities of an Innovator and Behavioral Assessment surveys.

Fig. 20.1
figure 1

Timeline of activities and assessments during the 10-week Presidential Fellows Clinical Innovation Program. The program included a series of structured feedback and guided reflection opportunities, as well as surveys and semi-structured interviews used to inform the program organizers of shifts in skill development

Full size image

Students participating in this program spent different amounts of time working on the projects: students pursuing BSN degrees contributed a minimum of 6 h per week to their team projects due to obligations to clinical experiences and required coursework, whereas students from other STEM fields contributed a minimum of 30 h per week to their projects. As such, we hypothesized that STEM students would show greater improvement in their critical thinking, creativity, collaboration , and communication skills as compared to their BSN counterparts. Semi-structured interview data led us to further investigate differences in skills development, particularly in collaboration , within each project team.

Differences in skill perception of STEM and BSN students, as well as between project teams, were assessed using data from the Qualities of an Innovator survey collected at weeks 1, 6, and 10. These were analyzed using independent samples t-tests. Behavioral assessment survey data was evaluating using an average of weeks 3–4 as baseline (early in the team formation), an average of weeks 5–7 as a middle point, and an average of weeks 8–10 as an end point. Self-assessments and peer-evaluations were analyzed separately. Growth was measured using F-tests in a one-way ANOVA. All Qualities of an Innovator survey data were assessed between weeks 1 and 10 and between weeks 6 and 10. Similarly, behavioral assessment data was evaluated using the average of weeks 3–4 compared to the average of weeks 8–10 and the average of weeks 5–7 compared to the average of weeks 8–10. All statistics were run using SPSS.

4.2 Pilot Assessment Results

Importantly, all data collected from our pilot study was collected from just ten students, which limits the quantitative conclusions that can be drawn from our study. However, a number of observations were made. All students were highly confident in their innovation skills even at the start of the program. By the end of the program, students reported increased confidence in communicating and presenting ideas. Confidence in creativity also increased from the start to the end of the program, and no differences were observed between disciplines or the two project teams. We observed some behavioral differences between students in traditional STEM disciplines compared to those who were pursuing degrees in nursing. We predicted these differences based on differences in participation requirements as well as discipline-specific stereotypes. Even early in the program, STEM students who were required to spend more time on their projects were perceived as being more committed to their projects (t(8) = 6.45, p < 0.001) and as having stronger critical thinking skills (t(8) = 2.39, p < 0.05) by their teammates, as compared to the BSN student team members (Beno et al., 2020). The teammate behavioral assessments revealed that STEM students, were perceived as more effective communicators, being more committed to their projects, and having higher critical thinking skills than their BSN student counterparts from the start of the project to the end of the project (F(1,8) = 7.90, p < 0.03, F(1,8) = 16.1, p < 0.005, and F(1,8) = 33.26, p < 0.001) (Beno et al., 2020).

Interestingly, we also observed a difference in perceived commitment to the projects between teams. The team working on a virtual reality training tool reported increased commitment from the initial to final phases of the project (70% vs 84%) whereas the team working on a wearable device for vital signs self-reported decreased commitment (69% to 60%) from the initial to final phase of the project (Beno et al., 2020). Teammate behavioral assessments corroborated this observation.

Semi-structured interviews helped interpret these observations. The interview responses supported our findings with differences between disciplines and between teams. From the interviews, we discovered that some of the differences in project commitment might have resulted from differences in expertise. STEM students felt valued throughout both the virtual reality and wearable device project design process, but students pursuing BSN degrees reported that while their expertise was needed to develop the virtual reality-training program, they lacked the technical skills needed to participate in the wearable device project as it progressed. Furthermore, participants recognized that their disparate time commitments affected their perceived growth in requisite technical skills. The guided reflection questions indicated students valued the experience of giving and receiving feedback throughout the summer, despite finding it uncomfortable initially. They realized that their teamwork was greatly facilitated by honest interaction about barriers and challenges.

4.3 Pilot Observations

Our toolkit provided insight into differences between students from different disciplines and between teams. We found overall increased confidence in presentation and communication skills for all students from the beginning to the end of the fellowship program. Confidence in critical thinking skills and creativity trended higher, but was not statistically significant, likely due to a small sample size. This confidence in these three skill sets was maintained 3 months post-fellowship. Following the guided team reflections, we observed significantly higher confidence in collaboration and creativity skills, suggesting that frequent feedback in both formative and summative instances are crucial for skill development. Future use of this toolkit may give educators further insight into student inclination for innovation .

5 Implications of the Toolkit

The toolkit presented here can be used across disciplines to assess student growth from active participation in an innovation challenge, problem-based learning activity, or experimental opportunity. In university classrooms, instructors often teach collaboration using group projects (Beier et al., 2018) and communication through oral presentations (Braun, 2017; Parker et al., 2020). However, these skills are rarely formally assessed. The toolkit can be used to track changes in this skill set over time, to identify time points for instructor innovation , and to understand interdisciplinary team dynamics. This toolkit allows evaluators to determine what changes are maintained throughout the course of a project or experiment , which can create opportunities for intervention to ensure successful teaching of these key skills as opposed to merely waiting for students to develop them.

5.1 Toolkit Use for Assessment of Essential Skills in Biological Experimentation

The toolkit assesses collaboration , communication, creativity, and critical thinking and can be utilized at different time points to measure change. This toolkit was designed to measure both confidence in the ability to use this skill set and in the perception of successful use and development of these skills. These skills are essential in biological experimentation and for successful scientists.

Collaboration is often key to a successful experiment . Through collaboration , projects can be completed more efficiently, more quickly, and with greater precision. Most research requires collaboration and relies on technical expertise of different individuals to be done well (Pelaez et al., 2018; Vermeulen et al., 2013). Having the skills to effectively collaborate with others is needed in order to participate in the global scientific culture.

Likewise, scientists must be effective communicators. In the general population, science literacy remains challenging (Rosenthal, 2020; Scheufele & Krause, 2019). Scientists performing biological experimentation therefore must be prepared to communicate their ideas and findings in layman’s terms to the appropriate audiences (Brownell et al., 2013). Scientists must also be able to communicate within a project (Pelaez et al., 2017; Chap. 1 in this volume), as highlighted by our pilot study semi-structured interviews. These students often brought up the importance of developing better communication skills.

Of course, creativity, innovation , and critical thinking are also important within biological experimentation. Scientists need to be able to brainstorm new ideas, evaluate existing research, and synthesize new information by designing experiments (DeHaan, 2009). Troubleshooting, which is inevitable in science, requires both critical thinking and creativity. Communicate is a core component of the ACE-Bio Competencies (Pelaez et al., 2017; Chap. 1 in this volume) as it is crucial for scientists to convey their research to others. While creativity, critical thinking, and collaboration are not overtly represented in these Basic Competencies of Biological Experimentation, both creativity and critical thinking are represented by the skills outlined in each competency. For example, creativity is needed in Question, Plan, Analyze, and Conclude. Likewise, skills outlined in competencies Identify, Question, Plan, Conduct, Analyze, and Conclude are specific examples of critical thinking. While collaboration is not included in the core competencies, it is an essential skill in biological experimentation. The ACE-Bio competencies are complemented by these additional skills, and these skills are required for scientists and relevant in many disciplines.

5.2 Toolkit Use in Broad, Interdisciplinary Situations

The piloting of this toolkit was for an interdisciplinary innovation challenge that included students in a nursing program and students from neuroscience and engineering disciplines. Pieces of the toolkit have since been used in courses in various undergraduate disciplines at the University of Alabama at Birmingham. The skill sets assessed in the toolkit presented here are integral to a successful biological experimentation, but they are also important for professional development. Therefore, simple modifications to personalize the toolkit to a specific discipline will help instructors ensure that students are well prepared for success in their careers. The Qualities of an Innovator Survey and self- and teammate- behavioral assessments were recently used in diverse subjects across campus, including a course for education majors planning to teach social studies, a kinesiology course, a nursing course, and another class focused on teaching innovation . Instructors reported compatibility of this toolkit for their various courses and future research will investigate the skill development of students participating in different project-based learning activities.

6 Future Directions and Overall Importance

The use of assessment and feedback are of utmost importance for student learning (Andersson & Palm, 2017; Rushton, 2005). A critical point to successful feedback is to keep things constructive. When giving feedback, it helps to focus on something that can be improved (Hardavella et al., 2017). In course design, it is important to include both formative and summative feedback and this principle carries into skill development. In our pilot of this toolkit, students reflected that the consistent use of feedback helped them to recognize how they were perceived and guided methods to improving collaboration and teamwork. Consequently, the implementation of feedback sessions may be highly important in developing these skills quickly.

Successful innovators can be successful in a variety of careers. The skills identified for the innovation toolkit: creativity, collaboration , critical thinking, and communication are all skills that are necessary for success in many careers. In particular, to develop effective scientists through biological experimentation we must also relate these skills to ACE-Bio Basic Competencies of Biological Experimentation outlined by Pelaez et al. (2017 Chap. 1 in this volume). Importantly, it is also critical that students develop competencies in successfully working in diverse and interdisciplinary groups. Data show scientific papers with authors of diverse academic discipline, location, and gender are cited at higher rates (Adams, 2013; AlShebi et al., 2018). While laboratory-based courses often are limited to students within a specific major, instructors can encourage diverse groups by combining students with different career goals or interests. In many careers, project teams are built with experts of different skill sets. The toolkit presented here can be used to better prepare students for their future endeavors.

For instructors and/or education researchers hoping to utilize the Innovation Toolkit, we offer the following recommendations:

  • Utilize the Qualities of an Innovator survey at minimum at the beginning and end of the experience.

  • Behavioral assessment surveys should be used often to track student self- and peer-perceived growth in the innovation skill areas.

  • Midway through the experience, the use of guided discussion following feedback may serve as an intervention. Using a mediator (instructor or evaluator), ask students to elaborate on their greatest strengths and weaknesses.

  • Ask students for action items- how will they become more effective experimenters moving forward?

  • Provide ample opportunities for self-reflection and for feedback within teams