1 Introduction

The ability to engage in the creative process is an essential component of the engineering profession (Howard et al. 2008) due to the link between innovation and long-term economic success (Ayag and Ozdemir 2009). As such, engineering research has long since been devoted to increasing the creative abilities of engineering students and professionals through the development and testing of idea generation methods (see, e.g., Cardin et al. 2013; Chulvi et al. 2012; Oman et al. 2013; Sarkar and Chakrabarti 2014; Shai et al. 2013; Yang 2009). Despite the recognized importance of creativity throughout the engineering design process, there are few studies that have explored the role of creativity during the concept selection process. This is a vital area to explore because in order for innovation to occur, the creative concepts generated during the early phases of design must be recognized and selected during the concept evaluation process (Rietzschel et al. 2010).

A variety of formalized concept selection methods are often taught in engineering education (see, e.g., Ayag and Ozdemir 2009; Hambali et al. 2009; Jacobs et al. 2014; Okudan and Tauhid 2008). These methods and their merits and disadvantages have received considerable attention from the design community (Frey et al. 2009, 2010; Hazelrigg 2010). Researchers have also noted that these selection methods have been developed from various research strains that each approach the decision-making problem in vastly different manners (Reich 2010). However, research has shown that companies lack a coherent or formal process for selecting ideas (Barczak et al. 2009). Instead, the early phases of concept evaluation typically involve a screening process where the ideas generated in the early phases of design are narrowed down to a few key concepts through informal team discussions (Onarheim, and Christensen 2012). While these informal methods can be effective in various contexts, they are often subject to the biases associated with human decision making (De Martino et al. 2006). For example, factors such as preferences for visually complex designs (Onarheim and Christensen 2012), development time (Kruglanski and Webster 1996), organizational culture (Amabile 1996), designer personality traits (Kichuk and Wiesner 1998), and ownership bias (Onarheim and Christensen 2012) can influence decision making during informal concept selection.

Research on concept selection in normative brainstorming groups (Delbecq et al. 1975) has found that people often perform poorly at selecting creative ideas during the evaluation process (Rietzschel et al. 2010) due in part to biases toward self-generated concepts (Nikander et al. 2014), visually complex designs (Onarheim and Christensen 2012), and salient ideas (Harvey and Kou 2013). Similarly, recent research has shown that the type of logical reasoning used during decision making can affect the selection of creative ideas (Dong et al. 2012). In addition, research on individual creativity has found that individuals often have a bias toward familiar or conventional ideas during concept selection because of the risk associated with creative ideas (Ford and Gioia 2000; Rietzschel et al. 2010), demonstrating a close link between risk attitudes and perceptions of creativity (Mueller et al. 2011; Nicholson et al. 2005; Zuckerman and Kuhlman 2000). Although not studied in the context of concept selection, personality, which is closely related to risk (Eysenck and Eysenck 1977; Whiteside and Lynam 2000; Zuckerman et al. 1993), has also been linked to creative performance in idea generation tasks (Baer et al. 2007). While these studies identify attributes that may impact creative concept selection, they focus on individual concept selection tasks leaving to question how these factors influence decision making in a team setting. Without this knowledge, it is impossible to know what team-based factors impact the selection or filtering of creative concepts. This is important because design is being recognized and taught as a team process in engineering (Dym et al. 2003).

Therefore, the purpose of this study is to investigate the impact of team risk attitudes and personality traits on the selection of creative concepts during team-based concept selection practices in engineering education. In order to accomplish this, an empirical study was conducted with 37 engineering students in order to understand the impact of team personality, risk attitudes, and creative abilities on a team’s propensity toward creative concepts. The results of this study add to our understanding of team-based decision making during concept selection and allow us to provide guidelines for developing and training design teams to identify and select creative ideas. The following sections provide background and motivation for studying the factors that can affect creative concept selection in teams, and start with a section that explores the role of personality traits and creativity in the design process. Next, research that has investigated the impact of risk attitudes on the creative process is discussed, and lastly, the research questions that are investigated in this paper are presented.

2 Background and motivation

2.1 Personality traits and team creativity

Design is increasingly being recognized and taught as a team process in engineering (Dym et al. 2003), in part because products developed by teams have been shown to be of higher quality than those produced solely by an individual (Gibbs 1995) and in part because teams foster a wider range of knowledge and expertise which aid in the development of ideas (Dunne 2000). In addition, teamwork has been shown to increase classroom performance (Hsiung 2012) and encourage more creative analysis and design (Stone et al. 2006). Therefore, researchers have focused their efforts on identifying the factors that impact team-based creativity.

Studies conducted in these areas show that factors such as organizational culture, individual abilities, group diversity, and resources can greatly influence overall team creative performance (Agrell and Gustafon 1996; Woodman et al. 1993). While these factors are important in determining overall group performance, researchers have argued that the composition of team member personality and disposition is one of the most important factors in determining team performance and (Wilde 1997) creativity (Somech and Drach-Zahavy 2011). In fact, the Big Five Factors of Personality (five-factor model) framework (Costa and McCrea 1992) has been shown to be strongly linked to creativity (Feist 2006).

The five-factor model states that personality has five dimensions: neuroticism, extraversion, openness to experience, agreeableness, and conscientiousness. Researchers have linked the extraversion, openness to experience, and agreeableness personality traits to creativity at the individual level (Batey and Furnham 2006). Specifically, studies have shown that creative achievement is closely related to high levels of extraversion (Stafford et al. 2010) and openness to experience (McCrae 1987; Steel et al. 2012). Results on agreeableness, on the other hand, have had mixed findings; some studies have reported that high levels of agreeableness relate positively to creative ability (Feist 1998), while others have found that creative individuals have low levels of agreeableness and “do not adapt to others, but go their own way” (p. 254) (Hoff et al. 2012). Factors that influence individual creativity are important for group creativity because the creative process starts with individuals conceptualizing ideas and then deciding whether or not to share them with the team (Gilson and Shalley 2004).

At the team level, where aggregate scores of team member personality attributes are analyzed (Mohammed and Angell 2003; Reilly et al. 2001), researchers have found that high levels of extraversion, openness to experience, and low conscientiousness tends to lead to the creation of more creative ideas in design teams (Baer et al. 2007). However, the results on the personality traits that impact this higher level of creative concept generation have been mixed. Specifically, researchers have argued that teams with high conscientiousness and agreeableness levels are more motivated to achieve goals (Bell 2007) and thus tend to be more creative (Woodman et al. 1993), while others still have argued that agreeableness and neuroticism are required for group creativity (Goncalo and Staw 2006). However, there has been limited research on the role of team personality attributes and creative concept selection.

These studies highlight the impact of individual personality traits on team-level creativity, but also show conflicting findings on which personality traits significantly impact team creativity. In addition, most research conducted in this area investigates the impact of personality traits on a team’s ability to generate creative ideas, leaving little data on how personality traits affect a team’s tendency to select creative concepts. Therefore, the current study was developed to respond to this research void.

2.2 Risk-taking and team creativity

In addition to personality traits, it is also important to study the role of risk attitude in creative concept selection as prior work has shown that risk attitudes impact an individuals’ perception of creativity (Rubenson and Runco 1995) and their creative abilities (Dewett 2007; El-Murad and West 2003). In the context of creativity, risk can be used to describe the extent to which there is uncertainty about whether potentially significant or disappointing outcomes will be realized given creative effort (Sitkin and Pablo 1992). Researchers have argued that risk-taking is an essential element of creativity since it encourages the individual to push boundaries and explore new territories (Kleiman 2008). However, it has been shown that individuals often select conventional or previously successful options during the concept selection process (Ford and Gioia 2000) due to their inadvertent bias against creativity (Rietzschel et al. 2010). Recent research conducted in this space has found that student design teams typically base decisions on the technical feasibility of ideas (Toh and Miller 2015). Because people have a deep-seated desire to maintain a sense of certainty and preserve the familiar (Sorrentino and Roney 2000), individuals may prematurely filter out novel ideas during the concept selection process regardless of merit in order to reduce risk. Risk not only impacts the individuals’ creative level, but also impacts their larger role in the social structure. Specifically, Perry-Smith (2006) showed that individuals who play a central role in the team and who have fewer external ties are more likely to take risks in group settings and score higher on supervisor-rated creativity. Therefore, it is essential that we understand the impact of risk-taking during team concept selection activities in order to promote the flow of creative ideas throughout the design process.

In addition to risk aversion, ambiguity aversion has also been studied in the context of creativity. While risk aversion is often calculated in situations where outcomes have a fixed probability of occurring, ambiguity aversion is calculated in situations that are more uncertain or where outcomes have an unknown probability of occurring (Moore and Eckel 2003). Ambiguity is significant to the study of decision making since many realistic situations involve both risk and ambiguity (Heath and Tversky 1991). Therefore, researchers have focused on studying the link between ambiguity aversion and creativity. Studies such as those done by Charness and Grieco (2013) have shown that an individual’s tolerance for ambiguity is linked to creativity in problem-solving tasks. Similarly, other studies reveal that an individual’s tolerance for ambiguity is positively correlated with creative performance (Sternberg and Lubart 1991; Zenasni et al. 2008) and is often a requirement for creativity, especially in scientific domains (Csermelv and Lederman 2003). While it is clear that both risk and ambiguity aversion are important factors that impact creativity, little research has been conducted regarding the possible effects that these factors may have on creative concept selection.

One of the main obstacles to overcome when exploring the relationship between risk and creative concept selection is identifying a method for appropriately measuring individual risk attitudes in creative design tasks (Weber et al. 2002). While there are a variety of ways to measure risk attitudes such as through the calculation of risk propensity (Dewett 2006), engineering-domain-specific risk-taking (Bossuyt et al. 2012; Van Bossuyt et al. 2013), and preference of ambiguity to risk (Charness and Grieco 2013), their relationship to risk in a creative task is largely unknown. Due to the fact that no measure exists that assesses risk-taking in the context of creative concept selection, and since risk behavior has been shown to vary greatly across situations and domains (Weber 2010; Weber et al. 2002), it is unclear how existing measures of risk can be used to measure risk-taking in a creative domain. A common method of studying risk behavior is through the use of traditional behavioral economics measures such as utility theory (Boyle et al. 2011, 2012; Han et al. 2012) or variants such as prospect theory (Kahneman and Tversky 1979) that use financial lotteries to determine risk and ambiguity attitudes since these measures have a high adoption rate and familiarity in existing design research. However, these measures have not been tested for their relationship to risk-taking in creative tasks. Other measures such as psychometric domain-specific risk-taking should also be explored for their role in creative concept selection since researchers have shown that the perception of what constitutes a risky situation can be context dependent (Weber 1999). Risk behaviors in the financial, ethical, and social domain are of particular interest to the study of risk in engineering design since much of design occurs in team-based project settings. Therefore, work is needed that explores the relationship between traditional behavioral economics and psychometric domain-specific measures of risk attitudes on risk-taking in a creative context in order to bridge the gap between risk attitudes in these different domains.

2.3 Research objectives

The goal of this study is to identify factors that impact creative concept selection in engineering design teams through an empirical study. Specifically, the following research hypotheses are addressed:

Hypothesis 1

The creativity of an idea has no impact on its likelihood of being selected during concept selection. We anticipate this result since prior research has shown that individuals often select conventional or previously successful options during the concept selection process (Ford and Gioia 2000).

Hypothesis 2

Creative idea generation ability affects team propensity for creative concept selection. We anticipate that teams who generate creative ideas (a combination of novelty and quality) will have a higher propensity for selecting creative ideas since prior research in psychology has shown that individuals who generate ideas with higher novelty are more likely to select novel ideas during group discussions (Putman and Paulus 2009).

Hypothesis 3

Team risk-taking attitudes affect team propensity for creative concept selection. We anticipate that teams who are more risk prone will have a higher propensity for selecting creative ideas since prior research has shown that individual risk attitudes affect one’s perception of creativity (Rubenson and Runco 1995).

Hypothesis 4

Team personality traits (specifically agreeableness, conscientiousness, and neuroticism) affect team propensity for creative concept selection. This hypothesis is based on prior research that showed that teams with high conscientiousness and agreeableness levels are more motivated to achieve goals (Bell 2007) while agreeableness and neuroticism are required for group creativity (Goncalo and Staw 2006).

These hypotheses are built on our previous research that found that individual-level risk attitudes can affect creative concept selection and generation in design (Toh and Miller 2014b).

3 Methodology

To address our research questions, a controlled study was conducted with engineering design students at a large northeastern university. During the study, participants were tasked with completing an idea generation activity and a concept selection activity in design teams. The details of this study are provided in the following sections.

3.1 Participants

Thirty-seven engineering students (25 males and 12 females) participated in this study. Nineteen of the participants were recruited from a first-year introduction to engineering design course, while the remaining 18 participants were recruited from a third-year mechanical engineering design methodology course. Participants in each course were in 3- and 4-member design teams that were assigned by the instructors at the start of the course based on prior expertise and knowledge of engineering design (four 4-member teams, seven 3-member teams). This team formation strategy was used to balance the a priori advantage of the teams through questionnaires given at the start of the semester that asked about student proficiencies in 2D and 3D modeling, sketching, and the engineering design process. Thus, design teams were formed in such a manner that no single team was significantly more proficient at these design skills.

3.2 Procedure

One week before the study, participants were introduced to the purpose and procedure of the study and were given an informed consent form to complete. Participants were given brief information regarding the purpose and procedure of the study, but no specific details about the design task, purpose of risk and personality measures, or research hypotheses were disclosed to participants. Therefore, participants were not given any information that could enable them to prepare for the design task in any meaningful way. Once informed consent was obtained, participants were asked to complete an online survey that assessed individual risk aversion and ambiguity aversion using a set of 20 lottery questions (10 each for risk and ambiguity aversion), see “Metrics” section of this paper for a description of the questions. The lottery questions were developed and utilized according to established measures used in standard behavioral economics (Boyle et al. 2011, 2012; Han et al. 2012) in order to capture each individual’s level of risk aversion and ambiguity aversion. In addition, personality measures for each participant were captured using the Short Form for the IPIP-NEO (International Personality Item Pool Representation of the NEO PI-R™) online questionnaire (Johnson 2014). Participants were assigned unique participant identification code for use in the online surveys and subsequent design tasks in order to maintain participant anonymity.

One week after the online surveys were completed, participants attended a design session where they were asked to develop a novel device to froth milk. While the design session took place at the same time and location of the participant’s design course, the activities were led by the research team and were independent of required class-related activities. The design task used in this study was selected to represent a typical project in an engineering design course. Students in these courses typically redesign small, electromechanical consumer products that require minimal engineering knowledge or expertise (Simpson and Thevenot 2007; Simpson et al. 2007). In order to make sure that our task fits within this spectrum, the design task went through a round of pilot testing with other undergraduate students in order to identify a task that most engineering undergraduate students were neither familiar nor unfamiliar with. In order to ensure that our participants were equally familiar with the product being explored, our design task went through pilot testing with first-year students prior to deployment. Specifically, the design task provided to participants in the current study was:

Your task is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction. Focus on developing ideas relating to both the form and function of the product.

Participants were informed that the goal of the design task was to generate creative early-phase ideas to satisfy the design goal.

Each participant was then provided with sheets of papers and asked to generate as many concepts as possible for a novel milk frother. Participants were given 20 mins for this brainstorming activity and were asked to stop generating ideas at the 20 min mark. This brainstorming activity was conducted individually in order to facilitate the free flow of ideas without judgment and to avoid distractions that can occur in group brainstorming activities (Diehl and Stroebe 1987). Participants were instructed to sketch only one idea per sheet of paper and write notes on each sketch such that an outsider would be able to understand the concepts upon isolated inspection, as shown in Fig. 1. It should be noted that no financial compensation was offered for participation; participants were motivated, perhaps, by the grade received in the course that was based on the novelty and feasibility of the final design concepts.

Fig. 1
figure 1

Example concepts sketched by participant N03AX

Full size image

Following the idea generation session, participants were given a 3 h break. Next, the second design session was completed where participants were asked to individually review and assess all concepts that their design team had generated in the previous session. Participants then formed their design teams that were assigned by the course instructor at the start of the semester and were asked to categorize each concept as follows.

Consider: Concepts in this category are the concepts that will most likely satisfy the design goals; you want to prototype and test these ideas immediately. It may be the entire design that you want to develop, or only one or two specific elements of the design that you think are valuable for prototyping or testing.

Do not consider: Concepts in this category have little to no likelihood of satisfying the design goals and you find minimal value in these ideas. These designs will not be prototyped or tested in the later stages of design because there are no elements in these concepts that you would consider implementing in future designs.

These two categories were chosen to simulate the rapid filtering of ideas that occur in the concept selection process in industry (Rietzchel et al. 2006). The design teams were asked to discuss each concept with their team members and come to a team consensus on which concepts best addressed the design goal. During this discussion session, the teams were asked to physically sort the generated concepts into these two categories and rank the ideas in the “consider” category using post-it notes (1 being the best), as shown in Fig. 2.

Fig. 2
figure 2

Sorting of team generated concepts into the “Consider” category and “Do Not Consider” category by team 5

Full size image

3.3 Metrics

3.3.1 Creativity metrics

Once the study was complete, the generated designs were collected and two independent raters were recruited to assess the creativity of all ideas based on Shah et al.’s four creativity metrics: novelty, quality, variety, and quantity (Shah et al. 2003). Since the variety and quantity metrics are measures for groups of ideas, not individual ideas, only the novelty and quality metrics were used for the calculation of creativity in this study, as has been proposed in previous research (Oman et al. 2013; Sarkar and Chakrabarti 2014). However, unlike these previous studies that conceptualized creativity as an aggregate of novelty and quality, the approach used in the current study maintains a distinction between the novelty and quality metrics, treating them as two separate components of creativity. This was done in order to allow for the analysis of the novelty and quality components of creativity separately, since the conclusions that can be drawn from methods that increase the selection of novel ideas may be vastly different from the conclusions that can be drawn from methods that increase the quality of the selected ideas. Indeed, Shah et al. (2003) argue that “since each of them [creativity metrics] measures something different, we feel that adding them directly makes no sense. Even if we were to normalize them in order to add, it is difficult to understand the meaning of such a measure… We can also argue that a method is worth using if it helps us with any of the measures” (p. 133). Therefore, the two raters used a 24-question Design Rating Survey (DRS), to assess the novelty and quality of each design. This survey helped raters classify the features each design concept addressed, similar to the approach used in prior studies (Toh and Miller 2014a). The raters were undergraduate students in mechanical engineering who received extensive training on the design task and rating process. Specifically, the raters attended several training sessions where the rating questions were explained in detail to them, and practice ratings were conducted in order to ensure a satisfactory agreement between raters. The raters achieved a Cohen’s kappa (inter-rater reliability) of 0.88, and any disagreements were settled in a conference between the two raters. The results from these concept evaluations were used to calculate the following metrics.

Idea novelty, D j : This metric was developed to capture the amount of novelty in each of the generated ideas. Novelty is the “measure of how unusual or unexpected an idea is compared to other ideas” (Shah et al. 2003) and was calculated for each generated design using the feature tree approach developed by Shah et al. (2003). In order to accomplish this, the novelty of each feature was first calculated. This feature novelty is defined as the novelty of each feature, i, as it compares to all other features addressed by all the generated designs. The more frequently a feature is addressed, the lower the feature novelty score. Thus, feature novelty, f i , can then vary from 0 to 1, with 1 indicating that the feature is very novel compared with other features. The method of computing f i is shown in Eq. 1.

$$f_{i} = \frac{{T - C_{i} }}{T}$$
(1)

where T is the total number of designs generated by all participants and C is the total number of designs that addressed feature f i . The novelty of each design, j, is then determined by the combined effect of the feature novelty, f i , of all the features that the design addresses. Because D j is computed for all the features addressed by a design, the novelty per design, D j , is computed as an average of feature novelty, as shown in Eq. 2.

$$D_{j} = \frac{{\sum f_{i} }}{\sum i}$$
(2)

where f i is the feature novelty of a feature that was addressed in the design and ∑i is the number of features addressed by the design.

Task-related novelty: This metric was developed to capture the level of creativity present in each design team. In order to accomplish this, participant novelty metric was first calculated as the average design novelty of all the designs each participant generated (Shah et al. 2000, 2003), as shown in Eq. 3.

$${\text{Task}\text{-}\text{Related Novelty}} = \frac{{\sum D_{j} }}{N}$$
(3)

where N is the total number of ideas generated by the participant. Team novelty was then computed as the average of the design novelty scores for all concepts generated within each design team.

Propensity toward novel concept selection, P N : This metric was developed by the authors in previous studies to assess each team’s tendency toward selecting or filtering creative concepts during concept selection (Toh and Miller 2014b). In order to calculate this metric, first the average novelty of the concepts selected by the team during concept selection is computed. Next, the average novelty of all concepts available to choose from is computed. Lastly, the quantity from step 1 is divided by the quantity in step 2. This metric is shown in detail in Eq. 4.

$$P_{N} = \frac{{\sum\nolimits_{j = 1}^{k} {(D_{j} \times C_{j} )} }}{k} \times \frac{l}{{\sum\nolimits_{j = 1}^{l} {D_{j} } }}$$
(4)

where P N is the team’s propensity for selecting novel ideas during concept selection, k is the number of ideas selected by the team, l is the number of ideas in their set, and C j  = 1 if the idea is selected and 0 if the idea is not selected.

In essence, P N measures the proportion of novel idea selection out of the total novelty of the ideas that were developed by the design team. This metric has a value greater than 1 if the average novelty of the selected ideas is higher than the average novelty of all the generated ideas, indicating a propensity for novel concept selection. In contrast, P N can achieve a value less than 1, indicating an aversion for creative concept selection. A score of 1 indicates that the team chose a set of ideas that, on average, had the same level of novelty as the ideas that was generated, indicating no propensity toward novel concept selection.

Idea quality, Q j : Quality is defined as a measure of a concept’s feasibility and how well it meets the design specifications (Shah and Vargas-Hernandez 2003). Similar to Linsey et al. (2011), we measured quality on an anchored multi-point scale. However, we included an additional question to the quality scale in order to capture the improvement of the generated concept over the original design. The quality metric was calculated using the raters’ answers to the final four questions on the 24-question survey, as shown in Fig. 3.

Fig. 3
figure 3

Quality scores assessed using the four-point scale

Full size image

The design quality, Q j , of each design was then computed using Eq. 5, where q k is the answer to the kth quality question. q k  = 1 when the quality question is answered with a “yes”, and q k  = 0 when the quality question is answered with a “no”.

$$Q_{j} = \frac{{\sum\nolimits_{k = 1}^{3} {q_{k} } }}{3}$$
(5)

The quality score for each participant is then obtained by computing the average quality scores of all designs that the participant generated.

Propensity toward quality concept selection, P Q : This metric was developed by the authors to assess each team’s tendency toward selecting or filtering high-quality concepts during concept selection. In order to calculate this metric, first the average quality of the selected concepts is computed. Next, the average quality of all concepts available to choose from is computed. Lastly, the quantity from step 1 is divided by the quantity in step 2. This metric is shown in detail in Eq. 6.

$$P_{Q} = \frac{{\sum\nolimits_{j = 1}^{k} {(Q_{j} \times C_{j} )} }}{k} \times \frac{l}{{\sum\nolimits_{j = 1}^{l} {Q_{j} } }}$$
(6)

where P Q is the team’s propensity for selecting quality ideas during concept selection, k is the number of ideas selected by the team, l is the number of ideas in their set, and C j  = 1 if the idea is selected and 0 if the idea is not selected.

Task-related quality: This metric was developed to capture the level of creativity present in each design team. In order to accomplish this, participant quality metric was first calculated as the average design quality of all the designs each participant generated (Shah et al. 2000, 2003), as shown in Eq. 7.

$${\text{Task - Related Quality}} = \frac{{\sum Q_{j} }}{N}$$
(7)

where N is the total number of ideas generated by the participant. Team quality was then computed as the average of the design quality scores for all concepts generated within each design team.

3.3.2 Risk and ambiguity aversion metrics

In addition to measuring the creativity of the ideas generated and selected by each team, the teams’ risk attitudes were also measured. Since no measure exists that assesses risk-taking in the context of creative concept selection, and since risk behavior has been shown to vary greatly across situations and domains (Weber 2010; Weber et al. 2002), it was unclear whether, or how well, existing measures of risk could be used to measure risk-taking in a creative domain. Therefore, our work sought to understand the relationship between these exiting approaches for measuring risk-taking in a creative task by measuring participants’ risk attitudes according to two existing approaches: traditional behavioral economics measures of risk (risk aversion and ambiguity aversion) and psychometric domain-specific measures of risk (financial risk behavior, ethical risk behavior, and social risk behavior). While five domain-specific measures of risk were originally developed using this psychometric approach, the financial, ethical, and social domains of risk were used in this study due to their relevance to the social and risk–reward nature of team-based design tasks. On the other hand, the health/safety and recreational domains of risk were not used in this study since they do not capture relevant aspects of creative concept selection in a small team setting. Specifically, in order to calculate combined risk attitude scores for each team, the following methods were used:

Risk aversion: An individual’s risk aversion was measured using the 10 lottery questions (Cronbach’s α = 0.91) from the risk aversion online survey taken from research in standard behavioral economics (Boyle et al. 2011, 2012; Han et al. 2012). An example question is “Which would you prefer? $15 for sure, or a coin flip in which you get $ [an amount greater than $15] if it is heads, or $0 if it is tails?” Potential gamble gains vary randomly within the interval of $20.00–$300.00, where monetary increments were determined through a series of pilot tests with engineering students. The team’s combined risk aversion score was calculated as the mean of each team member’s risk aversion score, as is typically done when calculating aggregate attribute scores from individual attribute scores (Mohammed and Angell 2003; Reilly et al. 2001).

Ambiguity aversion: In addition to risk aversion, ambiguity aversion was also measured due to its significance in the study of decision making since many realistic situations involve both risk and ambiguity (Heath and Tversky 1991). It is important to investigate the role of ambiguity aversion in creative tasks since prior research conducted on ambiguity aversion has shown that an individual’s tolerance for ambiguity is linked to creativity in problem-solving tasks (Charness and Grieco 2013) and creative performance (Sternberg and Lubart 1991; Zenasni et al. 2008). Ambiguity aversion was measured using 10 lottery questions (Cronbach’s α = 0.85) from the ambiguity aversion online survey. The goal of the assessment was to identify the point at which an individual would take the gamble given unknown odds of winning the gamble (i.e., make the “uncertain” choice). An example question is “Which would you prefer? $15 for sure, or $20 if you win the gamble with unknown probability and $0 if you do not?” Ambiguity aversion was then calculated according to Borghans et al. (2009). Similar to risk aversion, the team’s combined ambiguity aversion score was calculated as the mean of each team member’s ambiguity aversion score.

Financial risk behavior score: In addition to participants’ financial risk aversion measured using lottery questions, participants’ financial risk behavior was measured from a psychometric perspective using eight survey questions (Cronbach’s α = 0.70) that assessed each participant’s self-reported likelihood of participating in behaviors that are risky in a financial context on five-point verbally anchored Likert scale (Weber et al. 2002) through the online survey, as shown in Fig. 4. While new seven-point scales have been developed for Weber’s psychometric assessment, the use of the five-point scale strikes a balance between validity and increases in variability that may arise from a larger number of points on a Likert scale (Friedman and Amoo 1999).

Fig. 4
figure 4

Example financial risk behavior question from Weber et al. (2002)

Full size image

Ethical risk behavior score: Ethical risk behavior was measured using eight survey questions (Cronbach’s α = 0.73) that assessed each participant’s self-reported likelihood of participating in ethically risky behaviors on five-point verbally anchored Likert scale (Weber et al. 2002) through the same online survey (e.g., forging someone’s signature).

Social risk behavior score: Social risk behavior was measured using eight survey questions (Cronbach’s α = 0.54) that assessed each participant’s self-reported likelihood of participating in risky social behaviors on five-point verbally anchored Likert scale (Weber et al. 2002) through the online survey (e.g., speaking your mind about an unpopular issue at a social occasion).

3.3.3 Personality trait metrics

Finally, personality scores were measured using the short five-factor model (FFM) online questionnaire [Short Form for the IPIP-NEO (International Personality Item Pool Representation of the NEO PI-R™) (Johnson 2014)]. The combined personality trait scores of each team were calculated as follows:

Team personality levels: In order to calculate the combined personality trait scores of each design team, the personality traits of each participant were used. Each participant received a score (ranging from 0 to 100) on every one of the five personality traits: extraversion, agreeableness, conscientiousness, neuroticism, and openness. The team’s combined score on each personality trait was then calculated as the average of all the team members’ individual scores, as is typical of team personality research (Mohammed and Angell 2003; Reilly et al. 2001).

4 Results and discussion

During the study, 22 ideas (SD = 6.4) were generated, on average, by each team, and eight ideas (SD = 3.02) were selected, on average, for further development. Examples of ideas that were categorized in the “consider” and the “do not consider” categories are given in Table 1.

Table 1 Examples of ideas in the “consider” and “do not consider” categories
Full size table

Before testing our research questions, a post hoc power analysis was conducted using the software package, GPower (Faul et al. 2007). Three predictor variables and a sample size of 11 were used for the statistical power analyses. For moderate to large effect sizes of R 2 = 0.70, the statistical power for this study was calculated as 0.902. Therefore, it can be concluded that there was adequate power to detect moderate or large effect sizes. Since this study has the primary goal of exploring any possible effects that behavioral economics measures of risk, psychometric measures of risk, and personality have on creative concept selection, no interaction effects were explored in the analysis.

In addition, it was also important to conduct some preliminary analysis of our P N and P Q ratio variables in order to identify their appropriateness for analysis. Specifically, in order to insure a linear relationship between the novelty/quality of the generated ideas and the novelty/quality of the selected ideas, two linear regression analyses were conducted. The results revealed that there was in fact a significant positive relationship between the novelty (R 2 = 0.53, R 2adjusted  = 0.47, p < 0.01) and quality variables (R 2 = 0.58, R 2adjusted  = 0.54, p < 0.01). Since these relationships were found to be linear, the P N and P Q ratio variables were found to be appropriate for use in the remainder of our statistical analysis.

In addition, to determine the impact of any confounding variables since prior work has demonstrated differences between education levels and creativity in engineering design (Genco et al. 2012), two ANOVAs were conducted, both using education level as the independent variable. The first ANOVA used team’s propensity for novel concept selection P N as the dependent variable, and the second ANOVA used team propensity for quality concept selection P Q as the dependent variable. The results revealed no significant relationship between education level and P N , F = 2.10, p > 0.18, and between education level and P Q , F = 0.51, p > 0.49, indicating that education level did not impact the teams’ propensity for selecting novel or quality concepts. Therefore, the data from both classes are analyzed for our analysis. SPSS version 20 was used to analyze the findings. A significance level of 0.05 was used in all analyses, and ordinary least squares methods were used for all regression analyses. The following sections present the detailed results of our analyses in the order of our research hypotheses.

4.1 Hypothesis 1: Creative ideas do not have a higher likelihood of being selected during concept selection

Our first research hypothesis sought to determine whether idea creativity, conceptualized as a combination of novelty and quality, would affect the likelihood of an idea being selected by team members during group concept selection activities. Since the dependent variable of this analysis is discrete (selected or not selected), a multiple logistic regression analysis was conducted on all the generated ideas, with the dependent variable being whether the idea was selected by the team or not. In addition, since creativity is operationalized as the combination of design novelty and quality, the independent variables used in this analysis were idea novelty and idea quality. The results of this analysis revealed that idea novelty and quality did not significantly affect the likelihood of the idea being selected during concept selection, χ 2(2) = 3.72, p > 0.16. This result indicates that idea creativity did not significantly affect the selection of ideas during the team concept selection activity. This finding suggests that even if a highly creative design is generated during the early phases of design, it may not be selected during the concept selection process. This result demonstrates that design teams do not show any preference for creative ideas during the selection process, even though creativity is touted as an important element of the design process (Howard et al. 2008). Since feasibility is an important element of creativity in this study, this result is contrary to prior work that has found that individuals tend to select ideas based on feasibility, rather than originality (Ford and Gioia 2000; Rietzschel et al. 2010). However, unlike previous studies, the selection activity was conducted in design teams and involved typical engineering design problems. Nevertheless, the results of the study show that individuals did not show a preference for creative ideas even though creativity is regarded as an important element of successful engineering design. That is, despite the fact that design educators and practitioners recognize the importance of creativity in design, the mere awareness of its importance does not guarantee creative idea generation and selection. Therefore, more focused and directed efforts aimed at highlighting the importance of creativity and encouraging creative activities are needed to increase awareness of creativity throughout the design process.

4.2 Hypothesis 2: Creative idea generation ability is related to the teams’ propensity for creative concept selection

Our second research hypothesis sought to determine the effect of team task-related creativity on team propensity for selecting creative ideas during concept selection. In order to address this, a multivariate linear regression analysis was conducted using team propensity for novel concept selection P N and team propensity for quality concept selection P Q as dependent variables, while team task-related novelty and quality scores were used as independent variables. The multivariate regression analysis revealed no significant relationship between the dependent variables and task-related novelty (Wilk’s λ = 0.86, F = 0.57, p > 0.59), and task-related quality (Wilk’s λ = 0.84, F = 0.65, p > 0.55). These results indicate that task-related creativity is not predictive of the teams’ propensity for selecting creative ideas. In other words, a team’s ability to generate creative ideas has no significant impact on their ability to identify and select creative concepts during the later stages of the design process.

This finding suggests that even if a design team generates highly creative ideas, they may not necessarily select these creative ideas during the concept selection process. However, this result is promising because it demonstrates that even if a team does not generate a high number of creative ideas, it does not mean they cannot identify and select the most creative concepts out of their set, and thus contribute significantly to the overall creativity of the design process. Thus, students and practicing engineers who are expected to be creative during the design process should focus on creativity during concept generation and selection in order to truly innovate and break convention. While adoption rates of formalized methods in engineering practice remain relatively low (Birkhofer et al. 2005), the development and study of new methods and techniques for encouraging creativity during the selection phase is essential for increasing design creativity, since prior research has shown that many existing selection techniques discourage the selection of innovative ideas (Dong et al. 2015). Research efforts aimed at developing and studying these new creativity methods can also help add to our knowledge of the concept selection process in design.

4.3 Hypothesis 3: Teams who are more risk prone will select more creative ideas during concept selection

Our third research hypothesis sought to determine the effects of team risk attitudes on team propensity for selecting creative concepts. To address this research hypothesis, traditional behavioral economics measures of risk (risk aversion and ambiguity aversion) and psychometric domain-specific measures of risk (financial risk, ethical risk, and social risk) were investigated for their effects on the teams’ propensity for creative concept selection. First, a multivariate linear regression was conducted with the independent variables being team risk aversion and ambiguity aversion and the dependent variables being team propensity for novel concept selection P N and team propensity for quality concept selection PQ scores. This analysis revealed that risk aversion (Wilk’s λ = 0.98, F = 0.08, p > 0.93) and ambiguity aversion (Wilk’s λ = 0.49, F = 3.71, p > 0.08) could not predict the combination of team P N and P Q scores; for summary, see Table 2. However, team ambiguity aversion scores have a statistically significant effect on P N scores (B = −0.12, p < 0.05). This result indicates that teams with a higher tolerance for ambiguity (lower scores on ambiguity aversion) tended to select more novel concepts, as shown in Fig. 5.

Table 2 Summary of multivariate linear regression analyses between team P N and P Q scores and risk measures
Full size table
Fig. 5
figure 5

Significant negative relationship between team propensity for novel concept selection, P N , and average team ambiguity aversion scores

Full size image

A second multivariate linear regression was conducted with the independent variables being team financial, social, and ethical risk behavior scores and the dependent variables being team propensity for novel concept selection P N and team propensity for quality concept selection P Q scores. This analysis revealed that financial risk behavior (Wilk’s λ = 0.91, F = 0.29, p > 0.76), ethical risk behavior (Wilk’s λ = 0.52, F = 2.77, p > 0.14), and social risk behavior (Wilk’s λ = 0.79, F = 0.82, p > 0.49) could not predict team P N and P Q scores; for summary, see Table 2.

These results highlight the important role that risk attitudes can play in a design team setting and show that teams with an overall higher level of tolerance for ambiguity (lower ambiguity aversion scores) are more likely to select novel concepts. This result is supported by prior research on team creativity that showed that new and original ideas tend to be viewed with skepticism in team settings, likely discouraging the selection of these ideas (Baer et al. 2007). However, teams that are more comfortable with making decisions under uncertainty and that are more willing to select ideas have unknown parameters are more likely to engage in the creative process, negating the general bias against creativity in team settings (Bradshaw et al. 1999; Camacho and Paulus 1995). The fact that no significant relationships were found between risk aversion, financial risk behavior, ethical risk behavior, social risk behavior, and team propensity for novel and quality concept selection in this study suggests that perceptions and attitudes toward ambiguity in design dominate in team concept selection tasks, outweighing team attitudes toward other domains of risk. In addition, the results of our study show that tolerance for ambiguity only plays a role on propensity for selecting creative ideas in the novelty dimension, and not in the quality dimension, suggesting that participants’ perception and preference for novelty may be more affected by team risk attitude factors compared with quality. Nevertheless, since novelty is often considered an essential criteria for innovation and invention (Slaughter 1998) and is one of the components of creativity (Shah et al. 2003), it is important to study the factors that may affect design teams’ preferences for novel ideas during concept selection.

4.4 Hypothesis 4: Student personality traits will predict the teams’ propensity for creative concept selection

Our fourth and final research hypothesis sought to investigate the impact of team personality traits on the teams’ propensity for selecting novel concepts, P N , and propensity for selecting quality concepts, P Q . In order to understand this relationship, a multiple linear regression analysis was conducted with the dependent variables being team P N and P Q scores and the independent variables being team personality trait scores on all five traits. The multiple linear regression analysis results revealed that team personality traits do not significantly predict the combination of both P N and P Q scores; for summary, see Table 3. However, P N scores alone could be significantly predicted by team personality traits (R 2 = 0.88, R 2adjusted  = 0.77, p < 0.02). Specifically, higher levels of team agreeableness (B = 0.001, p < 0.03) and conscientiousness (B = 0.002, p < 0.04) were found to relate to a higher propensity for novel concept selection in teams, as shown in Fig. 6.

Table 3 Summary of multivariate linear regression analyses between team PN and PQ scores and personality traits
Full size table
Fig. 6
figure 6

Relationship between team propensity for novel concept selection and team agreeableness levels (left) and team conscientiousness levels (right)

Full size image

These results show that personality traits are linked to team novelty during concept selection, supporting prior research that has shown that personality is related to creative idea generation potential (Stafford et al. 2010). However, the results of our study show personality traits only relate to a teams propensity for selecting novel ideas, not their propensity for selecting high-quality ideas. This result suggests that personality traits may play a larger role in affecting participants’ perception of novelty compared with quality. Specifically, our study found that agreeableness and conscientiousness personality traits are positively related to novel concept selection supporting by prior research that shows that teams with high conscientiousness and agreeableness levels are more motivated to achieve goals (Bell 2007) and, thus, tend to be more creative (Woodman et al. 1993). Interestingly, results from other studies that explore these personality traits at the individual level show that agreeableness personality trait is negatively related to creativity (Feist 1998), indicating that team-level personality traits may differ from individual-level personality traits at a fundamental level. In fact, researchers have acknowledged that individual attributes interact in complex and dynamic ways in teams, resulting in team outcomes that are simply more than an aggregation of team member attributes (McGrath 1998).

Our result suggests that team-based perceptions and preferences for novel ideas are ultimately a function of the composition and heterogeneity of the design team; teams who are composed of many individuals with highly creative potential may not necessarily select the most creative ideas and vice versa. In addition, the results of this study show that the composition of individual attributes in small design teams can affect the selection of novel ideas in a relatively simple design task, in an engineering education context. Thus, educational strategies that leverage the diverse distribution of individual attributes such as risk attitudes and personality traits should be implemented in order to encourage novel concept selection. In addition, more research efforts are needed to help identify design team configurations that encourage the most creativity throughout the design process.

5 Implications for engineering design research and education

The results of this study bear significant implications for research in engineering design and the instruction of design methods in engineering education. First, this study provides a better understanding of how concepts are initially screened during the design process, showing that highly creative teams do not necessarily select creative concepts. Our study also identifies that teaching or encouraging creative concept generation is not sufficient for ensuring the selection of these creative concepts during the later stages of the design process. Therefore, traditional methods of concept selection, such as those they rely on the expected utility framework for selecting concepts, do not take creativity into account and are insufficient for encouraging creativity during the concept selection stage of the design process. This is due to the fact that most concept selection methods do not include creativity as an important aspect of the design while assessing ideas during concept selection. Thus, research is needed to develop and study methods and techniques for encouraging creativity that go beyond the mere expected utility of an idea during concept selection in order to increase overall creativity in the design process.

Another important finding of this study is that personality traits and risk attitudes are linked to novel concept selection in design. The results of this study provide empirical evidence that team-level personality attributes such as agreeableness and conscientiousness affect a design team’s perceptions and preference for the novelty dimension of creativity. While there exists a wealth of prior research that has shown that these personality traits can greatly affect individual creativity (Batey and Furnham 2006; Furnham and Yazdanpanahi 1995), the effects of these personality traits on team creativity are much less studied (Mumford 2012). Some studies have shown that team-level personality traits can influence creative idea generation in teams (Baer et al. 2007; Bell 2007; Woodman et al. 1993), but few studies have explored team-level personality traits in the context of creative concept selection.

The results of this study also found contradictory results on the role of team personality and creativity; Baer et al.’s (2007) study found that high levels of extraversion and openness and low levels of conscientiousness in teams resulted in the generation of highly creative ideas, while our study found that high levels of agreeableness and conscientiousness resulted in the selection of more novel ideas. This is supported by prior research that states that the types of cognitive and social factors that influence these two stages of design are fundamentally different and involve different sets of mental processes (Reiter-Palmon 2009). Thus, the formation of teams that have diverse personality traits can help ensure that creativity is encouraged throughout the design process. This notion of beneficial diversity is not novel, as it has been argued by researchers to be crucial in building teams that have highly creative performance (Klein et al. 2006). However, this study highlights the need of this diversity during the concept selection process. Therefore, efforts to build the “perfect” team composed of individuals with personality traits highly associated with creativity can be seen as a practice in futility since different types of personality traits may be linked with creativity at different stages of the design process.

Finally, one of the main goals of this research was to draw a link between team-level risk attitudes and propensities for teams to select creative ideas. The results of this study show that social risk attitudes play an important role in the selection of novel ideas in teams, agreeing with prior research that has shown that creativity is heavily influenced by social factors in a team setting (Woodman et al. 1993). In this study, new and original ideas were likely viewed with skepticism in the team, likely discouraging the selection of these ideas. However, teams that are more comfortable with making decisions in ambiguous situations and that are more willing to select ideas have unknown parameters are more likely to engage in the creative process, negating the general bias against creativity in team settings (Bradshaw et al. 1999; Camacho and Paulus 1995). Thus, perceptions and attitudes toward ambiguity appear to dominate in team concept selection tasks, outweighing team attitudes toward other types of risk. The development and adoption of environments and practices that encourage student designers to embrace ambiguity and take risks can allow students to openly and feely discuss ideas can help increase team creativity (Edmonson and Roloff 2009).

While the results from the current study identify important links between propensity for creative concepts, risk-taking and personality traits in teams, future work is needed to understand the underlying factors of creative concept selection by investigating the role of individual attributes in the perception and preference for creative ideas in team settings. In addition, engineering design educators should focus on forming functionally diverse teams in order to encourage a well-rounded focus on creativity throughout the design process. Lastly, student designers should be exposed to environments and practices that encourage social risk-taking and open idea sharing in an effort to educate the next generation of design engineers that are creative and effective in teams.

6 Future work and limitations

While this study establishes a link between personality traits, social risk attitudes, and novel concept selection, several important limitations should be noted. Most important is that this study was developed primarily to explore engineering students’ concept selection process in teams in situ, through the lens of creativity. Future work should focus on studying design teams in industry to compare the results found in this study with design practice. Similarly, larger sample sizes may reveal a link between creative concept selection and risk attitudes, such as interaction effects between factors, where one was not found in this study. Future work that explores the impact of personality and risk attitude compositions in teams (overall level and spread of traits) using controlled laboratory studies where teams with specific compositions of factors are assigned can also help add to our understanding of how these factors impact creative concept selection. More research is also needed to develop and study risk measures that are appropriate for use in creative contexts, since existing measures of risk may not fully capture the risk-taking behaviors of designers during creative concept selection (low reliability scores for scales). In addition, while this study provides knowledge of how student designers select concepts for a design project where students were specifically asked to be “innovative,” future studies should explore how the concept selection process is impacted in tasks that require varying degrees of innovation (e.g., tasks that require working rather than truly novel solutions). Similarly, more studies are needed to examine the impact of explicit instructions to select ideas that are both novel and useful on designer behavior during concept selection, and to understand whether designers are selecting ideas that are more feasible in favor of ideas with higher novelty. Other areas of further investigation include examining the use of voting methods or prototyping activities during concept selection that may lead to a narrower scope of selected ideas and may impact creative concept selection in a different manner. Finally, the framing of the concept selection task could also lead to different results. For example, the impact of risk attitudes on creative concept selection may vary if designers are asked to choose their best concept, instead of a collection of their preferred ideas. Therefore, future work is needed to explore these interesting and challenging problems.

7 Conclusions

The current study was developed to understand the relationship between creative idea generation ability, personality traits, risk attitudes, and creative concept selection in student design teams. Our results highlight the fact that teams that generate highly creative ideas do not necessarily select creative concepts. It was also found that team personality traits and social risk attitudes relate closely to novel concept selection. However, financial risk and ambiguity aversion were not linked to creative concept selection, indicating that social risk perceptions dominate team-based concept selection activities. Our results serve as an empirical basis for further research on creative concept selection and are used to provide recommendations for design instruction in engineering education.