Abstract
While creativity is often seen as an indispensable quality of engineering design, individuals often select conventional or previously successful options during the concept selection process due to the inherent risk associated with creative concepts and their inadvertent bias against creativity. However, little is actually known about what factors attribute to the promotion or filtering of these creative concepts during concept selection. To address this knowledge gap, an exploratory study was conducted with 38 undergraduate engineering students. This study was aimed at investigating the impact of individual risk aversion, ambiguity aversion, and student educational level on the selection and filtering of creative ideas during the concept selection process. The results from this study indicate that individuals’ ability to generate creative ideas is not significantly related to their preference for creative ideas during concept selection, but individual risk aversion and ambiguity aversion are significantly related to both creative concept selection and creative idea generation. Our results also revealed that first- and third-year students’ creative ability is affected differently by varying levels of tolerance for ambiguity. These results highlight the need for a more directed focus on creativity in engineering education in both concept creation and concept selection. These results also add to our understanding of creativity during concept selection and provide guidelines for enhancing the design process.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Sir Ove Arup said it best when he stated that, “Engineering problems are under-defined, there are many solutions, good, bad and indifferent. The art is to arrive at a good solution. This is a creative activity, involving imagination, intuition, and deliberate choice” (para. 6) (Arup 2014). This is especially true in the field of engineering design where boundary-breaking advancements and innovations are heavily emphasized. As such, research in engineering design has focused on developing methods to enhance creative idea development during the early phases of conceptual design through the development and study of ideation tools [see, e.g., (Altshuller 1984; Eberle 1996; Kulkarni et al. 2012; Osborn 1957)]. Researchers in the field of creativity (Baer et al. 2007; Daly et al. 2014) widely accept the definition of creativity as the “production of novel, useful products” (Mumford 2003), or ideas that are both original and feasible. While the goal of these formal idea generation techniques is to help designers generate a large quantity of effective solutions and expand the explored solution space (Shah et al. 2003), the creative ideas developed through these methods are often rapidly filtered out during the concept selection process (Rietzchel et al. 2006) with few making it to commercialization. Since the concept selection process is primarily used to identify concepts that should move forward in the design process from a pool of candidate concepts (Kijkuit and van der Ende 2007), it can be seen as the “gate keeper” of creative ideas. Thus, it is important that research in engineering design shifts its focus to identifying factors that lead to the promotion of creative ideas through the concept selection process in order to increase the likelihood of innovation, which is crucial for long-term economic success (Ayag and Ozdemir 2009).
Concept selection is considered one of the most crucial components of the design process because the direction of the final design is largely determined at this stage (Hambali et al. 2009; King and Sivaloganathan 1999). This process helps designers narrow down the solution space (King and Sivaloganathan 1999) and select the most promising ideas for satisfying the design goal. Research has shown that highly creative ideas, often defined as ideas that are both novel and useful (Mumford 2003), contribute the most value to the design process (Fuge et al. 2013), but simply generating creative ideas does not guarantee the creativity of the final design (Nijstad and De Dreu 2002; West 2002). Indeed, researchers have argued that the “availability of creative ideas is a necessary but insufficient condition for innovation” (p. 48) (Rietzchel et al. 2006), leading to the conclusion that creative ideas must be identified and selected through the concept selection process. However, creative ideas are often filtered out during this process because of people’s inadvertent bias against creativity. This bias against creativity is typically attributed to the uncertainty and risk associated with novel concepts (Rietzschel et al. 2010). This is particularly problematic in engineering education since research conducted in a variety of disciplines in engineering education, such mechanical, aerospace, chemical, and civil engineering, has shown that creativity is heavily emphasized and integrated in the engineering classroom (Litzinger et al. 2011; Richards 1998; Stouffer et al. 2004; Sullivan et al. 2001), but little data exist on how creativity is integrated into the concept selection stage of design projects. In fact, researchers have shown that the engineering curriculum discourages student creativity (Charyton and Merrill 2009) and upper-class students tend to be less creative than first-year students (Genco et al. 2012). In other words, while creativity may be touted in engineering education and the early stages of the design process, individuals may be unable or unwilling to adopt creative ideas when evaluating concepts. For example, a small number of studies have shown that individuals often perform poorly at selecting creative concepts even when factors such as concept feasibility or productivity of the ideation activity are considered (Faure 2004; Putman and Paulus 2009; Rietzchel et al. 2006; Rietzschel et al. 2010). Other studies have shown that individuals prefer conventional solutions due to the uncertainty behind investing in and endorsing novel ideas (Moscovici 1976; Rubenson and Runco 1995; Whitson and Galinksy 2008). These studies provide a foundation for understanding perceptions and preferences for creativity, but leave a gap in the knowledge base regarding the factors that contribute to the selection of creative ideas during concept selection.
The goal of this paper is to provide the results of a study aimed at identifying how individual risk aversion and ambiguity aversion affect an individual’s openness to creative ideas during the concept selection process by studying 38 undergraduate engineering design students. In addition, this study is exploratory in nature in that it seeks to identify and investigate factors that have the potential to affect creative concept selection since research in this space is not yet clearly defined and there is limited prior research on the exact factors that can influence preferences for creativity in engineering design. The results from this study add to our understanding of what factors influence the selection of creative ideas during the concept selection process and outlines new research opportunities in this area.
1.1 Concept selection in engineering design
Concept selection is considered one of the most crucial components of the design process because the direction of the final design is largely determined at this stage (Hambali et al. 2009; King and Sivaloganathan 1999). Engineering designers and engineering companies who select high-quality and highly innovative concepts during this process increase their likelihood of product success and radical innovation, while those who select concepts that do not address the design goal eventually have larger expenses including redesign costs and production postponement (Huang et al. 2013). This neglect of creative ideas can greatly damage companies that are trying to survive in the fast-growing market that demands product innovations (Ayag, and Ozdemir 2009). Therefore, it is crucial that research efforts be geared at understanding what factors influence the filtering or promotion of creative ideas through the concept selection process (Rietzchel et al. 2006).
The first stage of the concept selection process occurs directly after concept generation when the design team is tasked with quickly screening hundreds of concepts and selecting the ideas with most promise to move forward in the design process (Kudrowitz and Wallace 2013). Concepts that were generated in previous stages need to be selected and synthesized into a final solution in order to address the design goal (Ulrich et al. 2011). Thus, initial concepts are evaluated for their strengths and weaknesses and for their ability to fulfill customer needs. Since the selection of concepts can be seen as a highly subjective process, various formalized methods have been developed by researchers in engineering design that aim to systematically select the most optimum concepts from a pool of candidate concepts. Examples of widely used concept selection methods include utility theory (Pahl and Beitz 1984), the analytical hierarchy process (AHP) (Marsh et al. 1993), Pugh’s evaluation method (Pugh 1991), the quality function deployment (QFD) matrix method (Ter Harr et al. 1993), and the fuzzy set method (Thurston and Carnahan 1992). Designers typically consider the design robustness, novelty, production cost, and effectiveness of solutions during this process (Busby 2001). However, while these concept selection techniques provide a means of comparing ideas based on their ability to meet design specifications (King and Sivaloganathan 1999), they often neglect to consider the creativity or uniqueness of each concept as important selection criteria. Thus, it is not clear how creative ideas are selected during concept selection or what factors affect the promotion or filtering of these creative concepts during the design process.
One factor that can affect the selection of creative concepts in engineering design is people’s bias against creativity. Specifically, while creativity is often set as an important goal, researchers have found that individuals in scientific institutions, organizations, and industry often select conventional ideas over creative ones (Ford and Gioia 2000; Staw 1995). This preference for conventional design alternatives is often done in an unconscious manner (Dijksterhuis 2004), and numerous research studies have found that people tend to have an inadvertent bias against creativity (Bower 1981; Mueller et al. 2011; Rietzschel et al. 2010). This is said to occur because while practical ideas are generally considered valuable, individuals tend to be more uncertain about whether a novel idea is practical, error free, or useful (Amabile 1996). Indeed, as individuals experience more uncertainty in a situation, their perceptions of creativity quickly become negative (Bower 1981), since individuals are strongly motivated to avoid uncertainty and failure (Whitson and Galinksy 2008). On a similar note, individuals perceive more risk associated with endorsing novel ideas (Rubenson and Runco 1995) because of the uncertainty regarding the success and social approval of their decisions (Moscovici 1976). This bias against creativity plays an important role in engineering education and is one of the multitudes of factors that can affect creative concept selection in the classroom. Researchers have found that students tend to be less creative and innovative when there is a risk of receiving poor grades (Linnerud and Mocko 2013). This is despite the fact that researchers and educators have long since acknowledged the importance of fostering creative thinking abilities and methods in addition to teaching key engineering concepts (Felder and Brent 2004). Indeed, researchers have shown that educators tend to dislike students who exhibit creative behavior, even though creativity is touted as an important element of learning (Westby and Dawson 1995). In engineering education, researchers have found that students do not feel encouraged by their instructors to be creative or open-minded, and often do not search for multiple solutions to a design problem for fear of failing or receiving poor grades (Kazerounian and Foley 2007).
Other confounding factors that can influence the selection of ideas and perceptions of creativity during this stage include ownership bias (Nikander et al. 2014), the bias to select visually complex concepts (Onarheim and Christensen 2012), confirmation bias (Hammond et al. 1998), satisficing (Ball et al. 2001), and task familiarity (Forster 2009). In addition, differences in expertise and learning experience in students of different educational levels may also influence students’ creative ability and perceptions of creativity since creativity is typically heavily emphasized and integrated in the engineering classroom (Litzinger et al. 2011; Richards 1998; Stouffer et al. 2004). Importantly, individual attributes such as personality traits can play a role in affecting the selection of creative ideas since the composition of team member personality and disposition is one of the most important factors in determining team performance (Wilde 1997) and creativity (Somech and Drach-Zahavy 2011). Recent research conducted in engineering design has begun to explore the impact of these factors on creative concept selection and show that team-level personality traits influence creative idea selection tendencies (XXX). These results add to our understanding of the factors that influence creativity in the design process, but do not provide information regarding individual-level attributes and their impact on decision-making. This is important since researchers have shown that team-level attributes are actually complex combinations of individual-level attributes (McGrath 1998), and the impact of team-level attributes on team performance or creativity can be affected by factors such as compatibility of individual attributes (Moos and Speisman 1962), diversity of attributes (Belbin 1981; Hoffman and Maier 1961), or creative confidence (Baer et al. 2007). This compounding effect of individual-level attributes in the team setting makes it challenging to draw conclusions regarding innate individual biases or the impact of individual attributes on perceptions of creativity from work conducted on team-level attributes.
Other factors such as an individual’s attitudes toward risk and ambiguity have also been shown by prior research to affect an individual’s perception of creativity (Rubenson and Runco 1995) and their creative abilities (Dewett 2007; El-Murad and West 2003). Both risk and ambiguity are important to study in design since many situations in practice involve a degree of uncertainty (Antonsson and Otto 1995; Bucciarelli 1988; Sarbacker and Ishii 1997; Weck et al. 2007), requiring the decision-maker to take risks during decision-making. Uncertainty refers to “both the probability that certain assumptions made during design are incorrect as well as the presence of entirely unknown facts that might have a bearing on the future state of a product or system and its success in the marketplace” (p. 1) (Weck et al. 2007). By extension, risk can be used to describe the extent to which there is uncertainty in outcomes given creative effort (Sitkin and Pablo 1992), where the decision-maker is required to make decisions with less than perfect information (Sarbacker and Ishii 1997). Research on individual attitudes toward risk is important to explore since risk-taking is stated to be an essential element of creativity due to its ability to encourage the individual to push boundaries and explore new territories (Kleiman 2008). While risk refers to situations where outcomes have a fixed probability of occurring, ambiguity refers to situations where outcomes have an unknown probability of occurring (Moore and Eckel 2003), created by missing information that is relevant and could be known (Fellner 1961; Frisch and Baron 1988). Research on ambiguity aversion during design decision-making is important since many realistic situations involve both risk and ambiguity (Heath and Tversky 1991), and recent studies have shown that an individual’s tolerance for ambiguity is linked to creativity in problem-solving tasks (Charness and Grieco 2013). Although both risk and ambiguity are important elements of design decision-making, prior measures of individual risk and ambiguity attitudes [e.g., domain specific risk attitudes (Bossuyt et al. 2013; Weber et al. 2002), and preference of ambiguity to risk (Charness and Grieco 2013)] were not developed for use in the context of creative concept selection. Thus, their relationship with risk-taking in a creative task is largely unknown. In addition, the use of more traditional and familiar risk measures such as utility theory (Boyle et al. 2011; Han et al. 2012) or prospect theory (Kahneman and Tversky 1979) that utilize financial lotteries has not been tested for their relationship to risk-taking in creative tasks. Therefore, work is needed that explores the relationship between traditional measures of personal financial risk attitudes on risk-taking in a creative context in order to bridge the gap between risk attitudes in these different domains.
While it is clear that both risk and ambiguity aversion are important factors that impact creativity, little research has been conducted regarding the possible effects that these factors may have on creative concept selection. The conflicting role of creativity in the concept generation and selection phases suggest that more research is needed to explore the factors that lead to the decreased role of creativity in the later phases of design. Prior research conducted in this area has shown that team-level ambiguity aversion scores impact creative idea selection tendencies (Toh and Miller 2016), but do not focus on investigating individual-level attributes and their impact on decision-making. Therefore, this research seeks to fill these knowledge gaps by exploring the role that individual risk attitudes and student educational level play in the concept selection process.
1.2 Research objectives
The goal of this paper is to explore the impact of individual risk attitudes on a creative task. Specifically, this study seeks to address the following research hypotheses:
Hypothesis 1
Individuals who are more risk prone will generate ideas with more creativity, and this relationship will be impacted by student education level.
Hypothesis 2
Individuals who are more risk prone will select more creative concepts, and this relationship will be impacted by student education level.
Hypothesis 3
Individuals who generate more creative ideas will select more creative concepts, and student education level will affect this relationship.
These hypotheses are built on our prior research that found that individual-level risk attitudes can affect creative concept selection and generation in design (Toh and Miller 2014b). The current research hypotheses are summarized in a research framework diagram depicting the variables of interest as well as potential confounds (see Fig. 1).
2 Methodology
To address these research hypotheses, an exploratory study was conducted with undergraduate engineering students. Because little data exists on which factors affect creative idea selection, the current study was developed to provide preliminary evidence of the individual factors that impact creative concept selection. During this study, participants were asked to complete an idea generation task and a concept selection activity. The details of this study are provided in the following sections.
2.1 Participants
Thirty-eight undergraduate engineering students participated in this study. Nineteen of the participants were recruited from a first-year introduction to engineering design course (9 males, 10 females), while the remaining 19 participants were recruited from a third-year mechanical engineering design methodology course (17 males, 2 females). The first-year introduction to engineering design course was a required course for all engineering students and introduced basic methodologies used throughout the design process to students and consisted of multiple small design projects. The third-year design methodology course focused on more detailed design methodology techniques and consisted of a single, more involved design project. Participants in each course were in 3- and 4-member design teams (five 4-member teams and six 3-member teams) that were assigned by the instructors at the start of the course in order to balance the a priori advantage of the teams through questionnaires given at the start of the semester that asked about student proficiencies in 2D and 3D modeling, sketching, and the engineering design process. Thus, design teams were formed in such a manner that the teams were comparable in their baseline design skills at the start of the course.
2.2 Procedure
The study was completed in two phases (see Fig. 2).
One week prior to the start of the study, participants were given a brief introduction to the purpose and procedure of the study and were asked to complete an informed consent document. Once the IRB form was completed, participants were directed to an online survey that assessed individual risk aversion and ambiguity aversion using a set of 20 lottery questions (10 each for risk and ambiguity aversion) (see the metrics section of this paper for a description of the questions and “Appendix 3” for a full list of the questions). Each survey was coded according to measures used in standard behavioral economics (Boyle et al. 2012; Han et al. 2012) in order to capture each individual’s level of risk aversion and ambiguity aversion. Participants were assigned unique participant identification code for use in the online survey and subsequent design tasks in order to maintain participant anonymity. This survey was given to the participants 1 week before the next phase of the study in order to give participants enough time to complete the survey before the design task. Since risk attitudes have been considered to be individual attributes that are relatively stable and constant (Goldsenson 1984; Wolman 1989), we did not anticipate any plausible bias as a result of this assessment prior to the start of the study.
One week after the survey was completed, the participants attended a design session in a typical engineering classroom. At the start of the session, the researcher introduced the outline of the day’s activities using a script and participants were asked to develop a device to froth milk. The design problem was given to students on sheets of paper, and participants were given an opportunity to ask questions about the design task (see “Appendix 1” for instruction sheets). The researchers answered questions regarding frothed milk, but provided no design suggestions or additional information to the participants. No prior task was assigned before the design session for both the first-year and third-year classes. One of the most elusive challenges in design research is selecting a task that is both representative of the design area and appropriate for the research questions being explored (Kremer et al. 2011). The design task chosen in the current study was selected to represent a typical project in a cornerstone, or first year, engineering design course. In these courses, students are typically directed to redesign small, electromechanical consumer products that are equally familiar, or unfamiliar, to the student designers (Simpson and Thevenot 2007; Simpson et al. 2007). This type of task is often selected because of the minimal engineering knowledge students have in these early courses. In order to ensure that our participants were equally familiar with the product being explored, our design task went through pilot testing with first-year students prior to deployment and has been used previously in other studies investigating creativity in design (Toh and Miller 2013a, b, 2014). Specifically, the design task provided to participants in this study was:
Your task is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction. Focus on developing ideas relating to both the form and function of the product.
Participants were given individual sheets of papers and given 20 min to individually sketch as many concepts as possible for a novel milk frother. No discussion was allowed during this individual brainstorming session. While a recent study on idea generation found that the most creative ideas emerge only after about 9 ideas have been generated (Kurdrowitz and Dippo 2013), participants in that study were only provided with 3 min to generate ideas for a significantly less complex problem involving short phrases as opposed to design sketches. In addition, related research done in cognitive psychology has shown that creative idea generation tapers off at around 9–10 min of ideation time, corresponding to the typical amount of time given to participants in creative idea generation tasks in this domain (Beaty and Silvia 2012; Parnes 1961). Therefore, participants in this study were given more than this usual amount of time in order to fully explore the extent of creative ideas that our participants were able to generate for this design problem. They were instructed to sketch only one idea per sheet of paper and write notes on each sketch such that an outsider would be able to understand the concepts upon isolated inspection (see Fig. 3).
Three hours after the brainstorming session, participants returned for a second design session. Instructions for this design session were provided to participants on sheets on paper (see “Appendix 1”). They were provided with a stack of ideas (anonymous) from one of their team members and were given 20 min to individually assess all of the concepts generated by their team members by categorizing each concept as follows:
-
Consider: Concepts in this category are concepts that will most likely satisfy the design goals; you want to prototype and test these ideas immediately. It may be the entire design that you want to develop, or only 1 or 2 specific elements of the design that you think are valuable for prototyping or testing.
-
Do not consider: Concepts in this category have little to no likelihood of satisfying the design goals and you find minimal value in these ideas. These designs will not be prototyped or tested in the later stages of design because there are no elements in these concepts that you would consider implementing in future designs.
These two categories were chosen to simulate the rapid filtering of ideas that occur in the concept selection process in industry (Rietzchel et al. 2006), such as Go/No Go screening (Cooper and Brentani 1984; Ulrich et al. 2011) (see example concept assessment sheet in Fig. 4). While other concept selection practices in design practice typically do not require designers to make strictly categorical choices such as this, the procedure was designed in this way since we were primarily interested in participants’ innate preferences and reactions to creativity during the concept selection process. Once the participant had completed ratings for all of the ideas from their team member, they shuffled the ideas in random order (to avoid any ordering bias) and then passed the ideas clockwise to the next team member. This process was repeated until all the design concepts generated within each design team were assessed by all team members, including each team member’s own ideas (see Fig. 4). Therefore, each team member assessed a total of four design sets, corresponding to each member in the deign team. This idea assessment was conducted individually. Participants did not share their concept assessment sheets during the activity, and team discussions were not allowed during this activity. This was done to avoid any potential bias that may arise from team-based discussions that can affect their initial impressions and opinions of the ideas.
2.3 Metrics
In order to investigate the impact of risk attitudes on the creativity of the selected designs, several metrics for measuring risk attitude, concept assessment, and design creativity were developed. These metrics are described in detail in the following sections.
2.3.1 Risk aversion and ambiguity aversion metrics
Risk and ambiguity aversion for each participant was calculated using methods developed by researchers in standard behavioral economics (Boyle et al. 2011; Han et al. 2012). These methods were used since no metric yet exists for measuring risk-taking in creative engineering design tasks. However, since risk behavior has been shown to vary greatly across situations and domains (Weber 2010; Weber et al. 2002), it is unclear if these traditional financial risk aversion measurements can be used to measure risk-taking in a creative domain. Therefore, in order to investigate the link between individual risk attitudes and creativity in concept selection, each participant was asked to complete an online survey that measured their individual risk attitudes 1 week prior to the start of the study. Specifically, in order to calculate each participant’ risk attitude scores, the following methods were used:
-
Risk aversion: An individual’s risk aversion was measured using the 10 lottery questions from the risk aversion online survey (Chronbach’s α = 0.91) taken from research in standard behavioral economics (Boyle et al. 2012; Han et al. 2012). An example question is “Which would you prefer? $15 for sure, or a coin flip in which you get $ [an amount greater than $15] if it is heads, or $0 if it is tails?” Potential gamble gains vary randomly within the interval of $20.00–$300.00, where monetary increments were determined through a series of pilot tests with engineering students. The individual risk aversion index was then calculated according Han et al. (2012) and had a range from 0 (risk prone) to 1 (risk averse). The complete list of questions for assessing risk aversion and method of calculating individual risk aversion can be found in “Appendix 3.”
-
Ambiguity aversion: Ambiguity aversion was measured using 10 lottery questions from the ambiguity aversion online survey (Chronbach’s α = 0.85). The goal of the assessment was to identify the point at which an individual would take the gamble given unknown odds of winning the gamble (i.e., make the “uncertain” choice). An example question is “Which would you prefer? $15 for sure, or $20 if you win the gamble with unknown probability and $0 if you do not?” Ambiguity aversion was then calculated according to Borghans et al. (2009). The complete list of questions for assessing ambiguity aversion and the method of calculating individual ambiguity aversion levels can be found in “Appendix 3.” Similar to risk aversion, the individual’s ambiguity aversion could range from 0 to 1, with lower ambiguity aversion scores indicating more tolerance for ambiguity.
2.3.2 Design creativity and assessment metrics
Once the study was complete, two independent raters were recruited to assess the creativity of all 149 ideas generated in this study based on Shah et al.’s 4 creativity metrics: novelty, quality, variety, and quantity (Shah et al. 2003). While many other methods for assessing creativity in design have been developed, such as the Usefulness and Unusualness (Moss 1966), the SAPPhIRE model of novelty (Sarkar and Chakrabarti 2011), the Evaluation of Innovative Potential (EPI) (Chulvi et al. 2012), the Creative Product Semantic Scale (CPSS) (Besemer 1998; O’Quin and Besemer 2006), and the Consensual Assessment Technique (CAT) (Kaufman et al. 2008), the Shah et al. (2003) metrics have been widely adopted in the literature (Lopez-Mesa and Thompson 2006; Nelson and Wilson 2009; Oman and Tumer 2010; Schmidt and Vargas-Hernandez 2010; Srivathsavai and Genco 2010) and have been regarded as a valuable fundamental approach for assessing idea creativity that is grounded in strong conceptual definitions and is relatively simple to implement (Hernandez et al. 2012). In addition, these metrics were chosen for this analysis due to the time required to assess a large number of generated ideas and the relative ease of implementing the assessment method for the current design problem (Oman et al. 2013). Since the variety and quantity metrics in the Shah et al. (2003) approach are measures for groups of ideas, not individual ideas, only the novelty and quality metrics were used for the calculation of creativity in this study, as has been proposed in previous research (Oman et al. 2013; Sarkar and Chakrabarti 2014). The use of novelty in the current study is also important given that previous studies have found that novelty is closely related to the variety of an idea set and that novelty scores “can be used as an indication of the variety score” (p. 14) (Jagtap et al. 2015) in design ideation studies.
In addition, while previous studies conceptualized creativity as an aggregate of novelty and quality, the approach used in the current study maintains a distinction between the novelty and quality metrics, treating them as two separate components of creativity. This was done in order to allow for the analysis of the novelty and quality components of creativity separately, since the conclusions that can be drawn from methods that increase the selection of novel ideas may be vastly different from the conclusions that can be drawn from methods that increase the quality of the selected ideas. Indeed, Shah et al. (2003) argue that “since each of them (creativity metrics) measures something different, we feel that adding them directly makes no sense. Even if we were to normalize them in order to add, it is difficult to understand the meaning of such a measure… We can also argue that a method is worth using if it helps us with any of the measures.” (p. 133) (Shah et al. 2003). Therefore, the two raters used a 24-question Design Rating Survey (DRS), to assess the novelty and quality of each design, see “Appendix 2.” This survey helped raters classify the features each design concept addressed, similar to the approach used in prior studies (Toh and Miller 2014). The raters were undergraduate students in mechanical engineering who received extensive training on the design task and rating process. They attended several training sessions where the rating questions were explained in detail to them, and practice ratings were conducted in order to ensure a satisfactory agreement between raters. The first 20 questions on the DRS were used to help the raters classify the features each design concept addressed, similar to the feature tree approach used in previous studies used to compute design novelty (Toh et al. 2012a, b). The remaining four survey questions were used to compute design quality and helped the raters identify the quality and technical feasibility of the design, similar to the process used by Linsey et al. (2011). The Cohen’s Kappa (inter-rater reliability) was 0.88 for the first 20 novelty questions and 0.86 for the remaining four quality questions. Any disagreements were settled in a conference between the two raters as was done in previous studies investigating creativity (Chrysikou and Weisberg 2005; Toh et al. 2012a, b). The results from these concept evaluations were used to calculate the novelty of the generated designs as follows:
-
Idea novelty: Novelty is the “measure of how unusual or unexpected an idea is compared to other ideas” (p. 117) (Shah et al. 2003) and was calculated in this study according to Shah et al. (2003). In order to assess the amount of novelty in the designs generated by each participant, the novelty of each feature was calculated first. This feature novelty is defined as the novelty of each feature, i, as it compares to all other features addressed by all the generated designs. Feature novelty, \(f_{i}\), can then vary from 0 to 1, with 1 indicating that the feature is very novel compared to other features. The method of computing \(f_{i}\), is shown in Eq. 1:
where T is the total number of designs generated by all participants and C is the total number of designs that addressed feature \(f_{i}\) (see Fig. 5 for an illustration of feature novelty). The novelty of each design, j, is then determined by the combined effect of the Feature Novelty, \(f_{i}\), of all the features that the design addresses. Because D j is computed for all the features addressed by a design, the novelty per design is computed as an average of feature novelty, as seen in Eq. 2:
where f i is the feature novelty of a feature that was addressed in the design and \(\sum i\) is the number of features addressed by the design. This computation resulted in a novelty score for each design that reflects each idea’s relative novelty compared to all other ideas generated in this study and thus reflects the degree of originality of each design compared to every other design generated in this study.
Once idea novelty was calculated, each participant’s tendency for selecting creative ideas was captured using the following metric:
-
Propensity toward novel concept selection, P N : This metric is defined as each participant’s tendency toward selecting (or filtering) novel concepts in the concept selection process. This metric was developed by the authors in previous studies to assess each team’s tendency toward selecting or filtering creative concepts during concept selection (Toh and Miller 2014b). First, the average novelty of selected concepts is computed. Second, the average novelty of all concepts available to choose from is computed. This step is completed in order to normalize an individual’s propensity based on the novelty of their teams’ generated ideas and account for the fact that different teams generate ideas with varying levels of novelty. Lastly, the quantity from step 1 is divided by the quantity in step 2. This metric is shown in detail in Eq. 3.
where \(P_{N}\) is the participant’s propensity for novel concept selection, k is the number of ideas selected by the participant, l is the number of ideas in their set, and C j = 1 if the idea is selected and 0 if the idea is not selected. In essence, \(P_{N}\) measures the proportion of novel idea selection out of the total novelty of the ideas that were developed by the design team. This metric can achieve a value >1 if the average novelty of the selected ideas is higher than the average novelty of the available ideas, indicating a propensity for novel concept selection. P N can also be <1, indicating an aversion for novel concept selection. A score of 1 indicates that the participant chose a set of ideas that, on average, had the same level of novelty as the ideas that were provided to them, indicating no propensity toward novel concept selection. Table 1 shows examples of high and low P N scores. All quantities in the calculation of each participant’s P N score excluded ideas generated by the participants themselves. This was done in order to remove any personal bias the participants may have had for or against their own generated ideas.
Since we were also interested in the relationship between developing novel ideas and selecting novel ideas during the concept selection process, the novelty of each participant was also calculated as follows:
-
Participant novelty: The participant novelty metric was used as a measure of each participant’s ability to generate novel ideas in the idea generation activity. Therefore, this metric determined as the average design novelty of all the designs each participant generated (Shah et al. 2000, 2003), as seen in Eq. 4.
where N is the total number of ideas generated by the participant.
-
Idea quality: Quality is defined as a measure of a concept’s feasibility and how well it meets the design specifications (Shah et al. 2003). Similar to Linsey et al. (2011), we measured quality on an anchored multipoint scale. However, we included an additional question to the quality scale in order to capture the improvement of the generated concept over the original design. The quality metric was calculated using the raters’ answers to the final 4 questions on the 24-question survey (see Fig. 6).
-
The quality of each design, j, was then computed using Eq. 5, where q k is the answer to the kth quality question. q k = 1 when the quality question is answered with a “yes,” and q k = 0 when the quality question is answered with a “no.” The quality score for each participant is then obtained by computing the average quality scores of all designs that the participant generated.
-
Propensity toward quality concept selection, P Q : This metric was developed by the authors to assess each participant’s tendency toward selecting or filtering high-quality concepts during concept selection. In order to calculate this metric, first the average quality of the selected concepts is computed. Next, the average quality of all concepts available to choose from is computed. Similar to the calculation of P N , this denominator was created in order to normalize an individual’s propensity based on the quality of their teams’ generated ideas and account for the fact that different teams generate ideas with varying levels of quality. Lastly, the quantity from step 1 is divided by the quantity in step 2. This metric is shown in detail in Eq. 6.
where \(P_{Q}\) is the participant’s propensity for selecting quality ideas during concept selection, k is the number of ideas selected by the team, l is the number of ideas in their set, and C j = 1 if the idea is selected and 0 if the idea is not selected.
-
Participant quality: This metric was developed to capture each participant’s level of quality in the generated ideas. In order to accomplish this, participant quality metric was first calculated as the average design quality of all the designs each participant generated (Shah et al. 2000; Shah e tal. 2003), as seen in Eq. 7.
where N is the total number of ideas generated by the participant. Team quality was then computed as the average of the design quality scores for all concepts generated within each design team.
3 Data analysis and results
Prior to testing our hypotheses, descriptive statistics were calculated for the developed metrics (see Table 2). In addition, an outlier analysis was conducted on the novelty and quality of the teams’ generated designs, and no outliers were identified. SPSS v.22 was used to analyze the findings with a significance level of 0.05. A post hoc power analysis was conducted using the software package, GPower (Faul et al. 2007). Three predictor variables and a sample size of 38 were used for the statistical power analyses. For moderate effect sizes of f 2 = 1.0, the statistical power for this study was calculated as 0.99. Therefore, it can be concluded that there was more than adequate power to detect moderate or large effect sizes. A summary of the significant statistical findings is presented in Table 3. The results of our statistical analysis followed by a discussion of the implications of our findings are presented in the following sections.
3.1 (Hypothesis 1) The relationship between individual risk attitudes and creative ability during idea generation
In order to address our first research hypothesis on the relationship between individual risk attitudes and creative idea generation ability, a multivariate multiple linear regression analysis was conducted with the dependent variables being participant novelty and quality, and the independent variables being risk aversion, ambiguity aversion, and the educational level of the student. This last independent variable was chosen in order to account for differences in experience and engineering knowledge that may exist between the students in the first-year introduction to engineering design course and students in the third-year design methodology course. In addition to investigating the main effects of the independent variables, interaction effects between both risk aversion and educational level, and between ambiguity aversion and educational level were also explored.
Our results revealed that when taken together, individual risk aversion, ambiguity aversion, and educational level were not significantly related to participant novelty (R 2 = 0.28, p > 0.10) or quality (R 2 = 0.19, p > 0.26). However, both ambiguity aversion (B = 0.16, p < 0.04) and the interaction between ambiguity aversion and educational level (B = −0.10, p < 0.02) significantly predicted participant novelty. Risk aversion, educational level, and interaction between risk aversion and educational level were not significant predictors of participant novelty. This result indicates that our participants’ ability to generate novel ideas was positively related to their individual tolerance for ambiguity (see Fig. 7). It was found that first-year students who were more tolerant of ambiguity generated ideas with more novelty compared to their less ambiguity tolerant peers. In contrast, the reverse relationship was found with more experienced students, where third-year students who were more tolerant of ambiguity generated ideas with less novelty compared to their less ambiguity tolerant peers. However, the students from the two different courses did not differ in terms of individual creative ability. This result confirms our first hypothesis and indicates that varying levels of tolerance for ambiguity affect first- and third-year students differently in terms of creative ability.
3.2 (Hypothesis 2) The relationship between individual risk attitudes and selection of creative concepts
To address our second research hypothesis on individual risk attitudes and the selection of creative concepts, a multivariate multiple linear regression analysis was conducted using propensity for novel concept selection (P N ) and propensity for quality concept selection (P Q ) as the dependent variables, and individual risk aversion, ambiguity aversion, and the educational level of the student as the independent variables. Similar to our previous analysis, interaction effects between both risk aversion and educational level, and between ambiguity aversion and educational level were also explored.
The results showed a weak but statistically significant relationship between the independent variables of individual risk aversion, ambiguity aversion, educational standing, and participant propensity for novel concept selection, P N (R 2 = 0.33, p < 0.03), and participant propensity for quality concept selection, P Q (R 2 = 0.34, p < 0.03). Specifically, the interaction effect of risk aversion and educational level was shown to significantly affect P Q (B = −0.134, p < 0.04). No significant relationships were found between the other variables. These results indicate a significant positive relationship between risk-taking in the financial domain and risk-taking in the creative domain as demonstrated through participant propensity for creative ideas during concept selection (see Fig. 8). It was found that first-year students who had higher levels of risk aversion tended to select concepts with higher quality. On the other hand, third-year students who had lower levels of risk aversion tended to select concepts with lower quality. This result confirms our second hypothesis and indicates that risk attitudes affect first- and third-year students differently in terms of propensity for creative concept selection.
3.3 (Hypothesis 3) The relationship between creative idea generation ability and the selection of creative concepts
Our final research hypothesis sought to understand if there was a relationship between one’s ability to generate creative ideas and preference for creative ideas during a concept selection task. To address this research hypothesis, a multivariate linear regression analysis was conducted using participant novelty, participant quality, and educational level as the independent variables and participant P N and P Q as the dependent variables. The interaction effects between participant creative ability and educational level were also explored. Our results revealed that participant novelty, quality, and interaction effects with educational level could not predict propensity for novel concept selection, P N (R 2 = 0.10, p > 0.65) and propensity for quality concept selection P Q (R 2 = 0.16, p > 0.36), disconfirming our final hypothesis.
4 Discussion
The main goal of this study is to investigate the relationship between individual risk attitudes, creative idea generation ability, and the selection or filtering of creative concepts. Our results revealed 3 major findings, presented in Table 4 with a summary of the possible reasons for the findings.
These results, details of their possible reasons, and their implications for engineering research and education are discussed next.
4.1 Individual tolerance for ambiguity is related to creative idea generation ability
One of the main findings of this study is that tolerance of ambiguity in engineering design students was found to be related to their ability to generate novel ideas. This finding is supported by studies in the other fields that show a positive correlation between an individual’s tolerance to ambiguity and creativity in a variety of contexts (Charness and Grieco 2013; Csermelv and Lederman 2003; Sternberg and Lubart 1991; Zenasni et al. 2008). However, an inverse relationship was found for first-year students in this study. In other words, first-year students that were more tolerant of ambiguity were less likely to generate highly novel ideas. This result may be attributed to confounding factors such as differences in problem scoping (Atman 2008), information seeking (Atman 1999), perceptions of achievement (Waterman and Geary 1974), and intellectual maturity (Pavelich and Moore 1993) between the two education levels. Other factors may have also contributed to this observed effect in this study, and prior work in other research areas provides a foundation for determining possible reasons for this finding. Specifically, students of different levels may have different perceptions of creativity in the since creativity heavily emphasized and integrated in the engineering classroom (Litzinger et al. 2011; Richards 1998; Stouffer et al. 2004). Therefore, first-year students may not yet be able to recognize the ambiguity present in a design problem, or may be unable to associate risk and ambiguity with creativity to the same extent as the third-year students in this study. Future work is needed to explore the role that expertise and experience play in the creative design process, particularly with higher-level graduate engineering students and industry professionals.
This study is the first of its kind to empirically investigate the link between individual risk aversion, ambiguity aversion, and creative ability in an engineering design context. While previous studies have shown that attitudes toward risk and ambiguity can play an important role in perceptions of creativity and creative performance (Charness and Grieco 2013; Csermelv and Lederman 2003; Rubenson and Runco 1995; Simonton 1988), this study provides empirical evidence for the presence of these factors during early stage design activities such as ideation and concept selection. However, it should be noted that the relationship between ambiguity aversion and creative ability was only moderate in this study, suggesting that financial measures of ambiguity aversion may not fully capture the relationship between tolerance for ambiguity and creative ability. Although prior research has shown that attitudes toward the unknown remain relatively stable through different situations (Weber 1999). However, individual perceptions of what constitutes ambiguity may differ depending on the context (Weber et al. 2002). Therefore, more appropriate measures and techniques for assessing tolerance for ambiguity in the creative context need to be developed and validated for use in creativity research. For example, the impact of the level of ambiguity found in the generated ideas on the selection of ideas during concept selection should be explored in order to understand if design-specific attributes can influence creative concept selection.
Another important finding of this study was that varying levels of tolerance for ambiguity affect first- and third-year students differently in terms of creative ability. That is, first-year students were more likely to generate creative ideas if they were more tolerant for ambiguity, agreeing with prior work that has shown that tolerance for ambiguity is positively related to creative ability (Sternberg and Lubart 1991; Zenasni et al. 2008). However, the reverse relationship was found with third-year students where students who were more tolerant of ambiguity generated ideas with less novelty. This result can be attributed to the different levels of creativity found in first-year and third-year engineering students (Genco et al. 2012) due to an engineering curriculum that discourages creativity as students progress through engineering programs (Charyton and Merrill 2009). Therefore, in introductory engineering courses earlier in the engineering program, first-year students who are more tolerant of ambiguity may feel encouraged to think creatively during ideation, whereas third-year engineering students may have adapted to the engineering curriculum that discourages creativity and may thus feel less encouraged to think creatively during ideation. This result shows that an increased emphasis on creativity during design needs to be placed throughout the design process and across educational levels in order to ensure that creativity is not discouraged or reduced in students who progress through the engineering program. While this study was conducted in an engineering educational setting, the results highlight the need to develop new studies directed at understanding the role of risk-taking in both educational and industrial practices.
4.2 Individual risk aversion affects creative concept selection
While prior work on risk attitudes has identified risk-taking as an important factor in encouraging creative performance, this study is the first of its kind of demonstrate a weak-to-moderate link between individual risk attitudes and the concept selection process. Specifically, our results indicate that third-year students who are more risk prone tend to select ideas that are more creative during concept selection, highlighting the role that risk plays in both creative idea generation ability and perceptions of creativity (Rubenson and Runco 1995). Even though people have a deep-seeded desire to maintain a sense of certainty and preserve the familiar (Sorrentino and Roney 2000), the results of this research indicate that third-year students who are more prone to taking risks perform less filtering of these novel ideas during concept selection, even though there may be uncertainty about whether a novel idea is practical, error free or useful (Amabile 1996). On the other hand, an inverse relationship was found for first-year students. That is, first-year students who were more risk prone tended to select less creative ideas in this study. While confounding factors such as differences in problem scoping (Atman 2008), information seeking (Atman 1999), perceptions of achievement (Waterman and Geary 1974), and intellectual maturity (Pavelich and Moore 1993) may have caused this observed difference between educational levels, other factors may have also been responsible for these findings. Specifically, first-year students in this study may have had a lack of awareness of what constitutes risk in a creative context compared to third-year students since creativity is heavily emphasized and integrated in the engineering classroom (Litzinger et al. 2011; Richards 1998; Stouffer et al. 2004). Indeed, prior research has shown that while attitudes toward risk remain relatively stable through different situations (Hsee and Weber 1999), perceptions of what constitutes a risky decision may differ depending on the context (Weber et al. 2002). Therefore, students of different levels may have varying perceptions of risk-taking in a creative task and may respond different based on their perceptions of risk. The results of this study add to our understanding of the impact that risk attitudes have on decision-making in design. Specifically, while recent research has found that team-level risk aversion scores do not impact creative concept selection (Toh and Miller 2016), the findings of this study show that individual perceptions and preferences for creative ideas during concept selection activities are impacted differently by risk attitudes and are affected by student education level. Therefore, further work is needed to identify differences in creative risk-taking with engineers of different education levels and expertise in order to better understand how risk affects decision-making in creative tasks.
The results of this study also show that ambiguity aversion is not significantly related to an individual’s propensity for creative concept selection, indicating that tolerance for ambiguity is important for creative concept generation, but not for creative concept selection for engineering design students. Similarly, participant educational level was found to not significantly predict their propensity for creative concept selection, suggesting that participants’ perception and preference for creative ideas are unaffected by the knowledge and learning experience gained in the engineering classroom. These findings are supported by previous research on individual ambiguity aversion levels that show that individual-level ambiguity aversion scores are related to creative idea generation ability but not creative concept selection (Toh and Miller 2014b). These preliminary studies show that ambiguity aversion may impact both individual and team creative idea generation and concept selection activities in a similar manner in design education. This result shows that even if team-level attributes are complex combinations of individual-level attributes (McGrath 1998), the impact of ambiguity aversion on creative idea generation is similar across levels. This research also provides a foundation for studying the impact of factors such as compatibility, diversity, and creative confidence on team-level attributes. Future work should consider both individual and team-level individual attributes holistically and their impact on creative concept selection in order to gain a better understanding of how these attributes interact in team decision-making settings.
Another important implication of this result is that traditional measures of financial risk aversion developed in behavioral economics can be used as a proxy for risk-taking in creative design tasks. While prior work in engineering design provides little basis for measuring creativity during concept selection, the risk aversion metric utilizing financial gamble gains was able to predict an individual’s propensity toward creative concept selection in an engineering design setting. This result establishes a relationship between traditional measures of financial risk attitudes and creative concept selection where there was none before, and supports prior work that has shown that attitudes toward risk remain relatively stable across situations and domains (Weber 1999). However, the fact that only a weak relationship was found between these factors indicates that there may be other aspects of risk-taking that are present during creative tasks that may influence an individual’s preference for creative ideas that should be explored in future studies. In addition, since the risk aversion and ambiguity aversion scores were significantly correlated in this study (r = 0.525, p = 0.001), more research is needed to investigate the independence of risk and ambiguity factors in creative concept selection.
From this study, several recommendations and directions for future research can be presented. First, methods that encourage risk-taking or identify what risk-taking is in early-phase concept develop in engineering education should be developed and implemented to encourage the selection of creative concepts. Second, this study provides empirical evidence regarding the link between traditional measures of financial risk aversion and risk-taking in creative tasks in engineering education. Therefore, by using traditional measures of financial risk aversion, design researchers can investigate the filtering and selection of creative ideas in the design process and develop methods and techniques that encourage the selection of these creative ideas.
4.3 Participant creative idea generation ability is unrelated to creative concept selection
To address the research gaps in the engineering creativity literature, our study sought to investigate the relationship between individual creative idea generation ability and the selection of creative concepts in an engineering education context. Our results indicate that creative idea generation ability is unrelated to an individual’s tendency for selecting creative concepts during concept selection. That is, an individual’s ability to generate creative concepts does not necessarily increase the selection of creative ideas in the later stages of the design process.
This result demonstrates that creative idea generation ability is not necessarily coupled with the ability to identify creative concepts during concept selection. Therefore, even if an individual is unable to generate highly novel ideas during idea generation, they may still be able to contribute to the overall creativity of the design process in the later stages of concept development. This finding is important since engineering students and professionals can be encouraged to identify and recognize creative ideas in order to support the overall creativity of a design project. This result indicates that teaching or encouraging creative concept generation is not sufficient for ensuring the selection of these creative concepts during the later stages of the design stage. Students and practicing engineers who are expected to be creative during the design process should focus on creativity during concept generation and selection in order to truly innovate and break convention. These results are supported by prior research on team concept generation and selection activities that shows that team-level creative idea generation ability is unrelated to creative concept selection, indicating that creative idea generation ability is not required for creative concept selection in both team and individual design activities. Therefore, methods and techniques for encouraging creativity that span across all phases of the design process is essential for increasing design creativity and future research should focus on developing frameworks and methodologies for assessing and selecting creative ideas during concept selection. In addition, the results of this study highlight the fact that methods of encouraging creative idea generation should also be evaluated for their ability to generate creative ideas that are both novel and usable (Amabile 1982) in order to ensure successful selection of these ideas during the concept selection phase of the design process.
4.4 Study limitations and future work
While this study showed relationships between risk attitudes and creative concept selection, there exist several limitations that should be noted. The most important of these limitations is the fact that the participants of this study were novice designers from first-year and third-year engineering design classrooms. In addition, future work should explore the generalizability of the results utilizing larger sample sizes and investigating factors such as team dynamics and participant motivation. Therefore, future studies should investigate the link between individual risk attitudes and creative concept selection in a controlled laboratory setting, with controls for potential confounds. In addition, further work is needed to investigate the impact of individual risk attitudes on creative concept selection with practicing engineering designers. Further work should also explore the selection of creative concepts in the context of discussion-based concept selection activities among team members, as is typically done in engineering design practice. Another aspect of concept selection that should be explored in future studies in the use of other types of design problems, namely problems of varying structure. This is important because it is still unknown if highly structured or open-ended design problems encourage the most creativity during concept selection, and research that explores this factor in creative concept selection will help add to our understanding of the role of risk-taking in concept selection. Similarly, research that explores design problems that require different levels of innovation will add to knowledge regarding the interaction of these two facets of creativity, since it was found that novelty and quality were not significantly related in this study (r = −0.043, p = 0.308), indicating that these two variables assess separate elements of creativity. Work that investigates creative concept selection using different creativity assessment metrics such as variety and quantity can also provide more knowledge on the impact of risk attitudes on other potentially differing aspects of design creativity. Future studies that explore the implications of design task timing and duration on creative concept selection will add to our understanding of decision-making in design. Lastly, since this study focused on the selection of other team members’ ideas, further analysis of each participant’s assessment of their own generated design concepts will provide insights into individual perceptions of creativity during the concept selection process and potential decision-making biases such as ownership bias.
5 Conclusion
The current study was developed to understand the relationship between individual risk aversion, ambiguity aversion, educational level, creative idea generation ability, and propensity toward creative concept selection. Our results highlight the fact that an individual’s attitude toward risk and aversion can affect their creative idea generation ability and propensity for selecting creative ideas. It was also found that student educational level plays a role in the relationship between ambiguity aversion and creative idea generation ability. However, the generation of creative ideas is not necessarily indicative of creativity during concept selection. Therefore, techniques for encouraging creativity in both concept generation and concept selection should be developed to increase design creativity as a whole.
References
Altshuller GS (1984) Creativity as an exact science: the theory of the solution of inventive problems, vol 320. Gordon and Breach Science Publishers, Luxembourg
Amabile T (1982) Social psychology of creativity: a consensual assessment technique. J Pers Soc Psychol 43:997–1013
Amabile T (1996) Creativitiy in context. Westview Press, Boulder
Antonsson EK, Otto KN (1995) Imprecision in engineering design. J Mech Des 117:25–32
Arup S (2014) Ambuja knowledge initiative. http://www.foundationsakc.com/people/legends/sir-ove-arup
Atman CJ (1999) A comparison of freshman and senior engineering design processes. Des Stud 20:131–152
Atman CJ (2008) Breadth in problem scoping: a comparison of freshman and senior engineering students. J Eng Educ 24:234–245
Ayag Z, Ozdemir RG (2009) A hybrid approach to concept selection through fuzzy analytic network process. Comput Ind Eng 56:368–379. doi:10.1016/j.cie.2008.06.011
Baer M, Oldham GR, Jacobsohn GC, Hollingshead AB (2007) The personality composition of teams and creativity: the moderating role of team creative confidence. J Creat Behav 42:255–282
Ball LJ, Lambell NJ, Reed SE, Reid FJ (2001) The exploration of solution options in design: a’ naturalistic decision making’ perspective. Designing in Context, Delft University Press, Delft, pp 79–93
Beaty RE, Silvia PJ (2012) Why do ideas get more creative across time? An executive interpretation of the serial order effect in divergent thinking tasks. Psychol Aesthet Creat Arts 6:309–319
Belbin RM (1981) Management teams: why they succeed or fail. Heinemann, London
Besemer SP (1998) Creative product analysis matrix: testing the Model structure and a comparison among products—three novel chairs. Creat Res J 11:333–346
Borghans L, Heckman JJ, Golsteyn BHH, Meijers H (2009) Gender differences in risk aversion and ambiguity aversion. J Eur Econ Assoc 7:649–658. doi:10.1162/jeea.2009.7.2-3.649
Bossuyt DL, Dong A, Tumer IY, Carvalho L (2013) On measuring engineering risk attitudes. J Mech Des 135:121001
Bower GH (1981) Mood and memory. Am Psychol 36:129–148
Boyle PA, Yu L, Buchman AS, Laibson DI, Bennett DA (2011) Cognitive function is associated with risk aversion in community-based older persons. BMS Geriatr 11:53
Boyle PA, Yu L, Buchman AS, Bennett DA (2012) Risk aversion is associated with decision making among community-based older persons. Front Psychol 3:205
Bucciarelli LL (1988) An ethnographic perspective on engineering design. Des Stud 9:159–168
Busby JS (2001) Practices in design concept selection as distributed cognition. Cogn Technol Work 3:140–149
Charness G, Grieco D (2013) Individual creativity, ex-ante goals and financial incentives. Department of Economics, UCSB, UC Santa Barbara
Charyton C, Merrill JA (2009) Assessing general creativity and creative engineering design in first year engineering students. J Eng Educ 98:145–156
Chrysikou EG, Weisberg RW (2005) Following in the wrong footsteps: fixation effects of pictorial examples in a design problem-solving task. J Exp Psychol 31:1134–11448
Chulvi V, Mulet E, Chakrabarti A, López-Mesa B, González-Cruz C (2012) Comparison of the degree of creativity in the design outcomes using different design methods. J Eng Des 23:241–269
Cooper RG, Brentani U (1984) Criteria for screening new industrial products. Ind Mark Manage 13:149–156
Csermelv P, Lederman L (2003) Talent, science, and education: how do we cope with uncertainty and ambiguities? In: Paper presented at the NATO Advanced Research Workshop, Budapest, Hungary
Daly SR, Mosyjowski EA, Seifert CM (2014) Teaching creativity in engineering courses. J Eng Educ 103:417–449
Dewett T (2007) Linking intrinsic motivation, risk taking, and employee creativity in an R&D environment. R&D Manag 37:197–208
Dijksterhuis A (2004) Think different: the merits of unconscious thought in preference development in decision making. J Pers Soc Psychol 87:586–598
Eberle B (1996) Scamper: games for imagination development. Prufrock Press, Waco
El-Murad J, West DC (2003) Risk and creativity in advertising. J Mark Manag 19:657–673
Faul F, Erdfelder E, Lang AG, Buchner A (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39:175–191
Faure C (2004) Beyond brainstorming: effects of different group procedures on selection of ideas and satisfaction with the process. J Creat Behav 38:13–34
Felder RM, Brent R (2004) The ABC’S of engineering education: ABET, Bloom’s Taxonomy, cooperative learning, and so on. In: ASEE
Fellner W (1961) Distortion of subjective probabilities as a reaction to uncertainty. Quart J Econ 75:670–694
Ford CM, Gioia DA (2000) Factors influencing creativity in the domain of managerial decision making. J Manag 26:705–732
Forster J (2009) Cognitive consequences of novelty and familiarity: how mere exposure influences level of construal. J Exp Soc Psychol 45:444–447
Frisch D, Baron J (1988) Ambiguity and rationality. J Behav Decis Mak 1:149–157
Fuge M, Stroud J, Agogino AM (2013) Automatically inferring metrics for design creativity. In: Paper presented at the ASME Design Engineering Technical Conferences, Portland, OR, August 4–7
Genco N, Holtta-Otto K, Seepersad CC (2012) An experimental investigation of the innovation capabilities of undergraduate engineering students. J Eng Educ 101:60–81
Goldsenson RM (1984) Longman dictionary of psychology and psychiatry. Longman, New York
Hambali A, Supuan SM, Ismail N, Nukman Y (2009) Application of analytical hierarchy process in the design concept selection of automotive composite bumper beam during the conceptual design stage. Sci Res Essays 4:198–211
Hammond JS, Keeney RL, Raiffa H (1998) The hidden traps in decision making. Harv Bus Rev 76:47–58
Han SD, Boyle PA, Arfanakis K, Fleischman DA, Yu L, Edmonds EC, Bennet DA (2012) Neural intrinsic connectivity networks associated with risk aversion in old age. Behav Brain Res 227:233–240
Heath C, Tversky A (1991) Preferences and beliefs: ambiguity and competence in choice under uncertainty. J Risk Uncertain 2:5–35
Hernandez N, Okudan Kremer G, Schmidt LC (2012) Effectiveness metrics for ideation: Merging genealogy trees and improving novelty metric. In: Paper presented at the international design engineering technical conferences, Chicago, IL
Hoffman LR, Maier N (1961) Quality and acceptance of problem solutions by members of homogeneous and heterogeneous groups. J Abnorm Soc Psychol 62:401–407
Hsee W, Weber EU (1999) Cross-national differences in risk preference and lay predictions. J Behav Decis Mak 12:165–179
Huang H-Z, Liu Y, Li Y, Xue L, Wang Z (2013) New evaluation methods for conceptual design selection using computational intelligence techniques. J Mech Sci Technol 27:733–746
Jagtap S, Larsson A, Hiort V, Olander E, Warell A (2015) Interdependency between average novelty, individual average novelty, and variety. Int J Des Creativity Innov 3:43–60
Kahneman D, Tversky A (1979) Prospect theory: an analysis of decision under risk. Econom J Econom Soc 47:263–291
Kaufman JC, Baer J, Cole JC, Sexton JD (2008) A comparison of expert and nonexpert raters using the consensual assessment technique. Creat Res J 20:171–178
Kazerounian K, Foley S (2007) Barriers to creativity in engineering education: a study of instructors and students perceptions. J Mech Des 129:761–768
Kijkuit B, van der Ende J (2007) The organizational life of an idea: integrating social network, creativity and decision-making perspectives. J Manag Stud 44:863–882
King AM, Sivaloganathan S (1999) Development of a methodology for concept selection in flexible design strategies. J Eng Des 10:329–349. doi:10.1080/095448299261236
Kleiman P (2008) Towards transformation: conceptions of creativity in higher education. Innov Educ Train Int 45:209–217
Kremer GE, Schmidt LC, Hernandez N (2011) An investigation on the impact of the design problem in ideation effectiveness research. In: Paper presented at the American Society for Engineering Education Conference, Vancouver, B.C., June 26–29
Kudrowitz BM, Wallace D (2013) Assessing the quality of ideas from prolific, early-stage product ideation. J Eng Des 24:120–139
Kulkarni C, Dow SP, Klemmer SR (2012) Early and repeated exposure to examples improves creative work. In: Leifer L, Plattner H, Meinel C (eds) Design thinking research. Understanding Innovation. Springer, Heidelberg, pp 49–62
Kurdrowitz B, Dippo C (2013) Getting to the novel ideas: exploring the alternative uses test of divergent thinking. In: Paper presented at the ASME design engineering technical conferences, Portland, OR, August 4–7
Linnerud B, Mocko G (2013) Factors that effect motivation and performance on innovative design projects. In: Paper presented at the ASME design engineering technical conferences, Portland, OR, August 4–7
Linsey JS, Clauss EF, Kurtoglu T, Murphy JT, Wood KL, Markman AB (2011) An experimental study of group idea generation techniques: understanding the roles of idea representation and viewing methods. J Mech Des 133:031008
Litzinger TA, Lattuca LR, Hadgraft RG, Newsletter WC (2011) Engineering education and the development of expertise. J Eng Educ 100:123–150
Lopez-Mesa B, Thompson G (2006) On the significance of cognitive style and the selection of appropriate design methods. J Eng Des 17:371–386
Marsh ER, Slocum AH, Otto KN (1993) Hierarchical decision making in machine design. MIT Precision Engineering Research Center, Cambridge
McGrath JE (1998) A view of group composition through a group-theoretic lens. JAI, Greenwich
Moore E, Eckel C (2003) Measuring ambiguity aversion. In: Paper presented at the economic science association meetings, Barcelona, June 21–24
Moos RH, Speisman JC (1962) Group compatibility and productivity. J Abnorm Soc Psychol 65:190–196
Moscovici S (1976) Social influence and social change. Academic Press, London
Moss J (1966) Measuring creative abilities in junior high school industrial arts. No. 20036, American Council on Industrial Arts Teacher Education, Washington, DC
Mueller JS, Melwani S, Goncalo JA (2011) The bias against creativity: why people desire but reject creative ideas. Psychol Sci 2011:0956797611421018
Mumford MD (2003) Where have we been, where are we going? Taking stock in creativity research. Creat Res J 15:107–120
Nelson BA, Wilson JO (2009) Redefining metrics for measuring ideation effectiveness. Des Stud 30:737–743
Nijstad BA, De Dreu C (2002) Creativity and group innovation. Appl Psychol Int Rev 51:400–406
Nikander JB, Liikkanen LA, Laakso M (2014) The preference effect in design concept evaluation. Des Stud 35:473–499. doi:10.1016/j.destud.2014.02.006
Oman SK, Tumer IY (2010) Assessing creativity and innovation at the concept generation stage in engineering design: a classroom experiment. In: Paper presented at the international design engineering technical conferences, Montreal, Quebec, Canada
Oman SK, Tumer IY, Wood K, Seepersad C (2013) A comparison of creativity and innovation metrics and sample validation through in-class design projects. Res Eng Des 24:65–92
Onarheim B, Christensen BT (2012) Distributed idea screening in stage-gate development processes. J Eng Des 23:660–673
O’Quin K, Besemer SP (2006) Using the creative product semantic scale as a metric for results-oriented business. Creat Innov Manag 15:34–44
Osborn A (1957) Applied imagination. Scribner, New York
Pahl G, Beitz W (1984) Engineering design. The Design Council, London
Parnes SJ (1961) Effects of extended effort in creative problem solving. J Educ Psychol 52:117–122
Pavelich MJ, Moore W (1993) Measuring maturing rates of engineering students using the Perry model. In: Frontiers in education conference, 1993. Twenty-third annual conference’ engineering education: renewing America’s Technology’, Proceedings. IEEE, pp 451–455
Pugh S (1991) Total design: integrated methods for successful product engineering. Addison-Wesley, Workingham
Putman VL, Paulus PB (2009) Brainstorming, brainstorming rules and decision making. J Creat Behav 43:29–40
Richards LG (1998) Stimulating creativity: teaching engineers to be innovators. In: Paper presented at the frontiers in education conference, Tempe, AZ, Nov 4–7
Rietzchel EF, Nijstad BA, Stroebe W (2006) Productivity is not enough: a comparison of interactive and nominal groups in idea generation and selection. J Exp Soc Psychol 42:244–251
Rietzschel E, Nijstad BA, Stroebe W (2010) The selection of creative ideas after individual idea generation: choosing between creativity and impact. Br J Psychol 101:47–68
Rubenson DL, Runco MA (1995) The psychoeconomic view of creative work in groups and organizations. Creat Innov Manag 4:232–241
Sarbacker SD, Ishii K (1997) A framework for evaluating risk in innovative product development. In: Paper presented at the design engineering technical conferences, Sacramento, CA, September, 14–17
Sarkar P, Chakrabarti A (2011) Assessing design creativity. Des Stud 32:348–383
Sarkar P, Chakrabarti A (2014) Ideas generated in conceptual design and their effects on creativity. Res Eng Des 25:185–201
Schmidt L, Vargas-Hernandez N (2010) Pilot of systematic ideation study with lessons learned. In: Paper presented at the international design engineering technical conferences, Montreal, Quebec, Canada
Shah J, Kulkarni S, Vargas-Hernandez N (2000) Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments. J Mech Des 122:377–384
Shah JJ, Vargas-Hernandez N, Smith SM (2003) Metrics for measuring ideation effectiveness. Des Stud 24:111–134
Simonton D (1988) Scientific genius. Cambridge University Press, New York
Simpson T, Thevenot H (2007) Using product dissection to integrate product family design research into the classroom and improve students’ understanding of platform commonality. Int J Eng Educ 23:120–130
Simpson TW, Lewis KE, Stone RB, Regli WC (2007) Using cyberinfrastructure to enhance product dissection in the classroom. In: Paper presented at the Industrial Engineering Research Conference, Nashville, TN, May 19–23
Sitkin SB, Pablo AL (1992) Reconceptualizing the determinants of risk behavior. Acad Manag Rev 17:9–38
Somech A, Drach-Zahavy A (2011) Translating team creativity to innovation implementation: the role of team composition and climate for innovation. J Manag 39:684–708
Sorrentino R, Roney CJR (2000) The uncertain mind: individual differences in facing the unknown, vol 1. Psychology Press, Hove
Srivathsavai R, Genco N (2010) Study of existing metrics used in measurement of ideation effectiveness. In: Paper presented at the design engineering technical conferences, Montreal, Quebec, Canada
Staw BM (1995) Why no one really wants creativity. In: Ford CM, Gioia DA (eds) Creative Action in Organizations: Ivory Tower Visions and Real World Voices. Sage Publications, Thousand Oaks
Sternberg RJ, Lubart TI (1991) An investment theory of creativity and its development. Hum Dev 24:1–31
Stouffer WB, Russel JS, Oliva MG (2004) Making the strange familiar: creativity and the future of engineering education. In: Paper presented at the American Society for Engineering Education Annual Conference and Exposition, Salt Lake City, UT, June 20–23
Sullivan JF, Carlson LE, Carlson DW (2001) Developing aspiring engineers into budding entrepreneurs: an invention and innovation course. J Eng Educ 90:571–576
Ter Harr S, Clausling D, Eppinger S (1993) Integration of quality function deployment in the design structure matrix. Laboratory for Manufacturing and Productivity, MIT, Cambridge, MA
Thurston DL, Carnahan JV (1992) Fuzzy ratings and utility analysis in preliminary design evaluation of multiple attributes. J Mech Des 114:648–658
Toh C, Miller SR (2013a) Exploring the utility of product dissection for early-phase idea generation. In: Paper presented at the ASME design engineering technical conferences, Portland, OR, August 4–7
Toh C, Miller SR (2013b) Product dissection or visual inspection? The impact of designer-product interactions on engineering design creativity. In: Paper presented at the ASME design engineering technical conferences, Portland, OR
Toh CA, Miller SR (2014a) The impact of example modality and physical interactions on design creativity. J Mech Des. doi:10.1115/1.4027639
Toh C, Miller S (2014b) The role of individual risk attitudes on the selection of creative concepts in engineering design. Paper presented at the ASME Design Engineering Technical Conferences, Buffalo, NY
Toh CA, Miller SR (2016) Creativity in design teams: the influence of personality traits and risk attitudes on creative concept selection. Res Eng Des 27:73–89. doi:10.1007/s00163-015-0207-y
Toh CA, Miller SR, Kremer GE (2012a) The impact of product dissection activities on the novelty of design outcomes. In: Paper presented at the ASME 2012 international design engineering technical conferences and design theory and methodology, Chicago, IL, August 12–15
Toh CA, Miller SR, Kremer GE (2012b) Mitigating design fixation effects in engineering design through product dissection activities. In: Paper presented at the design computing and cognition, College Station, TX, June 7–9
Ulrich KT, Eppinger SD, Goyal A (2011) Product design and development. McGraw-Hill, New York
Waterman AS, Geary PS (1974) Longitudinal study of changes in ego identity. Dev Psychol 10:387–392
Weber EU (1999) Who’s afraid of a little risk? New evidence for general risk aversion. In: Edwards W, Shanteau J, Mellers BA, Schunn D (eds) Decision research from bayesian approaches to normative systems. Kluwer Academic Press, Norwell, pp 53–64
Weber EU (2010) Risk attitude and preference. Wiley Interdiscip Rev Cogn Sci 1:263–290
Weber EU, Blais A-R, Betz NE (2002) A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. J Behav Decis Mak 15:263–290. doi:10.1002/bdm.414
Weck O, Eckert C, Clarkson J (2007) A classification of uncertainty for early product and system design. In: Paper presented at the international conference on engineering design, ICED, Paris, France, August 28–31
West MA (2002) Sparkling fountains or stagnant ponds: an integrative model of creativity and innovation implementation in wokr groups. Appl Psychol 51:335–387
Westby EL, Dawson VL (1995) Creativity: asset of burden in the classroom? Creat Res J 8:1–10
Whitson J, Galinksy A (2008) Lacking control increasing illusory pattern perception. Science 322:115–117
Wilde DJ (1997) Using student preferences to guide design team composition. In: Paper presented at the design engineering technical conferences, Sacramento, CA, September 14–17
Wolman BB (1989) Dictionary of behavioral science. Academic Press, San Diego
Zenasni F, Besancon M, Lubart T (2008) Creativity and tolerance of ambiguity: an empirical study. J Creat Behav 42:61–73
Acknowledgments
This material is based upon work supported by the National Science Foundation under Grant No. 1351493. We would also like to thank our undergraduate research assistants Arti Patel, Connor Disco, Kelly Gagnon, and Clayton Meisel and our participants for their help in this project.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Brainstorming and concept assessment instructions
1.1 Individual brainstorming instructions
Upper management has put your team in charge of developing a concept for a new innovative product that froths milk in a short amount of time. Frothed milk is a pourable, virtually liquid foam that tastes rich and sweet. It is an ingredient in many coffee beverages, especially espresso-based coffee drinks (Lattes, Cappuccinos, Mochas). Frothed milk is made by incorporating very small air bubbles throughout the entire body of the milk through some form of vigorous motion. As such, devices that froth milk can also be used in a number of other applications, such as for whipping cream, blending drinks, emulsifying salad dressing, and many others. This design your team develops should be able to be used by the consumer with minimal instruction. It will be up to the board of directors to determine if your project will be carried on into production.
Once again, the goal is to develop concepts for a new, innovative product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction.
Sketch your ideas in the space provided in the idea generation sheets. As the goal of this design task is not to produce a final solution to the design problem but to brainstorm ideas that could lead to a new solution, feel free to explore the solution space and focus on both the form and function of the design in order to develop innovative concepts. In other words, generate as many ideas as possible—do not focus on the feasibility or detail of your ideas. You may include words or phrases that help clarify your sketch so that your concept can be understood easily by anyone.
For clarity, please use the provided pen to generate your concepts (i.e., do not use pencil). Your participant number is included on each of the provided idea generation sheets. Generate one idea per sheet and label the idea number at the top of the sheet.
1.2 Individual concept assessment
During this activity, you will review and assess the concepts that you and your team have generated to address the design goal. Once again, the goal of this design problem is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. Your task is to individually assess all of the generated concepts for the extent to which they address the design goal effectively, using the following instructions (illustrated in the diagram below):
-
1.
Shuffle all of the concepts that you have generated in random order. Pass all of the designs you have generated to the team member sitting to your right.
-
2.
After receiving the concepts that were passed to you from the team member sitting to your left, rate each concept in the order that you received them using the rating table provided to you in this booklet. For each concept that you rate, record the corresponding participant’s number, idea number, and a brief description of the concept (e.g., “Double frothing attachments”). You will be given 5 min to interpret the designs that you receive without conversing with your team members. For your reference, definitions of the rating scale items have been provided below:
-
Consider: Concepts in this category are the concepts that will most likely satisfy the design goals, you want to prototype and test these ideas immediately. It may be the entire design that you want to develop, or only 1 or 2 specific elements of the design that you think are valuable for prototyping or testing.
-
Do not consider: Concepts in this category have little to no likelihood of satisfying the design goals and you find minimal value in these ideas. These designs will not be prototyped or tested in the later stages of design because there are no elements in these concepts that you would consider implementing in future designs.
-
-
3.
Repeat step 2, passing designs that are already rated to your right, and rate designs that are passed to you from the left. You will be given 5 min to rate each set of design ideas.
Finish rating all the ideas that your team has generated, including yours. You should end this activity with rating all of the ideas that you have generated.
Appendix 2: Design rating survey (DRS)
Appendix 3: Risk and ambiguity aversion measures and calculation
3.1 Measuring individual risk aversion
Risk aversion is measured using the 10 lottery questions (also found in the online survey link) used in standard behavioral economics (Han et al. 2012). The goal of the assessment is to identify the point at which the individual would take the gamble given fixed odds of winning the gamble (i.e., make the “risky” choice). Potential gamble gains vary randomly within the interval of $20.00 to $300.00.
3.2 Risk aversion questions
The following questions assess an individual’s risk aversion level. Answer the following questions regarding hypothetical lottery scenarios by specifying whether you prefer a fixed payoff of a specified value, or a gamble of fair odds with an uncertain payoff of a specified value (i.e., you are equally likely to win the gamble or lose the gamble).
3.3 Measuring individual ambiguity aversion
Ambiguity aversion is measured using the 10 additional lottery questions (also found in the online survey link) used in standard behavioral economics (Borghans et al. 2009; Charness and Grieco 2013). The goal of the assessment is to identify the point at which the individual would take the gamble given unknown odds of winning the gamble (i.e., make the “uncertain” choice). The individual’s risk aversion can then be calculated using the responses to the risk aversion questionnaire (see below for details). Potential gamble gains once again vary randomly within the interval of $20.00 to $300.00 (identical to risk aversion questionnaire).
The following questions assess an individual’s ambiguity aversion level. Answer the following questions regarding hypothetical lottery scenarios by specifying whether you prefer a fixed payoff of a specified value, or a gamble of unknown odds with an uncertain payoff of a specified value (i.e., it is not known how likely it is for you to win the gamble, as it could range from not at all likely, to extremely likely).
3.4 Calculating individual risk aversion
In order to estimate each individual’s risk aversion, the following computations will be conducted:
The gamble option payoff of the ith participant at the jth question, GP ij , is
where Gain j is the gamble gain for question j, and \(\gamma_{i}\) is the risk aversion coefficient for participant i.
The safe option payoff is then SP ij defined as
where Safe j is the safe gain for the jth question.
Then, the probability of subject i choosing the gamble at question j is linked to GP ij and SP ij through the following logistic function:
where Y ij is the response to the survey by the ith participant, for the jth question. To obtain an estimate of the risk aversion coefficient \(\gamma_{i}\), the maximum likelihood function of this logistic model is computed.
3.5 Calculating individual ambiguity aversion
In order to estimate an individual’s level of ambiguity aversion \(\delta_{i}\), the following formula is used:
where AC i is the gamble gain for the gamble question in the ambiguity aversion questionnaire that the ith participant first takes (i.e., the cutoff point where the individual prefers taking a gamble over the safe payoff). Similarly, RC i is the gamble gain for the gamble question in the risk aversion questionnaire that the ith participant first takes. This method is similar to the method used in Borghans et al.’s work (2009) except that the gamble gains are provided to participants in increments of 10 questions, instead of left up to the participant to decide.
Rights and permissions
About this article
Cite this article
Toh, C.A., Miller, S.R. Choosing creativity: the role of individual risk and ambiguity aversion on creative concept selection in engineering design. Res Eng Design 27, 195–219 (2016). https://doi.org/10.1007/s00163-015-0212-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00163-015-0212-1