Acting with Creative Confidence: Developing a Creative Agency Assessment Tool

Abstract

Universities around the world are quickly introducing new learning models aimed at developing creativity and innovation in students. A leading model is the experiential teaching of design thinking as a creative problem solving process aimed at enhancing students’ creative confidence. Although these programs exist, little is known about student outcomes. Furthermore, the criteria by which we evaluate student “success” is not well defined because these programs almost uniformly have ambiguously stated learning objectives. This research uses qualitative and quantitative data to capture and categorizes successful outcomes by examining alumni of these programs. Based on these data is a scale that measures creative agency, a fundamental outcome of teaching design thinking.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

This research focuses on a new model of teaching creativity and innovation through a design process, often called design thinking (Cross 2007). In particular, this paper focuses on one of the leading institutions in this field, The Hasso Plattner Institute of Design at Stanford University (“d.school”). Stanford University and the d.school have garnered a reputation for producing successful entrepreneurs and innovators according to a recent Stanford Entrepreneurship Survey, and there is widespread interest in replicating its educational methodology. For example, universities in Japan, Chile, Malaysia and China have founded or are in the process of creating educational programs modeled on the d.school. Corporations are also interested in teaching design and creativity; in Europe, Germany’s design group Palomar 5 and PICNIC in The Netherlands have created ambitious programs inspired by the d.school’s educational methods. While the outcomes of these endeavors have not yet been fully evaluated, previous educational trends have demonstrated that expensive, culturally grounded practices rarely work as expected when imported to new international contexts.

One reason that imported educational methods tend to “fail” in new contexts is that the expected results are often imperfectly understood and specified (e.g., “math achievement” or “innovation”). To help those interested in the d.school and design thinking avoid this pitfall, we first sought to describe the actual outcomes of the d.school’s educational practices using qualitative and quantitative research methods rather than anecdotal reports or case studies alone. Such impact questions are often motivated by goals of program evaluation, that is, identifying what is pedagogically effective for the sake of promoting or propagating it, and what is ineffective for the purpose of improving it. At this level, only what works is important; why it works is unimportant. On a more theoretical level, however, we were also interested in the psychological mechanisms underlying whatever learning happens within an educational setting and curriculum such as the d.school. Our research, therefore, occupies the intersection of psychological research and program evaluation: we wished to identify the psychological constructs relevant to effective learning experiences within a particular program (the d.school), with the goal of designing evaluations that would reflect a common learning framework that would apply across educational settings.

Naturally, describing the d.school’s impact is an impractically large research question. Any reasonably complex educational institution has myriad impacts and innumerable contributing variables. Furthermore, as institutions like the d.school are non-traditional and interdisciplinary, they often do not have specific or easily measurable target outcomes for students. This is quite different from a graduate program in Physics that teaches students to do research in one of several subfields, each with a known set of standards and criteria with which to determine alumni achievement. At the d.school, the only concrete outcome identified in the mission statement is the production of “future innovators” (d.school 2004). Therefore, a first step in investigating the impact of the d.school is selecting targeted outcomes for which research will be appropriate and useful. One outcome of the d.school, for example, could be graduates’ romantic relationships; however, from an educational perspective this is of relatively minor interest. Given that the prevailing motivations for studying and imitating the d.school stem from its apparent contributions to business and the economy (see, for example, the Design Ladder developed by the Danish Design Centre), we chose to focus on students’ professional outcomes after graduation.

A second step in designing research for understanding the impact of the d.school is narrowing the field of variables we examine as characteristic of the institution. The space of potential variables is vast, including, for example, geographical location, student population, instructor characteristics, material resources, and individual course curricula. Because the d.school is inherently concerned with learning and teaching (Beckman and Joyce 2012), and because its unique pedagogy is its most salient feature, we chose to study student outcomes related to the d.school’s signature problem-solving approach, design thinking.

2 Background

While design thinking has been defined in different ways (Buchanan 1992), (Cross 2007), d.school courses are based on a common pedagogy that focuses on an overall process with five core constructs: Empathy, Define, Ideate, Prototype and Test. On a more specific instructional level, nearly all d.school courses involve techniques such as interviewing, brainstorming, and rapid prototyping. Design thinking as it is taught at the d.school goes beyond explicit pedagogy. The goal of the d.school is to develop future innovators, who, in addition to performing discrete observable skills, also have a characteristic set of attitudes and dispositions that propel them toward creative activity and achievement.

According to a framework of a d.school teaching model (Fig. 1), mindsets and an overall sense of creative confidence are built on top of repeated practice and success with discrete techniques (Rauth et al. 2010) such as the design thinking process (dt.process) and its various associated methods. These mindsets include a bias towards action, radical collaboration, and being human centered. Other dispositions, not specified in the model but commonly promoted at the d.school, include constant reframing and rapid iteration.

Fig. 1

Model of creative confidence (Rauth et al. 2010)

The literature about design thinking outcomes is fairly sparse. A great amount of design methodology research focuses on engineering or design teams and how they perform (Eris 2004), (Brereton et al. 1996). Because the d.school does not train designers, but rather helps students from all disciplines work in a more creative way, the existing design research is not a sufficient base.

Kolb’s learning model (Beckman and Barry 2007) connects how the design process helps students develop a sense of integrated thinking. This is very valuable, but we are interested in learning how deeper behavioral aspects are affected, and a more holistic view of how individual skills and techniques learned work together to create the overall problem-solving-approach of design thinking. There is also research exploring teaching models that are similar to design thinking, such as Leifer’s (1996) work evaluating Product-Based-Learning Education, Gerber’s examination of Design-Based Learning (Gerber 2011), and the Cambridge-MIT Institute (Lucas and Cooper 2004; Lucas and Cooper 2005). Though such work sheds valuable light on how the d.school teaching mechanism affects students, it does not focus particularly on student outcomes beyond the learning experience.

In deciding on a research design, we also kept in mind that the outcomes believed to be related to design thinking as it is taught at the d.school are based on the d.school founders’ and instructors’ a priori intuitions and beliefs. It is quite likely that graduates’ skills, dispositions and attitudes are not fully specified by either the traditional design thinking model or that shown in Fig. 1. Therefore, the studies described here exhibit a tension between our a priori research focus on graduates’ professional outcomes as they relate to the d.school pedagogy, and our desire to remain open to discovering new variables of interest, in both outcomes/impact and contributing factors.

We chose individual alumni as the unit of analysis versus organizations that have d.school graduates because the d.school’s mission is that of personal behavior change and developing individual attitudes, not redefining how companies operate. That said, knowing how individuals who apply d.school learnings within a corporate context does inform the impact of design thinking on a company (Gerber and Carroll 2012).

As confidence to think and act creatively came up frequently in the qualitative data, an initial psychological construct we used to characterize this phenomenon was self-efficacy (Bandura 1977). Self-efficacy refers to an individual’s belief in his or her abilities within a particular domain, in this case, creative problem-solving. Another design thinking institution has done some work on how self-efficacy may be related to design pedagogy (Jobst et al. 2012). Self-efficacy, however, is only one part of a larger construct, agency, which Bandura (1982) defined as the means by which “people can effect change in themselves and their situations through their own efforts,” (p. 1175). Agency includes not only self-efficacy, but also beliefs about the world, context, physical and emotional states, social support, and other factors. Though it is more complex, we felt that agency better reflected the multifarious nature of the creative competencies that many d.school graduates exhibited. We defined creative agency as individuals’ capacity to effect change in themselves and their situations to support successful creative problem-solving.

Given that there is little research on the outcomes of programs that involve teaching design thinking through problem-based learning methodologies, we decided to take a wide-to-narrow approach starting with exploratory qualitative studies (an open-ended survey with follow-up interviews) and an initial conceptual model based on our observations. To test the model’s key psychological construct, agency, we did a pilot test using a new scale, revised it based on the data, and then tested a second version of the scale across multiple populations (study 3).

3 Study 1: Alumni Survey

Although we reported the background, methodology and initial results for Study 1 in last year’s report (Royalty et al. 2012), for this year, we analyzed additional data and applied a new coding scheme to several questions. The preliminary goal of this continued work on last year’s data was to generate a foundational set of descriptive results regarding d.school alumni, so that stakeholders, prospective students and the larger design education community can better understand the alumni population. The second, more research-oriented purpose of these additional analyses was to address the following new research questions:

• 1. To what extent does the d.school affect alumni professional outcomes (i.e. career choice)?

• 2. To what extent are alumni professional activities related to d.school pedagogy?

As this study was exploratory in nature, there were no a priori hypotheses. For the sake of completeness, we re-report the methods in the following section, but interested readers can go straight to the results.

3.1 Method

3.1.1 Participants

We defined a d.school alumnus as any person who had taken at least one d.school course, where a course was defined as a quarter (ten weeks) of unit-bearing instruction. Based on student records, the participants we surveyed were a close approximation of the entire population (approximately 670 alumni). The response rate was 28 % for a final sample of 184. Among those reporting gender, 56 % were men (n = 73). Nearly all participants had been graduate students at Stanford University (some had finished their degrees and others had not) and were affiliated with various schools and programs, including Business, Law, Arts & Sciences, Education, Medicine and Engineering. As the d.school officially offered courses beginning in 2006, participants had been out of school and employed full-time from zero to a maximum of 5 years. The majority (83 %) had graduated in 2008 or after.

3.1.2 Procedure

Given the large number of potential participants in this study, for ease of data collection and analysis, we chose a survey methodology and used online survey software (Qualtrics). Alumni were identified through student records and contacted via personal email as well as a general Facebook announcement (it is possible that some people received more than one notification or invitation to participate in the study). Invitations were sent in May 2011 and the survey was open for 2 weeks. Participants were free to take the survey at a time and place of their convenience. There was no explicit incentive to participate; however, respondents had the opportunity to join a mailing list that would notify them of upcoming alumni events.

3.1.3 Materials

All of the questions on the survey were designed specifically for this research. Participants were asked about the current and previous occupations and occupational plans, how they applied what they learned at the d.school in their professional lives (Table 1), and demographic questions. So as not to bias respondents towards reporting specific techniques or general dispositions, we did not use the term “design thinking” in any questions; instead, we referred to “what you learned at the d.school”. The survey questions also did not offer any definition or examples of “applying” what was learned at the d.school; respondents were free to respond based on their own interpretation of this term.

Table 1 Selected survey questions on outcomes of d.school pedagogy

3.2 Results

3.2.1 Alumni Professional Outcomes and Their d.school Experiences

By comparing participants’ self-reported planned occupation when they began their graduate studies with their current occupation, we found that among those who had a planned occupation, a majority experienced a career change. Table 2 shows the frequency breakdown of all respondents, including those who had career plans and those who did not when they began graduate study. Looking at only those participants who did have a planned employment field and/or role when they began at Stanford (n = 122), 62.3 % of these (n = 76) reported a difference between the job/career they had planned on when they began their graduate study, and their current employment.

Table 2 Categories and frequencies of career plans and outcomes for d.school alumni

A subset of participants (n = 70) reported a career change and also answered the question, “Thinking about the difference (if any) between your expected occupation when you started at Stanford, and your current occupation, how much did your experience at the d.school play a role in this change?” The mean response was 2.47 (SD = 1.46), which corresponds to approximately halfway between “A moderate amount” (scale point 2) and “A lot” (scale point 3).

Of the 76 career changers, 34.2 % (n = 26) explicitly (and spontaneously) mentioned their experiences at the d.school as a reason for this outcome. Even among those who did not report a career change or did not have an original plan, some spontaneously mentioned their d.school experience as having an impact on their professional path.

To understand any potential role that the d.school had on alumni career choice and professional outcomes, responses were coded from the question, “If your current job is different from what you expected to do when you started at Stanford, what factors played a role in this change?” Among participants who reported working in an occupation that they had not planned on when they matriculated, the most common reason was a change in professional interests or desires (Table 3). Furthermore, approximately half of those who reported a change in interests or desires for their job/employment (19 out of 37) explicitly mentioned the d.school as a reason for this change.

Table 3 Reasons for occupational change among d.school alumni

For those who mentioned the d.school as a reason for their change in occupation, the overall theme was that various aspects of the d.school experience (e.g., a particular class, teamwork) enhanced their self-understanding or their understanding of professional roles. Table 4 gives some examples of these open-ended responses.

Table 4 Example responses to question about change in occupation relationship of alumni professional activities to d.school pedagogy

To understand how specific pedagogical elements of the d.school’s instructional methods may have affected alumni professional outcomes, we focused on the open-ended questions; “What are the most important effects or outcomes of your experience at the d.school?” and “What are some examples of how you applied what you learned at the d.school in your professional life in the last month?”. As not all participants responded to these questions, the number of respondents for this analysis was 131.

From these responses, we coded for elements of the d.school design process, defined a priori. We also included teamwork as an a priori coding category, given its centrality to the design thinking learning process. We also coded for dispositions that correspond to key tenets of the d.school curriculum. “Creative confidence” is a new term that we used to describe a theme of feeling comfortable with creative endeavors and a sense of ability and self-efficacy in the creative domain (Table 5).

Table 5 Frequency of d.school alumni mentioning d.school pedagogies

3.3 Discussion

Re-analyzing the survey data with a particular focus on career changes and how alumni viewed the contribution of d.school pedagogy revealed moderate evidence for the claim that educational methods at the d.school have an impact on graduates’ professional outcomes. Across all participants, nearly twenty percent brought up the d.school as a factor in their professional paths and current occupation. It is important to note that these participants mentioned the d.school’s influence without prompting – the question about why they were working in their current occupations did not mention the d.school, and was placed before (and on a separate page from) the question that specifically asked about the role of the d.school. Furthermore, the results show that alumni perceive their d.school experience as a causally contributing factor in their occupational choices.

As participants varied in what they found most significant about their experience at the d.school, we cannot conclude that it was, in fact, d.school pedagogy rather than other factors that drove these outcomes. The data on alumni use of fundamental design thinking methods, however, do support a hypothesis that alumni apply d.school lessons in professional settings. Of the six a priori categories that we coded for, empathy, ideation and prototyping were the most commonly mentioned. This is unsurprising as these techniques are in sharp contrast with more traditional work methods, and apply in many situations. The methodology of “defining” a problem was mentioned less frequently, probably because it is a more subtle, procedural step that can be difficult to describe. It was, however, somewhat surprising that only a fifth of participants talked about teamwork, as this is central to nearly all d.school experiences.

Arguably the most important outcome of the survey was the emergence of several new coding categories. We view these as overall psycho-behavioral “dispositions”; in particular, the themes of creative confidence, comfort with seemingly negative states (such as failure or ambiguity), and having a bias towards action (rather than intellectual reflection or rationalization) were frequently observed. In order to better understand how alumni may have developed these holistic dispositions, in addition to more explicit design thinking/problem-solving skills, we decided to do in-depth follow-up interviews with a subset of the alumni surveyed in Study 1.

4 Study 2: Alumni Interviews

As Study 2 was described in the previous report (2011), we will just briefly review the methodology and major findings in order to describe the larger research arc. Sixteen d.school alumni (ten women) from Study 1 were interviewed in person or via teleconference (Skype). Seven participants currently worked in business (including healthcare, technology and entertainment), three were self-employed or entrepreneurs, three worked in education, two in consulting or research, and one was in engineering. The interviews were semi-structured and were 45–100 min.

The interviews probed how “successful” alumni viewed what they had learned at the d.school and how that may have influenced both their career choices and their professional activities. By “successful”, we merely meant those who said that they chose to, and were able to, use what they learned at the d.school (which they defined for themselves) in their working life. On a high level, we wanted to see how former d.school students described their learning path and their subsequent behaviors in order to better understand how the d.school may be contributing the formation of “future innovators”. More specifically, we focused on their observable behaviors and dispositions, and how alumni perceived the d.school as shaping these outcomes.

The overall result of the interviews was a better understanding of how some d.school alumni develop and demonstrate creative confidence. Beyond simply learning basic design thinking processes and techniques, participants who reported using what they learned at the d.school had a strong desire and confidence to actually apply skills and methods in their workplace. This can be a daunting and risky endeavor in some traditional work contexts, where normal or known behaviors are expected and job responsibilities are tightly codified. In the strongest case – seen in several interviewees – creative confidence in the professional realm even extended to using the design thinking process to launch or change one’s entire career. For example, one participant left her job in engineering to pursue teaching, and explicitly stated that it was what she learned at the d.school that gave her the confidence to “try out” or prototype a new career, even though it meant taking significant professional risks.

Another, unexpected example of how alumni demonstrated creative confidence that came up in multiple interviews was how they built creative environments around themselves at work and in their personal lives. This often took the form of manipulating physical space to allow for more creative working styles, and in some cases, making the space similar to the interior of the d.school. Some participants also reported teaching colleagues about design thinking as a way to enable better collaborations and facilitate their own creativity. Across most of the alumni we interviewed, the rich descriptions of how they used what they learned at the d.school went far beyond simple applications of basic techniques or tools such as brainstorming or rapid prototyping. Instead, their stories reflected an overall approach to work and life that was informed by dispositions to solve problems in an innovative manner, and propelled by the confidence to do so in multiple ways.

Successful alumni seemed to share a set of behavioral patterns and dispositions that went beyond what we had captured in Study 1. Although we could observe those characteristics in interviews, we wanted to define a single psychological construct that could be measured accurately and efficiently. In Study 3, we attempted to narrow and quantify what was observed in the survey and interviews, by creating a scale that attempted to measure the new construct of creative agency.

5 Study 3: Competency-Based Creative Agency Scale (CBCA)

In developing a scale measuring the impact of a design-thinking based pedagogy, we revisited the data from the surveys and interviews and coded for key competencies frequently demonstrated by successful alumni. Again, we defined “successful” as those who reported remembering and using “what they learned at the d.school” (self-defined) the most within our sample. We then created the scale directly from those factors. The questions were worded in an abstract manner rather than by referring to specific scenarios, in order to limit the influence of previous experience or gender differences. The key factors that emerged were:

• Sources (gathering information from external sources)

• Comfort (with ambiguity)

• Mastery (of one’s own creative process)

• Environment (developing creative environments)

• Anti-perfectionism (reducing a sense that everything must be perfect)

• Prototyping (developing a culture of prototyping)

• Perseverance (increased in the face of failure)

• Facilitation (confidence to lead a creative process)

• Openness (to changes in thought, direction, beliefs, et cetera)

• Process (being able to describe one’s own creative process)

• Creative Output (solving problems in creative ways)

5.1 Method

5.1.1 Participants and Procedure

• 1. Current d.school students (General). The scale was implemented in the Autumn quarter of 2011 with current d.school students drawn from two classes. The first was a large, introduction to design methodology course (n = 72) (95 % of class) and the second was a more specialized course exploring the intersection of design and society (n = 13) (76 % of class). The questionnaire was given at the beginning (September) and end (December) of the academic quarter and no incentives were offered for participation. Of the 85 pretest participants, 68 completed the posttest.

• 2. Current d.school students (Education). Thirteen students in a small, Education-focused d.school course took the questionnaire as a pretest and posttest at the beginning and end of the Spring 2012 academic quarter. The pretest was given at the beginning of (April) and the end (June) of the Spring quarter; eight students took the posttest. A $15 gift card was offered for participating.

• 3. Non-d.school students (Education). Twelve Education students who were not taking the Education-related or any other d.school course prior to or during Spring 2012 were recruited as a control group for the d.school/Education group. These students completed the questionnaire online.

• 4. Workshop participants: Business executives. Participants in a 3-day design thinking workshop for business executives received an email with a link to an online questionnaire, 2 weeks prior to the workshop in March 2012. 57 participants took the pretest (98 % response rate). To date, six participants have taken the posttest, which was also provided via email, 10 weeks after the workshop. Those who took the post-test received a t-shirt.

• 5. Workshop participants: Business executives and teachers. Participants in a 3-day design thinking workshop for in-service teachers and business executives took the questionnaire on paper, at the beginning and end of the workshop (n = 62) (52 % response rate at pretest), which took place in May, 2012.

5.1.2 Materials

There were 11 to 14 Likert-scale questions on the CBCA scale; all were created for this study. Eleven items concerned self-assessed confidence related to a set of general competencies in the area of creative problem-solving (Table 6). The CBCA-related Likert-scale items were presented as a group and preceded by the question, “How confident are you that you could…”. Unlike the items in Study 3, the new questions were written to be sufficiently general so as to apply to nearly any situation or professional domain. There were five response categories: “Not at all confident” (scale point 1), “A little confident” (2), “Moderately confident” (3), “Very confident” (4), and “Completely confident” (5). There were no “Don’t Know” or “Not Applicable” response choices; however, participants were free to leave any item blank.

Table 6 Items on the competency-based creative agency scale

After implementing the 11-item questionnaire with the first group of participants, all of whom were d.school students, we added three new items on topics relatively unrelated to competency-based creative agency. These items concerned comfort with technology, artistic ability, and goal achievement. They were added to provide data on whether any apparent outcome of d.school study was specific to the design thinking competencies we found in earlier studies, or a more general positive impact.

5.2 Results

While there was no overall effect of gender, there was some evidence of a trend for gender to interact with time, such that women started out with higher CBCA but ended at a similar mean score, F(1, 53) = 3.04, p = .087 (Fig. 2). The effect size for this possible interaction was, however, quite small, with a partial eta-squared of .05.

Fig. 2

Mean competency-based creative agency, by time and gender

There were too few participants in the non-d.school student “control” group for statistical analysis; however, among the eight who took both pretest and posttest, there was no apparent change in mean CBCA. At pretest, the control group mean was 35.25 (SD = 5.31); at posttest, the mean was 36.12 (SD = 3.68). Figure 3 plots the means and standard errors for the d.school students versus non-school students at pretest and posttest.

Fig. 3

Mean competency-based creative agency, by time and group

For the 23 teacher/executive workshop participants for whom we have both pretest and posttest scores, mean CBCA at pretest was 32.91 (SD = 7.13) and mean CBCA at posttest was 38.65 (SD = 8.50). A paired-samples t-test showed a significant effect of time, t(22) = −4.50, p < .001. Figure 4 shows the pretest and posttest scores for each participant – only three decreased in their CBCA scores, while the remaining 20 increased.

Fig. 4

Mean competency-based creative agency, by sample and time

5.3 Discussion

Study 3 tested a new scale, competency-based creative agency, with three main samples: d.school students, similar students not taking d.school classes, and teachers and executives who took a 3-day design thinking workshop. As the scale items were based on empirical research on the applied skills of successful creative problem-solvers who were d.school alumni, it was hypothesized that d.school experience would be associated with greater CBCA (see Fig. 4). The results provided evidence for this relationship, as d.school students increased in CBCA from the beginning to the end of the course, across two different academic quarters and in three different d.school classes. In one of those quarters, a small group of non-d.school students also took the scale; they did not show any significant change in CBCA. While a larger control group would, of course, provide better support for the hypothesis, these data do suggest that the increase in CBCA seen among d.school students was not merely due to maturation or demand characteristics of the scale items.

Results from the teacher/executive sample also supported the hypothesis that d.school experience increases CBCA. Although the design thinking workshop was only 3 days, participants reported a significant increase in CBCA from the beginning to the end. Data from this sample also showed that the scale could be used with a non-students sample and retain its internal validity as well as its usefulness in showing change over time.

It is interesting to note that the average pre-test score of the teachers and executives was slightly lower than the average pre-test score of the non-d.school students, which in turn was marginally lower than the average pre-test score for d.school students. There was, however, an apparent difference between the mean starting CBCA for d.school students and teachers/executives, with d.school students reporting higher CBCA at pretest. Taking a design thinking course shows willingness to take risks by studying a topic that is relatively new and not a regular part of any degree program; a higher pretest score among these students, then, is not surprising.

6 Implications

The three studies reported here have led us toward a larger model of how individual creativity develops and is subsequently expressed. One of the original concerns of our research was to investigate how the d.school and other educational institutions can foster the kinds of skills that lead to real-world innovations. Our initial model of creative outcomes, as observed in our research, includes four constructs: self-efficacy, agency, output and impact (Fig. 5).

Fig. 5

A model of multiple creative outcomes

Creative self-efficacy is belief in one’s own creative abilities, and it is developed in many ways, such as mastery experiences (successfully completing creative activities) and vicarious experiences (learning from others who are creative). Self-efficacy is the most important part of creative agency, which goes beyond mere belief in oneself and extends to real-life application of creative problem-solving. Those who have high creative agency are capable of producing creative output – real work, artifacts, and ideas. Creative output determines creative impact, this is the change made due to a creative effort.

This model is not intended as a linear or temporal path, as there is multi-path feedback among the nodes. Self-efficacy naturally leads to greater agency, which may result in greater output, which strengthens both self-efficacy and agency. This can result in more creative output, which can strengthen agency that can lead to even more creative self-efficacy. The model is also not intended to be exhaustive: given the work being done to better understand structures of the creative brain, we may soon include segments to the left of creative self-efficacy, representing more internal structures. Likewise, there may be multiple segments to the right of creative impact, when one considers the impacts of not just individuals, but also institutions and other large systems.

7 Work in Progress

We are currently in the early pilot stages of three studies aimed at the creative output node of our model, that is, the ideas, work, artifacts, procedures and other “deliverables” that creative agents make.

8 Conclusions and Future Work

Understanding the impact of any educational institution or method is a daunting task. By focusing on the unique problem-solving approach taught at the d.school, design thinking, and grounding our work in an understanding of the characteristics that d.school alumni report, we have begun to provide evidence for the impact of d.school pedagogy on a newly identified construct, creative agency. Instead of focusing on more traditional outcome measures, such as salaries, patents, or awards, we chose to look at the competencies, capacities and habits that those demonstrated by alumni who say that they use what they learned at the d.school in their work and lives.

What emerged from our qualitative research is a sense that many d.school alumni are equipped with a strong sense of their own ability to be creative in problem-solving contexts (self-efficacy), and that this propels them to act to change their own thinking and their environments in order to perform better as creative problem-solvers (creative agency). These actions are characterized by multiple competencies, first learned and practiced at the d.school, and then carried into the workplace. These include beliefs (anti-perfectionism, failure as opportunity), knowledge (of one’s creative process), dispositions (towards prototyping, radical collaboration), and skills (teaching others, shaping one’s own environment). We also observed that these capacities were largely domain-agnostic, and were reported by teachers, engineers, entrepreneurs, and even chefs in equal measures.

In our quantitative research, we moved towards an efficient and accurate way to represent and measure the outcome we observed among the alumni, by designing and testing two scales. While the first did not function well as a singular-construct scale, we did see interesting results, such as, more experienced d.school students said that they were more comfortable starting a company than students with less d.school experience. Our second scale, which built upon the lessons of the first, was more successful at providing a valid and reliable measure of the impact of d.school pedagogy. By testing it with several populations, we found initial support for its use beyond the d.school. In the near future, we hope to repeat our studies with larger and more robust control groups, and implement the scale in other educational programs and institutions, both similar to and different from the d.school.

With more evidence for the validity and reliability of the Competency-based Creative Agency scale, we would like to eventually expand the research to its potential predictive validity, its neurological correlates, and ways to use it in controlled experiments. For example, using the CBCA in the workplace may yield insights into how workers with high creative confidence are recognized for their creativity and the benefit they provide to their companies (predictive validity). We are also collaborating with another research team on fMRI studies that incorporate the CBCA scale. Finally, to truly test the effects of the d.school pedagogy, we would need to run experimental studies that manipulate what we believe are key aspects of design thinking as it is learned at the d.school, and then measure how these components affect CBCA. This will take us closer to understanding the contribution of the d.school to educational methods, as well as how the successes of the d.school may be not only repeated, but also enhanced, at other organizations throughout the world.