Purpose

Despite an increase in complex problem-solving methods applied within technology-enhanced K-12 classrooms today that include case-, project-, or problem-based learning (Glazewski & Ertmer, 2020), the instructional strategies that K-12 teachers use to teach design are not widely known (Crismond & Adams, 2012; Li et al., 2019). Although dedicated books and articles on the topic of design processes and methods exist (Cross, 2000; Jones, 1992; Reigeluth, 1999; Schön, 1987), there is an ongoing “need for greater consideration of the pragmatic creation and use of methods to support designers” (Gray et al., 2022). To date the instruction of design strategies has largely focused on adult or undergraduate learners (e.g., Wrigley & Straker, 2017) rather than children. Teachers are in need of support to overcome difficulties in using design as a component of teaching and learning (Waite et al., 2020). One comprehensive framework that addresses dimensions of design practices and instructional approaches specific to K-16 educational contexts is the Informed Design Teaching and Learning Matrix (IDTL Matrix; Crismond & Adams, 2012); however, since its publication, no formal iteration of the IDTL Matrix has occurred despite the emergence of theoretical publications (Hrastinski, 2020) and empirical studies where the IDTL Matrix served as an evaluative or conceptual framework at the university (e.g., Adorjan & de Kereki, 2013; Goldstein et al., 2019; Karabiyik et al., 2020; Taleyarkhan et al., 2018), high school (Goldstein et al., 2019), middle (Goldstein et al., 2021), and elementary level (English, 2019). In this article, we put forth a revised IDTL Matrix to serve as a guide for developing students’ design strategies in K-12 learning contexts. A key premise of the IDTL Matrix framework is that children have the capability of becoming “informed designers” (a level between that of a novice and an expert) prior to university studies in design. This is critically important as design faculty have emphasized the need for the intentional development of students’ design skills prior to college (Scupelli et al., 2020) and teaching design can lead to improved learning outcomes in domain-specific areas, such as programming (Waite et al., 2020).

The research question posed in this study focused on identifying the assumptions and limitations of the IDTL Matrix through practice-based insights of high school teachers in varying design domains (e.g., computer science, 3-D design, digital design). Specifically, we sought to examine how the specified and enacted pedagogies (Nind et al., 2016) of high school design educators aligned to or diverged from the instructional framework for teaching informed design. To say this in another way, by observing how high school design educators facilitated student learning in design, we wanted to identify the specific design strategies that design teachers used to support student learning in order to parse out which strategies were represented within the IDTL Matrix and which strategies were omitted. To carry out this study, we conducted a multiple case study analysis. Data collected included teacher interviews, multiple classroom observations with each interview, and analysis of student artifacts.

We identified the reported and observed instructional practices that aligned and diverged from the recommended teaching strategies within the IDTL Matrix at a categorical level. We found three major design strategies relevant across teaching contexts that were not present in the IDTL Matrix: collaborative learning, peer communication, and the integration of tools and materials for purposes of designing. In this study, all participants directed students to work collaboratively and to employ self-regulation across design teams, skills considered essential to collaborative learning (Hadwin et al., 2010). The reasons cited by teacher participants for group-oriented assignments included (a) time efficiencies, (b) modeling professional practice, and (c) support for students who lacked pre-requisite skills. For peer communication, participants encouraged communication via peer review, informal peer assistance, and modeling of self-critique. Additionally, students were directed to (a) learn how to use tools and materials properly; (b) select appropriate design tools based on design intent/available resources; and (c) demonstrate technical skills.

Based on the research findings, we argue that revisions to the IDTL Matrix are merited. Modifications could include the identified instructional practices not currently within the framework. Additionally, the revisions to the IDTL Matrix could further enhance its relevancy to high school design domains outside of engineering education contexts.

Contribution to the field

First, this study addresses a problem in the lack of scholarship centered on instructional practices specific to teaching students how to design at the high school level in the United States. Although design education has been examined at the university level in the U.S. and at the K-12 level in international contexts, study of the teaching practices specific to U.S. K-12 design contexts has been limited, possibly due to the fact that design is not currently a content area in which K-12 educators can earn teaching licensure or certification, or because design has yet to be recognized as its own tradition (Nelson & Stolterman, 2012). This study was undertaken to (a) provide insight into current instructional practices of design educators in high school contexts and (b) address how high school design educators perceive that design should be taught, including examination of factors that influence those perceptions. This study addresses, in a small way, an overall dearth of studies describing experiences and content that support teaching in multidisciplinary design domains (Arya et al., 2021; Brophy et al., 2008). It also provides insight into student learning in material- and tool-rich (see Dalsgaard, 2017) learning contexts.

Second, this study recognizes the lack of preparation provided for teaching in design contexts and offers insights into what might constitute design pedagogy in teacher education, and professional development based on the practices and perceptions of current high school design educators. Since licensure and certification for teaching design in K-12 contexts do not currently exist in the U.S., this study can inform preservice teacher education programs that may be interested in preparing educators to teach design across varying domains. Since the first design experiences that students encounter can be formative in their development of a perspective toward a discipline (Davis, 2005), teacher preparation in K-12 design education is imperative. This study clarifies the instructional methods that high school design educators employ, allowing teacher education programs to review which teaching practices design educators value and why.

Third, including design domains like programming and software development, computer science, web and digital communications, visual arts, technology and film in those used to examine the teaching strategies of high school design educators, we expect to improve the existing IDTL Matrix created by Crismond and Adams (2012). By using the IDTL Matrix as a lens for examining high school teachers’ instructional practices, we sought to present a more inclusive representation of what design education entails in the U.S. within varying design domains. Since high school students are expected to design as part of learning objectives specified within national standards, such as the Common Career Technical Core and Next Generation Science Standards, this study advances conversations regarding national core standards, which have particular meanings and consequences for teaching and learning (Delandshere & Petrosky, 2004).

Literature

Given that design is a key term used throughout this study, it is necessary to establish a definition for the term based on design theory. Based on our analysis of the positions of various design theorists (Alexander, 1964; Friedman, 2003; Gibbons, 2013; Lawson, 1997; Nelson & Stolterman, 2012; Schön, 1983; Simon, 1996), for purposes of this research study, design can be described as an intentional act of creation, often enacted through a process specific to a design tradition and context, to achieve a particular aim, goal, or purpose. A challenge in presenting this definition of design, like any definition of design, is that it can be challenged on the basis of its scope (e.g., too broad) and utility (e.g., too vague). We present this definition of design as a way to orient readers to design who may come to this research study without a familiarity of design as its own tradition (Nelson & Stolterman, 2012). Design, like the fields of art, science, politics, and technology, has its own traditions (Nelson & Stolterman, 2012) and, as an academic subject matter, it has its own methods, domains, principles, and practices. It is not to say that the definition that we present is right or wrong, as no “singular satisfactory definition” of design will encompass the “complexity of the matter” (Lawson, 1997, p. 31); however, if design as a term remains ambiguous, it can create confusion (Schön, 1983). Therefore, we put forth a definition of design to provide a point of context for how the term is used in this study.

Although no singular definition of design is definitive, one’s conception of design always has consequences (Nelson & Stolterman, 2012). Previous research has found there is “substantial variation in the conceptions held by both students and teachers about what design is and how it should be learned” (Davies & Reid, 2000, p. 178). In the case of design educators, who are tasked with teaching a subject matter that can lead to a career in a design field, conceptions of design may not be explicitly articulated, but embodied in other ways. For example, design educators, like educators in other fields, make instructional design decisions regarding curriculum design, lesson development, implementation [or non-implementation] of standards, instructional practices, assessment creation, and feedback decisions (Porter, 2002) within a subject area that is unique in its own right. Those aspects of teaching design, to date, have not been examined in high school design education contexts. By understanding what high school design educators think about teaching design and what influences those perspectives, scholars can better understand how that subgroup conceive of design and how those conceptions are embodied within their instructional practices.

Design education at the college level

Given the unique context of design education in high school settings, it is worthwhile to examine design education at the college level as a point of comparison. Design programs in higher education tend to reflect established design domains. According to Friedman (2003) design typically falls into six general domains: the natural sciences, humanities and liberal arts, social and behavioral sciences, human professions and services, creative and applied arts, and technology and engineering. Depending on the nature of the project or problem to be solved, design “may involve any or all of these domains, in differing aspects and proportions” (Friedman, 2003, p. 509), which highlights how design domains can and do overlap. In addition to the six design domains presented by Friedman, Simon (1996) noted that “design is at the core of all professional training” (p. 111), meaning that fields such as business, education, law, and medicine also engage in educating students in design. The scope of design’s reach is broadened even further when considering the view of design presented by Schön (1983), who stated that any professions “engaged in the act of converting actual to preferred situations are concerned with design” (p. 77). The range of design programs at the post-secondary level, which include architecture, urban design, regional planning, engineering, product design, graphic design, media science, computer science, industrial engineering, and engineering specialists (Mitchell, 1993; Nelson & Stolterman, 2012; Schön, 1983), reflect the breadth of university design programs.

Evolution and growth of college design programs

Although design programs are typically domain specific, design professions continue to evolve (e.g., user experience designers, interaction designers, digital design engineers). Within the past decade, specialized interdisciplinary design programs have emerged (e.g., the Hasso Plattner Institute of Design at Stanford [d.school]), alongside MBA and MA design degree programs (Hanover Research, 2013) in areas such as Design Leadership (e.g., Johns Hopkins Carey Business School, Maryland Institute College of Art), Strategic Design and Management (e.g., Parsons The New School for Design), and Design Strategy (e.g., California College of the Arts).

The field of design education has grown considerably within the past three decades, a trend reflected by the increasing number of design degrees conferred. According to the U.S. Department of Education’s National Center for Education Statistics (NCES), the following fields of design had degree completion rates (pre-baccalaureate certificates to post-doctoral studies) increase at an annual compound growth rate of 8.8% between the period of 1987 to 2012 (Hanover Research, 2013, pp. 14–15):

  • 50.0401 Design and Visual Communications, General.

  • 50.0402 Commercial and Advertising Art.

  • 50.0404 Industrial and Product Design.

  • 50.0406 Commercial Photography.

  • 50.0407 Fashion/Apparel Design.

  • 50.0408 Interior Design.

  • 50.0409 Graphic Design.

  • 50.0410 Illustration.

  • 50.0411 Game and Interactive Media Design.

  • 50.0499 Design and Applied Arts, Other.

A report of emerging trends in U.S. and international post-secondary design schools indicated four key findings regarding design education (Hanover Research, 2013). The report noted that (1) design education curriculum should provide design students with foundational study in design theory and history, including precedent from other design disciplines; (2) as design education practices shift from a focus on artifacts and objects to human-centered interactions and systems, design students require new skills to address problems that encompass social challenges, cultural values, and technological opportunities; (3) design education would remain a hands-on studio-based and hands-on, although the focus would shift from assignments to problems of inquiry; and (4) there remains a continuous need for design students “to communicate outside their discipline” (Norman & Klemmer, 2014, p. 4), reflecting the trend for the collaboration across disciplines present in programs of design at the master’s level (Friedman, 2012) and undergraduate level (Self & Baek, 2017).

Design education at the high school level

Design has been described as an “interdisciplinary, integrative discipline” (Friedman, 2003, p. 508) that requires an element of design expertise on the part of the instructor to teach (Cross, 2004; Lawson & Dorst, 2009). Design education at the high school level within the United States is structured differently than university coursework in design. High school courses with a focus on design are primarily situated within either Career/Technical Education (CTE) or in the academic category of the Fine Arts (see Fig. 1) within the Secondary School Taxonomy (SST) structure. The SST was developed in the 1980s as a framework for analyzing and aggregating high school transcript data, and was revised in 2007 (Bradby & Hudson, 2007) to reflect changes within the CTE categories.

Fig. 1
figure 1

Top-Level Categories of the 1998 Secondary School Taxonomy (SST) Structure, with the 2007 Career/Technical Education Revisions Noted

Full size image

As noted in Fig. 1, the categories of Secondary School Taxonomy (SST) place career/technical education (CTE) and the Fine Arts in two fundamentally different areas within a school curriculum structure. Contrary to common perception, the field of CTE has expanded beyond traditional vocational education to include coursework in technology and career fields, including STEM subjects (NASDCTEc, 2012).

Career/technical education

High school career technical education pathways with design standards

The types of CTE courses that high schools offer and how those courses are organized into subjects, sequences, and programs is a decision determined at the state level. According to the National Association of State Directors of Career Technical Education Consortium (NASDCTEc, 2012), 46 states use Career Clusters as a foundation for their CTE standards or as an organizing framework for communicating about their CTE programs. CTE standards are defined by the NASDCTEc as “clear expectations of what students should know and be able to do at the end of a CTE program or course” (p. 9). Across the 16 Career Clusters within CTE programs, there are six Career Pathways where design is a component of an end-of-program standard at the highest level (see Table 1). It should be noted that the organization that oversees the National Career Clusters Framework seeks to revise its cluster names and groupings, a process will be finalized in 2024 (Advance CTE, 2023).

Table 1 CTE career pathways, organized according to corresponding career cluster, with selected end-of-program design standards noted
Full size table

The Career Pathways where design performance standards are situated include: Programming & Software Development Pathway, Web & Digital Communication Development Pathway, Design/Pre-Construction Career Pathway, Engineering & Technology Career Pathway, Visual Arts Career Pathway, and A/V Technology and Film Career Pathway. Each of the six pathways reference design in at least one overarching performance standard, with four of six Career Pathways including design in at least two top-tier standards. Although the performance standards themselves do not specify how teachers are to design their instructional activities, teachers must interpret the standards, along with the term design.

Instructional practices in design teaching and learning at high school level

Teaching design at the high school level requires content-specific knowledge and familiarity with processes inherent to design (Lammi et al., 2018). For high school design educators, a challenge that they face is that they may not be familiar with particular practices and the culture of specific design domains (Lammi et al., 2018), nor are they necessarily prepared to provide high school students with design experiences that are representative of the fast-paced technological growth within industry (Woods & Berry, 2022). Additionally, a national design organization (e.g., AIGA, IDSA) specific to high school teachers does not exist, limiting the professional development opportunities related to design teaching and learning at the state and national level. Not surprisingly, design educators often teach from what they know, based on their own experiences, passing along the same information, books, and resources used when they were students themselves (Berry, 2022).

The instructional practices in design teaching and learning at the high school level are in need of further research and support (Lammi et al., 2018). Brosens et al. (2023) recently conducted a systematic review that examined how design students should learn through teaching and learning activities. Although not specific to pre-college settings, the authors detailed a trend of design teachers as a facilitators of learning rather than experts of domain specific knowledge and the shift in studio environments to collaborative, peer learning. Within studio teaching, Brosens et al. (2023) cited authors who advocated for the iteration and refinement of solutions (Cennamo et al., 2011), creativity (Thoring et al., 2018), and fostering peer learning (Dominici, 2017; Micklethwaite & Knifton, 2017). Within specific learning activities, the systematic review identified critique, research through design, and sketching and prototyping as valued areas in need of support. Brosens et al. (2023) concluded that as it pertains to teaching and learning activities in design education, recommendations remain “quite vague” and “most research limits their conclusions to what should be instead of explaining how to get there” (p. 677). In other words, efforts need to be made to identify best practices for introducing and teaching design, particularly at the pre-college level (Lammi et al., 2018).

Research questions

To better describe how design is taught in high school contexts and what might influence particular approaches to design teaching, we posed the following research questions in this study:

  1. 1.

    How are the instructional design practices of a purposeful sample of high school educators teaching in varying design domains similar to or divergent from the instructional framework for teaching informed design (i.e., design capabilities between novice and expert)?

  2. 2.

    What do high school teachers of design within career/technical education and media arts think about teaching design, and what influences those perceptions?

Conceptual/theoretical framework

The theoretical framework guiding this study was the educational/instructional theory of informed design proposed by Crismond and Adams (2012). These authors describe the concept of informed design as applying to a designer whose level of competence is somewhere between that of a novice and expert designer, also known as an “expertlike novice” (Bereiter & Scardamalia, 1993) or “competent performer” (Dreyfus & Dreyfus, 2005). Since expertise can take 10 or more years to accumulate (Hayes, 1989), it can be argued that achieving the performance level of an informed designer is an appropriate end point for students in K-16 learning settings, as they are “not likely to accumulate the level of authentic practice necessary to acquire expert-like behaviors” (Crismond & Adams, 2012, p. 743). To state this in another way, children are not expected to become experts in designing during their school age years, yet they do not need to remain novices, as design strategies can be taught in such a way that students can become informed designers.

In order to bring their theory to K-16 instructional practice, Crismond and Adams (2012) coupled their notion of informed design with four characteristics of instructional design theory described by Reigeluth (1999). The first characteristic of an instructional design theory is design orientation, which is intended to offer guidelines as to what method(s) to use to obtain a given goal. A second characteristic is the identification of methods of instruction, including the situations in which particular instructional methods should and should not be used. A third characteristic of an instructional design theory is that it can be broken into more detailed component methods, providing additional guidance to educators. Fourth, the methods presented within a given instructional design theory are probabilistic rather than deterministic. In other words, presented methods in an instructional design theory are intended to increase the chances of obtaining a goal, but will not ensure goals will be obtained by students.

Building from a meta-analysis created by Duncan and Hmelo‐Silver (2009) that examined performance dimensions associated with foundational learning progressions (e.g., learning while designing, making and explaining knowledge driven decisions), Crismond and Adams (2012) created a conceptual and instructional framework known as the Informed Design Teaching and Learning Matrix (IDTL Matrix; see Table 2). Within the IDTL Matrix, key student performance dimensions of design practices within K-16 engineering and STEM educational contexts are articulated. The IDTL Matrix highlights nine design strategies that are grounded in design theory to illustrate its conceptual base. The design strategies presented within the IDTL Matrix are coupled with contrasting pattern statements that juxtapose beginning designer practices with the practices of an informed designer, previously described. The authors also suggested instructional approaches for each design strategy to facilitate the learning goals related to specific design practices. In Table 2, the nine design strategies (Column 1) are presented along with the contrasting patterns and statements of how beginning designers versus informed designers (Columns 2 and 3) carry out those strategies, which are aligned relevant learning goals (Column 4) and instructional approaches (Column 5) that teachers can use.

Table 2 The Informed Design Teaching and Learning Matrix
Full size table

Adapted from “The informed design teaching and learning matrix,” by D. Crismond and C. Adams, 2012, Journal of Engineering Education, 101(4), 748–749. Copyright 2012 by John Wiley & Sons, Ltd.

Crismond and Adams (2012) argued that their work was both an educational theory and an instructional theory of informed design, yet we argue that the IDTL Matrix is a conceptual and instructional framework. This is because articulating design strategies and presenting the ways in which those strategies could or should be taught are not representative of educational and instructional theory which encompass one’s theoretical and epistemological perspectives.

The IDTL Matrix (Crismond & Adams, 2012), as a conceptual and instructional framework, was created to address two specific needs. First, the authors cited that the field of design teaching and learning lacked “a coherent representation of design pedagogical content knowledge (Design PCK)” (p. 739). Their intention in the creation of the IDTL Matrix was to depict Design PCK (see Shulman, 1986), which they defined as “content-specific, specialized teacher knowledge associated with instructional techniques that are particularly suited to teaching effectively with design tasks” (p. 740). The second articulated need for the creation of the IDTL Matrix was to bridge educational research in design with K-16 teaching practices in a form that was useable for everyday classroom teaching, although it was not meant to represent an ideal of design pedagogical content knowledge. Ultimately, the framework was intended to (a) aid in instructional reflection and (b) serve as a practical instructional tool to improve students’ design learning.

Method

Since Crismond and Adams (2012) proposed guidelines for improving students’ design learning, we sought to examine the ways in which teacher participants’ instructional practices aligned to or diverged from the IDTL framework and how those results would challenge or refine the IDTL Matrix. The strategy used to do so was multiple case study analysis (Stake, 2005). A case study, according to Stake (2005) is not “a methodological choice but a choice of what is to be studied” (p. 443). The focus of qualitative case studies is not the methods used, but the case itself (Stake, 2005). In this study a “case,” a term that is often taken for granted in social science research (Ragin, 1992), describes a contemporary phenomenon (how teachers teach high school students to design) that is examined in depth and in a real world context within the scope of an empirical inquiry (Yin, 2014). Case selection, according to Stake (2005), should be based on “various interests in the phenomenon, selecting cases of some typicality but leaning toward those cases that seem to offer the opportunity to learn” (p. 451). We sought to select instrumental cases of practicing secondary teachers from six different Career Pathways: Programming & Software Development, Web & Digital Communication, Design/Pre-Construction, Engineering & Technology, Visual Arts, and A/V Technology and Film (CCTC; NASDCTEc/NCTEF, 2012).

According to Stake (2005) an instrumental case is examined “to provide insight into an issue or to redraw generalizations” (p. 445) and in a multiple case study, an instrumental study is extended to several cases. The rationale for the number of multiple-case designs to be selected should be based on literal and theoretical replications, with a minimum of two cases within each subgroup Yin (2014). Within this study, the cases examined were intended to provide insight into the instructional practices of secondary educators in differing design domains in order to conduct cross-case theme analysis. Given the identified Career Pathways, we sought to recruit, at a minimum, six total teacher participants across two different career pathways. We examined and compared, therefore, multiple cases of design teaching across various design domains within one large urban public high school district in the U.S.

Setting and participants

The participants within this study were practicing high school teachers of design, specifically those who taught within one of the six career pathways of the Career/Technical Education (CTE) Career Cluster classification system where design is noted as an end-of-program standard at the highest level within the Common Career Technical Core (CCTC; NASDCTEc/NCTEF, 2012). Identified Career Pathways included Programming & Software Development, Web & Digital Communication, Design/Pre-Construction, Engineering & Technology, Visual Arts, and A/V Technology and Film. Teachers of media arts are situated within the Visual Arts Career Pathway based on the structure of the Secondary School Taxonomy system (Bradby & Hudson, 2007). To identify potential teacher participants, we reviewed Chicago Public Schools websites for teacher profiles in the identified subject areas and contacted potential candidates by email. Teachers who indicated an interest in participating in the study were asked to confirm the design courses they taught through reply email, in case of inaccurate website information. Of the 45 teachers contacted, 15 individuals responded to the invitation. Nine individuals expressed interest in the study. Seven of the nine interested participants committed to initial interviews, with six participants completing all study procedures (see Table 3).

Table 3 Demographics of Teacher Participants in the Study
Full size table

Data collection

Given the critical nature of case selection, setting, and participants of the study, the data collection procedures and methods of analysis were carefully considered. Face-to-face interviews with six teacher study participants served as the initial source of data collection. Interviews provided these high school teachers of design the opportunity to speak to their perceptions of teaching design and their instructional practices. The semi-structured questions (see Online Appendix A) posed to the teachers during the interviews were crafted with the IDTL Matrix (Crismond & Adams, 2012) as a guiding framework, although the questions were designed such that new practices beyond the IDTL Matrix were possible. For example, for the design strategy of troubleshooting, teacher participants were asked, “When students become stuck on a design problem/challenge, how do you help them troubleshoot the situation?” followed by the question, “What do students tend to do when they become stuck on a design problem/challenge?” Depending upon participants’ responses, follow-up questions were posed to elicit more details specific to the suggested teaching strategies (i.e., diagnostic troubleshooting, cognitive training in troubleshooting, troubleshooting stations, teacher modeling of troubleshooting).

The purpose of the initial interviews was to allow participants to speak to their design teaching through concrete examples of their classroom instructional practices (e.g., If you could speak to one project in any of the classes that you teach, could you lead me through that project from beginning to end?). The semi-structured questions were intended to draw out various instructional approaches employed by the teachers. Responses provided by the teachers offered insight into the design strategies the teachers sought to foster within their students during their instruction.

To increase the trustworthiness of the data collected during interviews, a minimum of three half-day observations were conducted in the classroom of each study participant. In addition to detailed field notes, each teaching observation was audio recorded to supplement the notes, and digital images of student work were captured to provide visual documentation related to instructional practices. At the discretion of each teacher participant, digital artifacts were collected for data triangulation, including instructional materials, unit/lesson plans, and student work examples. Details of the context of each case were recorded within the field notes, as activities are influenced by contexts (Stake, 2005). Exit interviews with teacher participants followed the conclusion of observations. Table 4 summarizes the data collected over the course of the study.

Table 4 Summary of Data Collection Sources by Type and Period of Their Collection
Full size table

Data analysis

Coding and analytic memo writing were carried out within Atlas.ti concurrently with ongoing examination of the data corpus. The central analysis focused on the alignment and divergence of teachers’ instructional practices with the IDTL Matrix. Emergent patterns, categories, themes, concepts, and assertions were recorded through extended analytic memo writing. A code book (see Online Appendix B) was developed from the nine design strategies presented within the IDTL Matrix and sub-codes reflected suggested learning goals, teaching strategies, and patterns of informed design. The nine design strategies from the IDTL Matrix that were coded included: Understand the Challenge, Build Knowledge, Generate Ideas, Represent Ideas, Weigh Options & Make Decisions, Conduct Experiments, Troubleshoot, Revise/Iterate, Reflect on Process. All data that appeared to fall outside the suggested categories of the IDTL Matrix were categorized into broad themes (e.g., peer communication, group collaboration, tool and material use) in order to account for all the data within the study. Code frequency did not necessarily indicate significance (Saldaña, 2015). While looking within and across cases, the prominence of themes in each case were estimated (Stake, 2006). Themes were discussed with a second coder and an instructional design scholar to clarify theme prominence, an indication of case relevance (Stake, 2006).

The coding of individual teacher participant data occurred in the order in which data were collected. For each teacher participant within the study, the transcripts of pre-observation interviews were coded first and followed by the audio recordings of classroom observations as they transpired in the field (see Fig. 2). Data analysis for each teacher participant concluded with the coding of the exit interview transcript. Field notes were coded in correspondence with the observation audio and interview transcription data. Documents and images were not coded independently as the documents were intended to support the analysis rather than be the subject of the analysis.

Fig. 2
figure 2

Screenshot of Coding of an Audio File within Atlas.ti

Full size image

Themes emerging from the data that were relevant to the participants’ instructional practices, but that were not encompassed in the coding scheme developed from the IDTL Matrix, were also recorded. For example, during interviews with teacher participants, the teachers would speak about administrative support, existing curriculums they were expected to follow or develop, student expectations or engagement with the course content, and other contextually relevant information. To address these types of pedagogical activities, we used the dimensions of pedagogy identified by Nind et al. (2016) in the subset of codes that were developed. Nind et al. (2016) argued that pedagogy can be viewed as:

  • Specified (i.e., what is assumed to be an accepted or appropriate way to teach and learn within a particular domain of learning);

  • Enacted (i.e., how specified pedagogy is interpreted and carried out by an individual who has unique experiences, competencies, and power dynamics with others);

  • Experienced (i.e., how the pedagogy is experienced by teachers and learners in ways that encompass affect and transformation)

Nind et al.’s (2016) dimensions of pedagogy provided a means to categorize pedagogical activities cited by teacher participants that existed outside of the IDTL Matrix. As an example, when a teacher participant cited curriculum or standards they intended to teach, this was coded as #pedagogy: specified (unit/project goals), whereas if a teacher participant shared how they assess student work, it would have been coded as #pedagogy: enacted (assessment).

Major findings

For all design strategies presented within the IDTL Matrix, the areas where instructional practices aligned and diverged from the recommended teaching strategies at a categorical level were identified. In the section below, the alignment and divergence of each design strategy are detailed. A summary across all strategies is presented in Table 5.

Table 5 Summary of the alignment and divergence of the instructional design practices of teacher participants specific to IDTL matrix
Full size table

Summary of the alignment and divergence of instructional practices for understanding the design challenge

The ways in which teachers’ scaffolded elements of Understanding the Design Challenge varied within field observations; however, at the high school level, comprehending the problem statement and problem framing/scoping were highly verbal in nature and incorporated peer interactions through both large and small group-based discussions. For example, the Physical Computing teacher explained how he used discussions as a way to support students thinking through problem statements as follows:

It’s [my course] completely inquiry-based. It’s no longer about showing them syntax.... The first thing we do when we approach a new topic is talk about why that topic needs to be examined... Why do we need something like inheritance? You know, what does it bring to the table that we didn’t have before?... I do pair-share, then group-share, then, you know, think-pair-share. They journal, then they share in a pair, then share out with the whole group... The whole idea is that with that model, everyone gets to contribute in some way, even if it’s contributing just into our journal. So, whenever we approach a new topic, the first thing they do, I ask them to do is they answer some short question about, you know, “In the world around you, how are products created? How do you interact with your TV?” Questions like that get them thinking about, um, why we create the things—, why do we create the things that we have? Why is a smartphone the way it is? Why is your remote control at home the way it is? And usually questions like that, it gets them thinking about the design process, um, for the physical device, but that also can be connected to how programming works. You know, why do we design software packages the way they are, why do we design programs the way they are? Um, and once that discussion is had in class and I feel like the kids have a pretty good grasp of the whys, then we start to look at some code. (initial interview, Mr. Law).

In certain instances, problem framing also encompassed hands-on activities, such as photography “cold reads” and time-based design challenges with limited resources. Only one teacher participant reported using formal design briefs, which was a suggested teaching strategy within comprehending the problem statement. Functional descriptions were not observed across contexts. Instructional practices that diverged from the IDTL Matrix included considerations of user experiences and user interactions when undertaking new design challenges, as the IDTL Matrix only suggests that informed designers consider user needs.

Summary of the alignment and divergence of instructional practices for building knowledge through research

Across teacher participant contexts, teachers directed their students to conduct focused information searches. Teacher participants engaged their students in studying prior art to varying degrees within design activities including withholding prior solutions, withholding parts of prior solutions, and modifying prior solutions. Product dissections and reverse engineering were observed from all teacher participants. For example, the Digital Design I teacher, required his students to reverse engineer creating a commercial before they wrote, recorded, and edited an entire film. A form of product dissection, he explained, “We start by doing found footage. So, I give them, like, an old car commercial and they have to appropriate it and recontextualize it” (initial interview, Mr. Cihlar).

The use of case-based reasoning with catastrophic and other examples and writing a product history report were not observed nor were they addressed by teacher participants except for Ms. Smith who used case-based reasoning in two of her courses. Researching users as an instructional strategy was observed in only one teacher case. Strategies observed but not suggested within Building Knowledge through Research included examining primary sources, researching current trends, speaking with experts, and sharing background experiences.

Summary of the alignment and divergence of instructional practices for generating ideas

All teacher participants within this study incorporated brainstorming and divergent thinking into their instructional practices. However, the degree to which students generated ideas in each learning context varied. Although teachers would encourage students to generate a range of ideas and to consider alternate viewpoints, idea generation proved to be, at times, challenging for students. To aid in idea generation, teachers directed students to work both independently and in groups. For example, a computer science principles teacher, Ms. Smith, shared that she had her students brainstorm independently for homework, and during the next class meeting, students would share their ideas within a small group brainstorming session. Ms. Smith also had students incorporate idea sketching as part of the group brainstorming exercise, using both hand sketching and digital wireframe. Generative database searches and starter versus final project challenges were observed among select teacher participants and employed in specific instances. The instructional strategies of constraint relaxation and dream designing were not observed.

Summary of the alignment and divergence of instructional practices for representing ideas for deep inquiry

During classroom observations, all of the suggested instructional strategies for Representing Ideas for Deep Inquiry were employed in different ways; however, instructional practices varied depending upon the focus of the class and the tools available to the students. Teacher participants, across all contexts, engaged students in messing about with given models, usually through exploration of materials or software features. For example, in Digital Design I, Mr. Schroeck described in his initial interview what he called “bootcamps,” in which students completed a two-day exercise where they were introduced to a software-based tool that they could explore within a small assignment. An example of a bootcamp exercise cited by Mr. Schroeck involved students exploring how to seamlessly embed a pop culture figure into existing video footage. Virtual drawing and computational modeling were techniques that students applied in the development of 2-D and 3-D physical representations of designs (see Fig. 3), but models were not limited to software-based tools as students incorporated physical materials (e.g., cardboard, glass, clay) and electronic materials as well (see Fig. 4). Descriptions of design ideas were often coupled with sketching. Structured reviews of design ideas, which we refer to as formative feedback, transpired in all teacher participant cases through verbal presentations to the entire class, small group discussions, or personal feedback sessions with teachers. Student use of artifacts and gestures as stand-ins for drawings were also observed, although evidence of such were limited to the images and field recordings, as only audio data of dialogue took place.

Fig. 3
figure 3

Student-Created Representation of a 2-D Interactive Animation Created in Processing

Full size image
Fig. 4
figure 4

Student-Created Prototype of a Glove Intended to Aid Those with Limited Movement Through the Integration of a Flex Sensor

Full size image

Summary of the alignment and divergence of instructional practices for weighing options and making decisions

As students weighed options in their designs, they would often seek input from their teachers. The ways in which teacher participants responded to the students varied as some teachers encourage students to pursue more challenging or easier design paths. While there was some difficulty in seeing how design judgments transpired within classroom interactions, students were willing to share what they did within their design work and why. Explanation-based designing was valued by all teacher participants. For example, Mr. Law (the physical computing teacher) shared that the outcome of his students’ designs do not hold the same weight as their reasoning throughout the project. He stated that he tries not to:

. . . penalize them for their design. Um, ‘cause it’s not part of what they’re trying to learn. Not yet, right? For a 15, 16, 17, 18-year-old, I think it’s more important that I give them feedback on their process to their final design because, at this point, it’s not important to have the most efficient program. . . . For them, what’s more important is . . . how did they get from point A to point B? If their reasoning is sound, no matter what their decision is, I think that’s a good decision. (initial interview)

Design values, such as pragmatism and simplicity, and guidelines, while not overt, were embedded across teaching contexts. Emotions and their role in decision making were not explicitly addressed by teacher participants, although teacher participants with backgrounds in the fine arts did encourage their students to express themselves within the work they created. Decision diagrams were not observed. The only formal decision-making records kept by students were in courses taught within IB contexts as two teacher participants had their students keep “change logs” throughout the progression of their designs. For example, Ms. Smith explained (initial interview) that it took her students time during the school year to develop the capacity to explain changes within their designs well. In her words, she stated:

I do hold them accountable to how did your final project/product hold up to your original plan? And if it doesn’t, why did you change it? What changed? Why did you add this element or remove this other element? . . . They have to do the reflection on it. And it’s just getting them to realize that, “Okay, why did I change this?” And start with that slowly and then they start talking about it. And so now, at halfway through the year, they can actually tell me, “Well, I wanted to do this, but I saw this instead,” you know. “I saw that this one particular thing instead. This interested me more.” (Smith, initial interview)

Summary of the alignment and divergence of instructional practices for conducting tests and experiments

Of all the design strategies within the IDTL Matrix, conducting tests and experiments in the context of optimizing a desired outcome was coded less frequently than the other design strategies. Only two teacher participants, Mr. Law and Mr. Cihlar, asked their students to carry out projects that required formal experimentation of their designs (see Fig. 5). All teacher participants encouraged students to test out their own designs during development. For example, in one observation that I conducted in the media computation course, I watched as Student A tried to experiment with the command else in his code to see if it worked and it did not. By process of elimination, Student A determined the variable he needed was if. After reviewing his partner’s code, Student A shared the findings from his experiment with his partner, Student B. This example is representative of several exchanges I observed during studio work times in which informal testing resulted in peer advice given. Across teacher participant contexts, there were no instances in which teachers presented unfamiliar devices to their students for product comparisons or for investigation/redesign tasks.

Fig. 5
figure 5

Image of Testing Conducted Outside to Assess Efficacy of a Student-Designed Arduino Project

Full size image

Summary of the alignment and divergence of instructional practices for troubleshooting

Students engaged in troubleshooting efforts in every class observed across all teacher participant contexts throughout the duration of data collection efforts. If there is one constant in design education, it is the diagnostic troubleshooting of one’s own work (see Fig. 6). Students possessed a strong capacity for identifying problems that emerged within their own work in class, although there were instances in which students did not want to convey the issues they encountered with teacher participants. Students attempted to diagnose problems independently, with the help of peers, and/or sought out the assistance of their teacher. In certain instances, students were able to explain the cause of issues and, in other cases, the teacher and/or peers pointed out why the problems were happening. Remedying of issues was, in all instances, carried out by students with varying degrees of verbal guidance from teacher participants. Teacher modeling of troubleshooting involved questions posed to aid students whereas other teacher participants provide more pointed direction and helpful suggestions. For example, a teacher participant, Ms. Novak, detailed her instructional approach to troubleshooting issues within her 3-D Design course, stating:

If you can see around this room [gestures to the classroom space], I don’t even have a desk. . . I’m constantly kind of going around and like, um, just providing feedback and when students are like, “I don’t know how to do this,” I’m like, “Well, what have you done so far? Well, why don’t you think that’s successful? . . . I really try to massage out of them as much information and try to get them to resolve as much as they can. And then when they’re like, “I just can’t figure it out,” then, um, my whole thing is like, if it’s like a drawing schematic, like I bring over a separate piece of paper and like I walk them through how to do it. If it’s an actual material schematic, then I bring over an extra piece of material and I show them. Instead of manipulating their form, like I give them that one-on-one construction and feedback in order for them to be like, “Oh, okay. I get it now.” You know? Because for me, I remember being a student and I would HATE when a teacher would come over and be like, “Let me fix that for you,” and I would be like, “Damn it!” Like, you didn’t let me do it myself! You know? And so it’s really crucial for students to have that ownership over their things. And what I’ve found is, um, those students who aren’t like that . . . it becomes almost like a learned helplessness, like, “Can’t you just do it for me? Can’t you just do it for me?” And I’m like, “No, the point is for you to learn how to do it yourself. So here I can show you. I can give you this model to follow.” And like, usually, there’s like a little bit of saltiness after that, but then, they’re able to figure it out or at least they’re trying harder.

Fig. 6
figure 6

Example of Diagnostic Troubleshooting of Vibrations in Student Designed Game

Full size image

Cognitive training in troubleshooting was not observed as an instructional practice, as teacher modeling of troubleshooting took precedence. Troubleshooting stations were not observed within any context, which is understandable as students had enough troubleshooting tasks to contend with throughout the duration of their work.

Summary of the alignment and divergence of instructional practices for revising and iterating

All the teaching strategies presented within the Revising and Iterating strategy of the IDTL Matrix, apart from design storyboards, were observed across all teacher participant contexts within this study. Design storyboards were used primarily for planning purposes related to video creation, web design applications, and animation design, although alternate methods of having students record project progression were noted. Project and time management scaffolds served a critical role within classroom instruction. Teacher participants distributed written checklists, monitored work electronically, and offered verbal reminders to keep students on track with the progression of projects. Certain teacher participants spoke of the need to shorten or lengthen the amount of time allotted for project-based work. For example, in interviewing Ms. Wozniak, she shared the case of one student who wanted to add an additional component to his animation project:

He realized it was just going to be way too complicated, and would take way too much time. So, being able to say, “Okay, I can't do that, I have two days left, that’s not going to happen” [is an important skill]. Rather than trying to start something, and not being able to really implement it.

In this quote, Ms. Wozniak emphasized that a project management skill that students should possess is the ability to judge what one can reasonably complete within a given timeframe. Like time management skills, risk taking and iteration were strongly encouraged by all teacher participants. Instruction and scaffolding for systemic design were also observed within verbal teacher-student interactions and usually brought about through progress check-ins. Instructional practices that diverged from the suggested approaches of Revising and Iterating included peer feedback as a means of guiding student iterations on work, as well as the incorporation of smaller foundational exercises to facilitate project progression. For example, the entire first semester of the Physical Computing Lab was dedicated to building skills related to electronics and wiring through the use of SparkFun electronic kits (see Fig. 7), which were foundational scaffolds necessary to carrying out more complex group projects in the spring.

Fig. 7
figure 7

Example of a SparkFun electronic kit assembled by students

Full size image

Summary of the alignment and divergence of instructional practices for reflecting on process

The suggested instructional strategies within Reflecting on Process observed in teacher participants contexts included the use of design journals for idea generation, note taking, and reflection purposes. The comparison and contrasting of student work transpired through “gallery walks,” although teacher participants were more inclined to have students share their work through final presentations and engage in peer critique sessions. In certain instances, teacher did incorporate computer-supported structured reflections. The divergence of instructional practices from Reflecting on Process included the sharing of students’ designs to the larger school community through public displays of students’ work. Despite the integration of various forms of peer critique, the IDTL Matrix does not suggest critique as a studio-based pedagogy that can enhance reflective thinking, encompass structured prompts/questions, and bring about reflective social discourse. Students did not create any formal portfolios in any of the teacher contexts observed, nor did students watch videos or attempt similar design tasks done by others.

Additional findings

In addition to the findings represented within Table 5, three major design strategies emerged from the review of teacher participants’ instructional practices that were not encompassed within the IDTL Matrix: peer communicative acts, collaborative learning, and the integration of tools and materials for purposes of designing.

Teacher participants cited peer communication as a desirable skill for their students to learn, and they encouraged it through peer review, informal peer assistance, and modeling of self-critique. Since the majority of students’ design work was completed during class time, the studio orientation of the classroom learning environment fostered an ideal setting for teacher-student-peer communication. Students relied on each other heavily for troubleshooting efforts and would pose questions to nearby peers during independent work time. Teacher assistance was typically sought only when neighboring peers could not assist. More knowledgeable students, who embodied the role of a peer teacher, were observed “floating” to aid in troubleshooting efforts, assisting their classmates and/or the instructor, as requested. Students used asynchronous communication (e.g., Google Docs, Slack) to discuss design projects outside of class time, when needed.

Across all teaching contexts for this study, teacher participants encouraged and directed students to work with partners and within groups for design activities. Students started out the school year working more independently, but as the year progressed, teacher participants provided students with opportunities to design with their classmates. Each project observed required collaborative efforts and self-regulation across design teams, skills considered quintessential to collaborative learning (Hadwin et al., 2010). The reasons for group-oriented assignments cited by teacher participants included (a) efficiencies of time, (b) a desire to reflect professional practice where team-based work projects would be expected, and (c) as a means to provide support to marginalized groups and to students who lacked pre-requisite skills. Teachers incorporated peer grading/evaluations, designated roles, and soft competitions into collaborative learning activities to ease potential problems associated with group work.

The integration of tools and materials for purposes of designing was evident across all participant cases. Teacher participants within the study noted the necessity for students to (a) learn how to use the tools and materials properly; (b) select appropriate design tools based on the intent of their designs and the resources available to them; and (c) acquire and demonstrate technical skills within a project-based context. Access to tools and materials within and outside of school factored into the instruction that teacher participants designed. Teacher participants encouraged students to select their own tools and materials for design projects, although more commonly, teacher participants directed tool choice (e.g., students were required to use Processing to create their animations). When students had a choice of design tools, given the group orientation of projects, tool selection was a negotiated decision.

Teachers’ perceptions of teaching of design

The findings from the analysis of teachers’ perceptions of teaching design, the second research question posed, indicated that background experiences and formal education experiences shaped their conceptions of design, but also that those conceptions are not fixed. For example, educators with art backgrounds created design projects for self-expression, whereas computer science teachers took a different approach, stressing coding first, design second. Factors that influenced teachers’ perceptions are summarized in Table 6.

Table 6 Summary of the Factors that Influenced Teacher Participants’ Perceptions of Teaching Design
Full size table

Discussion

Based on the research findings, we argue that revisions to the IDTL Matrix are merited. Modifications could include the identified instructional practices not currently within the framework (see Table 5). Since the IDTL Matrix only considers student design activities at the individual level, we assert that the IDTL Matrix should encompass collaborative design learning given the group orientation of project-based tasks observed. Furthermore, we argue that the IDTL Matrix should be expanded to encompass tool use as a design strategy, as scholars have argued that functional tools can frame and shape acts of inquiry (Dalsgaard, 2017) and students’ design processes (Chao et al., 2017).

There is merit in refining instructional frameworks to study design education; these tools may be based in theory, but are not theories themselves. While they do not guarantee a clear and direct relationship between educational research and teacher practice (Farley-Ripple et al., 2018), they can serve to structure research and support educators. A challenge for scholars who create research products that are intended to inform practice, like the IDTL Matrix, is that a bidirectional process between researchers and practitioners (e.g., principals, teachers, school administrators, interventionists) ideally should exist where “research can inform practice and practice can inform research” (Farley-Ripple et al., 2018, p. 242). Proposed theory intended to improve K-12 settings needs to be explicitly examined against teachers’ instructional practices.

Implications

The suggested revisions to the IDTL Matrix are intended to generate renewed thinking and conversation about how design is taught in the United States at the high school level. By examining teacher participants’ instructional practices and their perceptions on teaching design at the high school level, we have identified potential ways in which the IDTL Matrix could be improved. Additionally, revisions to the IDTL Matrix could further enhance its relevancy to design domains outside of engineering education contexts at the high school level. The results of this study may also inform individual design educators who seek to reflect on design in the context of their own instructional practice. The teacher participants in this study noted that they rely on support mechanisms through community partnerships, professional development, and administrative support, which are potential arenas where dedicated space for reflective practice on design and designing could be cultivated, perhaps using a revised version of the IDTL Matrix as a guide. Indeed, the IDTL Matrix was intended to help teachers reflect on their own understanding of design content knowledge and to consider how the suggested instructional practices might apply to their own learning contexts.

Future research

Teachers do undertake design in ways similar to other designers, but need training and support (Bennett et al., 2017). For a revised version of the IDTL Matrix to be used in practice, its “dimensions of use and the potential gaps in assumptions and perspectives” between teachers and researchers needs to be articulated (Farley-Ripple et al., 2018, p. 241). In other words, additional research and scholarly dialogue is needed to investigate how educators would anticipate applying the framework and its relevancy to current issues faced within one’s own teaching context.

High school design education

Design educators at the high school level may benefit from examining strategies within the revised IDTL Matrix that they do not currently use, with specific attention to their own perceptions of how design should be taught. For example, accounting for user experience and user interactions as part of design activities was not frequently observed, yet teachers may find that approach critical to teaching their students how to design. Another implication is that design educators may not trust the IDTL Matrix if it does not include the strategies that they currently practice. Discussions with practicing high school design educators through professional development opportunities could provide a means for feedback on suggested revised strategies within the IDTL Matrix or bring to light additional omissions that are critical to address.

Education field as a whole

The relevancy of the work of Crismond and Adams (2012) at this moment in time is particularly salient. In many K-12 contexts, students are being encouraged to engage in ill-structured problem-solving (Glazewski & Ertmer, 2020; Law et al., 2020; Tawfik et al., 2020), meaning-making (McTighe & Silver, 2020), and individual knowledge representation, which naturally provide students the opportunity to design things for themselves. The IDTL Matrix might be a starting point for educators across domains who are incorporating design-based activities into learning without much support regarding the designing portion of those activities. As an “interdisciplinary, integrative discipline” (Friedman, 2003, p. 508), design requires an element of design expertise on the part of the instructor to teach the subject well (Cross, 2004; Lawson & Dorst, 2009).

Limitations

Due to the qualitative nature of this study, the findings from this study are not reflective of the instructional practices or perspectives of a large population of secondary teachers working across various design domains. The small numbers of teacher participants in this study worked at high performing schools overall, so the findings are not indicative of low-performing school settings in other areas of the same district. As relates to the IDTL Matrix, the findings presented in this study are based on moments observed, but for the IDTL Matrix to be improved, an ongoing scholarly conversation regarding potential changes to the framework is required. For example, if additional teachers participated in this study, it would have been possible to identify different approaches to teaching design and thus a more comprehensive iteration on the Matrix could have been presented. It should be noted that it was never the intent of Crismond and Adams (2012) to have the IDTL Matrix represent any teacher’s design pedagogical knowledge; rather, it was “designed to help teachers reflect on and develop their own Design PCK” (p. 779). However, the findings from this study are intended to provide insight into the actual instructional practices that aligned to and diverged from the IDTL Matrix as a means of refining the framework further.

Another limitation of this study is that Crismond and Adams (2012) specified that the teaching and learning strategies within the IDTL Matrix are specific to STEM and engineering design contexts, with specific attention given to engineering education. Certain teaching strategies within the IDTL Matrix are more commonly found within engineering domains than in other design domains. As a result, we did not suggest that any suggested instructional practices be removed from the IDTL Matrix. This is a limitation of this study as the approach is additive without being subtractive, but again, to take out teaching strategies suggested within the IDTL Matrix is contextually dependent on the specific learning context, as certain design strategies would be more or less applicable depending upon the design domain. For example, conducting formal experiments may be extremely relevant in engineering education, but the approach may not be readily practiced or practical within the field of instructional design.