Keywords

1 Introduction

At the onset of the 21st century, the narrative of an emerging global aesthetic economy signaled a pivot towards a renewed emphasis on haptic engagement. Product designers were thus prompted to focus on the role of tactile sensation and its influence on user experience [1,2,3]. Since 2019, however, the emergence of pandemic-induced social restrictions has necessitated a shift towards increased reliance on digital interfaces, resulting in a concomitant reduction in the diversity of physical material interactions [2, 3]. This transition has not only altered sensory engagement but also raised questions about the future of corporeal sensation and tactile consciousness, igniting the researchers’ interest in exploring uncharted territories of haptic cognition in this altered landscape.

Sensory-derived aesthetic experiences act as a vital channel connecting somatic sensations and self-perception to cognitive recognition [2]. The medium of tangible materials plays a crucial role, engaging individuals through direct tactile interaction, potentially culminating in a profound sense of qualia [2]. Beyond their physical properties, materials possess a more profound, almost spiritual resonance and can evoke extensive semantic networks through the interplay of physical sensations and perceptual awareness [2, 4]. Designers, therefore, must possess a profound comprehension of material experience nuances and haptic cognitive processes to enhance the intricate interrelation between materials and the human sensorium.

This study aims to analyze the emotional feeling and semantics expression from the material experience and assist educators and designers reasonably in realizing and extracting these haptic elements during the product design process. Moreover, the authors further try to implement the constructs of material experience (MX) and haptic semantics (HS) integrated into a haptic-driven design (HDD) framework, identifying the essential components for design innovation.

2 Literature Review

2.1 Material Experience

Over the years, the discussion around the “Material Experience” framework has been a focal point for scholars and designers, leading to the evolution of numerous strategies aimed at augmenting the design process with innovative material considerations [5,6,7]. Despite the product industry's primary focus on functionality for market success, a cohort of researchers advocates for a broader evaluation of materials, suggesting that they should also be appraised for their capacity to engender profound user experiences [6, 8, 9]. Within this spectrum of investigation, material-driven design (MDD) has emerged as a critical methodology, distinguished by its cohesive framework. In 2015, building upon this research, Karana and colleagues put forth the MDD approach as a means to foster the creation of innovative design processes and the conception of novel product materials that offer viable alternatives to conventional choices, such as sustainable materials, bio-based innovations, and intelligent materials [10]. The goal was to harness the potential of the MDD approach to empower designers to deeply engage with and reflect on their design practices, thereby cultivating a quest for meaningful material experiences. Karana et al. proposed four steps of the MDD method: (1) Understanding the material and extracting the technical experiential characterization, (2) Creating the vision of material experience, (3) Manifesting materials experience and matching patterns, (4) Designing product concept from material experience [10].

In the elaboration of material experience, Karana and colleagues delineated its composition through four distinct yet interconnected dimensions: the sensorial, which encompasses the aesthetic facet of experience; the interpretative, referring to the imbuing of materials with symbolic meanings; the affective, which involves the emotional resonances elicited by material interaction; and the performative, highlighting the active engagement and utility of materials in human experiences [8, 10, 11]. Specifically, at the sensorial level, individuals discern materials through tactile qualities such as hardness, texture, and temperature. Materials embody attributes like wit, allure, or generosity at the interpretative dimension, thus ascribing personality to the inanimate. Emotionally, the affective dimension acknowledges that materials can provoke a spectrum of responses, from joy to anger to surprise. Lastly, the performative aspect underscores the necessity of recognizing materials’ functional roles in fostering a holistic grasp of their utilization and significance in haptic cognition [1]. This study, therefore, integrates these four dimensions as fundamental to advancing our understanding of haptic cognition through the lens of material experience.

2.2 Haptic Cognition Model

Individuals experience sensations in response to external stimuli, a phenomenon extensively examined in studies focusing on human sensory perception and its underlying mechanisms [12]. This exploration not only elucidates the framework of the sensory system but also delves into how such stimuli trigger profound perceptions and cognitive processes [12]. Scholarly discourse since the 1980s has underscored the simultaneous occurrence of user engagement and experiential sensations during direct interactions with objects [13,14,15,16]. Based on the previous studies [1, 8, 10, 11, 17,18,19,20,21,22], this study explores the Haptic Cognition Model (HCM) [1], as shown in Fig. 1, within the realm of design theory and practice. Central to our investigation is the dynamic interaction between designers’ conceptualizations and users’ cognitive processes, mediated by haptic experiences. The HCM model elucidates two interrelated models guiding this communication process, facilitating the transition from material experience (MX) to product design through haptic semantics (HS). Material experience encompasses sensorial, interpretative, affective, and performative dimensions, while haptic semantics serves as the intermediary language for translating these experiences into tangible design elements. Through a qualitative methodology incorporating theoretical insights and empirical observations, our study unveils the nuances of the encoding and decoding processes within the HCM framework. Designers traverse three hierarchical levels—visceral (technical stratum), behavioral (semantic stratum), and reflective (effective stratum)—during encoding. In contrast, users engage in a decoding process, interpreting haptic stimuli across material sensation (aesthetic experience), semantic cognition (meaningful experience), and predicted effect (emotional, active experience). Integrating haptic semantics augments users’ cognitive engagement, fostering deeper interactions with designed products. This research underscores the transformative potential of haptic interactions in enriching design practices and enhancing user experiences, paving the way for future investigations into embodied cognition's cultural and contextual dimensions in design.

Fig. 1.
figure 1

Haptic Cognition Model (HCM) [1].

Full size image

2.3 Tactile Prompting and AI-co-Creation

Individuals apprehend the external environment through their bodily senses and articulate their emotions or ideas through verbal language or lexical constructs. Language serves as a potent tool enabling humans to imbue significance into their experiences [1]. Contextual descriptions not only reflect our comprehension of the world around us but also represent a mechanism for cultural evolution. As highlighted by Lakoff & Wehling, the meanings embedded within words and sentences transcend mere visual perception [23]. Moreover, this study champions the utilization of haptic semantics as a catalyst for enhancing designers’ comprehension and reinterpretation of materials by tactile prompting generated from the material experience.

Artificial Intelligence Generated Content (AIGC) currently finds widespread applications spanning various sectors, including media, education, entertainment, marketing, and scientific research [24]. This underscores the technology's capacity to offer users efficient, streamlined, and tailored content services [25]. AI functions as a collaborative ally, fostering a symbiotic partnership with designers and methodically guiding their innovative ideation [26,27,28]. A prime example is the case of Midjourney, where this tool has achieved significant advancements in the fundamental components of AI co-design and has seamlessly integrated into the design innovation process [24]. Nonetheless, AI-co-creation fosters interdisciplinary collaboration and communication, compensating for deficiencies in sketching abilities while concurrently reducing design time [24]. Research demonstrates that this technology enhances the precision and diversity of design solutions. Nevertheless, AI-co-creation fosters interdisciplinary collaboration and communication, bridging skill gaps in sketching and significantly reducing design timelines [24]. Prior studies demonstrate that this technology elevates the precision and diversity of design outcomes [29].

3 Method and Material

3.1 Research Method: HDD

Based on the preceding study of the Material Driven Design (MDD) and Haptic Cognition Model (HCM) [1, 8, 10, 11, 17,18,19,20,21,22], the authors further propose and conduct the process innovation in product design via the principles of Haptic Driven Design (HDD), as shown in Fig. 2. The HDD comprises six steps: (a) Choose and manifest the target material to create the personal experience. (b) Reflect and document the material experience via haptic vocabulary with sensorial, interpretative, affective, and performative expressions. (c) Employ tactile prompting with AI-co-creation in product design. (d) Conduct the design project and accomplish the qualia product. (e) Evaluate the correlation and affection between haptic semantics and design innovation. (f) Develop further Material Experience (MX) and explore the advanced HDD application.

Fig. 2.
figure 2

Six steps of Haptic Driven Design (HDD).

Full size image

3.2 Stimuli and Empirical Design Project

Based on the Haptic Driven Design (HDD) framework, this study schemes the design project, which explores and applies haptic semantics (feeling, imagery, impact, action) derived from material experience (sensorial, interpretive, affective, performative). Six kinds of materials (wood, cement, wool, RP resin, glass, and ceramic) were chosen as the stimuli by 10 designers. Then, this empirical design project developed 10 series of haptic topics and related objects, which were brought out based on designers’ material experiences. The project organizer directs these young designers to develop their haptic products, focusing on the haptic discussion according to the hand’s touch. Furthermore, these 10 extending objects are the advanced stimuli for experts’ evaluations.

3.3 Participants and Procedures

This research invites 6 experts to execute the Focus Group Discussion (FGD) and to be the consultors who are the senior experts in craft creation, product design, and art education, advise this experimental project previously, and evaluate the design outcomes later. Besides, this case study recruits 10 designers to participate in this practical project. The young designers are students from the Department of Arts and Design in the University with mature craft creation and product design experiences who design diverse products by applying the haptic cognition approach with abundant material experiences. The project director is a skilled crafter and design educator who organizes and promotes HDD applications in this case study. The HDD project comprises three stages: reflection, execution, and evaluation, as shown in Fig. 3.

  • Reflection Stage: A public exhibition was conducted to share the design outcomes of the MX-to-HS design project with the broader audience. This event also facilitated a collaborative dialogue among participants and designers on their ‘material experience’ and the extended expressions of ‘haptic semantics.’ This process guides designers in reflecting on their operational journey within the HDD framework, discussing the challenges encountered and potential solutions. Additionally, it involves suitable adjustments in further projects.

  • Execution Stage: Guide participants in extending and developing suitable haptic topics based on their prior experience in the MX-to-HS design project. It encourages the ten participating designers to continue utilizing materials from the initial phase (wood, cement, wool, RP resin, glass, ceramics) as stimuli samples for advanced material experiences. Furthermore, they are to iteratively refer to and fill out the application checklist, reviewing and revising the selected haptic vocabularies and control variables for tactile sensation. This process aims to complete the design and creation of ten sets of haptic products (PP01–PP10), which will serve as the stimuli for further evaluation.

  • Evaluation Stage: Six experts (EE01–EE06, comprising three with expertise in craftsmanship, two in product design, and one in art education) were invited to conduct empirical evaluations on the design outcomes of the HDD Project (consisting of ten haptic works). These evaluations were facilitated using the outcome assessment form, employing two assessment modes: V (visual evaluation) and S (synesthetic evaluation of sight and touch). Subsequently, an analysis and discussion of the expert evaluation data results were carried out.

Fig. 3.
figure 3

Three stages of HDD project.

Full size image

3.4 Research Instrument

This study proposed the “HDD Application Form” as the research instrument that guides the haptic-driven design procedures, as shown in Table 1. The designer records four parts of haptic semantics via sensorial, interpretive, affective, and performative keywords that connect to the description of material experiences in outer perception, extensional meaning, inner emotion, and associated scenarios. Besides, the experts evaluated the correlation between tactile prompting and results via the “HDD Assessment Form,” as shown in Table 2.

Table 1. HDD Application Form ( take D10 as an example).
Full size table
Table 2. HDD Assessment Form (take PP06-EE02 as an example).
Full size table

4 Results and Discussions

4.1 The Creation of HDD Project

This study, during the “Phase II: HDD Project” (spanning 10 weeks), recruited the same 10 designers who had previously participated in “Phase I: MX-to-HS Experiment” (spanning 12 weeks). Through the exhibition of initial outcomes, a discussion and observational learning of each other's preliminary application results were facilitated among all participants, also extending their foundational experience with the application of material experience established in the earlier experiment. This allowed for a nuanced retrospection of existing material experience and enabled the extension of haptic semantics, further expanding the development of qualia products. Consequently, the participating designers in this phase II (HDD Project) will be well equipped with essential knowledge and skills in the Haptic Driven Design (HDD) method. Based on their previous design experiences with the haptic object of cup sleeve, the designers refined the ten haptic topics developed during Phase I to “angular, gravelly, coarse, fluffy, puffy, bumpy, smooth, brittle, sandy, and gentle,” along with corresponding keywords as tactile prompting in four parts (sensorial, interpretive, affective, and performative semantics). Observations from the interaction between the project advisors and the designers revealed that repeated selection or careful adjustment of tactile prompting facilitated the designers’ ability to employ the HDD methods. This assistance was instrumental in manifesting subsequent haptic imageries, aligning more closely with the desired quality of material experience and design development vision, culminating in the completion of ten distinctive products (including pen, candle holder, planting pot, pillow, clutch, brush bottle, lampshade, jewelry box, fragrance cup, and ikebana ware), as shown in Fig. 4.

Fig. 4.
figure 4

Overview of the HDD project outcomes.

Full size image

4.2 Varying Capabilities of HDD Application

Contrasting the data from the “Evaluation results of the HDD project,” as shown in Table 3, indicates that designers exhibit diverse approaches and capabilities in translating and implementing the HDD method across different tactile knowledge dimensions. Overall, the ten designers performed better in the “Performative/ Associate Scenario “ level (mean: 4.10); conversely, they need to improve more in the transformation of the “Affective/Internal Emotion” level (mean: 3.63).

Table 3. Evaluation results of the HDD project (PP01 ~ PP10 by EE01 ~ EE06).
Full size table

Taking visual assessment (associated tactile perception) as an example, when PP07 (RP resin lampshade) received a higher evaluation (4.04), it also achieved the highest preference mean (4.17), along with the best ranking for the average change in order among the top three (3.00). On the other hand, when PP03 (concrete planting pot) received a lower recognition (2.83), it led to the lowest preference score (2.50).

Furthermore, excluding PP07 (RP resin lampshade), the data reveals a consistent trend across nine out of ten works that followed the same pattern: the means from “ Synesthetic assessment of sight and touch,” which included physical touch, were consistently higher than those means from “Visual assessment,” which only provided visual images. However, examining the overall means for tactile quality assessment across all ten products, there is an observable general improvement: the overall means across the four tactile dimensions increased from 3.72 (V) to 3.96 (S); also, the overall means of preference score rose from 3.52 (V) to 3.82 (S).

4.3 The Innovation via Tactile Prompting with AI-co-Creation

While initiating the HDD project, the director convened an FGD (Focus Group Discussion) to re-examine and modify research instruments accordingly. Consequently, an “AI-co-Creation” column was added to the “HDD Application Form” (Table 1). In the further stages of design development practice, designers participating in advanced phases were guided to establish their core haptic topics and plan the tactile knowledge dimensions during the sketching stage of design ideation. Subsequently, they integrated relevant haptic keywords with artificial intelligence image generation software (Midjourney) using Tactile Prompting, achieving innovative outcomes. Through the interactive sharing between project advisors and participating designers, the integration of “AI-Assisted Design” into the development process of qualia design was found to facilitate the external manifestation of internal tactile imageries within Haptic Driven Design (HDD). This process not only enriches the references available during design ideation but also accelerates the evolution of the design development process.

5 Conclusions

Based on the previous studies [1, 8, 10, 11, 17,18,19,20,21,22], the authors explore a design innovation connected with the material experience (haptic expression) and AI-co-creation (tactile prompting). Furthermore, the results of the Haptic Driven Design (HDD) project confirm the proposed methods and procedures. The research findings and conclusions are as follows: (a) Designers who have undergone the initial training of the ‘MX-to-HS Design Project’ will revise the haptic descriptors during the ‘HDD Project,’ making them more aptly reflect the desirable qualities and developmental vision of material experiences. (b) The precise selection or careful adjustment of haptic descriptors by designers aids in the subsequent manifestation of haptic imagery. This allows for the judgment or application of relevant reference materials or the flexible incorporation of other design methods. For instance, utilizing AI-co-creation enables more evident selection and mastery of necessary design information to optimize the quantity and quality of design sketch ideation, improving the efficiency from design development to completed work. (c) Individual designers possess distinct strengths in capturing different haptic quality aspects of design performance, which affects the preference level, acceptance, and ranking when evaluating haptic quality products. (d) In evaluating haptic quality products, the assessment that includes physical touch assistance (sympathetic haptics) yields higher ratings than evaluations based solely on visual observation (associative haptics). In other words, incorporating real material experiences in visual-haptic evaluation enhances experts’ appraisal of haptic products. Finally, the authors address that employing the haptic semantic could connect and enrich the material experience, cultural elements, and design presentation. This study proposes a new prospection for product design by enhancing the connection between material experience and haptic semantics. The authors suggest that subsequent research may focus on exploiting the haptic semantics with the cultural context and relevant application of AI-co-creation for design innovation.