Keywords

1 Introduction

Due to the increasing competitive pressure in product development, user-centered design can provide a decisive competitive advantage [1]. The competitive advantage is that products are optimized to the user requirements [2, 3]. To be able to do this, for the UCD it is necessary to identify and analyze the user needs [3]. Studies with subjects (user studies) are required for this. This can generate knowledge about user needs. To properly understand the needs of users, investigations with individuals must be organized in such a way as to ensure this. The theory, how user studies are designed, does not belong to classic engineering education. Therefore, it often presents a challenge. Product developers usually have no previous experience with studies in which subjects participate and no experience of what influence the study conduct has on the results of the study and therefore what special attention must be paid to. For engineers which perform user studies this unknowledge in this field can easily lead to errors, high effort, and a reduction in the quality of studies. Without consideration of humans and their influence in studies, significant changes in results [4], and misestimations may occur during studies. Developers are often unaware of this and draw false conclusions from user studies for product development. This may lead to incorrect identification and analysis of user needs and thus to the development of products that do not meet the customer´s core needs. Properly conducted and interpreted user studies lead to more usable products [5]. Traditionally, there is a great deal of knowledge, including methods and experiences of user studies in other disciplines that focus on humans. These areas include psychology and social sciences [6]. This knowledge of other areas of expertise is hardly used in product development. It is useful and necessary for developers to have relevant knowledge for designing effective user studies to perform user studies effective and efficient.

Since product development belongs to engineering sciences and is thus considered an interdisciplinary science, there is nothing to prevent the use of knowledge and methods from other sciences and specialist disciplines [7]. A main reason for the low use of the knowledge of other disciplines is the lack of awareness that this knowledge and experience exists and that product developers could benefit from it. Another important reason is the difficult access to this expertise and methodology for product developers. This is because the available literature is very extensive and extends over different areas of expertise, making it difficult to identify suitable basic literature as a non-expert. The degree of detail of available literature is also often inappropriate, which makes it difficult to identify suitable literature. Specific literature addresses the needs of the product developer with a lot of off-topic information, relevant content is difficult to identify. Non-specific literature can do little to help product developers design, perform and evaluate user studies. On the other hand, access to existing literature is hampered by the large amount of specialized vocabulary in the literature of other sciences, as well as the terminology used inconsistently across the different disciplines and research focus, making the content difficult to understand.

What is lacking is a support that structures, compacts and helps to understand existing knowledge from other areas of expertise that could be useful to the product developer, enabling him to perform user studies more effectively and efficiently. This will allow product designers to develop products which better meet user needs.

In this publication, therefore, the aim is to answer the following research questions:

  • Question 1: What knowledge from other disciplines is relevant for user studies in user-centered product development?

  • Question 2: What should an assistance look like to provide product developers with knowledge in a structured, compact, and understandable way?

To achieve this, the following approach was applied: Specific problems, typical sources of error, and challenges for user studies in product development were identified by interviews. Subsequently, disciplines that focus on empirical studies with a focus on humans were considered. These were analyzed for solutions to the specific problems, typical sources of error, and challenges previously identified. Solutions proposals have been developed and additional literature content has been added to ensure the quality and effectiveness of an investigation. This content has been structured, merged, and transferred into a process that supports the developer along with a survey. And finally, the whole process was opened up with the knowledge that was provided: understanding, consistency, detail, and language checked and adapted.

2 Methods

What knowledge from other disciplines is relevant for User Studies in user-centered product development?

To answer the first question, it was first clarified how user studies currently take place in user-centered product development and what challenges product developers face consciously and unconsciously. It was then possible to infer what knowledge is relevant to solve these difficulties and challenges. To identify specific problems, typical sources of error, and challenges of product developers at user studies, telephone interviews with seven researchers were conducted in user-centered product development. The researchers all had prior experience with user studies. For the interviews, a methodological mix was chosen as the method of data collection to obtain an impression as broad as possible of the experiences of the interviewees. Eid and Diener (2006) recommend multi-modal captures [8]. All conversations were recorded and then used for evaluation. The interviewees were informed about this at the beginning of the interview and asked for their consent. The interview consisted of two parts, a narrative interview at the beginning, and afterwards a guideline interview. In the narrative interview, a qualitative explorative study, the interviewer provides a narrative impulse and lets the interviewee tell in peace. At the end, questions are being asked [9]. The aim was to learn how user studies have been carried out, what experiences existed, and what caused difficulties for the researcher to derive a potential need to improve user studies, both directly and indirectly. The specific call for narrative was: “Please tell me in detail what you are investigating with or about people, how have you dealt with user studies so far, and what are typical challenges for you.” The guideline interview is a semi-standardized interview, which consists of punctual questions. These questions were based on previously identified relevant aspects of the subject matter of the investigation, which were contained in an interview guide:

  • Experience in user studies

  • Difficulties in the investigation

  • Use of existing assistance during the investigation process

  • Used literature

  • Main information on investigation processes

  • Wish to help respondents to user studies [9]

These aspects were included in the interview guide in the form of questions. The questions were similarly asked in each interview so that comparability between the interviews could be established. The interview was evaluated by a content analysis using a category system. In this context, upper categories, based on the relevant aspects of the thread interviews, were deduced from the material and subcategories from the data collected were developed. Thus, the given responses of the guideline interview, as well as the narrative of the narrative call, could be categorized.

After evaluating the interview results, the sciences, which deal with empirical studies of human focus, were analyzed. A literary research has been carried out to determine which areas of expertise and methods provide appropriate knowledge and methods for user studies. Literature hints on the quality, effectiveness and error avoidance of user studies within these areas were collected, analyzed, and understood as a relevant basis for necessary assistance. Following, the collected concrete problems, typical challenges, and frequent sources of error from product developers at user studies were used in the interviews to capture content in areas identified as relevant, which offer appropriate solutions. Topics were collected and supplemented by topics that ensure the quality and effectiveness of user studies. The collected subject areas were edited based on relevant basic literature [9,10,11,12,13,14,15,16,17,18].

The basic literature was read in parallel, contents were collected and checked. The content was reviewed for overlap, consistency, technical terms, detail, and relevance to user-centered product development. In this way, every step of the process of a user study was looked at in detail and the knowledge was brought together. In addition, content that had been identified as particularly relevant to user-centered product development for user studies had been included in a higher degree of detail than less relevant topics. Within an iterative process, the selected knowledge was structured, compressed, and linguistically adapted. This knowledge had been sorted in an order in which it was necessary in the investigation process to performed it effectively and with high quality. This allowed for a gradual acquisition of knowledge in parallel with the tasks to be performed for the investigation. In conclusion, the complete knowledge was checked and adjusted for understanding, consistency, the introduction of technical terms, and logical order. In addition, a design concept was developed to facilitate access to content and knowledge-taking.

What should an assistance look like to provide product developers with knowledge in a structured, compact and understandable way?

The answer to the second question was also carried out based on the interview results. This was a targeted examination of the assistance that people had been using to date for user studies, whether they were criticized and what additional help they wanted. On the other hand, there was a literature search. It was considered what assistance was available in the state of the research on user studies that included proposed solutions to the identified challenges and how these were structured [19,20,21,22,23,24,25,26,27,28]. The nature of assistance was based on the following question: “How well can the existing support for user studies be transferred to user-centered product development?” This assessment had been able to estimate a direction for the type of assistance. It was supplemented by the requirement to structure, and compactly and understandable present the relevant knowledge and to provide references from the state of research. Hereby, the term “understandable” stands for the reduction or introduction of technical terms from social sciences or psychology is meant, so that it was possible to get into subjects quickly.

3 Results

The interviews conducted had an average of 43 min and a time range of 23–69 min. On a five-level scale (not at all, very little, little, much, and very much needed), 57% of the subjects consider the need for user study support with “very much.” Cumulatively, all seven people consider the need for support “something” to “very much.” In Table 1, the category: “Types of typical challenges” is listed. It is particularly striking that 86% of the interviewees listeted challenges, which identified as “human challenges” or “research challenges.” 71% of the interviewees said that when investigating subjects, they select subjects by availability without pre-defined sampling criteria. 57% of the interviewees said they were aware of the presence of disruptive influences on user studies. They tried to capture or control them in the past, but had no systematic approach to it. Free narratives and responses to the guide interviews were taken into account. Multiple names are possible. The percentage describes how many interviewees mentioned at least one item in this category. The following areas of expertise have been identified from the extensive bibliography of user studies: Medical, psychology, social sciences (sociology included), media and communications (marketing included), sports, economics, and philosophy (anthropology included). The empirical research methods in psychology and social sciences have been identified as the basis beyond the references to methods, quality, effectiveness, and error prevention of user studies. Topics that provide solutions to identified concrete problems, typical challenges, and frequent sources of error from product developers at user studies are shown in Table 1.

Table 1. Identified Typical Challenges of User Studies (a. Consideration of boundary conditions in the planning phase; b. Create investigation plan; c. Ethical Guidelines; d. Duty to inform trial participants; e. Selection of subjects; f. Selection of a suitable research question for a study; g. Selection of an empirical research method; h. Knowledge about correct performance and evaluation of a research method; i. Generating, formulating and embedding hypotheses and theories; j. Operationalization of variables; k. Consideration of quality criteria of empirical studies; l. Consideration of confounding variables and confounding factors; m. Potential influencing factors; n. Consideration of scale levels; o. Data analysis and hypothesis testing; p. Interpretation and discussion of results)
Full size table

Identified, particularly relevant topics are presented in bold. In addition, the following in-depth content was found relevant, as the interviewees were all concerned with it:

  • Data collection focusing on experimental and correlative studies

  • Selection and number of subjects to be studied (with determinations)

  • Basic data analysis: descriptive and inductive statistics

In response to the second question, a process approach was identified in combination with an in-depth guideline, which provides knowledge in a structured, compact, and comprehensible way. For the development of the process approach (see Fig. 1) and the guide, the identified thematic areas from Table 1 are used.

Fig. 1.
figure 1

Process approach (Specifications in the state of the research are listed in the references 29–31, 9, 10, 6, 21, 32–37, 14, 38, 8, 15, 12, 39, 13, 40, 24, 17, 41–49)

Full size image

The process approach (see Fig. 1) serves to guide and support the design and the entire process of user studies. The dashed box highlights the process steps to be considered during the entire user study. Overall, it accompanies the product developer through the phases of planning, implementation, and evaluation of the empirical user studies. The depth of content with access to concrete knowledge on the topics is provided by the guideline. The often difficult access to literature of other sciences is provided by many literary recommendations within the guide and allows for specific research.

It provides pooled and refined expertise. In addition, language barriers will be reduced and a holistic view of the research process will be promoted through references and consistent technical terms. The guideline is publicly available on the Open Acces platform: KITopen [50].

On the one hand, it is possible to give the developer an overview of the investigation process and guide him/her through its content step by step. On the other hand, the developer is provided with selected, aggregated, and processed content as a source of knowledge. In addition, different levels of detail can also be taken into account here, as well as potential effects of humans as subjects to be reflected in the studies. This combination of knowledge provision and transformation of various specialties, in line with the step by step explanation of the individual process stages of the user studies in the user-centered product development, is a unique feature.

4 Discussion

The problem has been confirmed, that product developers usually have no previous experience with user studies, the possible influence of subjects in studies and therefore what special attention must be paid to. Product developers who have a lot of experience in the field of user studies can acquire the necessary knowledge through a lot of effort.

However, the engineering perspective determines the perception through their education studies, which makes the change of perspective difficult due to bias. In order to save this effort, the already existing methods and knowledge of other sciences can be used. With the following procedure an access was compiled.

Interviews were conducted to identify typical sources of error and challenges in user studies in user-centered product development. Based on this, literary-based solutions were developed, which were structured, compact and intelligible in a process approach and a guide. The research questions were answered:

  • Question 1: What knowledge from other disciplines is relevant for user studies in user-centered product development?

  • Question 2: What should an assistance look like to provide product developers with knowledge in a structured, compact, and understandable way?

On the one hand, relevant knowledge could be identified by the interview results in the fields of social sciences and psychology. This knowledge consists of expertise and methodological knowledge about user studies with a focus on humans. An example of this are the control techniques of potential disruptive variables in investigations. By using control techniques in a targeted way, disturbing influences on user studies can be specifically taken into account or eliminated. On the other hand, a process approach and a guideline for deepening could be developed as an appropriate help to provide product developers with knowledge in a structured, compact and comprehensible way.

The process itself gives a guidance on what steps to take within an investigation. The complements to the empirical research process steps by recommended literature from the state of the research and the developed guideline, which details the process steps with reference to literature, offer a possible added value. The product developer shall be progressively made available to him/her in a structured and prepared manner, which is selected chronologically for the examination procedure. In the guide, according to the analyzed typical error sources and challenges of user studies in user-centered product development, different levels of detail are realized in terms of content. This process approach aims to provide a supporting orientation within the User Studies. It is intended to help product designers, especially in the planning phase in combination with the in-depth Guideline, to concentrate on the important contents of the study and to provide suitable step-by-step knowledge for the correct implementation. Thus, a kind of guide for user studies as well as a reference book is provided at the same time.

Limitation and Future Research Needs

The interviews conducted, which are the main source of the selection of topics of the project approach and the guide, are among the qualitative data collection. Data collection is assessed as objective and reliable. The internal validity is considered limited because no weightings were used to consider the typical challenges. The results include the number of people who have addressed challenges in one category. However, whether a subject has identified one or five challenges within a category has not been reflected in the interpretation of the interviews. External validity is considered to be limited because a limited representativeness of the examined sample is assumed. This is because of the group size of the respondents of seven, the choice of subjects based on voluntary behavior and a shared background of the respondents, all working as researchers. Other quality criteria: Procedural documentation, rule-guideness, closeness to the subject, communicative validation are considered to be complied with. The argumentative validation of interpretations as well as the triangulation are rated as complied with a limited extent, since the interpretations of the results were not noted in writing and no further methods were used to compare the results.

In the future, the feasibility of the process approach and the guide, as well as the impact of their use, should be verified and validated. The aim should be to verify the transferability of power of expression to product developers in industry. This has not happened so far. The desired benefits of saving time and effort, avoiding mistakes and improving the quality of the user studies should be analyzed in detail. One possibility for verification is the assessment of the process approach and the guide during and after use in user-centered product development by engineers under field conditions. Both quantitatively and qualitatively, the influence can be divided between the different phases: planning, implementation and evaluation of user studies are considered and examined for structure, content preparation and understanding. The field of empirical studies with subjects into whom user studies are classified is very extensive. The prioritization of content is based on the average needs of the product developer. This was estimated on the basis of the interviews conducted. Individual needs are not taken into account. The basis for the process approach and the guide is derived from the german-speaking environment. Demographic or cultural differences were not taken into account. Transferability to user studies in the international environment should be considered in future. Differences in ethical and legal guidelines are expected. The process approach in combination with the guide is intended to be an advantage to the state of research, which is to be demonstrated by this. This is because the content is tailored, aggregated and processed to the needs of product developers, so that consistent support is provided during the course of user studies.

5 Conclusion

To remain competitive in the market, user-centered deisgn (UCD) is getting more and more important [1]. For UCD it is necessary to identify and analyze the user needs. But there are difficulties in identifying the real customer needs. Performed user studies often miss their goal to identify the customer's needs. Potential development goals for optimized user-centered product design thus are not achieved. This is because most product developers lack knowledge in the field of empirical investigations for more targeted use of methods, as well as time for familiarization and study performing. It is usually not known which knowledge and which methods are relevant for designing, performing and evaluation of user studies. This easily leads to errors, high expenditure and a reduction in quality of the studies. To prevent this, a process approach was developed. In a summarized and elaborated form, it provides knowledge, focused on psychology and social sciences about user studies along the ideally typical history of an empirical study, adapted to the needs of the product developer in user-centered design. Research-state bibliography complements the process. A guide has been developed along with the process approach to providing the in-depth knowledge needed to conduct user-centric product development studies.

The following procedure was used for the development. First, typical error sources were analyzed in user studies. In addition, interviews were conducted that show that engineers wish support in the field of empirical methods and what typical challenges they face in user studies. In order to identify approaches to support at these challenges, sciences with a high level of expertise and methods in the field of user studies have been identified: Social sciences and psychology. The most relevant knowledge for user-center product development studies has been identified:

  • Consideration of boundary conditions in the planning phase

  • Selection of a suitable research question for a study

  • Consideration of quality criteria of empirical studies

  • Consideration of confounding variables and confounding factors

  • Consideration of scale levels

  • Data collection with focus on experimental and correlative studies

  • Selection and number of study participants (with significance for study)

  • Basics of data analysis: descriptive and inductive statistics

It is possible to integrate different topics identified as relevant for user-centered product development into a process. A comprehensive bibliography of the relevant subject areas was conducted, content was merged, filtered by overlap and consistency, and checked for relevance to user studies in user-centered product development. This created the process approach. The developed process approach and the guidance it provides must be verified and validated in future studies.