Keywords

1 Introduction

The number of software startups and their role in technical and economic development have increased globally. Many recent success stories, such as Facebook, Spotify, and LinkedIn, originated from startup companies [1]. Studies have explored software startups from different viewpoints, such as challenges, success factors, startup processes, and models [2,3,4,5,6,7,8,9]. In recent years, the lean startup approach [5] has gained popularity among researchers, presenting principles for developing a business model built on a relevant problem/solution and product/market fit. Some derivatives of lean startup have been created [7, 10], fine-tuning the original ideas. Steinert et al. [6] proposed a similar concept focusing on seeking the great idea.

In software startups, self-destruction is a bigger cause of failure than the competition [4, 11]. Broad and reliable validation of a product idea from innovation to prototype and creating the first product is a crucial period in a software startup’s evolution, independent of the model or process that the work is following [4]. The importance is further increased due to the challenges a startup faces, such as limited resources, inexperienced teams, and dependency on a single product [1]. Studies on suitable practices for validating ideas are, however, missing, as shown in a thorough mapping study of software startups [1].

In this paper, we study the practices used in a sample of European software startups for idea validation, the effect of available competencies on the idea validation process, and the role of the practices during requirement gathering. We broaden the principles of the build-measure-learn process defined in [5] from the validation of a business case to cover also the technical aspects of the idea. While validating the idea a startup gathers requirements for the product. To highlight that we map our findings to a requirement gathering model presented in [12].

We define the key concepts of our paper as follows: Idea validation refers to all the actions and steps that are directly targeted to improve the idea and validate its technical and commercial feasibility. An idea validation practice refers to the elements of the idea validation process. Competencies refer to the skills and knowledge needed to conduct the process successfully [13].

The research was conducted as a multiple case study [14]. We interviewed a sample of software startups in four European locations and analyzed the collected research data via thematic analysis following the guidelines of [15].

The rest of the paper is structured as follows: Sect. 2 describes the background of and motivation for the study; Sect. 3 presents the research method and data analysis technique used; and Sect. 4 presents the empirical results. Section 5 discusses the answers to the research questions. Section 6 concludes the paper, briefly describing the limitations of the study and directions for future research.

2 Background and Motivation

Innovative startups play an important role in the economy because of their potential to grow through rapid expansion even in highly competitive markets. However, internal problems are a bigger cause of startup failure than the competition [3, 4].

2.1 Startup Models and Processes

Increasing interest in startups has led to the development of models describing the evolution of a startup. Crowne [9] introduced a startup model with four phases: startup, growth, stabilization, and maturity. Paternoster et al. broadly studied software startups in [16] and introduced a greenfield model of software startups. The lean startup approach [5] and the hunter-gatherer model [6] deal with business model creation by focusing on finding a winning innovation.

The lean startup model proposes practices for managing the uncertainty that characterizes startups’ business prospects by defining the minimal viable product (MVP) and the build-test-learn loop for finding a problem/solution fit and a product/market fit for a product idea. Bosch et al. in [7] used lean startup as a basis and proposed an early stage software startup development model (ESSDM) while exploring the distinctive challenges of a startup in searching for a product idea worth scaling. Wang et al. [17] identified a set of challenges perceived during the idea conceptualization stage, such as building the product, creating a business model, and building an MVP.

Two recent mapping studies by Paternoster et al. [1] and Klotins et al. [18] on software engineering in startups conclude that startups typically don’t follow strictly defined processes.

Coleman et al. [8] concluded in their grounded theory study that the key persons’ earlier experiences act as the basis of the process development in startups. The same phenomenon was identified in our prior research on the initial team of a software startup [13].

In a startup, the early development steps, during which the idea is validated, build the basis for the next process steps, requirement engineering and product specification. According to [12], a typical product development project in an established company is organized so that the marketing department of the company acts as the customer whereas the development department acts as a supplier. In the context of a startup, there are seldom separate departments and strictly specified roles, but the initial team takes care of all aspects of the product development, including the steps that validate the idea and bring it forward [13].

The current literature in the context of software startups demonstrates only limited knowledge of the actual work done to validate the idea, for instance the lean startup [5] describes building an MVP and measuring its value in fairly abstract terms. That leaves a research gap, how the companies conduct the work and how they acquire the knowledge needed in validating the idea.

3 Research Design

This study aims to address the research gap identified in the previous section: the process run in software startups to validate the innovation’s technical and commercial feasibility. We also study how the competencies of the founder affect that idea validation. To fulfill the research objective, we propose the following exploratory research questions [19].

3.1 Research Questions

RQ1: What practices are utilized when validating an idea in software startups? The objective of the first research question is to find what practices are used as elements of the idea validation process.

RQ2: In what ways do the prior competencies of the innovator/founder affect the idea validation practices? The aim of the second research question is to understand how the prior competencies of the innovator/founder affect the idea validation process.

To answer the research questions, we carried out a multiple case study on a sample of software startups following the guidelines set out in [14].

3.2 Case and Subject Selection

We collected the research data by interviewing a sample of software startups in May and June 2015. We opted to collect a sample of companies with different backgrounds, products, business cases, and evolution phases. We used local startup incubators to help finding candidates on random basis. The sample included nine startup companies in four European locations: Bolzano, Italy; Trondheim, Norway; Oulu, Finland; and Helsinki, Finland. Out of the sample, four case companies had embedded products while five were developing pure software products. We included embedded cases because validating the idea of an embedded product may be different from a pure software product due to needed electronics and mechanics. The case companies, their product types, business cases, and current statuses are summarized in Table 1.

Table 1. Descriptions of the case startups.
Full size table

Eight case companies were ordinary startups and one was an internal startup. We opted to include an internal startup in our sample to find out possible differences to ordinary startups. The size of the case companies ranged between four and twelve employees, in most cases between five and seven. The operational age was between 12 and 60 months. In some cases, the original idea was refined and tested several years before the company was founded.

3.3 Data Collection Procedure

We selected interviewees via the key informant technique in order to collect rich qualitative data [20]. The interviews involved the founders, chief operating officers (COOs), and chief technology officers (CTOs) of the case companies. We used direct techniques in the form of semi-structured face-to-face interviews [21] and created a thematic interview guide before we conducted the interviews. The interview guide contained questions that broadly covered the early phases of the startups from the original idea to the present situation.

In case companies B and C, we interviewed two persons from the same company, and in one case, the same interviewee covered case companies D and E. Thus, the total number of interviewees was ten. All the interviews were conducted in English. They were recorded and later transcribed by a professional transcription company.

3.4 Data Analysis Procedure

We opted to analyze the empirical data by using thematic synthesis as defined in [15]. We followed the recommendations of [15] to utilize the integrated approach by combining inductive and deductive coding, a method that determines an initial set of codes and defines new ones during the coding process when new topics emerge from the research data.

As the first step, the interview recordings were transcribed to MSWord documents. The documents were read thoroughly by the first author, and a decision was made to include all interview data for coding in order to utilize the benefits of the inductive-deductive approach.

In the thematic synthesis, the interview data was analyzed sentence by sentence by using NVivo11, and the data related to innovation validation were identified and coded. The codes were then collected into ten themes summarizing the idea validation practices. The themes were further incorporated into three categories, engineering-related, business-related, and combined, as shown in Table 2. The classification was based on our research data, for example, on the context and purpose in which each practice was used in our sample companies. Several practices were deployed in both engineering-related and business-related domains.

Table 2. Identified practices for idea validation.
Full size table

Though creating prototypes may be a part of creating MVPs, we classified the former as engineering-related practices and the latter as business-related ones. The reason was that in our sample prototyping was done mostly for validating engineering solutions, while an MPVs is meant for measuring the business value, as defined in [5]. Similarly, we do not classify changing the technical solution identified in some case companies as pivoting, as defined in [5]. Changing the technology solution doesn’t necessarily mean a change to the business case, as pivoting by definition does.

Three practices—expert support, host company support, and educational support—deal in our sample only with broadening the knowledge and skills available in a startup. They were handled separately from each other because the sources of the support and the contexts when utilizing them were different.

To find answers to RQ2, we identified codes related to the founder’s product creation competencies. The founder’s perspective was selected based on the findings of [8, 13], which indicate that the founder is the key person in conducting innovation-related work in startups and that the key person’s previous experiences strongly affect the process development. The codes identified in the research data were incorporated into three competency-related themes as shown in Table 3.

Table 3. Themes for competencies.
Full size table

Because this study focused on software startups, the key areas arising from the research data were competencies in software development and in the application domain. The theme software development covered competencies in all software development areas, such as analysis, design, implementation, testing, requirement engineering, and related process development. Similarly, all competencies related to the application area of the planned product were gathered under the theme application area competencies.

Because four out of nine case companies were developing embedded products, we included competencies in disciplines other than software. In that theme, we ranked other technology areas needed for product development, such as hardware and mechanics development.

We compared the competency themes and the idea validation themes for each company in order to identify the relationships between the competency and the idea validation themes and to find answers to the research questions.

4 Results

In this section, we discuss the startup cases and describe the identified idea validation practices and the effect of the competencies on the idea validation practices.

4.1 Case Description

Case company A developed a web-based service for multimedia sharing in a university environment. The product was a pure software product developed by a team of students. The founder was a professor with very strong competencies in software development, software engineering, and innovation but no prior experience in the application domain.

Case company B developed a web-based ticketing service. The product was a pure software product. The founders had just graduated from the university with degrees in non-software-related topics. The founders did not have prior competencies in any competency areas of interest.

Case company C developed a smart emergency call service that provides the emergency call center with access to the caller’s relevant health information in addition to automatically providing the location of the caller. The idea emerged from an existing similar system in the US and from the founder’s own experiences after an accident.

Case company D tried to develop an embedded device that measures online the human body’s fat burning rate during physical exercise. The measurement was based on analysis of certain marker substances in the user’s exhaled breath. The development of the sensor technology failed, and the company was dissolved. The founder had prior competencies in software, hardware, and mechanical development but no competencies in the application domain.

Case company E was developing an embedded device that measures the work and effort of a human muscle in physical exercise. The founders had a background in health care and medicine and thus had prior competencies in the application domain. However, they did not have any competencies in the technology areas needed to create a product. The product idea emerged from the founders’ daily work and the solution principles were copied from existing electromyography (EMG) devices.

Case company F was an internal startup within a bigger company with a focus on software services. The product was an embedded Internet of Things (IoT) device that integrated a multitude of sensors and communication solutions. The host company had almost a hundred very experienced developers and managers with strong prior competencies in the application area and all technology areas needed to create the product.

Case company G was developing a software tool for improving aircraft maintenance at a big aviation company. The product was a pure software product. The founder had just graduated from the university but had strong prior competencies in software development. He had also worked on a temporary contract in the maintenance department of the customer company.

Case company H was creating a graphical user interface platform for smart devices, especially focusing on smart watches. The idea was much older than the company and had emerged from the founder’s earlier work. The founder had done two prototyping rounds with two different implementation approaches before finding the final one and founding the company. The founder had strong competencies in the application area and in software development.

Case company I developed an embedded ultrasound device with complex software, hardware, and mechanics. The idea emerged from a similar device the founder had learned about. The founder was a seasoned entrepreneur who had strong competencies in software development, reasonable competencies in hardware development, and some competencies in the application domain.

4.2 Idea Validation Practices

We summarized the results of the thematic analysis in a multidimensional chart combining the idea validation practices and the founder’s prior competencies per company, as shown in Fig. 1. In the chart, a small square at the intersection between a company and an identified theme shows (a) on the left-hand side what idea validation practices were used in the case company and (b) on the right-hand side what the founder’s prior competencies were. The vertical light yellow bars highlight the most frequently used idea validation practices, and the horizontal light blue bars indicate the effect of the founder’s prior competencies on the utilized practices.

Fig. 1.
figure 1

Idea validation practices vs. founder’s competencies

Full size image

The chart in Fig. 1 shows that use of the idea validation practices varied considerably between the case companies. Most of the companies utilized several practices to validate the product idea. Copying existing products, prototyping, and utilizing expert support, together with customer cooperation were the most frequently used practices. The business-related practices for idea validation, utilizing a market study, utilizing an MVP, and pivoting, were in the research data strongly tied to the practices being more engineering-related. Practices from both categories were utilized in parallel, mutually supporting each other.

4.3 Effect of Prior Competencies on Idea Validation Practices

The results indicate that the founder’s prior competencies in the application domain were the key difference between the case companies. The better the application knowledge competencies, the smoother the progress from idea to a product. Also, the palette of idea validation practices was the most focused, while the broadest variation of practices was in the cases when the founder did not have prior competencies in the application domain. In four out of five cases without application domain competencies, utilizing expert support belonged to the practices palette, broadened by copying existing products, customer cooperation, and prototyping.

Utilizing expert support was common for the three cases without prior competencies in software development. In all those cases, the experts’ role was to compensate for the founder’s missing competencies. Out of the six cases with existing software development competencies, expert support was utilized in two cases, both developing embedded products with highly specific technology.

Prior competencies in disciplines other than software were mainly relevant to companies that developed embedded products. Expert support was a common idea validation practice independent of existing competencies in the other discipline category,

The internal startup F differed from all other case companies by being self-sustained both in validating the idea and acquiring all necessary competencies from the host company.

5 Discussion

In this section, we discuss the findings of our research. First, the answers to the research questions are presented. Then, our findings are discussed in the context of development processes and requirement gathering. Finally, the relevance of our findings to the research and to practitioners is discussed.

5.1 Answering the Research Questions

Based on the results of the study, we conclude the following answers to our research questions:

RQ1: What practices are utilized when validating an idea in software startups?

From our case sample, we identified ten idea validation practices: close customer cooperation, MVP, copying existing products, market study, prototyping, technology feasibility study, pivoting, support from home company, support from educational institute, and expert support. We further found that copying existing products, prototyping, and utilizing expert support, together with customer cooperation, were the most often used practices in the cases.

Early and close customer cooperation was commonly utilized in our sample, especially in cases where the customer was easily identifiable and accessible.

In case companies A, D, and E, where the product was targeted to the mass market, close customer cooperation was not identified. That may indicate that following the recommendations proposed in [5,6,7, 10] may be difficult in a practical situation if good representatives of a customer base cannot be found.

In case companies F and H, late customer cooperation was identified. In both cases, the business-to-business (B2B) customer base was broad, and the companies had excellent competencies for developing and testing the product.

Possibly the most surprising practice for idea validation in our sample was copying existing products. Copying was identified in five out of nine companies. Copying was tied to business cases aiming at developing a cheaper or better product to compete with an existing product, developing a product for a specific customer base, broadening the use of known solutions to a new application area, or developing a local copy of a product already used in another country.

In our sample, the utilization of the methodology proposed by the lean startup method [5] was surprisingly small, although the interviews revealed that the approach was known. There may be several reasons for that. In cases where copying from existing products was utilized, the basics of the product idea, its business case, and the potential customers were probably already well enough known in early phases of the startup’s evolution. Pivoting, as proposed in [5], was identified only in case companies B and F.

We separated prototyping from MVP according to the purpose. While an MVP’s goal was early measuring of the customer value of an idea [5], prototyping was utilized for company internal validation and optimization of the technical solutions.

RQ2: In what ways do the prior competencies of the innovator/founder affect the idea validation practices?

The founder’s missing competencies in the application domain and in software development led to the utilization of many idea validation practices in parallel.

The chart in Fig. 1 shows that expert support was a frequently utilized practice to compensate for the founder’s missing competencies. It was utilized especially when embedded products were developed and when the founder did not have prior competencies in the application area or in software development.

Support from an educational institution and from the host company were utilized in companies A, C, and F when the founders studied or worked at the supporting organization.

Copying from existing products was utilized both when prior competencies in the application area existed and when they were missing. In the cases when copying was utilized, the dependency on the targeted business case was clearer than the dependency on the available competencies.

Close customer cooperation was an equally utilized practice independent of the prior competencies of the founder. However, in the cases of very competent founders and development teams, customer cooperation was first initiated at a later stage of the development.

In the sample, prototyping was a common practice for idea validation independent of other practices and the founder’s prior competencies.

5.2 Idea Validation Process

The results of our study highlight the complexity of the idea validation process and thus confirm empirically the related findings of [1, 3, 8, 18]. A smooth and linear process from the idea to a product was identified in two cases, companies F and G. In company F, the reasons may have been a fairly clear business case for the IoT device and the host company’s strong experience in developing such a product. In company G, the founder had direct work experience in the customer organization and hence excellent personal contacts. He also succeeded in deploying a simple but convincing MVP at a very early stage. The other companies faced different challenges and a nonlinear process during the idea validation.

The idea validation processes identified in our study in different case companies can be described as ad hoc, although the individual validation practices were well known ones in product development. The ad-hoc nature comes from three main elements derived from Sects. 4.2 and 4.3: (1) the combinations of different practices varied between the companies, (2) utilizing different practices was strongly context dependent, and (3) the founder’s prior competencies affected the set of practices deployed. Thus, our empirical results are in line with the earlier findings of [1, 8, 18], and they don’t allow us to determine any general process model for idea validation.

5.3 Idea Validation Practices and Requirements Gathering

As discussed above, the work done for refining and validating the original idea is a part of the overall product development process of a startup. In this section, we explore how the identified idea validation practices link to requirement gathering. As the framework for the exploration, we utilize the four-level requirement model proposed in [12]. In the model, the requirements are classified in four levels: goal, domain, product, and design.

Figure 2 shows the proposed research-data-based mapping of the identified idea validation practices onto the model.

Fig. 2.
figure 2

Idea validation practices and requirement gathering level

Full size image

At the goal level, requirements related to the business objective of the product are created and verified [12]. To gather the requirements of the goal level, copying and customer cooperation were utilized. Pivoting as defined in [5] is a retrospective decision and a sign that the targeted business case was not strong enough for continued development.

At the domain level, requirements usually focus on the key functionality of the product. It is important that the correct functionality is carefully recognized and the planned product’s ability to support it is ensured [12]. As Fig. 2 shows, most idea validation practices, including copying, customer cooperation, expert support, technical feasibility study, market study, and MVP, were utilized at the domain level.

At the product level, requirements defining the product’s physical and logical boundaries, its interfaces, and its inputs and outputs are described without focusing on the actual implementation [12]. For the requirements at this level, prototyping was used, and customer cooperation continued.

At the design level, prototyping was continued. Company I, developing a demanding product, continued conducting small-scale trials separate from the main-stream prototype development. Thus, we include technical feasibility study in the idea validation practices for the design level.

Because of missing detailed research data, we do not map educational support and host company support onto the model shown in Fig. 2.

5.4 Validity Discussion

We discuss the validity of the study in terms of construct validity, internal validity, external validity, and reliability as described in [14]. Construct validity deals with taking full provisions in the data collection procedure so that the collected data line up with the research questions. We designed an interview guide before we conducted the interviews to ensure the interview questions covered the research topic and the research questions in a broad manner. Experienced people from the case companies were selected for the interviews based on the key informant technique.

Internal validity is mainly used for explanatory and causal studies in which causal relationship between outcomes and intervention is examined in order to find the explanation for a given condition or problem [14]. Our study explored phenomena in software startups without addressing causal relationships. Thus, we don’t consider internal validity. External validity refers to whether the findings from the study can be generalized outside the studied cases. Our results are based on nine software startups in Europe. To generalize the results further empirical research across other regions and a bigger sample is needed.

Study reliability is concerned with how the data analysis depends on the particular researchers. In the initial phase, the first author created a case study protocol in order to have a systematic research procedure. The data analysis was performed by the first and second authors with the qualitative data analysis tool NVivo11 in order to ensure that various aspects of the idea validation process in the software startups were addressed.

5.5 Relevance to Academia and Practitioners

The results of this study reveal interesting viewpoints for academia and industry. Recently, startup research has focused on big innovations and methods for seeking them [5, 6]. Our results may, however, indicate that among the famous but few big innovation startups, there is a dense undergrowth of others creating business and furthering development, though in smaller steps. For academia, broadening the focus of research to that undergrowth may create valuable knowledge supporting the economic growth that takes place outside the big but few innovation cases.

From the practitioner’s point of view, the study identified a set of idea validation practices that were deployed in real-life startups. The results highlight the complexity of the idea validation process, the importance for a startup to realistically recognize and acknowledge the competency weaknesses, and the importance of seeking ways to compensate for those weaknesses.

Though it is not reasonable to set any specific priority order onto the practices we identified, some general guidelines can be given. Close customer cooperation is recommended in cases where the potential customers are easily accessible and willing to cooperate. Close customer cooperation combined with active prototyping covers both the business-related and engineering-related issues of product development.

The value of following the recommendations of lean startup [5] was identified in our study, as well. A well-fitting MVP opened the business case of company G to a big institutional customer that otherwise might have been difficult to reach. Timely pivoting might have helped company A get back on track from the dead-end of its original product idea.

Our mapping of the identified idea validation practices onto the requirement gathering [12] highlights for practitioners the importance of conducting validation practices in an appropriate way, though no generic process covering the practices was defined.

6 Conclusions and Future Research

In this paper, we explored empirically the practices deployed in idea validation in software startups. We used a multiple case study method with nine software startups. To collect the data, we used semi-structured face-to-face interviews. During the data analyses, we found ten practices that affect the idea validation process. Of the ten practices, the most frequently utilized ones were copying existing products, cooperating closely with customers, utilizing expert support, and prototyping.

The results indicate that the case companies utilized the practices in varying, context-dependent ways. No uniform and systematic process from idea to product was identified. The results also indicate that the previous competencies and experiences of the founder affect the utilization of different practices. From those perspectives, our empirical study was in line with the prior literature, confirming our findings.

A finding of our study is that the founder’s prior knowledge, skills, and competencies in software development and application area tend to make the process of moving from an idea to a product more linear and straightforward. In the opposite case, a key learning is the need to use all possible ways to fill the knowledge and competency gaps.

The research was based on several startup companies located in various regions across Europe. The results from this study need to be further empirically verified across other regions and a bigger sample of startups in order to validate the generalizability of our results. Other interesting future work would be to examine in more detail how the idea validation process continues to systematic requirement engineering.