1 Introduction

The relationship between competition and innovation has remained a puzzle in industrial economics. Thanks to the availability of better data and inspiration from new theoretical models, the research agenda has gained momentum in recent years. Though astonishing progress has been made in terms of analytical rigour and precision from the early works by Schumpeter (1911, 1942) to Arrow (1962) or Aghion et al. (2005), no general consensus has emerged on one of the most fundamental questions in economics: is increased competition conducive or obstructive to innovation?

The situation is aggravated by the fact that any kind of relationship appears to be possible in both theoretical and empirical analysis, as the many surveys of the literature show (e.g. by De Bondt and Vandekerckhove 2012; Cohen 2010; Gilbert 2006; Aghion and Griffith 2005; or Reinganum 1989). While theoretical models are becoming more refined, they are also yielding increasingly conflicting results and relying more on variables that hardly relate to the available empirical data. Conversely, many empirical findings lack robustness, which is frequently due to unresolved problems of endogeneity between innovation and competition. The lack of robust findings is not without consequences. Referring to recent examples from U.S. antitrust, Shapiro (2011, p. 6) warns that “a misleading ‘complexity proposition’ has taken root and threatens to become the conventional wisdom” in the practice of competition policy.

A consensus can be found, however, in the need for more empirical testing and scrutiny based on perspicuously structured models that tackle endogeneity more thoroughly. This is where we aim to contribute and advance beyond the current empirical literature on the inverted-U hypothesis. Using a unique micro-panel database with an exceptionally rich set of variables on innovation behaviour and intensity of competition, we estimate a simultaneous system of three equations. First, the innovation opportunity function determines the impact of competition on the firm’s research effort. Second, the innovation production function captures the transmission from research effort to innovation outcome. Third, the innovation impact function shows how the difference between creative vs. adaptive entrepreneurship affects the intensity of competition in terms of the firm’s number of competitors. While the model is relatively simple, we believe that its focus on basic relationships between empirically observable variables enhances its value for policy practice.

We apply a three-stage, least-square estimation (3-SLS) and use three complementary taxonomies of technological regimes as instruments. These instruments are new and exhibit several particular strengths. First, they directly address the repeated concern that the relationship between innovation and competition is dominated by the specific technological and market environment within which firms operate (see Cohen 2010; or Gilbert 2006). Second, based on innovation theory and empirical cluster analyses, they serve as a multi-dimensional representation of different factors, thus capturing a more varied picture than single-variable indicators do (Peneder 2010). Finally, the exogeneity of the instruments is guaranteed by the fact that the taxonomies have been built using European micro-data which does not include the country from which the firm sample for the current analysis was drawn. Robustness of the results is tested and confirmed through an alternative choice of instruments.

The empirical analysis is based on a panel of Swiss firms observed across four periods (1999, 2002, 2005, 2008). The data were collected by the Swiss Economic Institute (KOF) at the ETH Zurich in the course of four postal surveys using a comprehensive questionnaire. The questionnaire included information on firm characteristics, innovation activities and the number of principal competitors, among other variables. The survey data allow us to control for technology potential, capital intensity, human capital, the expected development of future demand, past demand growth, firm size, foreign ownership, export activities and firm age. Furthermore, we control for industry and time fixed effects.

We report detailed results for all three equations. While research effort is positively related to innovation outcome, the latter is also shown to have a significant and consistently negative impact on the number of the firms’ principal competitors. As regards the impact of competition on the firms’ actual R&D activities, the simultaneous system depicts a robust and nonlinear inverted-U relationship. At low levels of initial competition, an increase in the number of competitors raises the firms’ probability of conducting own R&D, but it does so at a diminishing rate. Intermediate levels of competition provide the largest incentives for research. Whereas, if initial competition is already high, the incentives decrease with the number of competitors.

When we split the sample into two groups: ‘creative’ firms with own innovation and ‘adaptive’ firms, who either pursue new technology from external sources or do not innovate at all, the inverted-U proves robust, but much steeper for the group of ‘creative’ firms. This suggests that the research effort of creative firms is more sensitive to changes in the intensity of competition than that of adaptive firms.

Solving the system numerically and discussing the likely dynamic paths of adjustment reveals three configurations of particular interest. One stable equilibrium would be the corner solution of an uncontested monopoly with low innovation. When the market is contestable, innovation rises and is attracted to a stable solution of the system that provides for high innovation in combination with a small number of competitors. Another possible but inherently unstable equilibrium is characterized by low innovation and high competition. Any slight deviation can attract the firm towards either the previous equilibrium of high innovation and low competition or a corner solution of no innovation with very high competition.

Besides the theoretical and practical advantages of applying the technological regimes as instrumental variables, and the use of a rich and comprehensive firm-level database, we consider in particular our simultaneous system an important advance and novel contribution to the literature. It draws attention towards the joint determination of separate functions and away from the often misleading interpretation of single equations. For example, without a system approach, the typical conclusion drawn from an inverted-U relationship is that an intermediate degree of competition is most conducive to maximize innovation. But this interpretation ignores that only under very specific circumstances the system will ever settle for a maximum of innovation. Since innovation breeds cost, a maximum of innovation is neither an objective of the firm nor desirable for the system as such. In contrast, our solutions to the simultaneous equations highlight that (under the influence of the same inverted-U shape relationship), the system will typically settle either with an intermediate degree of competition and innovation, or very high competition and no innovation at all.

The remainder of this article is organized as follows. In Section 2 we explain the theoretical framework based on a discussion of the related literature and then present our simple structural model. In Section 3 we discuss the data and variables, followed by the econometric results in Section 4. In Section 5 we conjecture about likely mechanisms of dynamic adjustment in the system. Section 6 presents simulations of variations in our exogenous variables. Section 7 presents a brief summary and conclusions.

2 Theoretical framework

2.1 The inverted-U relationship

Most studies refer to Schumpeter (1942) and Arrow (1962) as fundamentally conflicting hypotheses, reduced to the prediction of a negative ‘Schumpeter’ and a positive ‘Arrow’ effect of competition on innovation. At least implicitly, it is regularly assumed that these apply to the entire range of the initial intensities of competition. However, this crude simplification ignores that Schumpeter mainly addressed the logical impossibility of endogenous innovation within a model of perfect competition. He never posited a linear relationship, nor was he specific about any functional form or precise range. Schumpeter only argued that the anticipation of a certain degree of market power is necessary for and conducive to innovation.Footnote 1 Moreover, in his considerations monopoly was always contestable due to the ongoing rivalry for technological leadership and the threat of being displaced by new entrants. Schumpeterian models therefore place emphasis on competition for innovation as drivers of dynamic R&D processes.Footnote 2

Arrow (1962) explicitly acknowledged the impossibility of perfect competition in the knowledge-producing industry and considered the case of a temporary, contestable monopoly as competitive. In contrast, he was interested in how non-contestable monopolies which are protected by entry barriers affect the incentive to innovate. Compared to this benchmark, he argued that competitive markets result in more innovation, because a successful innovation by the monopolist will replace its own rent previously held. The net gain is therefore less than it is for a new entrant, who can displace the incumbent in a contestable market. Compared to Schumpeter, Arrow’s finding thus applies to the opposite end of the spectrum of possible intensities of competition. Taken together, both make a strong case that neither perfect competition nor uncontested monopolies provide a market structure that is conducive for the creation of new knowledge.

In the literature, this subtle complementarity of Arrow and Schumpeter has largely been ignored and the two have been portrayed as antagonists. The most frequently recurring finding has been a negative impact of competition on innovation. Examples of this can be found in Kamien and Schwartz (1972, 1974), Loury (1979), Dasgupta and Stiglitz (1980), Gilbert and Newberry (1982, 1984), or Delbono and Denicolo (1991). The opposite hypothesis of a positive impact of competition on innovation is, for example, supported by Lee and Wilde (1980), or Reinganum (1985). Vives (2008) demonstrates a negative effect of decreasing entry cost or an increasing number of firms, but a positive effect of increasing product substitutability (without free entry) on R&D effort. As summarised by De Bondt and Vandekerckhove (2012), the game theory models produce a highly diversified set of mechanisms and outcomes which depend, among other factors, on the static vs. dynamic nature of the game, whether R&D is modeled as a fixed or variable cost, the mode of competition (Bertrand vs Cournot), the nature of innovations (incremental vs radical), and the structure of rewards (winner-take-all vs. leader-follower patterns).

In the empirical studies, predominant negative effects have been found. Examples are the studies by Mansfield (1963), Kraft (1989), Crépon et al. (1998), Artes (2009), Hashmi and Van Biesebroeck (2010), Santos (2010), or Czarnitzki et al. (2011). Support for a positive relationship is provided, for instance, by Geroski (1995), Nickel (1996), Blundell et al. (1999), or Gottschalk and Janz (2001). Tang (2006) showed that high competition in terms of high perceived substitutability of products has a negative impact on R&D and product innovation, whereas the rapid arrival of novel products and production technologies has a positive effect. In an experimental setting, Darai et al. (2010) observe a negative impact of an increased number of players on R&D investments, and a positive impact of a switch from Cournot to Bertrand competition. Castellacci (2010) reports that competition negatively affects R&D, but enhances the positive impact of innovation on productivity.

Those who advocate enhancing competition in order to foster innovation increasingly tend to argue for a nonlinear relationship. They find support, e.g. in analyses by Tishler and Milstein (2009), Scott (2009), Schmutzler (2010) or Sacco and Schmutzler (2011). While the latter also demonstrates the theoretical possibility of a U-shaped relationship, most debate and inspiration has been drawn towards the idea of an inverted-U shape. Strikingly consistent with a literal reading of both Schumpeter and Arrow, the inverted-U implies that neither perfect competition nor a full monopoly can provide the optimal market environment, and that instead some intermediate degree of rivalry is most conducive to innovation.

Scherer (1967a, b) was the first to observe an inverted-U shape. Kamien and Schwartz (1976) provide an analytic model of the inverted-U relationship, further elaborated by De Bondt (1977). More recently, De Bondt and Vandekerckhove (2012) have discussed the model by Kamien and Schwartz (1976) and provide an illustration of its predictions. Other empirical findings that support an inverted-U relationship have been reported by Levin et al. (1985), Aghion et al. (2005), Tingvall and Poldahl (2006), Alder (2010), Van der Wiel (2010), or Polder and Veldhuizen (2012). Correa (2012) has re-estimated the Aghion et al. (2005) data and reports a structural break which renders the relationship insignificant for the period after the early 1980s (and positive before).

Problems of endogeneity can explain much of the variation among the empirical findings and have raised the attention given to the question of proper instrumentation. Since the choice of instruments depends much on the respective measures of competition and innovation, we also find much variety in the approaches. To give only two of the most notable examples, Aghion et al. (2005) investigated the competition innovation relationship at the industry level. They used citation weighted patents in order to measure innovation, the Lerner-index to measure competition, and also controlled for time and industry fixed effects. In order to instrument competition, they used policy variables, exploiting external shocks to British industry such as the EU single market programme, the Thatcher era privatization, or new developments in the UK Monopoly and Merger Commission. In contrast, the instruments of Czarnitzki et al. (2011) reflect the different measures of competition and innovation used. The former is captured in terms of entry barriers and the latter by R&D expenditures. They used the ratio of total industry sales per firm as a proxy for the minimum efficient scale. Together with the importance of advertising these serve as an instrument for entry barriers at the industry level. As an additional instrument, they used the degree of product substitutability at the firm level. Overall, the literature demonstrates that the effectiveness of the identification strategy depends very much on the respective variables for competition and innovation and on data availability. For the paper at hand we use proxies for the technological regime as instruments. These mirror theoretical notions about the relationship between the instruments and the instrumented variables and are thus firmly embedded in the overall structure of the model.

The recent surge of interest in this relationship must be attributed to the work of Aghion et al. (2005). They extend the Schumpeterian growth model of Aghion and Howitt (1992) by distinguishing between the firms’ pre- and post-innovation rents, relating them to the relative proximity of firms to the technological frontier. The ‘rent dissipation effect’ involves a negative impact of competition on post-innovation rents, which implies that competition is expected to be high even if the firm successfully innovates. A positive ‘escape competition effect’ occurs if competition reduces pre-innovation rents more than post-innovation rents, thereby raising the incremental returns to innovation. Their key prediction is that the positive ‘escape effect’ of competition on innovation dominates at low levels of competition, while the negative ‘dissipation effect’ dominates at high levels of competition. The trade-off depends on the technological characteristics of the industry, in particular the technological distance between firms. In their duopoly framework, they call industries leveled if both firms producing an intermediary product have the same technology and competition is therefore ‘neck-to-neck’. Conversely, unleveled industries are characterized by competition between a technological leader and a follower. Leaders can be ahead only by one step and followers can only catch up with but not overtake them within one time period. The inverted-U relationship results from a composition effect, i.e. the distribution of leveled versus unleveled sectors.

The specific theoretical framework of Aghion et al. (2005) cannot easily be transposed to the micro-econometric setting of our analysis. The distinction between firms and sectors clearly makes a difference in the implied mechanisms and predictions, especially when these reflect a composition effect.Footnote 3 Moreover, for our empirical application we have to realise that the majority of the firms in our sample do not operate within an environment that corresponds with the specific assumptions of many game-theoretical models. Even where they do, we hardly have the empirical data to verify them and discriminate our observations accordingly. While game-theoretical models apply to very specific markets with a few well-defined competitors, in our sample most firms have only limited knowledge of the precise information set and intricate strategic aspects of their rivals’ choices. The many duopoly models clearly do not apply, since the vast majority of our firms has more than one rival. It is even hard to justify applying predictions from more general oligopolistic models, as about 32 % of the firms in our sample report having more than 16 competitors and 45 % report having more than 11 principal competitors (see Table 1). Moreover, many of the game theoretical models specifically refer to process innovations, whereas in our sample 51.4 % of innovating firms report having introduced novel products. It is precisely from game theory that we have learned just how sensitive predictions are with respect to these assumptions. Consequently, we share Cohen’s (1995, p. 234) concern that, for our purpose, most of the “game-theoretical models of R&D rivalry do not provide clear, testable empirical implications”.

Table 1 List of variables

In our case, the older decision-theoretical model by Kamien and Schwartz (1976) provides a more appealing analytic setting due to its straightforward intuition and good match with the variables available in our data. They have modeled an innovation race in which firms seek the development period that maximizes the expected present value of an innovation. The firm faces a trade-off: a longer development period reduces the cost of innovation but also the accordant stream of revenues, which depends on the growth of demand, the development period and the mark-up. Competition enters the firm’s decision problem in the form of a subjective belief about the exogenous (and positive) hazard h, which is the probability of preempting innovations by a rival. Without additional information on the innovation strategies and capabilities of competitors, firms assign equal probabilities of innovation to each of these and the constant 1/h depicts the expected introduction time of a rival innovation. Within this information setting, the hazard h directly relates to the number of firms in the market C i .

Maximizing the expected net return of R&D, greater rivalry increases the risk of preemption and hence incites more research effort for low-to-intermediate ranges of that hazard. However, when the risk of rival preemption becomes sufficiently large, firms start to reduce their effort. The inverted-U relationship results from the fact that increasing competition raises the risk of preemption by rivals, as well as the cost of defending against it. Up to a certain degree of competition the threat of preemption spurs own R&D. However, when competition is too intense, lower returns from imitation become more attractive than risky returns from own innovation, causing firms to become more cautious and invest less in R&D.

2.2 A system of three equations

The inverted-U relationship is a hypothesis on how competition affects innovation. However, innovation and competition are mutually dependent, with causality going both ways.Footnote 4 To deal with endogeneity, we add analytic structure by distinguishing between the reported research effort and the actual innovation outcome. Given the high uncertainty of success in combination with the high heterogeneity of firm capabilities, research effort and innovation outcome should not be considered equal. We therefore do not treat innovation as a single state, equally affected by and itself affecting the intensity of competition, making endogeneity inherently more difficult to control for. Instead, we separate two distinct causal mechanisms. The first mechanism deals with how competition affects the firm’s incentive to invest effort in innovation. The second mechanism addresses how successful innovation affects the degree of competition. To close the system, we add a third mechanism, which relates research effort to innovation outcome.

Figure 1 summarises the basic structure of the model, while Annex 1 provides the analytical solution of the system in reduced form. All the three equations are simultaneously determined and consistent with the model of Kamien and Schwartz (1976). In particular, K-S capture the impact of competition on R&D incentives by the firm’s changing beliefs about the probability of a rival introduction of the innovation. We label this mechanism the ‘innovation opportunity’ function. The second mechanism requires an ‘innovation production’ function. In the K-S model this corresponds to the assumption that more expenditures on R&D buy a sooner completion date and hence raise the probability to win the innovation race. Finally, our ‘innovation impact function’ draws on the assumption that innovation produces rents from increased market power. As in the K-S model, these depend on exogenous characteristics of markets and technology and can either be fully appropriated by the innovator alone, or there can be a mixed ecology of innovation leaders and followers, who imitate and earn lower returns.

Fig. 1
figure 1

Basic causal structure. Endogenous variables for firm i: C = competition, E = research effort, I = innovation outcome; Confounders for firm i / industry j: X = vector of control variables (see Table 1); Instrumental variables for industry j: O = opportunity conditions; j = cumulativeness of knowledge, A = appropriability conditions

To begin with, the ‘innovation opportunity’ function specifies for firms i how competition affects research effort E i and estimates the impact of the number of competitors C i together with a vector of control variables X i . By adding a nonlinear term \(C_i^2 \), our particular aim is to test the hypothesis of an inverted-U relationship at the micro-level.

We use a sectoral taxonomy of ‘opportunity conditions’ O j , which was derived from EU micro-data (not including any Swiss firms) as an instrument. It takes account of the empirical fact that R&D investments do not only depend on endogenous firm specific choices, but also on exogenous sectoral contingencies. The distinct profiles in the distribution of firms with different innovation activities capture the characteristics of the technological regime at the sector level, which correlate with the probability of the individual firms to invest in own R&D. The impact of opportunity conditions on innovation outcome is only indirect, i.e. due to variations in innovation effort. As a consequence, the instrument is not correlated with the error term in the following equation for the innovation production function.

The empirical specification of the opportunity function relates very closely to the theoretical rationales of Kamien and Schwartz (1976). In their model, intensity of competition is given by the number of competitors C i , and demand growth g is a critical exogenous variable (also included among our controls X. Consistent with their prediction of an inverted-U, we expect a positive sign for β 1 , and θ to be negative.

$$ E_i =\alpha_1 +\beta_1 C_i +\theta C_i^2 +\gamma_1 X_i +\delta_1 O_j +\nu_{1i} $$
(1)

Second, the ‘innovation production’ function relates the innovation outcome I i to the firm’s innovation effort E i and a vector of control variables X i . The straightforward hypothesis is that more innovation effort raises the probability of being a ‘creative’ firm reporting own innovations. Similarly to Crépon et al. (1998) and Cohen and Levin (1989), we consider technology potential as well as demand factors and firm size to be important determinants of the innovative outcome. Additionally, the estimates tell us about the impact of further control variables such as age, exports or foreign ownership on innovation success, conditional on the jointly determined level of effort.

Our instrument is a sectoral taxonomy, which depicts the ‘cumulativeness of knowledge’ M j , and was again derived from the EU micro-data. For the given status as R&D performer, we expect that increasing returns to knowledge creation have an impact on and are therefore correlated with the probability of innovation success. Conversely, the impact on the intensity of competition can only be indirect, i.e. dependent on whether the innovation is indeed successful. As a consequence, the cumulativeness of knowledge at the sector level is not correlated with the error term in the following innovation impact function.Footnote 5

$$ I_i =\alpha_2 +\beta_2 E_i +\gamma_2 X_i +\delta_2 M_j +\nu_{2i} $$
(2)

Finally, the ‘innovation impact’ function captures the effect of the innovation outcome \(I_{i}\) and a vector of control variables X i on the number of competitors C i with the appropriability conditions A j (again derived from EU micro-data) as the instrument. We consider that individual appropriability measures also depend on exogenous sectoral contingencies, which correlate with the endogenous choices by the individual firms. Furthermore, the causal structure of our model implies that appropriability conditions affect innovation incentives only indirectly, that is if they actually have an influence on the intensity of competition. For the same reason, they are uncorrelated with the error term in Eq. 1.

Ever since Schumpeter (1911), economists have understood that firms invest resources in innovation to earn a positive rent from market power, which is another way of saying that they pursue innovation in order to ‘escape’ more intense competition. This has become a quintessential assumption in the ‘Schumpeterian’ models of endogenous growth (Aghion and Howitt 1992, 2009). We consequently expect a negative impact of innovation on the number of competitors.

$$ C_i =\alpha_3 +\beta_3 I_i +\gamma_3 X_i +\delta_3 A_j +\nu_{3i} $$
(3)

3 Data and variables

The estimations are based on a panel of Swiss firms observed across four periods (1999, 2002, 2005, and 2008). The data were collected by the Swiss Economic Institute (KOF) at the ETH Zurich, in the course of four postal surveys using a rather comprehensive questionnaire (available from www.kof.ethz.ch Footnote 6). Observations come from a stratified random sample of firms having at least five employees within all relevant industries in the manufacturing, construction, and service sectors. The stratification covers 28 industries and, within each industry, three firm size classes (with full coverage of the upper class of firms). Responses were received from 2,172 firms (33.8 %), 2,583 firms (39.6 %), 2,555 firms (38.7 %), and 2,141 (36.1 %) for the years 1999, 2002, 2005 and 2008, respectively. The firm panel was highly unbalanced. Due to missing values in some questionnaires we could not use all observations. The final econometric estimations are based on 8,656 observations.

Table 1 provides detailed descriptions of the variables used, while Table 2 summarises the data.Footnote 7 Table 8 in the Annex 2 provides a detailed breakdown of the sample by industries. Among the three endogenous variables, competition is measured by the number of principal competitors in the firm’s main product category as reported by the respondents of the innovation survey, and these had to fall into either of four mutually exclusive classes (the cut-off points are 5, 15, and 50 competitors). Of course, the number of competitors is only an imperfect proxy for the intensity of competition and the subjective nature of our variable may add some noise to the data. However, as argued e.g. by Tang (2006), subjective measures have the advantage of capturing the intensity of competition as felt by individual firms. In contrast to industry-based measures, such as conventional market concentration or Boone’s (2008a, b) profit elasticity (as well as industry level price cost margins), this measure takes account of the fact that, even within narrow industry classifications, relevant markets are typically further segmented—with firms supplying different goods and services to different customers. Compared to measures of market concentration, it has the additional advantage of capturing rivalry from both domestic and international competitors, which is particularly important in a small, open economy such as that of Switzerland.

Table 2 Mean values and standard deviations (sd) for our variables across the different waves of the survey

Innovation effort is measured by the R&D activities of a firm. The variable takes the value 1 if the firm has no R&D activities. It takes the values 2, 3, and 4 if the sales share of R&D expenditures is lower than 1.5 %, between 1.5 % and 5 %, or above 5 %, respectively. Following Peneder (2010), the initial intention was to include information on the external acquisition of new knowledge (e.g., buying machinery, licences, or external R&D) for a multidimensional representation that also considers innovation effort other than own R&D. Unfortunately, these data were not available for the Swiss sample.Footnote 8

The variable for innovation outcome takes the value ‘1’ if the firm has not introduced any new technologies. Apparently, these firms have pursued opportunities other than those arising from technological innovation. The value is ‘2’ if a firm merely adopts a new technology. A value of ‘3’ indicates that product or process innovations are predominantly developed in-house, even if not considered new to the market. Finally, the entrepreneurial status takes the value ‘4’ if the firm has made product innovations that are new to the market. Following the terminology of Schumpeter (1947), we associate the first two groups with ‘adaptive’ behaviour and the latter two groups with ‘creative’ behaviour.Footnote 9

The three dependent variables are affected by a number of confounders. In Fig. 1 we extend the framework by the impact of a vector of control variables X i that may simultaneously exert an influence on competition as well as research effort and innovation outcome. We consider, in particular, the perceived technological potential tp i , capital intensity k i , human capital hc i (proxied by average wages), the perceived growth of demand in the past 3 years g i as well as the expected demand growth in the coming 3 years \(g_i^e \), firm size s i , foreign ownership f i , export status e i , firm age a i as well as time dummies T t and industry dummies \(I_{t}\) (see Table 1 for further details on the variables).

To control for endogeneity, we seek instrumental variables Z i that are correlated with the endogenous variable and not with the error term.Footnote 10 For that purpose, we apply three complementary sectoral taxonomies, which characterize the prevalent technological regime in which firms operate (Peneder 2010). They were built from European CIS micro-data at the Eurostat safe centre. Statistical clustering algorithms were applied to the standardized distributions of heterogenous firm types. In Eq. 1, where the individual firm’s research effort is the dependent variable, we apply the typical sector distribution of opportunity conditions O j among the EU countries. In Eq. 2, where we aim to explain the transmission from research effort to innovation outcome, our instrument is the typical characterization of a sector in terms of the cumulativeness of knowledge M j . The latter had been identified by combining information on innovation outcome and the relative importance of external vs. internal knowledge for creative and adaptive firms. In Eq. 3, where we estimate the number of competitors conditional on innovation success, we take the sectoral appropriability conditions A s as the instrument. This taxonomy was clustered from differences in the distribution of EU firms applying patents or other formal and strategic means to protect their innovations.

The three sector taxonomies offer valid instruments. They are strictly exogenous to the dependent firm variables: first, because firms are too small (or industries defined too broadly) for any reverse causality; second, the Swiss firms studied here were not included in the EU micro-data used for the clustering of the technological regimes in Peneder (2010). All of them are correlated with the endogenous variable (Anderson canonical correlation test (under-identification test)), while the fact that they are predetermined guarantees (by assumption) that they are uncorrelated with the error terms. In estimations with more than one instrument, the Sargan Test (over-identification test) has also been passed by the instruments (see Tables 3, 4 and 5).

Table 3 3 SLS estimations for the innovation opportunity function (innovation effort E i is the dependent variable)
Table 4 3 SLS estimations for the innovation production function (innovation outcome I i is the dependent variable)
Table 5 3 SLS estimations for the innovation impact function (number of principal competitors C i is the dependent variable)

There is, however, more than a technical side to those instruments. In his survey of the literature, Gilbert (2006, p. 162) complains that “one reason why empirical studies have not generated clear conclusions about the relationship between competition and innovation is a failure of many of these studies to account for different market and technological conditions”. Cohen (2010) makes the same point. ‘Opportunity conditions’, ‘appropriability’, and the ‘cumulativeness of knowledge’ are prominent examples of such ‘technological conditions’ (see, e.g., Winter 1984; Malerba and Orsenigo 1993, 1997; Malerba 2007). Among the most notable empirical applications of technological regimes, Breschi et al. (2000) demonstrate the impact of the technological regimes on the structural characteristics of markets for innovation. More recently, Castellacci and Zheng (2010) show that these characteristics help to discriminate between the different role of technical progress and efficiency improvements in explaining productivity growth.

Sensitivity analysis has shown that the stated relationships between competition, R&D and innovation are robust. We used a different set of valid instruments other than the taxonomies. The new set of instruments is measured at the firm level, they are time-variant, and pass the above mentioned overid-test and underid-test (see Table 9 in the Annex 2). Since one might think that the squared competition variable \(\left (\mathbf {C_i^{squared}}\right )\) is also suspected to be endogenous, we have instrumented the squared competition variable in a reduced form, inserted the estimated values into the main function and bootstrapped the standard errors. The innovation obstacle of ‘too high taxes’ as reported in the survey was used in order to instrument \(\mathbf {C_i^{squared}}\). ‘Too high taxes’ are clearly beyond the influence of a single firm and hence exogenous to their behaviour. Moreover, in this dataset it is correlated with \(\mathbf {C_i^{squared}}\). In this procedure the observed relationships between competition, effort, and innovation also do not change (see Table 10 in the Annex 2); for the reduced form please refer to Table 11 (in the Annex 2).

While the basic structural model shows to be robust to alternative instrumentation strategies, our preferred choice are the technological regimes. Not only does the empirical innovation literature highlight their importance, but also the assumed causal linkages reflect important theoretical considerations and thereby enhance the model. Furthermore, the particular way the instruments were constructed supports their assumed exogeneity. As a general note of caution, one should however mention that some causal influence of Swiss firms in the sample on the sector characteristics of other European countries is still a theoretical possibility. But any resulting correlation with the error term is extremely unlikely to be of a significant statistical magnitude.

4 Econometric estimates

Least-square estimation would be both biased and inconsistent, because the error terms are correlated with the endogenous variables. Hence, we apply a three-stage, least-square estimation (3-SLS). In the first stage, the reduced form of the model is estimated. In the second, the fitted values of the endogenous variables are used to get estimates of all the equations in the system (2-SLS). In the third and final stage, the residuals of each equation are used to estimate the cross-equation variances and co-variances, and generalized least-squares parameter estimates are obtained. By taking into account the cross-equation correlations, the 3-SLS procedure yields more efficient parameter estimates than the 2-SLS (Madansky 1964).

Tables 3, 4 and 5 report detailed results for all three equations from the simultaneous system. Here, we only summarise the main and robust findings, all of which are statistically significant. Beginning with the innovation opportunity function, our simultaneous system depicts a robust and nonlinear inverted-U shaped effect of competition C on research effort E. A higher number of competitors increases the firms’ probability to conduct own R&D, but does so at a diminishing rate. While R&D expenditures reach a maximum at intermediate levels of competition, they decrease with the number of competitors when initial competition is high. When we further split the sample into the two groups of ‘creative’ and ‘adaptive’ entrepreneurs, the inverted-U is still a robust observation for both groups, but much steeper for the ‘creative’ entrepreneurs. This implies that their research effort is more sensitive to changes in the intensity of competition.

Among the control variables, the perceived technology potential and growth of demand for the main product, firm size, and exports have a positive impact on R&D expenditures, whereas foreign ownership has a negative effect. While different in size, the sign of all these effects is consistent for both ‘creative’ and ‘adaptive’ entrepreneurs.

With regard to the innovation production function, research effort E associates positively with innovation outcome I. High R&D expenditures raise the probability of being a creative firm with own innovations, whereas no R&D expenditures indicate that entrepreneurs seek their profits from sources other than technological innovation (Peneder 2009). Among the exogenous variables, firm size, age, exports, and the cumulativeness of knowledge only have a positive impact for creative entrepreneurs. In contrast, we find a significant negative impact of exports, technology potential and the cumulativeness of knowledge for adaptive entrepreneurs, with a positive effect of foreign ownership.

Finally, turning to the innovation impact function, the effect of innovation I on the number of principal competitors C is also straightforward and consistently negative. In other words, creative firms with own innovations face the lowest number of competitors. Technology adopters tend to operate in an intermediate range of competition, and firms pursuing profits from sources other than technological innovation have the largest number of competitors. Among the control variables, the number of competitors increases with firm size (presumably due to larger aspired markets), firm exports, technological potential, and an increase in demand for the main product.

In a next step, we apply the empirical estimates to the analytically reduced form of our system (Annex 1). Figure 2 presents the estimates for the two samples of ‘creative’ entrepreneurs with own innovation, and ‘adaptive’ entrepreneurs, who either pursue new technology from external sources or do not innovate at all. On the x-axis we have the categories for the number of principal competitors; on the y-axis we have innovation effort (after substitution of the innovation production function into the innovation outcome variable). The quadratic innovation opportunity function is depicted by E = f(C). The linear innovation impact function is expressed as C = f(E), after substituting innovation outcomes I by the innovation production function (see Eq. 2). Due to the quadratic nature of the opportunity function, the numerical solution displays two possible equilibria where both functions intersect. Both are within a valid value range. Characteristically, in the one equilibrium firms perform higher innovation and face low-to-intermediate competition, whereas in the other equilibrium firms display lower innovation with more intense competition. Figure 2 also illustrates that the inverted-U shape is much steeper for creative entrepreneurs than for adaptive ones. This means that, for both the positive and the negative slope of the schedule, any change in competition will affect the R&D expenditures of entrepreneurial firms more strongly than those of adaptive firms. Second, the higher intercept at the y-axis for both functions demonstrates that entrepreneurial firms generally exhibit a higher level of innovation. Finally, the innovation impact schedule is steeper for entrepreneurial firms. The implication is that for any given change in innovation, the impact on the number of competitors is greater for adaptive firms than for entrepreneurial ones.

Fig. 2
figure 2

Estimated model for adaptive and creative entrepreneurs (split sample). The y-axis pictures the firm-level R&D effort, whereas the x-axis depicts the respective categories for the number of competitors

5 Forces behind dynamic adjustment

Our simple simultaneous model does not explain the dynamics of how firms and markets find either of the possible equilibria. In this section, we briefly conjecture about the likely forces that drive the adjustment mechanism.Footnote 11 Inevitably, any such process depends on critical assumptions. In our case, we start by acknowledging that only research effort is a parameter of choice for the individual firm. In contrast, intensity of competition is the outcome of the joint interaction of all firms in the market. An individual firm can only influence this indirectly via its own innovation. Consequently, for a given intensity of competition, the firm chooses the research effort according to its position relative to the innovation opportunity function. If the firm’s actual position is below the function, it aims to increase innovation; if it is above the opportunity function, it tends to decrease its innovation effort. These forces are represented by the vertical arrows in Fig. 3.

Fig. 3
figure 3

Forces behind dynamic adjustment

The market reaction in terms of a changing number of competitors is captured by the slope of the innovation impact function (into which we have already substituted the estimates from the innovation production function). When a firm finds itself below the impact function, the consequence will be an increase in the number of competitors. The reason is that the firm’s research effort is not sufficient to protect its position. This results in a horizontal shift to the ‘east’ in Fig. 3. Conversely, if a firm is above the impact function, the number of competitors will decline, since better innovation performance buys more market power.Footnote 12 This implies a shift to the ‘west’. These forces are represented by the horizontal arrows in Fig. 3. Finally, we assume that a firm’s R&D expenditures will generally rise (fall), if it initially starts from below (above) both schedules. This allows us to focus on the more interesting areas that lie on or between the two graphs in Fig. 3.

Depending on whether the opportunity function lies below or above the impact function, and whether it has a positive or negative slope, we can now associate all possible initial positions outside equilibrium with any of four different cases. Table 6 summarises the directions of change for innovation activity and the number of competitors, respectively.

Table 6 Four patterns of dynamic adjustment

To begin with area I to the very left of Fig. 3, the innovation opportunity function lies below the innovation impact function. Innovation is too low to keep rivals out of the market and the number of competitors will increase. Since the slope of the opportunity function is positive, growing competition leads to an increase in research effort until the two functions intersect. At the intersection the system is in equilibrium E1, because on the opportunity function the firm has no incentive to alter its effort, and the number of competitors will also not change, because it is already consistent with the impact function.

While the above rationale explains the general direction of adjustment, the exact moves and their sequence depend on further details that we do not need to specify for the purpose of this analysis. To provide an example, if the market reaction is slow and/or hardly anticipated by the firm (i.e. firms are very myopic), we should expect an initial drop in the firm’s research effort until it hits the opportunity function. Once there, the market pressure towards more competition (i.e. a however tiny move away from the opportunity function to the ‘east’) will incite the firm to gradually raise its R&D until it again reaches the intersection of the two functions. Conversely, less myopic firms that are better at anticipating the market reaction may take a shorter route towards equilibrium within the described area.

Turning to that part of Fig. 3, where the impact function lies below the opportunity function, we must distinguish between area II, which is characterized by a positive slope of the opportunity function, and area III, where that slope is negative. For any given level of innovation above the impact function, adjustment works to the ‘west’—that is, the number of competitors tends to decrease. Because of the positive slope of the opportunity function, in area II the decrease in competition implies fewer incentives for innovation. Thus, any firm in area II will move towards equilibrium E1. Conversely, if the firm is located in area III, the negative slope of the opportunity function implies that any decrease in the number of competitors incites an increase in R&D. From this it follows that even a small deviation to the ‘west’ of the second intersection (equilibrium E2) will attract the firm farther away from it and to the ‘north-west’ of the graph. At the peak of the opportunity function the same forces still drive the firm ‘west’, i.e. out of area III into area II. Now the slope of the opportunity function turns positive, which suggests that a decline in the number of competitors leads to a reduction in R&D expenditures and carries the firm further towards E1. Consequently, equilibrium E1 is the stable attractor for all initial positions in the area below the opportunity and above the impact function, i.e. for both areas II and III.

Again, our argument provides for the general direction and outcome, whereas the exact moves within the two areas depend on how well firms anticipate change and how quickly markets react. For example, if markets react slowly and myopic firms start in area III, they will first increase innovation to the level of the opportunity function and then move further up along the curve to its peak, apparently trying to escape the high level of competition. Only after the peak will they start to realize that they have overinvested in innovation and subsequently reduce their effort. Because the firm’s level of innovation is still above the impact function, the number of competitors will decline, despite the firm conducting less innovation. This mechanism operates until the firm is in equilibrium E1. Of course, this process is costly and the described overinvestment is a high price to pay for being myopic. If the same firm is able to better anticipate the market reaction, it prefers to move towards E1, closer along the impact function, thus avoiding excessive R&D expenditures.

In area IV the opportunity function is again below the impact function, which means that innovation is too weak to stave off competitors and competition increases. In contrast to area I, the slope of the opportunity function is negative, which implies that the incentives for innovation also decline. Consequently, the firm drifts farther to the ‘south–east’ until it hits bottom at zero innovation. Equilibrium E3 is thus another stable corner solution where firms are trapped in a situation of no innovation and extremely high competition.

As a consequence, the unstable equilibrium E2 is a saddle point which defines the watershed between two basins of attraction. While itself depicting a consistent configuration of innovation and competition, any slight deviation into areas III or IV , would set the firm moving towards the high-innovation equilibrium with low competition E1, or towards the ‘no-innovation trap’ with very high competition E3, respectively.

Before we summarise the various equilibrium configurations, let us briefly reflect on the opposite corner solution of an uncontestable monopoly. If a monopoly is contestable, i.e. incumbents must fear displacement by new rivals, the same forces discussed for area I apply and equilibrium E1 is the stable attractor. However, what happens when the monopoly is legally protected? Arguing from our simple model, this would suggest that the impact function no longer matters. The firm only considers the value of its opportunity function, which will be either zero or very low, pointing towards another hypothetical outcome E0 in Fig. 3, which however is not strictly part of our system. Still, since no market forces threaten to drive the firm out of this position, one may easily conjecture that this is another stable corner solution and characterized by no competition and no or very low innovation.

In short, by distinguishing between the slope of the opportunity function and its position relative to the impact function, we can explain four distinct processes that cover all possible initial positions outside equilibrium. In area I, where the opportunity function is below the impact function and has a positive slope, competition invariably increases. Innovation may initially drop (until it hits bottom at the opportunity function), but must then increase to defend the firm’s position in the market. In area II, where the opportunity function is above the impact function and has a positive slope, both competition and innovation decrease as a consequence of the initial overinvestment in innovation. In area III, where the opportunity function is also above the impact function but has a negative slope, innovation tends to rise and competition must decrease. Finally, in area IV , where the opportunity function is again below the impact function and has a negative slope, innovation decreases and competition grows.

Based on these rationales, we may conjecture three possible solutions to the system two of which are stable. First, we find a stable equilibrium E2 that is characterised by high-innovation and low competition. Second, the unstable equilibrium E2 combines low innovation with high competition. Third, equilibrium E3 constitutes a stable corner solution in which firms are trapped with no innovation and very high competition. The unstable equilibrium E2 indicates the watershed between the two basins of attraction. For a lower number of competitors, firms are attracted towards the high innovation and low competition configuration E1. Conversely, if the intensity of competition is higher than in E2, firms drift towards the ‘no innovation trap’ E3 where competition is extremely strong. Finally, not covered by our data but rather straightforward to conjecture, there is the possibility of a corner solution E0 for a legally protected monopoly with low or no innovation.

6 Simulation of exogenous change

In addition to the individual estimated coefficients discussed in Section 4, we wish to know how exogenous changes affect the endogenous variables, when all the interactions of our system are simultaneously taken into account. Focusing on the stable non-corner solution E1, we successively change the values of an exogenous variable by one unit (only the categorical variables were significant in the regressions), and ceteris paribus leave all other exogenous variables unchanged. Table 7 displays the simulation results for the total sample as well as the two subsamples of firms characterized by an adaptive or creative entrepreneurial regime in terms of the induced average percent change of the endogenous variable.

Table 7 Simulated impacts of increase in exogenous variables (average effects in percent)

To begin with the subjectively perceived technology potential, an exogenous change by one unit mainly alters the equilibrium configuration by increasing R&D expenditures by 5.6 % on average within the total sample. Similarly, we observe a higher innovation outcome of 3.3 % on average in terms of the entrepreneurial status of the firm. For both variables, the effect is somewhat stronger among creative entrepreneurs than it is among adaptive ones. In contrast, the new configuration makes practically no difference in terms of the number of competitors.

For demand growth over the past 3 years, the impacts are generally weak but strongest with respect to research effort and the outcome of adaptive firms. In contrast, expected demand growth over the next 3 years appears to affect the innovation of entrepreneurial firms more strongly. The impact of demand growth on the number of competitors is generally weak, but consistently negative. This suggests that the higher incentives for R&D tend to dominate the potential effects of a larger market size on the number of competitors.

When we shift our attention from technology potential and demand growth to the specific characteristics of the firm, our structural model confirms the traditional Schumpeter hypothesis of a positive effect of firm size on research effort. Moving up the size classes generally increases the probability that R&D and innovation will rise, but leaves the overall number of competitors unaffected. The export dummy has a consistently positive impact on research effort and innovation outcome, slightly more so for entrepreneurial firms than for adaptive ones. Again, we find little impact on the number of competitors. This is also true for foreign ownership, which however negatively affects own R&D. The negative impact on innovation outcome is less pronounced, indicating the importance of knowledge transfer within multinational enterprises.

Turning to the three instrumental variables capturing aspects of the technological regime at the sector level, we also observe strong effects on our subjective, firm-level measure of competition. This is most pronounced for the sectoral taxonomy of opportunity conditions, where a generally high level of innovation activity causes the number of competitors to decline. In contrast, a high appropriability of new knowledge tends to increase the number of competitors, probably because of the better protection of returns among innovative small and medium sized companies. This would also explain why high appropriability increases both R&D expenditures and innovation outcome. This finding nevertheless came as a surprise and calls for further research. Finally, a change in the cumulativeness of knowledge makes the least difference, but tends to raise the number of competitors for adaptive firms.

Our results on the technological regimes complement the findings of Breschi et al. (2000), who estimated the impact of opportunity, appropriability and cumulativeness on market structure measured in terms of innovation success (i.e. the share of new innovators as well as the rank correlation and concentration ratio of patenting firms). Like them, we add evidence for the explanatory power of technological regimes and demonstrate their importance for the empirical analysis of innovation and competition.

To summarise, our estimates of the main relationships among the three endogenous variables are embedded in a system of exogenous forces that affect the firms’ research effort and innovation outcome as well as the intensity of competition. While our simple structural model certainly has its limitations and casts aside many important questions (for example, with respect to refined strategic interactions and precise information sets), it has offered a comprehensive and meaningful frame for the empirical studies.

7 Summary and conclusions

Based on a rich firm-level database for Switzerland, we estimate a simultaneous system of three equations. In the first equation, the innovation opportunity function tests the presumed inverted-U relationship between the intensity of competition, as measured by the number of principal competitors reported by the firms, and the firms’ research effort. Second, the innovation production function controls for the relationship between research effort and innovation outcome. The final innovation impact function provides the estimates of how successful innovation affects the number of competitors.

We apply 3-SLS system estimates to control for endogeneity. The findings confirm a robust inverted-U relationship, where a higher number of competitors increases the firm’s research effort, but at a diminishing rate. Technology potential, demand growth, firm size, and exports have a positive effect, while foreign ownership has a negative impact on innovation. Splitting the sample by firm types, the inverted-U shape is steeper for creative firms than for adaptive ones.

In recent years, new and inspiring theoretical models, together with the diffusion of advanced econometric methods and a broader availability of micro-level data, have fueled a rapidly growing literature on the relationship between competition and innovation. At the same time, competition policy is increasingly concerned with the lack of robust findings and a growing ‘complexity trap’ (Shapiro and Lerner 2011). When opposing conclusion can be supported by varying a few assumptions within increasingly refined theoretical models, and little data is available to test for their empirical validity, the paradoxical consequence is that no conclusions can be drawn—at least none that would be sufficiently resilient for the purpose of policy-making.

We have therefore based our analysis on fairly general and straightforward decision-theoretic rationales. Empirically, we have placed emphasis on the robustness of our results, in particular with respect to variations in the control and instrumental variables, the use of industry dummies, and how we dealt with the quadratic term in an endogenous model. These variations could change the signs of a few control variables or turn significant instruments into weak ones. However, as long as we estimate our structural model in a simultaneous system, the inverted-U relationship proves strikingly robust. Still, we must acknowledge that the findings are strictly valid for the population of Swiss firms only. Further studies using a similar set-up with firm samples from other countries are warranted to boost confidence in our findings.

Our analysis suggests two general lessons for economic policy, which we expect will also hold in environments outside of Switzerland. First, with regard to competition policy, the inverted-U relationship implies that we only find a negative impact of competition on innovation at high levels of initial competition. In other words, the negative Schumpeter effect does not arise in typical situations involving antitrust authorities. In markets with few competitors we should generally expect a positive impact of competition on innovation. Therefore, one should be critical about the incentives for innovation when used against the enforcement of antitrust measures.Footnote 13 On the contrary, in highly concentrated markets, antitrust measures tend to increase both competition and innovation. Second, the simultaneous possibility of different equilibria—for example, one with high innovation and an intermediate-to-low degree of competition, and the other with no innovation and very high competition—may provide a rationale for industrial policies that can help propel the system out of a ‘no-innovation trap’ and gear it towards a higher innovation trajectory.

As a final note, we wish to address the decade-long contest to prove the dominance of either a negative Schumpeter effect or a positive Arrow effect of competition on innovation. Our discussion has revealed three stable outcomes. In the first instance, monopoly is legally protected and hence uncontestable. Here, innovation will be low or nonexistent. In contrast, another stable solution is characterized by low competition and high innovation. Moving from a monopoly to a degree of (still low) competition increases innovation, and is thus consistent with the way in which Arrow (1962) framed his argument for a positive effect of competition on innovation. In contrast, the third stable equilibrium is characterized by no innovation and very high competition. Comparing the second with the third equilibrium, our estimates are also consistent with Schumpeter’s negative impact of competition on innovation, illustrating his point that own innovation is impossible within a market of ‘perfect competition.’ Acknowledging that Schumpeter always discussed monopoly as contestable through new innovation, and that Arrow considered contestable monopolies to be competitive, the two effects almost naturally fall in line with their respective ranges of initial competition. The inverted-U relationship as modeled by Kamien and Schwartz (1976) or Aghion et al. (2005) manages to integrate them into a common framework.