Supporting the Evolution of Research in Software Ecosystems: Reviewing the Empirical Literature

Abstract

The field of software ecosystems is gradually transiting towards an established means of software development and distribution, counting numerous areas of applicability. However, research in software ecosystems, although the activity of over 10 years, is still characterized as premature with significant lack of software ecosystem specific theories that are solid, mature, generic, and detailed enough to be measurable and transferable. In this study, we intent to come closer to an evolution of the field by supporting the “localization” of research, i.e. the focus on specific types of software ecosystems. To do so, we investigate the literature of empirical, non open source ecosystem studies and intent to identify the various aspects and perspectives studied.

In total, we review 56 empirical studies that investigate 55 software ecosystems. Our analysis confirms the assumption that proprietary software ecosystem studies lack deeper investigation of technical and collaborative aspects. Moreover, we identify an increased focus on organizational aspects and a rather limited focus on business. Furthermore, we identify common technology as the component investigated most in the ecosystems, both from the technical aspects, but also as means of applying orchestration. Finally, comparing the main areas with the overall ecosystem literature, we identify that empirical studies lack representation of health, motivation, actor activity, reusability, integration, and quality of ecosystems.

1 Introduction

The field of software ecosystems has arguably moved from a new and upcoming field to an established means of developing and distributing software products, functions, or services. Currently, it is the most viable option of software product development in several domains and is counting examples in numerous other. Although the field has been active in research in the course of more than ten years, it can be argued that research is still scratching the surface of the field. The field can be characterized as one that is counting numerous and constantly increasing studies that might go into depth in a specific aspect and/or type of software ecosystems, but find it challenging to make contributions that robust, while abstract enough to be applied to different types of ecosystems.

The case of software ecosystem health is a representative example of the evolution that theories in the field have been following. Software ecosystem health can be described as “the ability of the ecosystem to endure and remain variable and productive over time” [1]. It has been defined in the context of natural (biological) ecosystems and has appeared in software ecosystems mainly inspired by business ecosystems health¹. Health, has been of focus and an important aspect from the early times of software ecosystem research [7], however the work on this aspect can be mainly characterized as either (a) too abstract, and thus not directly applicable (e.g. [1, 8, 9]), or (b) too specific, and thus challenging to be transfered to other (types of) ecosystems (e.g. [10–12]). Similar challenges can be noted in the governance ² of software ecosystems [15–18].

This has as a result that, although theories and concepts evolve in the field, software ecosystems are still lacking a basic level of knowledge that is tailored to the specific needs of problems in the field. These theories are not solid and specific enough to allow for measurable results, while being abstract enough to allow for transferability (i.e. applied on ecosystems of different characteristics). This becomes magnified when taking into consideration the big variability in and differentiation of types of ecosystems existing.

This lack of specific theories is something that is also noted in the most recent and extensive systematic literature study of software ecosystems. [19] studies the literature of the field consisting of a total of 231 papers, spanning form 2007 to 2014. While examining the evolution of the field to characterize, among other, the field maturity, it is identified that the existing literature can be categorized as: empirical but specific, where one or more ecosystems are studied as means of addressing a problem, while the problem or the solution being highly coupled to these ecosystems; temperature measuring, where different theories, tools, or methods, usually imported from another field, investigate a problem that results in interpreting results based on assumptions. Furthermore, two steps for the evolution of research in the field towards its better maturity are proposed: (i) research scoping, where research should set more focus on defining the specific ecosystem parameters that study results are applicable, and (ii) theory building, where research should focus on defining theories that are designed for the specific characteristics and problems of software ecosystems. In order for (i) to be accelerated and have better results, some “ground-work” should be done on identifying and defining sets of parameters that separate the different types of ecosystems and their variability.

In this study, we intent to contribute towards an arguably higher level of maturity in the field by supporting work towards a better scoping of future research, as mentioned in (i). Our aim is to investigate the work studying existing ecosystems and identify what aspects of (existing) software ecosystems are (empirically) studied. One of the most common differentiation of ecosystem types is the separation between ecosystems that are driven or supported by free and open source software (FOSS) and ecosystems that are driven or supported by proprietary software. Literature studying FOSS ecosystems, as identified by [7], tends to have deeper study of technical and collaborative aspects but might lack organizational and business perspectives. On the other hand, proprietary ecosystem studies tend to have do the opposite. In reality, the boarders of this polarization tend to be more obscure, as there are several ecosystems that support both FOSS and proprietary contributions or are based on both FOSS or proprietary common technologies.

In this study, we focus on proprietary ecosystems and intent to identify the various aspects and perspectives studied. More specifically, we review the empirical literature of proprietary software ecosystems, i.e. the literature of that studies some aspect(s) of an existing non-FOSS software ecosystem, to identify what ecosystem characteristics are defined. We do so, by reviewing 56 papers that study a total of 55 existing and non-FOSS software ecosystems. Our results reveal an increased focus in organizational aspects of software ecosystems, a restricted focus on business, with rather limited aspects of revenues and monetization. Moreover, our analysis confirms the view of proprietary ecosystem studies having limited access to proprietary information, such as source code, with a distinct lack of studies of the software perspectives of ecosystem contributions. The literature puts the most focus on the common technology both from the technical perspective but also as a means to apply orchestration to the ecosystem. Another main focus of the studies is the actors of the ecosystem and the relations among them. Finally, we compare the main areas of the empirical studies with the overall ecosystem literature and identify that ecosystem aspects such as health, motivation, actor activity, reusability, integration, and quality are not represented.

2 Related Work

The field of software ecosystems counts a number of secondary studies, but to our knowledge, none with primary focus on investigating the implications of empirical studies of existing software ecosystems. The systematic literature review of [7] identifies, among other, 42 software ecosystems in a literature body of 90 papers from 2007 to 2012. Manikas [19] expands this list to 108 for a literature body of 231 papers from 2007 to 2014.

In the context of secondary studies, Barbosa and Alves [20] conduct a mapping study of the literature of software ecosystems up to 2010 identifying 44 papers. Among their findings, they note that 10 studies were based on case studies. Santos et al. [21] combining the literature from that study and their previous study [22], identify four dimensions of software ecosystems: technical, business, social, and management - engineering. Hansen and Dybå [23] intent to build an overview of theories used in the literature. By reviewing a literature body of 40 papers, they identify a set of theoretical areas while using the concept of “organizational ecology”. Handoyo et al. [24] use the roles identified [7] and [20] to create a classification of ecosystem roles. Manikas and Hansen [1] focus on the context of health and review the literature of software ecosystem health and related areas. They find that software ecosystem health is heavily inspired from business ecosystems and propose a framework for the measurable definition of software ecosystem health. Fotrousi et al. [25] map the literature of software ecosystems to identify key performance indicators used in software ecosystems. They map 34 papers from software and digital ecosystems and identify right measurement attributes spread across seven entities. Franco-Bedoya et al [26] review part of the literature of open source software ecosystems (a total of 17 papers) to identify quality measures and provide input to their proposed quality model.

3 Method

In this study, we review the empirical literature of software ecosystems that is not build on a FOSS ecosystem, i.e. the academic literature that includes the study or investigation of an existing proprietary software ecosystem. This literature was identified as part of the analysis of the software ecosystem literature in the systematic literature study of [19]. This literature study was designed according to the guidelines of Kitchenham and Charters [27], using a similar protocol with the second most recent and extensive systematic literature study of software ecosystems [7]. This protocol includes the literature search in a list of academic libraries³. Moreover, the literature body was expanded with the papers from the International Workshop of Software Ecosystem (IWSECO) for years 2007-2004, the Workshop on Ecosystem Architectures (WEA) for years 2013-2014, the special issue on software ecosystems of the Journal of Systems and Software, and the special issue on software ecosystems of the Journal of Information Technology. All the identified literature contains the words “software ecosystem(s)” in either of the fields title, abstract, or keywords.

After we define the collected literature, we analyze it using the three structures of “software ecosystem architecture”, proposed by [28], and the ecosystem components, proposed by [29].

Christensen et al. in [28] investigate means of modeling software ecosystems where, based on the design of a software ecosystem, they propose the ecosystem analysis and modeling using the concept of software ecosystem architecture. This concept consists of three main structures that are necessary for the design and well-functioning of a software ecosystem:

• Organizational structure. That covers aspects of the ecosystem related to the orchestration of the ecosystem elements, such as actor and software elements, as much as possible connections and interactions among these elements.

• Business structure. That covers aspects of the ecosystem related to the creation of value. This is examined both from the perspective of the ecosystem, i.e. how is value added to the ecosystem, and the perspective of the ecosystem element, e.g. how is an actor gaining value from the contribution to the ecosystem.

• Software structure. Covers aspects that relate to the software elements of the ecosystem, such as the structure of the common technology or the contributions to the ecosystem.

Moreover, in order to be specific and identify what elements of the ecosystems are studied, we use the approach of the ecosystem components. Knodel and Manikas in [29] propose a typification of software ecosystems challenging the existing ecosystems definitions. In this work they identify a number of components that ecosystems are consisted of⁴. They describe that an ecosystem is build on top of a common technology, that supports the interaction of a set of actors. The actors are part of the ecosystem by having an activity that results in one or several contributions to the ecosystem. The contributions can be of variable nature such as a (software) product or component, a service, or data (information). Each actor’s activity in the ecosystem is motivated by one or several incentives. Moreover, the ecosystem exists and operates on a specific environment. The environment might include the domain of the ecosystem and the physical or digital aspects surrounding the ecosystem, while it can pose different requirements, or constraints to the ecosystem.

We use the three ecosystem structures and the five ecosystem components to analyze the ecosystem studies. Each study was analyzed and categorized according to what structure(s) it addresses and what are the main components investigated. One study can be categorized in more than one structure (e.g. both organizational and business) and have up to three components. The components classification was prioritized, e.g. a study can primarily focus on the common technology of an ecosystem with (secondary) focus on contributions.

4 Analysis

Our literature body includes 56 studies spanning from 2008 to 2014⁵. In total the literature studies 37 different, existing, named, software ecosystems and 18 anonymized.

Figure 1, shows how the papers are distributed in the three structures and what ecosystem components are the main focus for each structure. As it can be seen, the organizational structure has the largest focus with roughly $61\,\%$ of the total literature, while business has $37\,\%$, and software $39\,\%$ ⁶.

Fig. 1.

Number of papers and ecosystem components according to ecosystem structures (Color figure online).

The most common component analyzed in the studies focusing on organizational structures is the ‘actor’, being the main focus of approximately $47\,\%$ of the studies within this structure. The most common second component appearing after actor in the organizational structure is ‘incentive’ and the third ‘common technology’. This gives a good indication that many of the studies have been focusing on the network and relationships of actors and examining those from two main perspectives: the actor incentives and the common platform as means of facilitating actor relationship. In the business structure, it is not a surprise that the most common component is ‘incentive’ ($33\,\%$ of the studies in the structure). As expected, the most common component that comes with ‘incentive’ is ‘actor’. In the software structure, the most common component is ‘common technology’ ($77\,\%$) with most of those studies having ‘contribution’ as the second component.

If we examine the components independently from the architecture structures, we note that ‘common technology’ is the most common component⁷. Common technology usually has as a second component ‘contribution’ and that is mainly in the studies belonging to the software structure. These studies typically investigate the influence of software engineering aspects, such as software architecture, to the ecosystem. Another component that comes (to a less extent than the contribution) is ‘environment’. These studies typically investigate the technical aspects of the common technology and how it poses additional restrictions, limitations, or specific rules to the ecosystem. The second most common component studied is ‘actor’. Actor usually has a second component ‘incentive’ and, to a less extent, ‘common technology’. What might come as a surprise in this context is the fact that contribution is not one of the most common components coming after actor. So there is not many studies⁸ that investigate the actors and their contributions in this data set. This can be explained by two reasons: (i) the nature of the empirical sets, i.e. non-FOSS ecosystems, make it hard to study specific contributions in detail and (ii) the contribution component is in general not very studied (possibly also due to (i)).

Table 1. Analysis of software ecosystem structures and components.

Table 1 contains the identified main foci of the studies according to the ecosystem structures and components. Looking at the table, our first remark is that incentive and environment do not appear in the software structure. Moreover, the contribution component is not very studied in this structure. That, in combination with what is actually studied in the software structure, makes the statement of the nature of the ecosystems not allowing for deep analysis into contributions even stronger.

Examining the organizational structure, we note that actor, environment, and common technology are of focus. The representation of the common technology here provides the view of a common technology used as a means of imposing orchestration rules and strategies.

Looking at the business structure, we note a more spread distribution in the foci of components with the other two structures. This implies that different aspects of the business and value creation on software ecosystem are investigated. From the other side, it is notable that the central aspects of business, i.e. monetization and revenues, are arguably under-represented. While, intersection of business structures and organizational structure is more emphasized in the studies. This is also explained by the fact that many ($52\,\%$) of the business structure studies are also categorized as organizational structure.

Finally, when examining the focus areas of all the studies, we notice that there are several perspectives⁹ that appear across components and structures. Furthermore, we notice that some of the ecosystem aspects that the literature has been focusing, including non-empirical literature, are not represented here. Comparing with the analysis of [19], that analyzes the literature and identifies trends, we note the following:

Organizational structure is lacking or is under-represented in studies of the aspects of health, and actor activity.

Business structure lacks focus on motivation, process, and innovation.

Software structure is focusing on the software architecture only in the level of the common technology (not contribution). Moreover, aspects like reusability, evolution, integration, and quality are not adequately represented.

5 Summary

In this study we review the empirical literature of existing proprietary (non-FOSS) software ecosystems to identify studied ecosystem aspects and perspectives. We identify a literature body of 56 empirical studies, studying a total of 55 software ecosystems. Our analysis includes the use of the concept of software ecosystem architecture and the three structures of software ecosystem modeling: organizational, business, and software structures. Moreover, we identify the main components studied in software ecosystem using the five ecosystem components: actor, incentive, common technology, contribution, and environment.

Our study confirms the assumption that proprietary software ecosystem studies lack deeper investigation of technical and collaborative aspects. Moreover, it reveals an increased focus on organizational structures and a rather limited focus on business with lack of revenue and monetization aspects. The most investigated ecosystem component is common technology, that is studied both as a technical but also as an orchestration element. Furthermore, actors, their incentives, and their influence to and from the common technology is also of focus in the studies. Finally, we compare the main areas of the empirical studies with the overall ecosystem literature and identify that ecosystem aspects such as health, motivation, actor activity, reusability, integration, and quality are not represented.

Appendix: Literature body

[9, 17, 30–83]