Keywords

1 Introduction

In recent years a large part of software development companies have changed focus from developing software as a product (SaaP), where companies developed the software and delivered a finished product to a customer that then installed and ran it locally, to develop software as a service (SaaS), where software is centrally hosted on a cloud infrastructure and accessed through for example a web-browser [1], or other channels that delivers it directly to a customer’s machine or device [2]. The use of it is then offered through licensing and subscriptions. With SaaS, the customers do not control the underlying cloud infrastructure or the application’s functionality [1], as that is done by the provider. This gives the provider the opportunity to continuously improve and deliver their software without having to redistribute it to all their clients as they simply update the software on their own cloud infrastructure. This modern software engineering process of developing while continuously integrating and delivering software is complex. Continuous integration (CI) means to automatically integrate new code from several developers into the same version of the software and at the same time, check for errors [3]. Continuous Delivery (CD) means to deploy new software to production, with the differing factor from traditional software deployment being the frequency of deployment, which can happen multiple times every day [3]. “Continuous delivery enables businesses to reduce cycle time so as to get faster feedback from users, reduce the risk and cost of deployments, get better visibility into the delivery process itself, and manage the risks of software delivery more effectively.” [4]. These processes require a large number of tools and information systems [5]. These processes, tools and systems are often managed by independent operations teams [6]. Many challenges when implementing CI/CD resulted from lack of collaboration and communication between the operators and developers [2, 3, 6, 7]. Attempts at overcoming these challenges have resulted in a concept, termed DevOps [2].

DevOps is described as the “conceptual and operational merging of development and operations’ needs, teams, and technologies” [6]. This merging is meant to align the priorities of the development teams and operations teams so they work together towards a common goal of successful project execution [6] by cooperating on software development and deploying that software into production [2]. This can be done by involving operations in all development stages, by developers and operators collaborating to solve problems, make processes and products that can be automated, and agree on and develop metrics that everyone can make use of [8]. This reflects the four main principles of DevOps: culture, automation, measurement and sharing (CAMS) [2, 4].

As DevOps has become more popular, many organizations are adopting the practices associated with it. However, a survey by the HPE Security Fortify team [9] from 2016 shows that while many believe that security should be a part of DevOps, security is not something many DevOps programs have included as part of their process. Gartner estimates in [10] that less than 20% of “enterprise security architects have engaged with their DevOps initiatives to actively and systematically incorporate information security into their DevOps initiatives”. [10] points to management, developers, and operators viewing security as an inhibitor to the agility and speed required in DevOps practices, like CI and CD, as one reason for this.

The need for security in DevOps is met by DevSecOps. This concept is an attempt at creating and including modern security practices that can be incorporated in the fast and agile world of DevOps. It promotes an extension to DevOps’ goal of promoting collaboration between developers and operators by involving security experts from the start as well [11].

Since DevSecOps is a new trend, it is important to obtain an overview of the practices and experiences accumulated on the subject. There is not a lot of research on DevSecOps, but a search of available literature shows: [12] is a study that through Internet artefacts and a survey looks at practitioners experiences with DevSecOps and the practices they perform, [11] is a systematic mapping (SM) study on what is being researched in the field and it showed research was being conducted on the aspects of “a definition, security best practices, compliance, process automation, tools for DevSecOps, software configuration, team collaboration, availability of activity data and information secrecy”. A search for systematic reviews or mappings on continuous processes (CI/CD) used in DevOps, resulted in a several results. Examples are: [13] examines the impact agile release engineering (ARE) and the continuous processes involved had on software engineering. [14] maps literature related to CD and provides an analysis of the benefits and challenges related to CD. [15] uses a literature review to show differences in how CI is done for different cases. None of the literature we found gives a collected overview of DevSecOps and what it is.

To the best of authors’ knowledge, there is not a systematic literature review on DevSecOps or a large body of scientific work related to DevSecOps. This absence of works devoted to the topic lead us to the need to work on the topic using a tool like multivocal literature review, intended to bridge the gap between professional and scientific literature. By this mean, authors examine the concept of DevSecOps, how it has evolved since it was first introduced, and the challenges and benefits DevSecOps brings to an organization.

The rest of this paper is structured as a systematic literature review. In Sect. 2 the methodology for the research is presented. In Sect. 3 we present the results from our study. In Sect. 4 we conclude on our paper, summarize the results and suggest future work.

2 Research Methodology

In this section, an overview of our research methodology is presented followed by an overview of the systematic approach used to gather relevant literature.

2.1 Multivocal Literature Review

After an initial search on literature to learn more on the topic of DevSecOps, we could not find a substantial body of academic research on the topic. We therefore decided to conduct a multivocal literature review (MLR). Multivocal literature is defined as all accessible literature on a topic [16]. This includes, but is not limited to: blogs, white papers, articles and academic literature. By using this variety of literature the results will give a more nuanced look at the topic, since it includes the voices and opinions of academics, practitioners, independent researchers, development firms and others with experience [16].

Previously published MLRs include but is not limited to: [17] is an MLR on automated software testing and the proposed guidelines from practitioners and researchers for when and what to automate. [18] is an MLR providing an overview of DevOps. [19] is an MLR on software test maturity assessments and test process improvement.

To the best of our knowledge, this is the first MLR on the topic although it is not the first for DevOps.

[20] points to the importance of MLRs in software engineering (SE) fields by stating that SE practitioners produces multivocal literature on a great scale, but that it is not published in academic forums. They mention however, that not including that literature in systematic reviews means researchers miss out on important current state-of-the-art practice in SE.

2.2 Research Questions

This MLR is conducted to obtain an understanding of what DevSecOps is, how it has evolved and the challenges and benefits of adopting such an approach. To specify the goal of this paper, four research questions were formulated:

Research Question 1: How does the literature define DevSecOps?

Research Question 2: What are the characteristics of DevSecOps?

Research Question 3: What are the main expected benefits and challenges of adopting DevSecOps?

Research Question 4: Since it was first mentioned, how has DevSecOps evolved?

2.3 Study Protocol

The study protocol describes the systematic way we found the literature used in our study. This section lists the databases used in the search, what search strategy was used to find related literature, the inclusion and exclusion criteria used to find the most relevant literature, and the process in which we catalogued the literature.

Databases. For this MLR we used Google’s search engines to find relevant literature:

Google’s search engines was chosen over more precise search engines (like Springer Link, ACM Digital Library, IEEE Explore etc.) because DevSecOps is a very new topic and very little academic research is available. We therefore knew beforehand that this literature review would rely mostly on the grey literature it would find, which Google’s search engines would be able to locate.

Search Terms. DevSecOps is a new term based on adding the term “SECurity” to DevOps which stands for “DEVeloper” and “OPeration”. There is not a consensus in the field on the ordering of the words, so the search terms must cover all possible permutations. The search string must also be made to find relevant literature according to the RQs. The search string used is therefore as follows:

figure a

Study Selection. Once initial search results were retrieved, a procedure to exclude irrelevant papers were conducted using the following inclusion and exclusion criteria:

  • Inclusion criteria:

    • Literature that explicitly discuss DevSecOps.

    • Literature that explicitly discuss DevOps and Security, particularly the challenges and benefits.

    • Literature discussing the present challenges to DevSecOps.

    • Literature discussing the benefits of DevSecOps.

    • Literature that discuss the definition of DevSecOps.

    • Literature published after 2014.

    • Include only the 5 first pages on Google Search.

  • Exclusion criteria:

    • Literature that is inaccessible.

    • Results Google Search deems to similar to other results.

    • Vendors tool advertisements.

Fig. 1.
figure 1

An overview of the search process to find relevant literature for this study.

Full size image

Search Procedure. The process is as follows: First we perform an advanced search in Google Search and Google Scholar. To let Google’s search engine put primary focus on the different RQs, the term will be split into 5 parts, each focusing on all permutations of DevSecOps and one of the words related to the RQs. The 5 search strings can be seen in Fig. 1. For each search we then read the literature systematically applying the remaining inclusion and exclusion criteria, selecting only relevant literature for the primary study. The process is visualized in Fig. 1:

3 Results

In the following section we show the results from executing our search followed by our review of the literature in conjunction with our research questions.

3.1 Search Execution

The search was performed on during the first quarter of 2017. The initial search gave 284 results. After applying the inclusion and exclusion criteria 52 results were left. Table 1 summarizes the search:

Table 1. Summary of search results for primary study
Full size table

3.2 RQ1 - How It Is Defined

In the literature we reviewed there seemed to be a consensus on what DevSecOps is seen as. DevSecOps is seen as a necessary expansion to DevOps, where the purpose is to integrate security controls and processes into the DevOps software development cycle [21] and that it is done by promoting the collaboration between security teams, development teams and operations teams [11].

3.3 RQ2 - DevSecOps Characteristics

When reviewing the literature, the features that stood out as characterizing DevSecOps were the principles seen as the basis and reasoning for DevSecOps and the practices used when implementing security into their software development processes.

Principles: The principles that characterize DevSecOps are based on DevOps and the CAMS principles [4, 22], culture, automation, measurement, and sharing, but with the addition of adding security from the start:

Culture: A DevOps culture promotes collaboration between development teams and operation teams [4], where they all accept that they are responsible for delivering software to an end-user [2]. DevSecOps means to include collaboration with the security team as well as promote a culture where operations and development also work on integrating security in their work [11, 23]. That means involving the security team from the planning stages, and making sure everyone agrees that security is everyone’s responsibilities [24, 25]. To get everyone to think security, practitioners points to creating a common mindset with regards to success by developing a set a metrics that everybody agrees on and can use [25], promote customer focus by creating an alignment of business and security strategies to ensure just right and enough security that everyone in an organization can support and implement [26, 27].

Automation: In DevOps the automation of build, deployment, and testing is important to achieve rapid development, deployment [4, 22], and feedback from end-users [2]. DevSecOps promotes a focus on automating security as well, to be able to keep up with the speed and scale achieved by DevOps. The aim should be 100% automation of security controls, where the controls can be deployed and managed without manual interference [28]. It is important to implement automatic security in a way that does not hinder DevOps’ agility in any way, which can cause friction [10, 27, 29].

Measurement: In DevOps measurements include monitoring business metrics such as revenue and key performance indicators, like the effect new releases have on the stability of a system, in order to know the current state and finding out how to improve it [2, 4]. DevSecOps promotes the use and development of metrics that track threats and vulnerabilities throughout the software development process [10]. Automatic security controls throughout the software development process means metrics are available to track threats and vulnerabilities in real-time and that allows the organization to verify how good an application is on demand [22].

Sharing: In DevOps developers and operators share knowledge, development tools, and techniques to manage the process [2, 4]. DevSecOps promotes the inclusion of the security team in the sharing promoted in a DevOps environment. By letting security teams know about the challenges faced by operators and developers, and vice versa, the security processes they develop will be improved [22].

Shift security to the left: In the traditional software development process, security is a step close to the end of the process. DevSecOps promotes a shift to the left for security, where it is to be included in every part of the software development process [23]. This means that security teams are involved from the very first planning step and is part of planning every iteration of the development cycle [29, 30]. It also means security is there to help developers and operators on security considerations [24, 28, 31].

Practices: Several practices for DevSecOps were pointed to in the literature:

Threat modeling and risk assessments: Practicing secure DevOps means that organizations have to develop expertise and processes to best discover, protect against, and find solutions to threats and risks [32], preferably ahead of time [25]. Performing risk assessments from the first planning stage and continuously before every iteration is important as a way to prioritize risks, examine controls already in place and decide which are needed going forward [21, 33]. Threat modeling is another method where you attack your system on paper early in the development cycle to identify how an attack can occur and where it is most likely to happen [34].

Continuous testing: Automatic security controls at every part of the software development process is important for security assurance and allows tests to continuously scan code for changes [32, 34], continuously detect anomalies, and automatic rollback of code when needed [21, 24].

Monitoring and logging: When automating security controls throughout the software developing process it is important for those involved to be able to generate evidence on demand that controls are working and that they are effective [35]. To that end, it is important to monitor every part of the inventory and to log every resource [21, 25, 33].

Security as code: This means to define security policies, for example integration testing, and network configuration and access, and write scripted templates or configuration files that can be implemented into the development process from the start of the project. These codified security policies can then be activated automatically according to schedules or be activated by user (simple push of a button), and be stored in a central repository for reuse on new projects [36].

Red-Team and security drills: To stay ahead of possible attackers, practitioners of DevSecOps create a Red-Team that runs security drill on the deployed software. They have the task of finding and exploiting vulnerabilities in the system [25, 34]. This not only helps to find security flaws, but improves measurements, and helps the organization find solutions [26]. The point of the Red-Team is to have people that never claim something can’t possibly happen.

3.4 RQ3 - Benefits

The following section provides an overview, according to the literature, of the benefits gained from DevSecOps and its practices:

Shifting Security to the Left: By involving security experts from the start of the development process it is easier to plan and execute integration of security controls throughout the development process without causing delays or creating issues by implementing security controls after systems are running [29].

Automating Security: This allows security controls to be fast, scalable and effective thus making it possible to keep a high pace for detecting errors, alerting about the errors, fixing the errors, finding countermeasures for future errors and forensics to identify why an error occurred [37]. This not only helps to lower risk and time spent on errors, but also makes it easier to understand risk and create policies and procedures [38]. The automation allows processes to be consistent and repeatable, with predictable outcomes for similar tests, it allows logging and documentation to be automatic [39] and letting security tests be run at the push of a button frees up developers time to write code instead of running tests [40]. This also reduces the risk for human error [39]. The ability to store security policy templates that is created during a development process in a central repository means that security teams don’t need to manually configure every new environment when starting a new project which frees security experts from manual, repetitive and unproductive work [36].

Value: [38, 41] points to how security missteps can be very expensive and that it is cheaper to implement security from the start than to wait for something to happen. [38] points to a survey that mentions how high-performance organizations spend 22% less time on unplanned work and rework. The ability to monitor and measure for security flaws early in the process ensures that bugs that prevent a delay in the deployment are caught and quantified [38]. This decreases the cost of making mistakes, finding them, and fixing them [36].

3.5 RQ3 - Challenges

The following section provides an overview of the challenges an organization faces from DevSecOps. The challenges are connected to the speed and agility needed not to slow down other DevOps practices, organizational changes, tools and practices:

Keeping up with DevOps: Using traditional, manual security methods heavily impairs the speed and agility of DevOps. This means security methods have to be more agile, and these agile security methods have to be understood by security teams and accepted by development teams [42] to make sure they contribute meaningfully to the DevOps movement without hampering their development speed and service delivery [43].

Organizational: Getting started with DevSecOps means the organization has to adopt change. Skills, culture, tools, processes, standards and practices must be considered as a possibility for implementing security [29]:

  • There will be a need for skills in areas such as encryption and logging standards etc. [29].

  • The organizational barriers between security teams and the rest of the organization must be broken down:

    • Developers and managers can be frustrated with the added time it takes to produce code, when adding security [21, 33]. Developers and operators think of security as a hindrance to their goals, which is to deliver functionality fast, while security teams are focused on making sure the functionality is secure and robust [44].

    • The security teams not being properly trained on tools developers and operators use, hinders them from being able to integrate security in a transparent and understandable way for other users, which would limit collaboration between teams [31].

    • Organizations see security as a costly activity, and not something that generates revenue [32, 35].

  • There will be a need for new standards for security prevention, detection and response [29].

Tools and Practices: The dynamic environment when practicing DevOps means that security functionality has to be available in tools that work on the right platforms. There is a lack of available tools [21]. Any security functionality not automated in the available tools will create friction in the DevOps cycle. The users need to be properly trained when using advanced tools. [38] points to developers that had difficulties writing secure code because they couldn’t use the tools efficient enough to keep up with DevOps’ speed.

3.6 RQ4 - The Evolution of DevSecOps

The need for security to be integrated in DevOps was first mentioned in a blog by Neil MacDonald, a Gartner analyst, in a blogpost called “DevOps Needs to Become DevOpsSec” in 2012. DevSecOps has since become more and more acknowledged as a necessity. Table 2 shows the increase in number articles per year, which is evidence that awareness, recognition and use of DevSecOps is on the rise.

Table 2. Overview of number of results per year
Full size table

3.7 Limitations of Results

This research is based on multivocal literature, and most of the material has not been subject to the rigorous peer-review academic research usually is. The literature has instead consisted of blogs, white-papers, industry reports and academic research.

DevSecOps is a very new term, and the term has not even been agreed upon. It varies between SecDevOps, DevSecOps, DevOpsSec, Secure DevOps, and Rugged DevOps. In this research paper we have consistently used DevSecOps (with exception to where I am referring to other sources and their titles). The fact that it is as new as it is, means the results from this MLR can become outdated as best practices change.

4 Conclusion

This MLR presents the research we did on DevSecOps to find out how DevSecOps can be defined, what doing DevSecOps means for an organization in regard to what principles and practices they should adhere to, what challenges they would face attempting to adopt DevSecOps, the benefits if it’s done successfully and how it has evolved from the need to implement security in DevOps to what could seem like a movement on its own.

We used Google Scholar and Google Search to locate literature and after applying our inclusion and exclusion criteria, 52 artefacts were found to be relevant to our search terms. Only 2 of those were academic research papers. The remaining 50 consisted of white papers, blogs and articles.

We found that DevSecOps is defined by many as the integration of security processes and practices into DevOps environments, that DevSecOps promotes a set of principles meant to shift the mindsets of all participants in the software development process so everyone participates and do what they can to ensure security in the project and a set of practices that can ensure security in the project based on the idea of planning and implementing security from the start and as code.

We identified a set of challenges and benefits to implementing DevSecOps. The challenges we identified should not be seen as deterrents to implementing DevSecOps, but a symptom of its youth. As DevSecOps matures, better methods, practices, tools etc. can probably overcome them. The benefits we identified indicates it is maturing, by for example resulting in less unplanned work and a decrease in manual labour.

As future work, it would be interesting to conduct surveys on organizations to possibly expand this study’s coverage on DevSecOps. It is also of interest to investigate this study’s suggested practices: observing practices effects on the surrounding environments (development, operations, business, customers) to find best practices. A possibility would then be to investigate and propose possible architectures or frameworks for implementing DevSecOps, [45] for example looks at continuous software engineering while using a microservices architecture which could be an alternative for “security as code” in DevSecOps.