Keywords

1 Introduction and Motivation

It is not without reason that the idea of agile development, as well as its congruous implementation, have until today been seen as revolutionary compared to traditional development methods. In many ways, tasks are approached differently: with a different philosophy, a different understanding of cooperation and responsibility, different starting conditions, intermediate goals, and acceptances. However, the similarities are concise: in both the agile and traditional worlds, development aims to deliver the desired products on time, budget and to quality.

The traditional development is characterized by a phase-oriented, highly controlled product development, in which first the waterfall model and then also the V-model were established. In theory, this means requirements and plans are either worked out in one pass – or in a few cycles and then implemented. In practice, of course, there are always variances and improvement loops. Either way, the development team proceed phase by phase – which proves a successful process model – provided requirements and plans can be determined in advance of implementation. Changes are possible but result in natural disruptions to the originally planned process [1, 2].

It is not a coincidence that the traditional V-model approach to development is well established in the automotive industry. Processes can be well controlled (and thus highly regulated) through the implementation of required standards and their achievement. They can be trimmed to high performance – to effectively, efficiently and reliably ensure high quality and low risk. There are proven standards for the improvement and evaluation of system and software development organizations, with the help of which the processes are further and further optimized in order to meet the constantly increasing demands on product development: ever more complex functions are to be developed in ever less time, distributed over more locations, with more quality requirements. If the framework conditions, requirements, or plans change in this environment, the disruptions often have severe consequences.

The increasing drive for innovation has sparked interest in the agile methodology – and in return is now leading to a rise in demand that agile development environments need to face. Due to the completely different holistic approach, agile methodologies have a decisive advantage: they are much better equipped to develop effectively and efficiently with initially only partially clarified requirements, priorities, and plans. Changes to requirements and priorities as well as clarifications of requirements or framework conditions are comparatively easy to introduce and cause much less disruption to the development process.

Another decisive advantage is the significantly increased appreciation of the people involved both internally and externally. This leads to a significant reduction of demotivating factors and thus to better performance of both individuals and the teams. These advantages, as well as the complex disruptions in traditional development, lead to the fact that a large part of the industry is now leaning towards more agility and experiencing organizational transformations. Step by step, more and more agile elements are being integrated into development processes, and (partially) agile solutions are visibly establishing themselves.

Early literature ventured into interesting analogies between product development and team sports [5], comparing the traditional development to a relay race (run forward only after the previous stage is completed and the handover is formally received) and the then upcoming agile methodology to a rugby game (see the distance between the ball and the bar as a unit, work as a team, run back and forth if required, be creative and change your tactic according to needs). This analogy contributed to building the opinion, that the two product development methodologies are incompatible to each other and subsequently led to misunderstandings, which in some cases still remain difficult to remove today.

More recent works set about to clarify the existing myths, on one side considering that at high level, process standards describe the “what” and agile practices are more oriented to the “how” [6], on the other side deriving a systematic approach to compare agile practices with standard models, particularly with Automotive SPICE [7,8,9].

During the last two years we contributed to the intacs Working Group Agile SPICE, developing the add-on assessment model Agile SPICE - currently released as Version 1.1 [10].

Based on our work within the intacs working group - and with the encouraging results of the first pilot assessments as well as several discussions held between assessors, those being assessed and experts in the industry - we have found that the traditional and the agile development approaches are far from being incompatible. Rather, if suitably combined, they mutually support the achievement of high process quality and reduction of product risks.

This paper is structured as follows. In Sect. 2 we first outline the expected user of the model and our motivations to develop a new Process Assessment Model (PAM) - complementary to Automotive SPICE. The new model is developed according to the ISO 33004 [27] standard requirements for a Process Assessment Model. It can be applied alternatively to assess processes according to ISO 33002 [11], as well as to evaluate capabilities and necessary improvements in an agile or partially agile project organization. The three already published Agile SPICE processes are briefly illustrated. In Sect. 3 we give an overview on the first pilots and on the very early results gathered during the last months. Finally, Sect. 4 gives a short summary of the next steps and upcoming work challenges.

2 Agile SPICE: The “Bridge”

Increasing complexity and rapid changes during development are driving the increased use of agile approaches [3, 4]. At the same time, OEMs are ensuring the overall quality and safety of their products, leading to increased pressure to give evidence for Automotive SPICE capabilities. Although Automotive SPICE has historically been used primarily in a traditional development environment, it can still be applied and interpreted in its current form for any type of development organizations, including agile or partially agile development environments. However, both agile organizations and those currently in agile transition struggle to interpret and implement Automotive SPICE. Equally so, many Automotive SPICE assessors have difficulty mapping agile development appropriately to the Automotive SPICE model, despite existing guidelines [12]. Misinterpretations lead to wrong, and for the assessed organization unsatisfactory, assessment results. There is a need to reduce misunderstandings and to give a clear interpretation of the terminology for both the agile and the V-Model driven traditional world.

Existing analyses consider how agile practices support Automotive SPICE [8, 9, 13]. These analyses are mainly based on a statistical approach and do not consider how Automotive SPICE itself supports agile practices or the need for assessment objectivity.

2.1 The Agile SPICE Approach

The intacs Working Group, Agile SPICE, made up of experienced experts in agile development and Automotive SPICE has formed to fill existing gaps and pursue the motivations illustrated above. Our aim is to build a bridge and provide a model, which is suitable to perform ASPICE conforming process improvement and assessment, to help increase the acceptance of Automotive SPICE in the agile community and to reduce the interpretive diversity by Automotive SPICE assessors.

The medium-term goal of the intacs Agile SPICE Working Group is to demonstrate that process analyses and assessments according to Agile SPICE can be applied in a reliable and recognized way to a process assessment according to Automotive SPICE and thus demands for Automotive SPICE process capabilities can also be met with Agile SPICE process analyses.

Based on the extensive process assessment and process improvement experience in the working group, three exemplary use cases were evaluated before the start of the development of the Agile SPICE model:

  1. 1.

    An agile organization can often meet customer demand for process capability. To do so, it needs internal and external acceptance of process capability requirements, even in agile environments, especially in the automotive industry. This kind of organization needs an assessment model that clearly describes the requirements in terms of what is expected, not how. Only then can Agile SPICE support the implementation of agile good practices and the simultaneous achievement of the process attributes. What needs to be dismantled are previous misconceptions about how to implement and assess relevant base practices in agile environments.

  2. 2.

    A traditional organization that is currently transforming incrementally to an agile organization must maintain existing capabilities as it moves towards agile approaches. It learns agile terminology and must shape the change without losing its process maturity and process capability. It must avoid typical pitfalls in an agile transformation, such as aiming for a minimum amount of governance.

  3. 3.

    Certified Automotive SPICE assessors need a very broad understanding of existing processes and practices in agile organizations. They also need to understand the terminology and be able to map practices correctly to the model. Mapping of assessment indicators between traditional and agile approaches is needed to ensure comparability of assessment results between agile and traditional approaches in development work.

The consideration of the three use cases highlighted above was based on real assessment results gathered in different contexts. This led to the definition of the purpose and the outcomes of the processes, as outlined in Sect. 2.2, 2.3 and 2.4 for the already published AGL processes.

In our definitionFootnote 1 “Agility is the timely adaption of an organization (or team) to an ever-changing environment while continuously delivering value to their customers at sustainable pace” [10]. Based on this definition, the first step was to adapt the agile principles [14] specifically for the automotive worldFootnote 2, in order to have a common basis shared by the whole group as a starting point for developing the add-on process reference and assessment model (Table 1).

Table 1. Agile principles for automotive [10]
Full size table

Existing examples of agile practices [13] - without favoring any particular one (e.g. Scrum, Kanban, SAFe®, LeSS, Nexus, Scrum of Scrums…) - have been then condensed to become independent from any of these agile models and give us the possibility to focus on selected practices instead of considering a particular “whole” agile methodology.

Agile practices are therefore mapped to existing Automotive SPICE practices to ensure comparability of ratings between agile and traditional approaches to development work, thus achieving acceptance of ratings by OEMs based on agile practices.

The approach is used for the definition of the agile specific assessment indicators for the process performance attribute (process dimension). Existing process attributes and generic practices on Capability Level 2–5 (capability dimension) are retained without changes [11]. The resulting Process Assessment Model is compliant to ISO 33004 [27]. The complementary assessment model Agile SPICE is currently released as Version 1.1 [10] available at intacs and ready for piloting.

2.2 AGL.1 Agile Work Management

The use of agile project management in different industries was recently analyzed by several authors [16, 17] and seen as a productive ground for the application, even experimentation, of specific models and methodologies to meet the agreed project goals in a fast and continuously changing environment, characterized by volatility of requirements and complex interfaces. Agile practices give the teams the ability to better overcome changes and turn them into chances to improve and further evolve their approach, providing a suitable balance between the human need for stability and the necessary level of flexibility required.

However, any agile methodology is made to fit each type of project and the successful implementation and use of agile practices usually depend on the organizational context and culture.

On a higher abstraction level Automotive SPICE requires that for each project the scope of work is defined in terms of goals, motivations and boundaries while, considering the traditional V-development cycle, requirements are fixed, and effort and duration are variable. The agile approach, where requirements are not clearly defined and effort and duration are fixed, leads to a different approach of work management: the need for understanding of customer demand and work boundaries remains, therefore it is based on collaboration and agreement between stakeholders both internal and external to the organization.

Agile approaches are people-centric and built on the collective intelligence of the individuals involved, so in practice “… an Agile team is a cross-functional group of 5–11 individuals who define, build, test, and deliver an increment of value in a short time box….” [18]. On a large-scale, different types of teams sharing a common approach are defined, typically the technical and business teams. “Both types of teams strive for fast learning by performing work in small batches, assessing the results, and adjusting accordingly” [19].

Given that the requirement’s granularity increases step by step from long-term to short-term iteration cycles, the team (or team of teams) must be self-organizing and able to define the work approach appropriate to reflect the level of complexity of the product to be developed. The continuous self-learning capability aimed to optimize predictability, to better control risks and to increase the quality of the incrementally delivered value, is a measure of the evolving team maturity. Our understanding of agile team maturity encompasses high productivity and continuous learning and improvement, related to the ability of adoption of new practices while keeping lean agile processes.

There is no violation of the Automotive SPICE requirement for project life cycle, definition and continuous control and adaption of the work breakdown structure and the activity plan: high maturity of the team assumed, the frequency and different rhythm of iteration from long- to short-term give breath to the project, ensuring the planning of manageable work portions inside predefined time boxes. Management of interfaces and consistency between customer demand and incrementally delivered value are supported by the continuous control intrinsic to the ritual events with fixed length, typical of the agile approach chosen.

Reality, however, looks different, and we mostly encounter organizations still in the transition phase of an agile transformation. Single practices are eventually established but necessary completeness of the work approach is not yet recognizable and the agile team maturity not advanced enough. Malfunctions at the project interfaces may reveal further, organizational limitations to the work approach and may compromise the transition process.

Particularly in such cases, where the work environment is partially traditional and partially agile, the complementary relationship between AGL.1 and Project Management (MAN.3 as defined in ISO12207 [20] and as used in Automotive SPICE [21]) helps the practitioner to better understand “how” to reach a holistic agile work approach. A typical example is a complexly distributed system development where part of the subprojects or even the main project are still managed in a traditional way. In this case it is suitable for the organization to refer to AGL.1 instead of MAN.3 to improve process activities and fulfil Automotive SPICE requirements. On the other side the assessor would face a transition situation and would be able to rate AGL.1 or MAN.3 in a complementary and comparable way, depending on the approach adopted by the main project and the subprojects.

2.3 AGL.2 Partner Collaboration Management

The last years have shown an increasing tendency to adopt an agile approach by the management of outsourced development projects [3, 4]. It is manifest, that due to the industry aim for sustainable technological innovation – particularly in case of system and software development – the established business models and the typical customer-to-supplier hierarchical structure can no longer support the fast-growing need for collaboration and agile transformation at all levels: new kinds of development partnerships between different departments of the same company or cross-company, are shaped to achieve the highest level of quality and customer satisfaction. Shared agreed joint progress instead of obstinate control is the new mindset.

Based on the four prerequisites of a successful agile collaboration - transparency, direct communication, frequent delivery, valuable scope - different approaches for agile collaboration have been recently elaborated and are currently available as a set of recommendations or as referencing industry best practice. Two examples, both applicable to agile approaches on an arbitrary scale, are briefly illustrated.

The VDA Agile Collaboration [22] handbook provides a set of recommendations on how an agile collaboration across company borders could work. Alongside five levels of agile collaboration, it defines processes for initiation, agreement and execution, as well as typical roles, responsibilities and rules for collaboration. Several outlined use cases represent a valuable toolbox to agree scope, objectives, roles, artifacts, communication as well as infrastructure, and support the congruous implementation with regard to the particular collaborative environment.

Another recently presented model also defines 5 levels of engagement for agile collaboration with suppliers [23]. It is a valuable process-oriented industry best-practice, applicable to any agile environment at different scales, from individual to the team of teams. The engagement model was originally issued as alignment to SAFe [28], although it can be applied to any other agile approach and, in any case, if compliance to Automotive SPICE is strongly required for the project. The collaboration aspect is essentially based on agreement upon process ownership, where customer processes are appropriately tailored and trained according to the type of supplier agreement, and supplier-owned processes underlie assessment control according to Automotive SPICE. Based on the type of contract, the suitable level of the model works very well; although it maintains a kind of hierarchy between customer and supplier, fortified by regular control (as intended by Automotive SPICE ACQ.4 Supplier Monitoring [21]) with less emphasis on dynamic agreement, partnership and mutual learning as sought in an agile approach.

In both cases, the defined levels of agile collaboration or engagement refer to the increasing collaboration intensity and the scope of the project, not to growing agile maturity. Automotive SPICE compliance is required by both, even though the base practices as defined by Automotive SPICE ACQ.4 do not really match with the agile mindset.

The purpose of Partner Collaboration Management as defined in Agile SPICE is to achieve “common project goals in collaboration with a partner” [10]. The base practices - aimed to enable an objective capability determination in collaborative agile environments - are defined according to the automotive agile principle independently of any specific agile framework. The collaboration must be based on a common understanding regarding project roles, recurring events and work results. This means, it is characterized by teamwork to achieve a common goal, issued from continuous alignment and synchronization. Alongside the agreed ownership of processes and artifacts, the collaboration agreement must also consider technical and organizational interfaces as well as responsibilities, communication and escalation paths. Information is most notably shared in the context of the agreed meeting structure (backlog refinement, iteration/increment planning, reviews and retrospectives, inspect and adapt). The technical content is jointly developed and regularly reviewed, based on the agreed collaboration approach. The visualization of the joint progress in terms of quality, effort, costs and schedule requires a common platform where the team can detect deviations from the desired status early, and then act consequently. Likewise, the mitigation of risk and handling of impediments are collaboratively lived processes. This way, recuring retrospectives become the medium for continuous improvement.

2.4 AGL.3 Agile Quality Assurance

Partially influenced by industrial needs, the agile approaches have evolved and diversified during the last 30 years leading to the currently increasing tendency to the adoption of Scrum and SAFe [4, 13]. Taking both approaches as an example, we consider briefly how they handle the concept of quality.

The Scrum Guide [18] combines agile values and principles as stated in the Agile Manifesto [15] and defines a set of practices and techniques focused on the delivery of value to the customer, however it does not define explicitly any quality requirement, leaving this aspect to the commitment of the team, which is considered always accountable for “instilling quality by adhering to a Definition of Done”. The Definition of Done is then defined as “formal description of the state of the increment when it meets the quality measures required for the product” [18], requiring that quality remain constant throughout the iterations. Product means here the finally delivered product (e.g., software functionality), without reference to work products as expected intermediary results of an established development process as intended by Automotive SPICE.

According to SAFe [24] “teams apply Built-In Quality practices to drive disciplined content creation and quality. Built-In Quality practices ensure that each solution element, at every increment, meets appropriate quality standards throughout development”. Built-In Quality is considered a core value, has emphasis on “technical excellence and good design” and focuses also on the product as the final result of the development value stream. These concepts remain however generic and other aspects of quality are given as implicit.

Agile Quality Assurance emphasizes the need for collaboratively agreed quality objectives also considering criteria for review and approval of work products, which relate to the Definition of Ready and Definition of Done as embedded elements of the quality strategy of the agile project. According to ISO 33004 [27] Agile SPICE does not define any explicit role, however, the concept of independent quality assurance - already known from Automotive SPICE [21] - remains central in Agile SPICE where in AGL.3, the need for an independent and objective report and tracking to closure of detected quality non-conformances is emphasized. Without definition of an explicit role, we like to address agile quality assurance as a second independent voice giving the stakeholders outside the agile team an objective view on the project.

Concerning quality assurance, the misunderstanding that the agile and the V-Model oriented development approaches are incompatible to each other, seems to be more deep-rooted than for other processes. This probably relies on the definitions of quality provided (or not) by the available guides on agile methodology, which merely directly address external quality aspects instead of internal ones. We assume here external quality as visible to customers and any user of the final product, and internal quality as related to the product development processes and intermediary work products. Internal quality is therefore always a necessary precondition to achieve the required quality of the final product, it is however not directly tangible, and its importance remains difficult to perceive in a context where teams are focused on the creation of “usable” increments.

Beyond product and process quality, Poth et al. [25] introduced the concept of “agile team work quality” as a necessary third dimension contributing to the achievement of the required quality of the final product developed in an agile, team-centric environment and identified six aspects leading to team performance: “communication, coordination, balance of contribution, mutual support, effort and cohesion”. The concept can be considered complementary to the current agile methods and is new to the existing standards [20, 21].

Quality Assurance as defined in ISO12207 [20] and as used in Automotive SPICE [21] (SUP.1) addresses the internal quality of the whole product development process and makes it difficult to perceive its suitability in an agile context. In practice however, the Automotive SPICE requirements for Quality Assurance can be suitably fulfilled by adapting agile practices without loss of agility.

AGL.3 Agile Quality Assurance addresses the existing gaps between the standard approach of Automotive SPICE and the agile development methods, in order to give guidance on how to implement agile quality assurance practices and how to measure the achieved effectiveness of development processes and expected results. The defined base practices focus upon team collaboration and responsibility for the alignment of work product and process to the adopted work approach, while seeing non-conformances as a disruption to the incremental value flow. The importance of communication during and beyond scheduled regular agile events is emphasized. Finally, the learning aspect, fundamental to any team-centric approach, introduced and integrated as base practice and which gives the opportunity for the continuous improvement of collaborative work processes, represents an enhancement compared to SUP.1 and can further contribute to the establishment of an agile mindset.

3 Piloting Agile SPICE

In order to prove the suitability of Agile SPICE with respect to the three possible use cases outlined in Sect. 2, we promoted pilot activities to interested customers and amongst assessor colleagues. The pilots are conducted in the form of an assessment, which can be formal (conformant assessment) or informal (gap analysis). In both cases, the assessment is performed according to the requirements of ISO 33002 [11]. Only for the pilot purposes the assessments are performed using both Agile SPICE and Automotive SPICE in parallel.

Four questions led the pilot’s assessment activities:

  1. 1.

    Do the defined agile practices give a suitable representation of the implementation of agile practices in a project organization, independently of the project size and the adopted agile approach?

  2. 2.

    Does the “Agile SPICE-to-Automotive SPICE” relationship allow for a process analysis that leads to the same results as an Automotive SPICE process analysis of the corresponding processes?

  3. 3.

    Do those being assessed in an agile project better understand the requirements of the standard and the necessary process improvements, if the interviews are based on Agile SPICE instead of Automotive SPICE?

  4. 4.

    Does an assessment according to Agile SPICE say anything about the actual agility team maturity?

For this initial pilot phase, we are more interested in qualitative than in quantitative results, since our actual aim is to have a first proof of concept and to gather practical industrial experience using the model.

The technique used for the qualitative evaluation of the questions posed above was the detailed analysis of the assessment records followed by in-depth discussions with the assessees.

In the near future more pilot assessments are needed and the evaluation of results is required, to be able to answer the above questions in a meaningful statistical way. Furthermore, we are interested in experiences of using Agile SPICE for standard process improvement in agile environments when Automotive SPICE compliance is required. Experiences, that can be made in projects of different sizes and complexity as well as product development in different vehicle domains, would give insight on how the number, cultural background and size of the teams at different development sites could influence the agile transformation and the process evaluation.

This section provides an overview on current pilot activities and the very early qualitative results we gathered during the first month of this year. More pilot results are expected over the rest of the year.

3.1 Overview on Completed Pilot Assessments

Agile SPICE was piloted at OEM and Tier 1 projects for process analysis and process assessment. The projects run at one or more locations, in one case the locations are distributed over different continents, manifesting different organizational culture (Table 2):

Table 2. Overview on project context
Full size table

The assessed development projects differ significantly in the type of product developed. The assessments were carried out partly informally, partly formally according to the corresponding specifications. The assessment teams were composed of different assessors (Table 3).

Table 3. Overview on pilot type
Full size table

3.2 Very Early Results

The first pilot assessments on the projects highlighted in Sect. 3.1 were performed on AGL.1 Agile Work Management and the results were compared to MAN.3 Project Management. The evaluation of the assessment results, both formal and informal, have shown that:

  1. 1.

    The process practices as defined in Agile SPICE fit very well with the way teams work in the different project contexts, both in the case of a single agile team and when the team of teams approach is used on a large scale. It is noteworthy that Pilot 3 showed that AGL.1 Agile Work Management can also be successfully applied very easily in the case of very large projects with a distributed environment.

  2. 2.

    In all cases, very similar strengths and weaknesses are visible when addressing AGL.1 and MAN.3 in parallel. More specifically, the assessment of the process attributes for Pilot 2 and Pilot 3 using Automotive SPICE and Agile SPICE respectively, yielded almost the same rating percentage.

  3. 3.

    In the case of Pilot 4 the project, which develops a highly safety-critical product and is located at a large Tier 1 supplier in Germany with development support in India and was assessed according to Agile SPICE up to Level 1, the questions derived from the Base Practices could be well understood and answered by the project team. Based on the answers, the experienced assessors were able to assess the Agile Base Practices using the SPICE standard rating system NPLF. The rating of the process attributes and thus the determination of the capability level was well accepted by the agile team.

  4. 4.

    In two of the five pilots we recognized that, if a Base Practice of AGL.1 is rated N or P, it does not show a weakness according to Automotive SPICE, but an incomplete implementation of the agile methods or principles. This may have different root causes, to be searched in the organizational context, the history of the project, the poorly developed agility of the customer and of the involved suppliers. The process analysis showed the team and the sponsor those areas in which they were already successfully working agilely and those areas where they were not or not yet satisfactorily advanced. The Agile SPICE analysis result showed in which activities further agile alignment may be worthwhile.

The peculiarity of Pilot 5 was that the agile team decided to adopt Agile SPICE directly for the set-up of their own work management and that the assessment was performed later in the project life cycle. Also, in this case the team found it easier to comply with Automotive SPICE while working agilely and, concerning the rating of the process attributes, the assessment results showed no notable differences between AGL.1 and MAN.3.

4 Conclusion and Further Developments

Regarding the actual version of Agile SPICE, we can conclude that the model is more intuitive and acceptable for agile or partially agile organizations and teams facing the implementation of Automotive SPICE requirements to achieve compliance. Also, it fits its purpose to reduce or even avoid misunderstandings, giving clear interpretation guidance and a quantitative measure in terms of ratings of the correct or eventually still incomplete implementation of the agile practices, finally leading to a tangible measure of the agile team maturity.

For pilot purposes the two models were used in parallel, however achieving Automotive SPICE compliance in an agile environment, does not mean that both PAMs have to be used, in any case overlapping of requirements between the two models must be avoided.

Like the use of Automotive SPICE in a traditional V-Model environment, Agile SPICE may be used alternatively for process improvement and process assessment in an agile environment. In the case of an organization that is still in a transition phase, the decision on which model to apply will depend on the project context.

The feedback issued from the pilots and the first proof of concept outlined in Sect. 3 gave us a valuable contribution to improve the current model version and have encouraged us to pursue our work. The topics relating to the necessary interpretation guidelines for agile engineering and further support processes are currently in development and a new release of the Agile SPICE PAM is planned for the end of this year. An appropriate training for the suitable application of the new Process Assessment Model will follow.

5 Contribution to the SPI Manifesto

The SPI Manifesto [26] addresses “people”, “business” and “change” as core values. The present paper contributes to all three. Specifically, team and organizational learning leading to processes adaptable as needed, are central to achieve high agile team maturity and work quality; the agreement on agile quality objectives supports the achievement of organizational visions and goals, while the correct set-up of agile practices supports the management of process-related product risks. Finally, the need for mutual agreement at each level ensures that project work processes are fully “lived” by all involved parties.