Keywords

1 Introduction

For many years various software (SW) companies have put numerous efforts to mature SW development practices using different techniques and methods. The last decades were devoted mainly to the idea of software process improvement (SPI) [1]. SPI covers rather large number of approaches to improve software engineering practices. We discovered variety of SPI definitions but in this paper the following definition proposed by Hansen et al. [13] is used: “SPI is an applied academic field rooted in the software engineering and information systems disciplines. It deals with the professional management of software firms, and the improvements of their practice, displaying a managerial focus rather that dealing directly with the techniques that are used to write software”.

Practitioners have an opinion that areas of the software development lifecycle that benefit from SPI include project management, requirements management, configuration management, software development, quality assurance, quality control, testing, risk management, acceptance and ongoing maintenance. Initially, SPI has had its roots primarily in software engineering, nowadays this approach has grown and covers, for example, management of software companies and plays a significant role in organizational change concept. SPI is considered as a complex action involving not only process improvement but also the organizational improvements. The improvement per se requires commitment from several departments including finances, human resources and senior management [16]. In this paper we admit this wide perception of SPI.

Numerous organizations reported the success of SPI, including examples from many industry case studies such as Alcatel [7], Ericsson [6], Hughes Aircraft [14], Motorola [8], and Onion [4]. A number of scholars [10–12, 17, 18] studied positive factors – critical success factors (CSF) influencing SPI. In this paper we adopt the perception of CSF in SPI proposed by Espinosa-Curiel et al. [15]: “SPI factor is any knowledge, competence, behavior, attitude, perception, feeling, situation, condition, or activity at personal, social, technical, or organizational level that influences the results of an SPI initiative.” In other words, CSF is a factor that must present in SPI initiative or program. Scholars declare the organizational issues, change management and people involvement as the key elements of any SPI initiative. There are several distinct commonalities in the finding of aforementioned studies of CSFs which are discussed in section three of this paper.

The rationale of this article is to identify CSFs in SPI in the comprehensive literature, compare them and answer two research questions studied in the case company:

  1. 1.

    Does the case company manage critical success factors in software process improvement initiatives?

  2. 2.

    How does the case company manage critical success factors in software process improvement initiatives?

Why is it important to address those questions? The body of knowledge is CSF in SPI is well defined; however, the number of publications concerning the topic of management and stakeholder value recognition in CSFs in SPI is limited. We suppose that SPI initiatives are less effective if they are not useful to stakeholders. Stakeholders of SPI program or initiative are, for example, executives of all levels of organization, SW end-users or customers, programmers as well as other personnel of a company who are involved into SPI activities. Boehm [5] states that many failures in software projects and SPI initiatives are caused by value-oriented challenges when stakeholder values are not addressed or meat properly. In framework of this research stakeholder value is usefulness of offering SPI to its key beneficiaries, so they are fully involved into SPI activities which increases the success of those activities. Since many CSFs identified in the literature relate to people - SPI stakeholders, we plan to study in the case company are those factors important, what their current state is and how the case company manages them. This research is considered as a pilot case-study within one Finnish SW development company Developers Oy.

The paper is organized as follows: section two refers to the methodology used in the study, study limitations and case company description; section three identifies and analyzes the main studies on CSF in SPI; section four presents analysis of the case study; section five concludes the paper; section six suggests future research.

2 Methodology and Limitations

The body of knowledge on SPI CSFs is well shaped and developed. We studied the most relevant papers on the field to select and compare the identified CSFs for further investigation in the case company. The list of used scientific journals is limited to those: IEEE Software, Information and Management, Information and Software Technology, International Journal of Information Management, Journal of Empirical Software Engineering, Journal of Knowledge Management, Journal of Systems and Software, Scandinavian Journal of Information Systems, Software Process: Improvement and Practice, Software Quality Journal. Out of potentially interesting articles we included articles with direct focus on SPI critical success factors in general. As a consequence, we excluded articles focusing on more specific issues in SPI such as, for example, SPI success factors in small- and medium size companies, Web or agile software projects. We rationalize exclusion of the papers with narrow specialization because we assume the study done in this paper as a pilot one and general in its nature.

Table 1 shows the list of twenty two CSFs proposed by scholars and employed in this study. We analyzed the following factors, some factors are more general and some of them are more detailed. We combined few detailed factors with general ones and proposed thirteen of CSFs for rating and analysis in the case company.

Table 1. Critical success factors for SPI initiatives identified through selected publications
Full size table

This case company is Finnish middle size SW development organization running SPI activities for several years. Five company’s representatives: chief executive officer, SPI program manager, process engineer, SPI project manager, and senior product development manager participated in the evaluation. We asked them to estimate the importance of each factor for the company and current state each factor holds in the company at present. The scale from 1 to 5 is employed, where 1 is the lowest value and 5 is the highest. As the second part of the study we conducted an interview with senior product development manager who is in charge of SPI program to analyze the importance and current state of CSFs in the company and to report what managerial actions the case company makes to improve performance of those factors.

3 Critical Success Factors of SPI Presented in the Literature

Several studies have been undertaken on how SPI can result into success. However, as the goals of SPI vary according to context and company implementing it, the consensus on success factors may be problematic. Here, we review several key studies and report the success factors identified in each case. The accumulative Table 1 of factors found by researches stated below is presented in the end of this chapter.

Goldenson and Herbsleb [11] conducted survey of 138 respondents involved in 56 CMM programs in medium and large organizations. Based on that study they picked out more successful SPI initiatives. The factors proposed by them may be summarized as: senior management commitment; SPI goals distinction and understanding within the company; staff involvement (to what extend staff members participate in SPI initiatives); SPI personnel have to be of high respect within the company; dedication of resources and staff time; compensated SPI responsibilities.

El Emam et al. [10] conducted a study of factors influencing the success of SPI in 61 USA and Canada based medium and large software companies involved in SPI programs. They presented SPI success factors across organizational factors and process factors. Organizational factors are those that characterize the organization undergoing SPI and the characteristics of the organizational SPI effort itself. Process factors characterize activities of infrastructure that are believed to be necessary for successful SPI effort. Process improvement is more likely to be successful when there is a moderate amount of process focus. This research identifies several success factors that may influence successful SPI utilization: commitment (management interest and involvement into SPI initiatives); turnover (at the middle management and technical levels); politics (promoting of long-term benefits of SPI initiatives); respect (to what extend employees involved into SPI are respected in the company); focus (how deep the company is focused towards SPI activities.

Stezler and Mellis [19] reviewed case studies and reports of 56 medium and large organizations involved in SPI activities based on CMM or ISO 9000 improvement activities. They found the following factors that hold prospects for SPI success: change agents and opinion leaders (change agents supports the SPI activities at the company level, opinion leaders at the local levels); encouraging communication and collaboration (communication between the members of one team and members of different departments); management commitment and support (management interest and involvement into SPI initiatives); managing the improvement project (the level of effectiveness in SPI project management and control); providing enhanced understanding (existence of knowledge in current SW processes and business environment); setting relevant and realistic objectives (distinction and understanding of goals within the company); stabilizing changed processes (continuous support of SW processes, their maintenance and improvement at the local level); staff involvement (to what extend staff members participate in SPI initiatives); tailoring improvement initiatives (adaptation of SPI activities and needs of specific departments within the company); unfreezing the organization (readiness of the company to overcome “inner resistance” for change.

Hall et al. [12] conducted survey of 85 companies possessing different CMMI levels. They divided factors in across (1) human factors (for example, SPI leaders, management commitment, and staff involvement); (2) organizational factors (for example, communication and resources); and (3) implementation factors (for example, SPI infrastructure, setting objectives, tailoring SPI, evaluation). In their research they measured the use of implementation factors in the industry. Based on the study the success factors that may be considered are: resourcing (quantity of resources and time allocated for SPI in the company); quality of internal SPI staff; tailoring SPI (adaptation of SPI activities and needs of specific departments within the company); staff involvement.

Another study by Rainer and Hall [18] based on the survey of 85 companies complements the work of El Emam et al. [10], Stezler and Mellis [19] and Goldenson and Herbsleb [11]. Complementing factor was the specific attention to impact of success factors on levels of maturity in company. Factors found to be important in majorities of opinions are: reviews (regular review on current state of SPI program); standards and procedures (development of standards and procedures within SPI program); training and mentoring (training provided to the team of experts); experienced staff.

More matured companies considered the most valuable for successful SPI implementation the following: internal leadership; inspections (advanced reviews of the current status); internal process ownership (responsibility of particular process belongs to authorized person of team.

Dyba [9] performed quantitative research of 120 software and hardware companies of different sizes in Norway. The survey studied validity of several hypotheses presented by researcher and resulted in those key success factors for SPI initiatives: SPI success is positively associated with business orientation (the extent to which SPI goals are aligned with business goals and strategies); involved leadership (the extent to which leaders of all levels are committed and participate in SPI initiatives); employee participation (the extent to which employees use their knowledge and experience in SPI initiatives); concern for measurement – (the extent to which software company collects and utilizes quality data to guide and assess the effect of SPI initiatives); exploitation of existing knowledge; exploration of new knowledge. In general, study by Dyba [9] complements studies by Rainer and Hall [18], El Emam et al. [10], Stezler and Mellis [19] and Goldenson and Herbsleb [11]. However, it provides more detailed analysis of selected success factors.

Study by Niazi at el. [17] interviewed representatives of 29 medium and large companies involved into SPI programs. They identified success factors by empirical study and compare them to the factor the most frequently cited in the literature. However, authors state that in addition to success factors mentioned by El Emam et al. [10], Stezler and Mellis [19] and Goldenson and Herbsleb [11] there are two additional factors that may affect the success of SPI: awareness of SPI in the company (promoting of long-term benefits of SPI initiatives); defined SPI methodology (guidance on how to implement SPI activities).

One common observation from all studies presented in the section above is the necessity of senior management involvement into SPI initiatives. Additionally, the importance of clear vision and understanding of SPI goals should be considered. However, both technical and managerial human resources need to be involved in SPI program. All authors view staff involvement of all levels as extremely important success factor.

Dyba [9] suggests software organizations to “focus on both short-term and long-term alignment of SPI goals and business goals” and “share domain knowledge between software and business executives”. Even though involved leadership assessed as success factor for SPI initiatives in his study, its importance stated less significant than in studies by Ryner and Hall [18], Stezler and Mellis [19] and followed findings by Abrahamsson [2] that “many SPI activities do not require management commitment beyond obtaining the resources needed”. In Table 1 we unite factors of senior management commitment and involved leadership into one.

Technically it is possible to associate factor “focus” with such factor as “SPI goal distinction”. The author supports the view of El Emam et al. [10]. The view reads: “an organization cannot be focused in its SPI effort if its improvement goals are not clearly stated and understood”. So, first it is important to distinct SPI goals and then to focus the company into SPI effort. These are two different factors even though they are connected.

Niazi et al. [17] identify that awareness of SPI in the company is the novelty to the success factor list. However, Emam et al. [10] looked at the awareness issue, at least indirectly. For example, they report: “the general label for politically motivated activities and invectives that may promote or hinder SPI within an organization. This overlaps with the view of Niazi et al. [17] which reads: “by awareness we meant promoting through awareness events…among the higher management and the staff members of the organization”. Niazi et al. [17] find that the awareness of SPI becomes important because it is a long-term activity and consumes finance. These potential benefits are not clearly seen at the beginning. By explaining and promoting advances of new initiatives credibility of SPI raises among staff members. Defined SPI implementation methodology is also reported to be new important SPI success factor to managers as in some cases managers lacked the knowledge of implementation of SPI activities. Thus, if implementation methodology is well stated it may help managers coordinate and steer the SPI activity.

Rainer and Hall [18] mention that their findings are not well comparable with factors identified by El Emam et al. [10], Stezler and Mellis [19] and Goldenson and Herbsleb [11]. However, our opinion is that factors such as “executive support and leadership”, “change agents and opinion leaders”, “experienced staff and personnel respect” correspond with the similar factors identified in earlier studies. They are integrated with those factors in the Table 1.

Dyba’s [9] main contribution to the list of critical success factors in SPI is resulted in identification such factors as exploitation of existing knowledge and exploration of new knowledge. Exploitation of existing knowledge could be compared with key success factor “providing enhanced understanding” (existence of knowledge in current SW processes and business environment) by Stezler and Mellis [50] to some extent. However, the factor exploration of new knowledge is introduced by Dyba [9]. The main finding about those factors is that both forms of learning in organization are important for successful SPI initiatives and companies should find proper balance between them.

Table 1 accumulates critical success factors in SPI identified in the reviewed above studies and indicates authors that mentioned the factor.

4 Analysis and Discussion

Table 1 presents twenty two CSFs discovered by scholars. For evaluation in the case company Developers Oy we combine few detailed factors with general factors. Factor SPI personnel respect is united with factor change agents/opinion leaders; factor focus is united with factor defined SPI implementation; factors reviews and standards/procedures, concern for measurement are united with factor managing the improvement project; factors internal process ownership, turnover and compensated SPI responsibilities are united with staff involvement; factor exploration of new knowledge is united with factor providing enhanced understanding.

Five members of Developers Oy participated in the survey: chief executive officer, program manager, process engineer, SPI project manager, senior product development manager. They evaluated importance of each factor and current state each factor has in the company at present. The scale from 1 to 5 was used, 1 is the lowest value and 5 is the highest. Then we interviewed senior product development manager who is responsible for SPI activities to report what managerial actions the case company makes to improve performance of those factors.

Discussion below gives answers to our research questions.

“Management commitment” is management interest and involvement into SPI initiatives. It appears to be number one factor for success in SPI initiatives, it gets 4.4 average points. Practically all respondents perceive this factor as the most important. The current state in the company is evaluated as 2.8. Senior product development manager claims: “if you ask whether all managers ready to commit, the answer is no”. Several top managers understand the benefits and accept the necessity of SPI initiatives. They lead their teams towards the right direction. However, most business executives who run global operations, global marketing and sales are interested primarily in product deliverables. The main challenge is that they are not directly involved into SPI activities; they have general perception on SPI which is not enough for valuable commitment. Nevertheless, Developers Oy found the way how to motivate those managers who commit in abridged way by: “showing the small improvements we can get, recording them; this is how they see what we did. It is about the measures, we try to build dashboards for different levels of organization, not to the SPI but the deliverables, the improved process. Let us say more visibility and more control for the R&D. I think that is the key issue, the only thing that they actually understand and can commit to”.

The second most popular CSF is “unfreezing the organization” - readiness of the company to overcome inner resistance for change. Developers Oy evaluate this factor with importance equal to 4.4. However the situation in the company reaches the rate of 2.4. Almost all employees see the need for change, but as a cause of limited management commitment and rather low non-technical management involvement skeptical attitude toward SPI initiatives can be observed in some cases.

“Managing the improvement project” or the level of effectiveness in SPI project management and control is an important issue for the company, with average rating of 4 and 2.8 as for the current situation. Managers received training on SPI essentials: “we have given the training on the SPI model, how we are going to run it, how we are setting the target, how we are reporting the target, how we are doing those process improvements”. Training is essential, so managers have the knowledge base for competent company running.

“Staff involvement” indicates to what extent staff members participate in SPI initiatives; it is granted with 4 points and 2.4 for the current situation. In Developers Oy the personnel is conditionally divided into several groups, each group receives information selected for it: “we built a dashboard, the dashboard is one of the ways of giving this information, and the higher you go in organization, on higher level those goals and measures actually are”. In general, there is a process development team in the company, four people who are fully occupied with tasks related to SPI. Moreover, those employees have an option to develop SPI activities in some company’s projects by involving extra human resources. That results in 30 % of all company personnel involved into some SPI initiatives and one team of professionals entirely working on those issues.

“Training and mentoring” refer to training provided to SPI team by experts. It gets 4 in average importance and 2.4 for the current situation. The factor is highly graded by the company representatives; however the situation with SPI policy is special and influences this factor. Developers Oy pilots SPI activities in two big projects: one is related to requirements engineering, the second to “formal workflow for actual software development, or so-called configuration management”. Personnel working for these projects were specially selected for this purpose, meaning that: “they are not average people; they are well motivated and clearly understand advantages that the company receives by SPI initiatives”. Those people do not need too much special mentoring. On the other hand, company provided some training to middle level management, but it was not quite successful: “unfortunately, we have some kind of failure there; we have done some internal training and motivation. It happened because this training was not enough and people were not willing to have it further”.

“Change agents” normally support SPI activities at the company level, and “opinion leaders” at the local levels. This factor is evaluated as 3.8 by importance and 3 as on present. The overall attitude toward SPI specialists is ambiguous: “there are some people who are a little bit jealous to SPI team by not being involved; some who are afraid of changes and not willing to change any processes”. Developers Oy does not have so-called change agents but few people on executive level who are “eager, well-committed and able to explain benefits SPI activities are targeted at. They are highly respected in the company”.

“Awareness of SPI” shows how well SPI initiatives are promoted in the company as a short and long term beneficial activities. The factor is evaluated with importance of 3.6 and 3.2 for current situation. Developers Oy is rapidly growing SW company, getting more new employees and processes evolve, so people understand the necessity of change: “now when we have had some small evidence of success employees observe the benefits the whole company can get. There are concerns about technology and processes in use, but over all people see benefits of doing SPI”. Globally, benefits for developers are perceived as following: “developers become professionals; they understand the essences of SW development, not the programming language, not as-it-is status, but on how to develop in more efficient way and improve the quality at the same time, so they do not need to debug few times”.

“Resource allocation” defines how human and time resources are dedicated to SPI. The importance is graded with 3.6. Even though current state is the company receives only 3 points, senior product development manager declares: “in Developers Oy the situation is rather stable, we have enough resources”.

“SPI goals” important to be distinguished and understood within the company. Developers Oy assesses it as 3.6 and 2.8 for the current state. Management of the company has clear and well explained position on the company’s goals and future vision; business requirements normally derive from those goals, in turns processes appear from business requirements: “all SPI actions we perform are aligned with the strategy of the company”. Senior product development manager claims that employees of engineering processes and support department (partly responsible for SPI activities) are well motivated and trained towards SPI goals and their correlation with the overall strategy of the company. At the same time personnel not involved into SPI activities on daily basis have challenges which are not a positive sign. Developers Oy needs certain measurements and evidence of successful SPI initiatives from other departments as well.

“Stabilizing change process” indicates continuous support of SW processes, their maintenance and improvement at the local level. This factor gets 3.6 points for importance and 2 for the current state. That factor assumes regular feedbacks from employees and management support. Developers Oy does not have feedbacks as widespread practice: “the only things that we have are wiki and common workplaces where we have discussions and comment projects; this is not obligatory, but we can have valuable feedback during meetings and reviews”. Nevertheless, the company plans to introduce new questions on SPI to the mandatory internal satisfactory survey. As a motivation Developers Oy holds regular meeting on running SPI initiatives in the company where key players and opinion leaders present.

“Encouraging communication and collaboration” assumes communication between the members of one team and members of different departments. Developers Oy organizes regular meetings for the personnel and tries to increase interactivity and communication by introducing new and modern tools for expressing anonymous opinion and providing feedback. The company has kick off meetings for R&D department twice a year where they discuss achievements. Personnel exchange information primarily on project level, not much on business unit level.

“Providing enhanced understanding” correlates with a factor of “SPI goals” and means existence of knowledge in current SW processes and business environment. Some people in the company relate their contribution to SPI initiatives as to overall corporate mission and vision, some not.

“Tailoring improvement activities” is adaptation of SPI activities and needs of specific departments within the company. SPI activities effort to specific strengths and weaknesses of different teams and departments in Developers Oy: “it is a reality, all the teams are different and all of them have to be handled differently. It is done case by case, for example testing personnel has to be treated differently than program managers. Program managers are easy to explain why we do certain things, while testers mostly have distractive mindset and pessimistic to everything”. In marketing department the situation is also challenging: “we have to motivate them heavily to get their stake of involvement”.

“Defined SPI implementation methodology” is defined as guidance on how to implement SPI activities. Developers Oy organized seminar on SPI model used in the company disseminating how the company was going to run the model, how targets were settled, how the target would be reported, how process improvement was implemented and accepted to use. There two types of managers supporting SPI activities. First group is eager to share their practical knowledge by demonstrating how processes are performed; second group is well concerned about methods of work but delegate the guidance to responsible people.

5 Conclusions

In this research we conducted a pilot case-study on CSFs in SPI in Finnish software company Developers Oy to verify whether the company manages performance of CSF in SPI initiatives and how the company manages them.

By examining comprehensive literature we selected main papers on the topic. The CSFs in SPI found by scholars are different but mostly frequent mentioned are: senior management involvement in SPI initiatives, importance of clear vision and understanding of SPI goals, technical staff participation, and motivation for employees to be part of SPI initiative. Besides, both technical and managerial human resources need to be involved in SPI program. All scholars perceive the organizational issues, human involvement and change management as the key elements of any SPI initiative.

The factors selected from the literature generated the evaluation questionnaire on importance and as-it-is status of CSFs for the case company. Several case company representatives evaluated the list. CSFs were rated and commented by Developers Oy. We observe the main challenges with people and change management factors. The most important CSFs needed improvements are: management commitment, staff involvement, unfreezing the organization and stabilizing change process. Those factors are interrelated. Unfreezing the organization and overcome resistance for change within the whole company is problematic without proper management commitment. This commitment is needed at all levels of organization starting with higher management. In Developers Oy the situation on the higher level is rather promising, while middle management level is challenging one. The factor of staff involvement relates directly to management commitment. If managers do not motivate the personnel, do not provide support and personal example employees are not willing to take part into novelties and refuse changes. Therefore, it is basically impossible to have any positive feedback about SPI initiatives if regular workers do not get enough support. Based on the opinion of senior product development manager we conclude that one of the main challenges within SPI program is managing the change resistance. Currently, managers have no training how to handle various situations since all employees are different and should be treated differently. This kind of training is needed and for key SPI players as well. Managing the change resistance allows constructing initial improvement plan on that issue.

The main finding of this pilot study shows that proper management of CSFs increase usefulness of offering SPI to its key beneficiaries, thus stakeholders values are taken into consideration.

6 Future Research

The field of CSF in SPI is well developed and studied but there is limited number of research about CSF in SPI in small- and medium sized companies (SME). In addition we did not find many papers devoted to research in the field of CSF related stakeholders in SPI in SMEs. At the same time small – and medium size companies play significant role on the information technology world market. Based on the results of the current paper and domain observation we plan to conduct wide exploratory study with large sample of companies on CSF related to stakeholder in SPI activities in SMEs.