Application of GQM⁺Strategies in a Multi-industry State-Owned Company

Abstract

Technological applications have an increasingly important role in most companies. Investment in Information Technologies (IT) is also growing in most of them. The need to align IT-related goals with the company’s strategic goals becomes imperative. GQM⁺Strategies is an approach to align organizational goals, strategies, and measurements at different levels of an organization. This paper describes experiences learned from a GQM⁺Strategies implementation at a large multi-industry state-owned company. The implementation was conducted by an academic research team joined by representatives of the company. Results showed an improved alignment and integration of different goals. Moreover, a holistic goal visualization was achieved, even though the company works in two different industries. As a state-owned company, external pressures force continuous planning. Sometimes, inadvertently designing those plans is the only goal achieved (i.e., plans are never executed). Using GQM⁺Strategies, the research team leveraged the separation of goals and strategies, allowing identification of redundancies and replicated efforts across the company. The implementation of the GQM⁺Strategies approach in such a complex context was very valuable. However, it also required a large amount of effort from the researchers and company representatives.

1 Introduction

GQM⁺Strategies is a measurement approach that extends the GQM paradigm [1, 2], it promotes the creation of measurement programs that ensure alignment between goals in different levels of a company.

This paper describes experiences gathered while applying GQM⁺Strategies in a complex context (i.e., a large multi-industry state-owned company). The company has an intrinsically hierarchical structure. Moreover, industry segregation promoted a silo mentality within the company.

The implementation of GQM⁺Strategies in this context was triggered by a Software Process Improvement (SPI) initiative in the company’s IT department. This department serves both industries and an administrative and finance management. Moreover, it is in charge of promoting IT related controls and standards.

The main benefits of GQM⁺Strategies and similar approaches are: an explicit description of goals, strategies, context factors and metrics that allow the organizational alignment; a transparent instrument to improve communication both within the organization and to third parties; and measurement data that allows information-based decision-making [3].

This research has two goals: one industry-related and one academic. The industry-related goal was to align the different levels of a large multi-industry state-owned company and to help this company in their SPI efforts. The academic goal was to assess the applicability of GQM⁺Strategies in such a complex context.

GQM⁺Strategies demonstrated that several efforts within the company were replicated and some goals were promoted by both industries in which the company operates. Moreover, the alignment allowed to reduce segregation and provided a holistic vision of the company’s IT roadmap. Senior management saw potential on the GQM⁺Strategies Grid. They started to call it the company’s value map.

The rest of the paper is structured as follows. Section 2 introduces the theoretical background, delving into GQM⁺Strategies definition and similar experience reports of case studies (i.e., related work). Section 3 describes our implementation of GQM⁺Strategies including: Application context, research approach, execution and results. Section 4 describes the main lessons learned in this research. Section 5 describes similar experience reports and compares those experiences with the ones described on this paper. Finally, Sect. 6 presents some final remarks.

2 Background

2.1 GQM⁺Strategies

Goal-Question-Metric plus Strategies approach (GQM⁺Strategies® [4, 5]) was defined as an extension of the GQM paradigm [1, 2]. GQM is a measurement system that sets rules for interpretation of measurement data in three levels: Conceptual, operational and quantitative. The GQM process identifies goals, derives questions and specifies metrics [1].

GQM⁺Strategies proposes the alignment of the traditional GQM paradigm with organizational goals in three levels: Business, software and project. Moreover, GQM⁺Strategies describes an iterative process to achieve organizational goals alignment through measurement [3] and proposes that constant updates are necessary to reflect organizational, departmental or team goal changes [6]. Figure 1 shows the phases of GQM⁺Strategies and its sub-activities.

Fig. 1.

GQM⁺Strategies flow diagram. Based on phases reported on [3]

GQM⁺Strategies recommends the use of a Grid that links goals to strategies, questions, and metrics, easing communication of common goals in the organization. The Grid facilitates traceability of goal-related data and integration of the measurement program across different levels within the organization [7, 8].

GQM⁺Strategies considers context factors and assumptions that are made during the implementation of measurement programs. Therefore, it allows inspection of erroneous assumptions or context changes over time. GQM⁺Strategies helps organizations to define what, why and how to measure, and interpret those measurements [6].

3 GQM⁺Strategies Implementation

3.1 Context

The company in which GQM⁺Strategies was implemented has several distinct characteristics: (1) It is a large organization (more than 15.000 employees on the payroll), (2) has more than 60 years of existence, (3) it is a multi-industry company, and (4) it is a state-owned company. Furthermore, one of the industries in which the company operates recently transitioned from a monopoly to a competitive environment. Figure 2 shows a simplified company organigram.

Fig. 2.

Company organigram

Figure 3 illustrates the organization within the IT department. The IT department is part of the administrative and finance management, there are also two managements at the same level, one for each industry in which the company operates.

Fig. 3.

Organization of the IT department

Industries 1 and 2 have different dynamics, one is a very volatile (i.e., requirement change rates are very high), the other is very structured and passive. The IT department is in charge of IT across the company. However, each management has its local IT (self-funded) therefore a silo mentality has been established (i.e., departments or groups within an organization do not want to share resources (e.g., information, knowledge, funding, among others) with other individuals in the same organization.

The traditional approach towards measurement in the organization has been balanced scorecards [9] and Key Performance Indicators (KPI). Therefore, concepts such as learning and growth, business process, customer or financial perspective, and strategy mapping are deeply rooted in the company.

3.2 Research Approach

The GQM⁺Strategies implementation reported in this paper was performed collaboratively between the University of Costa Rica research team and the company representatives. The research team comprised twelve members: Four PhDs, one PhD-student researcher, three MSc (one researcher and two industry practitioners), and four BSc, during a two-year period (i.e., approximately four at any given time). The company representatives comprised: Three permanent members (all seniors at the organization). The main positions of the company representatives were: Company’s IT department manager, IT department’s strategy division manager, and another strategy division member. However, the application of GQM⁺Strategies incorporated the vision of over 12 people working in the IT department and its divisions.

GQM⁺Strategies implementation started from senior management’s desire of implementing a software factory (i.e., commitment to long-term, integrated efforts to enhance software operations [10]). The research team performed a diagnosis focused on: The company’s organizational scheme, software development culture and processes, human resources and technology used in the development process and other regulations.

The diagnosis was implemented through interviews (15 interviews/71 people), site visits (5 visits) and an online survey (281 participants). All these activities were carried with people from the IT department and the local ITs (Fig. 2). From the diagnosis 44 findings were reported. Findings can be categorized as: General, business vision, requirement management, quality assurance, release management, change management and configuration, project management, architecture, support, outsourcing management, IT governance, technology, people, and culture.

In several of these findings the measurement process is mentioned as a requirement before addressing the issues reported (i.e., measurement process must be implemented before fixing the problem), in order to ensure improvement and measure progress. This situation was crucial to boost GQM⁺Strategies implementation in the organization and convince senior management that a measurement approach that also aligns business goals would be very beneficial for the IT department and the company.

3.3 GQM⁺Strategies Execution and Results

The application of GQM⁺Strategies consisted mainly of a set of planning meetings with the company representatives. In the next subsections we will describe how we approached each phase of GQM⁺Strategies in the context of this implementation, the main results and difficulties found along the way.

Initialize

According to [3], required inputs for this phase include a motivational talk and tutorial. We conducted three sessions of motivational talks (due to the number of people that participated). In these sessions, the IT department manager and the company’s training manager introduced the planned efforts. Three cases were presented to motivate participants: A former software engineer at Google talked about working culture, a former software testing engineer at Microsoft Corp talked about the importance of high quality software products and a former software developer at Costa Rica’s Central Bank talked about software improvement processes in public institutions.

Commitment was a given, since the process originated from the organization. Moreover, the IT department manager was fully involved at this stage. A schedule of activities was prepared and the budget was calculated. At this stage training people is a task that should be executed. However, only two work sessions were conducted due to representatives’ insufficient time. Table 1 shows the training sessions, their coverage and dates.

Table 1. Joint events for sensitization and training, all sessions were conducted in 2015

The original GQM⁺Strategies scope was the IT department, leaving out the local ITs. However, once the process started, provided documentations included IT-related goals of the company’s three managements (See Fig. 2).

Characterize the Environment

In this phase, while assessing the applicability of GQM⁺Strategies, we discovered that IT department’s strategy division head compared the approach with balanced scorecards (due to their focus on measurement), this was a difficult to overcome problem, because balanced scorecards have been used for many years and their promotors did not want to change the way measurements were executed.

Since our application contexts is a public state-owned company, they are asked to provide detailed documentation of goals and investments. Therefore, there was a high availability of organizational structure, process, goals, and future plans documentation. At a certain point there was too much documentation. Later on, we will discuss on how plans for each management had duplications and ambiguities that made the GQM⁺Strategies refinement process very difficult.

Define Goals, Strategies and Measurement

We asked the company representatives to provide information regarding current goals, plans and measurements. Delivered documentation included: Current measurements for the business operative plan, 2016 IT plan, IT strategic framework (2015-2018) at company level, IT strategic framework for the administration and finance management (2015), IT strategic framework for the Industry 1 management (2014-2018), IT strategic framework for the Industry 2 management (2016), and IT Strategic framework, general strategic documentation (2014-2018), proposed metric for ITIL (2011), IT government statute, and the administration and finance management strategy (2015-2018).

The research team read and interpreted all the provided documentation and extracted IT-related goals. Figure 4 provides a structural overview of the GQM⁺Strategies Grid derived from the documental revision. Due to confidentiality reasons of the case company, the complete list of goals and strategies are not mentioned.

Fig. 4.

Structural overview of the GQM⁺Strategies grid (first version)

In the first version of the GQM⁺Strategies Grid, no difference was made between goals and strategies, everything was considered a goal since there were no further details on how the statements were going to be achieved.

A change in mental models was required to understand the GQM⁺Strategies proposal and differentiate between goals and strategies. This effort was very valuable to assess the implementability of many proposed goals in the organization.

In order to move from the first version of the GQM⁺Strategies Grid (Fig. 4) to the final version (Fig. 5), refinement events were carried with company representatives. They were asked to assess the Grid’s vision according with the scope of the implementation and the reviewed documents.

Fig. 5.

Structural overview of the GQM⁺Strategies grid (final version)

In Fig. 4, the left most boxes represent the company’s strategic goals. They were considered and incorporated in the Grid because one of the documents stated that the goal of IT in the company was to support all the company’s strategic goals.

Figure 4 contains twelve company strategic goals (non IT-related), seven high level goals (IT-related), and 193 lower level goals. Some of these goals were similar from each other. Nonetheless, the research team decided to keep them all, in order to provide evidence of the duplicated goals found across documents.

The first refinement session was conducted with the version of the GQM⁺Strategies Grid created from the goal extraction of the documentation. In this first refinement session, both the director of the IT department’s strategy division and a senior employee of that division participated. The GQM⁺Strategies grid was introduced and they started to define changes that should be incorporated.

Table 2 shows the GQM⁺Strategies Grid refinement sessions conducted during this GQM⁺Strategies implementation. After each refinement session a new version of the GQM⁺Strategies Grid was generated and used in the next refinement session. This did not occur in the sessions conducted on November 11, all these sessions were conducted with the same version of the grid and a set of changes was prepared to be incorporated after these three sessions.

Table 2. GQM⁺Strategies Grid refinement events carried during 2015

Even though the research team tried enforced the company representatives to make strategy decisions that would allow later strategy execution, this was not possible. In the context of a large state-owned company, individuals are not used to make decisions alone.

Once the GQM⁺Strategies Grid was ready (i.e., all apparent goals were included), the IT plan for 2016 was released and approved by the board of the company (i.e., it was engraved in stone). This document was facilitated to the research team in early 2016; therefore, the GQM⁺Strategies Grid was reviewed and refined. Again, no difference was found between goals and strategies in the IT plan. A session was conducted to identify how the IT plan and the GQM⁺Strategies Grid could be aligned and the representative’s answer was that the Grid was the one that required to be changed, because the IT plan was approved by the company’s board of directors.

The defined solution was to implement a mapping tool (initially an Excel document) to translate IT plan language into goals and strategies and incorporate them in the GQM⁺Strategies Grid. At this point, the GQM⁺Strategies Grid was transferred to be used internally in the company.

Plan Grid Implementation

In this phase, the research team decided to adapt an iterative approach toward improvement [11]. An improvement backlog was developed based on the GQM⁺Strategies Grid strategies, and an iterative improvement process began. In order to assure that measurements are aligned between them, the research team proposed measures for each goal incorporated in the GQM⁺Strategies Grid final version.

The GQM⁺Strategies Grid showed in Fig. 5 is the Grid in its current state. The Grid contains two (non IT-related) company strategic goals that are directly mapped with nine IT-related high level goals. Sixty-two low level goals were finally incorporated into the GQM⁺Strategies Grid. Furthermore, 149 strategies were linked with those goals. The analyze outcomes and package improvement phases of GQM⁺Strategies are still to be implemented after the strategy execution and measurement implementation.

In the next section, we describe some lessons learned of this GQM⁺Strategies implementation. We acknowledge that GQM⁺Strategies is not a novel approach (i.e., it has more than 6 years being applied); however, we did not found evidence (published in academic literature) of the application of this approach in such a complex context: Large multi-industry state-owned company. Therefore, we believe that the insights gathered from this experience could be beneficial for further implementations and researches.

4 Lessons Learned

The context in which GQM⁺Strategies was implemented had traditionally used balanced scorecards as the main measurement approach. Some issues were found in the implementation of balanced scorecards that reduced their value for the organization:

• Metrics were created to be centralized and generalizable. Lower levels of the company were not given the change to develop their own measurements.

By applying GQM⁺Strategies, we did not only allow lower levels to design measurements, but also to incorporate their goals into the GQM⁺Strategies Grid.

• The company worked with silo mentality, from managements to working departments, all areas work separately and do not share information. There was an entrenched lack of information integration.

By reviewing documentation provided by all management offices and both the IT department and local ITs, we took a step forward towards data integration in the company. GQM⁺Strategies, and specially the Grid provided so much value in the eyes of the strategy division director that he started calling it the company’s value map.

• Balanced scorecards were developed in different levels of the organization; however, they were not fully integrated.

As we mentioned, information was not integrated horizontally. Furthermore, the measurement process carried did not provide a vertical integration. Therefore, lower level measurements were not always proper to assess higher level goals. In some cases balanced scorecards used different indicators to measure similar things in different levels of the organization.

Delving into the GQM⁺Strategies implementation, we also faced some challenges and we learned some lessons while dealing with those challenges:

• A flexible tool to manage the GQM⁺Strategies Grid

We developed the visual representation of the GQM⁺Strategies grid using Microsoft’s Visio. In cases in which the grid is small, there are no major challenges to manage changes. However, in a Grid with over 200 components, changes were difficult to deal with.

A flexible tool designed for business and information management purposes (instead of a diagramming tool) would have been very beneficial for the process. During the implementation of GQM⁺Strategies, the company in which the approach was implemented bought a tool called MindManager, the first activity they performed as soon as the grid was delivered, was to change it from Visio to MindManager due to the flexibility it provided. We acknowledge that our use of Microsoft’s Visio was not the best, since we used a basic diagram, however, the use of a specialized tool made the process much easier.

• During the implementation of GQM⁺Strategies, a clear segmentation between the research team and the company representatives was made. The first version of the GQM⁺Strategies Grid was created by the research team by reviewing documentation. This approach was beneficial to avoid bias in the Grid; however, it also increased the cost and time required to finalize the first version of the GQM⁺Strategies Grid.

By creating the Grid from an outsider perspective, the research team was capable of identifying a large number of goals in the reviewed documentation. All these goals were incorporated in the first version of the GQM⁺Strategies Grid. However, once this version was ready to be assessed, we suffered from lack of participant identification and commitment in the refinement sessions. Moreover, we believe that they did not understand the process.

Through training and iterations in the refinement process, this issue was overcome. However, the costs increased. If the company representatives had participated in the process since the beginning this issue would not have appeared, but the unbiased perspective could not have been achieved.

• An automation tool could have been used to keep track of changes

During this implementation we used manual changelogs (to keep track of changes in the grid). A better tool to manage the GQM⁺Strategies Grid could have provided the mechanisms to automate tracking changes and avoid the costs of manually tracking them.

• The company is undertaking a change process. However, deep changes in the organizational structure of the company destabilize these efforts. We faced problems to maintain sponsorship during GQM⁺Strategies execution. The director of the IT department strategy division changed during the implementation. The new director saw potential in the results of the process and promoted it. This could have been the other way around, and sponsorship could have been lost.

The GQM⁺Strategies Grid allowed us to clearly distinguish the organizational changes over time. Excess of documents over the years were proof of those changes. It is hard to assess if GQM⁺Strategies should be implemented while changes so deep are occurring in a company. Furthermore, cultural characteristics were against the incorporation of practices that make visible information within the organization. Therefore, we faced resistance from some employees.

In order to face some of the main issues found during our GQM⁺Strategies implementation we used some mechanisms including: A changelog to keep track of changes between refinement sessions. This was necessary due to the complexity of the GQM⁺Strategies Grid initial version.

The changelog is composed of all the changes that arise during the refinement session. The choices of changes proposed to the participants were: Merge, eliminate, modify and create. Merge was used to unify two or more goals attending the same issue or executing similar tasks. Eliminate was available to discard goals that were not considered IT-related. Modify was used to adjust the original texts to either goal or strategy syntax. Create was used when a strategy was devised and it had no associated goal (i.e., the goal was created based on the strategy).

All this was conducted to obtain the first stable version of the GQM⁺Strategies Grid, after the refinement sessions. However, we also see fit to use the changelog as further reference. For instance, once a goal is achieved, it might be changed or discarded from the GQM⁺Strategies Grid, without trace of it ever being there. We believe that changelogs or version control might be implemented to provide the company with a memory of its goals.

To implement this proposal, the definition proposed by Münch et al. [12] might be used to characterize each goal, denoting the changes it has undergone: New goal, discarded goal, unchanged goal, revised goal, split goals, merged goal, established goal and linked goal.

The IT plan provided by the company for this GQM⁺Strategies implementation considers goals to have different maturity levels, including: Foundation establishment, positioning and consolidation/operation. These phases can be mapped to the GQM⁺Strategies Grid as it changes over time.

The changelog allowed us to effectively track and document the changes made after a refinement session. Also, we used them as a reference before we started a new refinement session. To show to the interviewee the evolution of the Grid and the most recent modifications.

Finally, as it was expected, visualizing such a large GQM⁺Strategies Grid was almost impossible. Navigation between goals was also a difficult task. Figure 6 shows the final version of the developed GQM⁺Strategies Grid.

Fig. 6.

GQM⁺Strategies grid traditional visualization

As it is evident, the traditional hierarchical visualization of the GQM⁺Strategies Grid does not work with so many goals and strategies. It is difficult to understand the context of each goal and navigate between them. We created a poster version of the grid (Fig. 7), dividing each branch at the higher IT-related goals. A highlighted section of the poster shows the highest level relationships. This visualization was very helpful for the company representatives, easing navigation and overall understanding of the GQM⁺Strategies results.

Fig. 7.

GQM⁺Strategies grid visualized as a poster

5 Discussion and Similar Experience Reports

Several experience reports and case studies have been reported in literature addressing the implementation of GQM⁺Strategies in different contexts. This section describes some of them, their similarities and differences with the experience described in this paper.

In 2010, Mandić et al. [13] proposed an approach to assess GQM⁺Strategies’ practical value and applied it in a Finnish ICT company. Authors concluded that GQM⁺Strategies had practical value for the company. However, authors only assessed a small department in the company. In this paper, only one department was assessed. Nonetheless, this department serves the whole company and shares goals three different managerial offices.

Also in 2010, Sarcia [14] reported that in order to apply GQM⁺Strategies in non-software development domains in required to be generalized (i.e., converted into a context free approach). Author assessed the application of GQM⁺Strategies in a military training domain in Italy. This work was an ongoing experience and no further details were found in literature. Even though the experience described in this paper was gathered from the application of GQM⁺Strategies in an IT Department, one of the industries in which the company works is not related to software development. Some of the high level goals (for this industry) had nothing to do with software development. Therefore, this experiences support the finding that GQM⁺Strategies is applicable and introduces business value in non-software development domains.

In 2011, Kaneko et al. [15] presented results and experiences from applying the GQM⁺Strategies approach at the Japan Aerospace Exploration Agency’s software research department. The efforts were joined with IESE’s experts. Three top-level goals are reported; 20 lower level goals are also identifiable in their GQM⁺Strategies Grid. Reported lessons learned include: (1) Reuse of previously developed measurement models is important to reduce efforts and (2) GQM⁺Strategies Grid can contribute to prioritize high level goals by assessing which ones are more feasible. In this experience, consultants had to force a separation between the previously used measurement model and GQM⁺Strategies, because management started to fall back into their traditional measurement practices, and this was considered a threat for GQM⁺Strategies implementation and the SPI initiative as a whole.

Also during 2011, Trendowicz, Heidrich, and Shintani [16], reported lessons learned from a GQM⁺Strategies implementation at the Japanese Information-technology Promotion Agency. Authors present an interesting perspective on how GQM⁺Strategies allows project evaluation regarding their contribution towards higher-level goals, and how clear rationale can be applied for rejecting or approving project proposals based on their contribution. To assess this perspective, the implementation of GQM⁺Strategies should be ingrained in the company and its practices. An evaluation of this contribution in the context of application presented in this paper will require more time and further analysis.

In 2013, Münch et al. [17] reported experiences on a five-month application of GQM⁺Strategies in an industrial company. Main lessons learned reported include: (1) The necessity of company representatives to have knowledge of GQM⁺Strategies and how to apply it, (2) find a suitable entry point for the approach implementation is crucial. In this case, the efforts presented in this paper are very similar to the ones described by Münch et al. [17]. However, the experience time period was larger due to the complexity of the company and the amount of information available that had to be revised. Moreover, the entry point was a key aspect to boost GQM⁺Strategies implementation. The sponsorship of senior management and participation of representatives with the proper knowledge enhanced sped up the GQM⁺Strategies implementation.

In 2013 Basili, Lampasona, and Ramírez [18], describe an overview of the application of GQM⁺Strategies in ECOPETROL (oil and gas industry). Their main goals were for alignment and formalization of goals, providing strategies with visibility, and defining operational measurement and interpretation of goals. Authors point the effort required to implement GQM⁺Strategies (i.e., the GQM⁺Strategies Grid requires the collaboration of several people representing different levels of the organization and is a non-trivial mental activity). This experience report is very similar to the one conducted at ECOPETRO. Different perspectives were incorporated, not only due to consultant request but also because the company representatives advised that other perspectives should be considered to create the GQM⁺Strategies Grid.

In 2014, Petersen et al. [19] proposed an elicitation instrument to gather stakeholder perspectives and incorporate it into the GQM⁺Strategies approach. In their paper, authors explain the case study performed at Ericsson AB (Telecommunications industry). We tried to apply this elicitation instrument; however, it was too structured for the first implementation of GQM⁺Strategies. This is the most similar case to the one presented in this paper, due to industry similarity. However, Petersen et al. [19], do not delve in the specific results of the implementations. Therefore, a comparison is difficult to conduct.

In this section we presented experience reports, case studies of GQM⁺Strategies conducted from 2010 to 2014. Those reports were compared with the one described in this paper.

6 Conclusions

In this paper we presented our experiences of applying the GQM⁺Strategies approach in a large multi-industry state-owned company. The application was triggered by an improvement initiative to implement a software factory.

GQM⁺Strategies was implemented by a research team jointly with company representatives. The approach was followed as proposed in [3]. The research team managed the process and company representatives worked as stakeholders.

The GQM⁺Strategies Grid was firstly derived from a documental review of the company’s information. Over 10 large (100 page or more) documents were reviewed and goals or action plans were extracted. With the list of goals, the research team conducted a process to unify them in one single representation. The first version of the GQM⁺Strategies Grid did not differentiate between goals and strategies.

Following a proposal to operationalize GQM⁺Strategies [19], we conducted refinement sessions. In each refinement session one company representative’s vision was incorporated, and doubts emerged from the process were revised and solved. The research team implemented a changelog to keep track of changes in order to modify the Grid and use an improved version in the next refinement session. We observed the following key benefits and advantages of applying GQM⁺Strategies in such a complex context:

• GQM⁺Strategies helped the company to get a complete vision of their goals and provided them with a tool to visualize those goals integrally.

• The approach implementation made visible the benefits provided by operative and low level tasks both to immediately above goals and to the company’s objectives.

• As a state-owned company, they suffer external pressures to plan their activities. Such pressures sometimes force them to plan in advance for several years. GQM⁺Strategies provided a tool to enforce alignment between plans (goals) and to define strategies to achieve those goals. Moreover, it allowed the identification of redundancies and replicated efforts.

• As a multi-industry company, they adopted a silo mentality. GQM⁺Strategies promoted the unification of the IT department goals for both industries and the administrative and finance management. Therefore, the segregation was reduced. Moreover, the research team specifically recorded which industry or management proposed the goals allowing the IT department to prioritize goals that have impact into more than one management.

One of the key lessons from our application GQM⁺Strategies is that goal very difficult and institutions that first try to implement the approach do not distinguish between goals and strategies. In general, the GQM⁺Strategies approach was very valuable in the context in which it was applied. However, the efforts invested were large. We expect for the results to have an impact as large as the effort invested to implement it.

One of the key lessons gathered from this effort was the help that GQM⁺Strategies implementation provided to visualize redundant goals within the company. This visibility allowed the research team to point out deficiencies in the budged investment and the company representatives to convince internal management that a change was necessary to avoid unnecessary expenses. Even though this is not a direct benefit from GQM⁺Strategies researchers could not achieved this result without following this approach.