Keywords

1 Introduction

Alignment between IT (Information Technology) and business goals is considered by both practitioners and researchers a management practice to enhance organizational performance. However, there is still lack of knowledge about what organizational actors really should do in practice for this alignment to happen [5]. There is a need for researchers to adapt and extend knowledge about what means IT to be aligned with business and how to measure it [6].

Measurement is a key process to support organizations in managing and improving processes, products, and services to achieve customer satisfaction [1]. Measures should be used to monitor the alignment of IT to business goals by providing useful information for decision-making [3]. However, managers face difficulties to define measures, evaluate if projects are bringing expected results to business and monitor results to keep alignment between IT and business goals [3, 4].

The first author of this paper works at IT team of a large global mining company. She was asked to help the director and the five managers to define measures for goals and to review initiatives (projects and operational activities) to achieve those goals. At that point, goals were qualitative and subjective, and the director was not able to verify if the initiatives were contributing to goals achievement. We have already successfully used GQM+Strategies [2] in other areas of the organization to aid in the alignment between goals and strategies through measurement [10]. However, the IT director was running a tight schedule and needed a fast approach, which did not require training or many phases. He asked us to use OKR (Objective Key Results) [7], a method to support defining and tracking goals and their outcomes, which has been increasingly used in industry. OKR has an agile appeal, while GQM+Strategies provides detailed knowledge on how to align goals and strategies through measurement. Thus, we decided to explore the combined use of the two methods in a way that they work complementarily. As a result, after three meetings, we defined OKRs to five IT goals and initiatives to achieve them.

This paper presents the study and its main findings. It is organized as follows: Sect. 2 provides the background for the paper; Sect. 3 presents the study planning and execution; Sect. 4 addresses the process that arose from the study, Sect. 5 discusses our findings and study limitations; and Sect. 6 presents conclusions and future work.

2 Background

IT-business alignment can be considered the level of fit and integration between business, IT processes, projects, and infrastructure of an organization [13]. Aligning goals and IT projects help focus resources and projects towards value creation and requires finding the connections between them so that the links are explicit and allow for analytic reasoning about what is successful and where change is necessary [2].

The GQM+Strategies approach [2] is an extension of the Goal-Question-Metric paradigm and helps control the success or failure of strategies and goals by using a measurement system. In GQM+Strategies, strategies refer to projects, actions, or other initiatives performed to achieve goals. The GQM+Strategies model relates goals and strategies at several organizational levels. One or more strategies can accomplish the same goal. Context factors and assumptions influence goals and strategies. A GQM+Strategies element includes an organizational goal, respective strategies, context, and assumptions that influence them. GQM+Strategies elements and related models are represented in a GQM+Strategies Grid, making goals and strategies explicit, as well as measures related to them, providing a transparent correlation between goals, strategies and measurement initiatives. The GQM+Strategies process consists of an initial phase and a repeatable cycle with three stages and six phases: Develop (phases 1 and 2); Implement (phases 3 and 4) Learn (phases 5 and 6) [2].

Objective Key Results (OKR) is a collaborative goal-setting protocol to help ensure that the company is consistently focusing and prioritizing efforts on the same issues throughout the organization [8]. An OKR has two components: the Objective, qualitative and inspirational, and Key Results, quantitative and measurable. The objective should be meaningful, significant, concrete, actionable, and inspirational. Key results gauge and measure how to achieve the objective and are quantitative, usually time-bound, verifiable, and realistic. The process to define OKRs consists in setting the objectives; determining the key results for each objective, executing actions to achieve the objectives; providing regular feedbacks.

3 Study Planning and Execution

Participative case study was selected as research method as the researcher was a member of organization, she observed the particular group of organization’ subjects, and was one participant in the process being observed [11]. The researcher had some control over some intervening variables and was a stakeholder in the process’ outcome, as she was part of the department and would work to achieve OKRs. The participative case study report attempts to capture and communicate the biased interpretation by stakeholders of their particular environment during a particular period in time. We followed two phases Planning, for case study preparation, and Execution, for data collection.

The organization where we carried out the study is a large global mining company operating in over 30 countries, with offices, operations, exploration, and joint ventures across five continents. Information Technology (IT) department is composed of five areas: Innovation and Projects, Architecture and Technology, IT Services, Business Partners, and Strategy and Planning. At the beginning of the year, the IT director defined a set of goals, and the IT managers elicited 140 initiatives to achieve them. In April, the director realized that the goals seemed non-measurable, and he was not able to verify if initiatives elicited by IT managers were able to achieve the defined IT goals. The IT director needed a fast approach to focus efforts on the right direction and had not enough time to spend on training or following many phases of a traditional goal-setting method.

Since OKR (Objective Key Results) [8] has been increasingly used by industry to support the creation of measurable and achievable goals to foster alignment, engage the team and follow a fast cadence, the IT director showed interest in using it. The OKR literature provides knowledge (examples, good practices, tips, concepts) to build OKRs and monitor results [8]. However, there is no practical direction or procedure about how to gather contextual information and turn a qualitative objective into a measurable goal for a key result. There is also no direction about how to elicit initiatives (i.e., strategies) to achieve goals. GQM+Strategies [2] provides this kind of knowledge. We had previous experience using GQM+Strategies [2] in other departments of the company [10], and thus, we decided to combine both methods. By doing that, we expected that OKR would satisfy the need for a faster approach, while GQM+Strategies would provide complementary knowledge to perform the activities. Next, we present information about the study planning and execution.

3.1 Planning

The goal of the study is to analyze the combined use of OKR and GQM+Strategies to support defining measurable goals, OKRs, and initiatives for IT goals. Aligned with this goal, we defined the following research question: How to combine OKR and GQM+Strategies to measure qualitative goals and support their achievement? The expected outcomes were (i) a list of OKRs agreed by both IT director and IT managers to measure the achievement of IT goals, (ii) a process to support defining OKRs.

The technique used to collect data was document analysis and three brainstorm meetings with the IT director and IT managers. When we received the following list of five IT goals (G) defined by the IT director, it became clear for us that goals could not be easily quantified without contextual information: (G1) Become the natural provider of Operational Technology (OT) support; (G2) Continue to streamline and improve services delivery; (G3) Improve customer experience through innovation; (G4) Enable Digital Transformation journey; (G5) Be a role model for digital transformation inside IT. We also received spreadsheets containing the initiatives elicited by the IT managers and the initiative´s deliverables. Meetings were scheduled to review goals in measurable terms, define OKRs, and review the initiatives.

3.2 Execution

We followed a plan of using practical work meetings, lasting between 1 and 2 h each. The IT director and the five IT managers participated in all meetings. We started by analyzing the IT goals under the perspective of OKR in order to verify if they were meaningful, significant, concrete, actionable, and inspirational. The main problem we found was that the IT goals were defined using qualitative terms (e.g., natural provider), without a rationale to explain them. This makes it difficult to measure goals achievement. We needed the information to express the goal in measurable terms. Thus, we followed practices from the Develop stage – Phase 1 of the GQM+Strategies process, which says that rationale, context factors, and assumptions characterize the environment and help define and understand goals. We asked questions to brainstorm discussion and get information to define aspects that could bring a basis to measure the achievement of qualitative objectives (e.g., aspects to explain and quantify what means to be a natural provider for the organization).

Once the aspects to be measured were identified, we used practices from the Develop stage – Phase 2 of the GQM+Strategies process to, first, define key results (KRs) and, then, elicit strategies to achieve them. To define KRs, we considered OKR guidelines (KRs should be quantitative, time-bound, verifiable, and realistic). When discussing being verifiable, the IT director and managers quickly defined how to collect data, as a brief measurement plan that would be further detailed. For sake of confidentiality, we used X and Y to represent current and target values, respectively, and omitted the time to achieve the result (Table 1).

Table 1. OKRs for “Become the natural provider of Operational Technology (OT) support”
Full size table

Once we had measurable goals expressed by OKRs, next step was to review elicited initiatives to verify and prioritize the ones aligned to OKRs. Due to the high number of initiatives and for the sake of confidentiality, we discuss only some of them.

First, we verified alignment between initiatives and OKRs by analyzing if the initiative deliverable could contribute to the achievement of the OKR. We also questioned the high number of initiatives (IT managers listed 140 initiatives) and their connection with the IT goals. Thus, we selected only the initiatives truly aligned with OKRs. For example, by analyzing the initiatives, it was noted that the initiative “Elaborating Software as a Services Contract Guidelines,” which includes benchmarking studies, architecture guidelines and contracts review, was not aligned to any OKR, since it was not able to produce deliverables that contribute to achieving the OKRs.

We also verified the need for new initiatives. For example, there were only two initiatives related to the goal “Become the natural provider of Operational Technology (OT) support,” namely: “Include scope for supporting OT users in outsourcing contract” and “Implementation of network standards to improve the security posture for OT sites.” When we defined measurable goals and created OKRs, the participants realized that those initiatives were not enough, and new initiatives should be created to achieve the OKRs. The OKR goal-setting protocol does not provide any mechanism to elicit initiatives to achieve KRs. Thus, we followed an approach based on GQM+Strategies [10] to fulfill this gap. This proposal suggests that in order to elicit effective initiatives, processes related to the goals to be achieved should be analyzed. Hence, we qualitatively analyzed processes associated with each KR and investigated root-causes of problems related to these processes that impact KRs achievement. As a result, new initiatives were created to support goals achievement.

For example, we analyzed the process performed to provide the required infrastructure foundation for a new location, and we found out that the main obstacles related to the OKR “Increase the number of OT locations with foundation implemented from X to Y” are related to network and support. So, we defined two initiatives: Implement network standards and Extend the outsourcing contract to support OT users. After reviewing the initiatives to achieve each OKR, we consolidated OKRs and respective initiatives in a GQM+Strategies grid [10] to visualize results and analyze conflicts.

4 Results

The OKR provides a simple way to define and track goals and measurable results, including agile principles to help define and monitor objectives and key results, and some ideas about techniques to use during meetings to define OKRs (e.g., Design Thinking) [7]. However, it does not provide a process to guide establishing quantitative key results for qualitative objectives. Moreover, OKR does not clearly address the initiatives to be executed in order to achieve the KRs. GQM+Strategies describes a process, including a Develop stage, which can be helpful when defining quantitative KRs. Besides, GQM+Strategies gives directions on eliciting strategies to achieve goals and, once again, can be helpful to define initiatives to achieve OKRs.

OKR and GQM+Strategies have some similarities (e.g., both are concerned with defining measurable goals) and also differences (as discussed previously). In this study, we combined both practices and, together with an IT director and five IT managers, we could define OKRs and initiatives to achieve them. Figure 1 illustrates the process that arose from the study, and we briefly explain it next. Although the process is linearly presented, there can be interaction between the phases.

Fig. 1.
figure 1

Process to support defining and monitoring OKRs and strategies to achieve them.

Full size image

Stage 1: What do we want? – Define objectives aligned to business (or review if they exist) being meaningful, significant, concrete, actionable, and inspirational [8].

Stage 2: What is behind this objective? – Identify measurable aspects to provide a basis to turn qualitative objectives into measurable objectives. Explore abstract terms like adjectives to understand what they mean for the organization by asking questions such as “why,” “how,” “for what.” Gather rationale, context factors and assumptions [2] behind each goal and find measurable aspects to be measured in key results [8].

Stage 3: Where do we want to go? – Create KRs for each objective using the measurable aspects as basis. KRs should be quantitative, usually time-bound, verifiable and realistic [8]. When building KRs current values for each measurable aspect are used to establish baseline (where we are today) and challenging (where do we want to go) but also realistic (where can we go) values defined as targets [8].

Stage 4: How are we going to get there? – Elicit strategies (i.e., projects, actions or other initiatives) to achieve KRs [2]. Includes reviewing existent strategies to verify if their deliverables contribute to OKRs achievement. Process analysis, involving root-cause analysis and Pareto techniques, can be used to find obstacles to be addressed in the strategies, and that can help prioritize them [10].

Stage 5: Let´s consolidate? – Elaborate a grid with OKRs and respective strategies to detect and remove any conflicts that can prevent an OKR from being achieved. Adjust the grid, if needed, and communicate to stakeholders. OKRs should be public [8] but many times, strategies may not be. Define a monitoring process instrumentation (e.g., emails, reports) and frequency in short cycles [8] to review OKRs results.

Stage 6: Are we getting there? – This stage is cyclic, as monitoring repeat following the frequency defined by organization. OKRs results, projects’ deliverables, business contextual information behind goals should be regularly monitored, preferably on a short period [8]. Consolidate information, align with teams, communicate OKRs and results to all organization, review what changed and create new OKRs if needed.

5 Discussion

The results of the participative case study have initial findings to show it is possible to use GQM+Strategies and OKR together to support creating measurable goals, OKRs, and initiatives for IT goals. When asking the IT director for feedback, he said “we were stuck before your help starting with questions to demystify some terms used in goals. From there, creating measurable goals was very practical and useful to clarify meaning and make explicit how to measure it.” He mentioned the approach was agile enough to provide expected results and clear enough to make the information explicit to the team.

IT goals were originally defined in a non-quantitative way, which was hard for IT managers to think about measurable attributes for them and select, from all initiatives, which ones could really deliver what was needed to achieve the goals. The culture of creating measurable goals needs to be spread through all the organization. OKRs can help with simple and actionable goals, constant monitoring, and agile changing for new OKRs when needed. By evaluating the deliverables of each initiative, we found only a few of them were truly strategic. The use of OKR and GQM+Strategies helped to make clear the alignment between initiatives and OKRs, providing a link between the actions performed by the teams and the goals the IT area wants to achieve. OKR literature suggests when OKRs are transparent, teams are senior enough to take ownership and get the work done [9]. During this study, we found a different scenario. Even for senior professionals, details about what have to be done to achieve the KRs were necessary. GQM+Strategies helped to satisfy this need. Aiming to make it easier to visualize the resulting OKRs, we built a grid. The grid was inspired by the GQM+Strategies grid proposed in [10]. Besides providing an overview of the defined OKRs and initiatives, it allows finding conflicts between them, as a monitoring and communication tool.

The process we followed to define OKRs can inspire other organizations on how measuring goals. Managers responsible for defining IT measurement processes can use information about how we defined OKRs, how we reviewed initiatives to guarantee alignment, then minimize difficulties during the definition of goals and initiatives and reduce the risk of failing in goals achievement. Furthermore, the study results can also be useful for researchers to identify practical issues to be addressed in future researches.

Regarding this study limitations, one of the biggest threats in this context is the ability to generalize from the case-specific findings to different cases [12]. Thus, the main threat to external validity in this study is about results’ generalization. In case-based research, after getting results from specific case studies, generalization can be established for similar cases. Participative case study is biased [11] and subjective as its results rely on the researchers. The first author of this paper primarily conducted the study collaborating with the practitioners. She has been working at the organization for eight years. Thus, she does not provide an external view of the situation. To reduce this threat, we involved other researchers as a steering group in discussing and reflecting on the study and results. Besides, the first author had previous experience with GQM+Strategies, which may have influenced its use along with the study.

6 Conclusions and Future Work

In this paper, we reported a preliminary experience of using GQM+Strategies and OKR practices together to define measurable goals, OKRs, and initiatives for IT goals. GQM+Strategies and OKR worked in a complimentary way, where OKR provided basic concepts, simplicity, and agility to the process, while GQM+Strategies provided useful knowledge to perform activities and define initiatives. We used an informal language to avoid communication barriers between academy and industry members.

As a result of this initial study, we created a first version of a process with six stages to define OKRs and initiatives to achieve them. We used provocative questions as What is behind this goal? to guide a brainstorm between practitioners and help them define measurable attributes for goals; Where do we want to go?, to incentivize practitioners to think about targets; How are going to get there?, to review if existent initiatives were able to achieve key results and elicit new ones; Let´s consolidate?, to group OKRs and initiatives; and Are we getting there?, to monitor results and check if goals are achieved by the elicited initiatives. This paper points out a direction for further studies to evaluate whether the proposed process could help other software organizations.

The process and knowledge provided from using OKR and GQM+Strategies practices together can be useful for practitioners to reuse or adapt the process, as well as to be inspired by our experience to define their own OKRs and initiatives. Researchers, in turn, can identify practical issues to be addressed in future research (e.g., the knowledge gaps in OKR). We did not find any work reporting the use of OKR in the IT domain combining OKR and GQM+Strategies. As future works, we intend to perform new studies applying the created process to get new data about its use and improve it.