1 Introduction

Previously, business operations in large companies were often supported by a number of isolated software systems performing diverse and specific tasks: from real-time process control to administrative functions. Today, many companies possess a highly complex enterprise-wide IT system; a large organization may employ hundreds of interconnected systems. The size of each single system varies extensively from enterprise resource planning systems to custom-made niche products, while the system interconnections remain numerous and heterogeneous. Typically, these emerging enterprise IT systems have not evolved through a planned approach. Rather, each business unit has independently developed and acquired the IT systems needed (Linthicum 2000; Johnson 2002). As a result, the enterprise-wide IT system is composed of a considerable number of poorly understood components, often storing redundant data and implementing similar functionality. The systems tend to interact by means of equally diverse and confusing connectors, and the applications are deployed on a wide variety of platforms while utilizing many different technologies (Linthicum 2000). Furthermore, the need for management of IT in contemporary organizations is driven by frequent changes in the general business environment such as mergers, acquisitions, strategic alliances, global partnerships, or dramatic economic changes and pressures (Reich 2003). Governing enterprise IT through this ever-changing environment is a challenging task for IT management and requires many decisions to be made in a timely and correctly manner.

The IT management function of an enterprise consists of one or more roles with overarching responsibility for decisions related to the enterprise IT. IT management responsibility includes areas such as IT-business alignment, IT investment decisions, and IT system quality assessment and improvement (Gottschalk and Taylor 2000; Cassidy 1998; The Open Group 2002). In the literature, IT management is sometimes instantiated by the CIO as the head of the IT department (Gottschalk and Taylor 2000; Enns and Huff 1999).

In the information systems literature, a number of models for the responsibilities of IT management have been published. These include Nolan’s Stage Theory (Renken 2004), the IT Management Strategic Grid (Applegate 2003) and Balanced Scorecards (Kaplan and Norton 1996; Van Grembergen 2004).

One approach suggested as an aid for the IT management’s decision-making process is Enterprise architecture: EA. Enterprise architecture is based on the architectural models of the enterprise information system and its context. During the last decade, EA has developed into an established approach for the holistic management of the IT in an organization. A number of Enterprise architecture frameworks have been proposed, including the Department of Defense Architecture Framework (DoDAF), The Open Group Architecture Framework (TOGAF), the Zachman Framework (Department of Defense 2004; The Open Group 2002; Zachman 1987). Enterprise architecture is model-based, in that the diagrammatic descriptions of the systems and their environment constitute the core of the approach. Referring to the domains featured in the Zachman Framework (Zachman 1987), the content of EA models include: scope and goals, business model, systems, technology, component configuration, and functions. It is clear that EA spans several areas including both IT and the business organization. However, not all of them are relevant for IT management. Given that EA is a broad discipline and that IT management uses EA models to make IT-related decisions, it is the belief of the authors that a framework which further delimits the EA scope to better suit IT management decision-making is needed.

The purpose of this paper is to present an assessment framework for scooping and determination of questions for IT management decision-making. It is an assessment framework in the sense that it guides IT management in helping determine the limits of the questions for the evaluation of EA and EA scenarios, c.f. Sect. 2.1 below. The identification of relevant questions is important since historically many EA initiatives have failed because current EA frameworks suggest the creation of models that are too ambitious (Lindström 2006). It is simply not possible to maintain a model containing detailed information on every single part of an enterprise. The plethora of information within the models hinders the extraction of relevant decision support. In order to model only those artifacts relevant for IT management decision-making, it must be clarified which questions the model should answer.

2 The assessment framework

The proposed framework consists of three top dimensions that represent the areas controllable by IT management, namely the IT organization, the IT systems, and their connection to the Business organization, c.f. Fig. 1. The separation of IT and business is a common distinction within Enterprise architecture. The renowned Zachman framework (Zachman 1987) separates between the Business model and, e.g. the System model and the Technology model. The TOGAF (The Open Group 2002) distinguishes the Business architecture from architectural concerns for Information systems and Technology. The separation of IT and business can also be found in the field of IT and Business alignment. For example Luftman (1996) bases his research upon this separation and in particular the problem of how to make IT deliver value to the business. A further distinction is made in the proposed framework by separating IT into the IT systems themselves, and the organization managing and maintaining the systems. These top domains are named IT systems and IT organization correspondingly. The IT organization generates business value to the business organization by ensuring that the IT systems operate correctly, and is responsible for long-term planning of the evolution of all IT within the enterprise. The IT organization can also generate business value direct to the business organization by training of users, support, helpdesk, providing documentation, etc. The IT systems generate business value when the systems are used in the Business organization.

Fig. 1
figure 1

The top dimensions of the assessment framework

Full size image

2.1 Using the assessment framework for Enterprise Architecture evaluation

The framework can be used in several ways:

  1. 1.

    To measure on the enterprise architecture as such. By further breaking down the dimensions of the IT systems and IT organization into specific questions, it is possible to measure the status for each dimension by collecting answers to the questions. As an example, information security for IT systems can be assessed via studies of the number of correctly configured firewalls, or virus protection update frequency (Johansson and Johnson 2005). Correspondingly, the IT organization may be assessed as the aggregated quality of each IT process. Such questions can be found in numerous sources (Weill and Ross 2004; ITGI 2005; OGC 2002).

  2. 2.

    To evaluate EA scenarios. An EA scenario is a model with a configuration of systems, applications, processes, etc. It describes a current or future state of IT within an entire organization or a subset of it and is used to represent decision alternatives. In order to evaluate EA scenarios, the IT system and IT organization dimensions are broken down into detailed questions. The assessments are done by collecting and aggregating the answers to the questions. Depending on business requirements, questions can be prioritized. For example, if a merge with another market actor is on the agenda, questions reflecting the IT systems’ modifiability might be allotted a high weight in order to simplify future system integration.

  3. 3.

    To demonstrate business value. By linking the dimensions of the IT systems or IT organization to the business organization, traceability of business value delivery can be achieved. Given that a systematic decomposition is done, it can later be shown how each of the dimensions is related to the business value. This systematical decomposition is done in order to demonstrate business value of different EA scenarios.

2.2 Outline

The three top dimensions in Fig. 1 are described in Sects. 4, 5, and 6, including a breakdown into dimensions, which in turn are broken down into sub dimensions that can be further detailed into specific questions. Section 3 presents the theoretical foundation and the literature base used to detail the three top dimensions. The breakdown of the IT organization is presented in Sect. 4, the IT systems in Sect. 5, and the Business organization in Sect. 6. The case study presented in Sect. 7 is an example of how the IT system dimensions are further broken down into specific, concrete questions in order to evaluate EA scenarios and assess the scenarios’ business value.

3 Theoretical foundations

In order to create the assessment framework, c.f. Fig. 2, extensive literature studies were carried out. This section presents the literature used when developing the framework. For all three top dimensions of the framework, systematic search queries were used on multiple databases to find the most relevant sources of information, and to ensure the completeness of findings. To assure the framework’s validity, the selected sources consist of highly cited academic articles.

Fig. 2
figure 2

The enterprise IT is divided into IT systems and IT governance. Both are providing value to the Business organization, express in the Business value dimensions

Full size image

The scope of the IT organization dimension is to explain how IT-related decisions are made, by whom, and about what. Since the field of IT governance aims at responding to those questions, a literature search on IT governance constitute the theoretical foundation for the IT organization dimension (Weill and Ross 2004; Roepke et al. 2004; ITGI 2003; Weiss and Andersen 2004). Results from the literature study have been published previously, c.f. (Simonsson and Johnson 2006; Simonsson and Ekstedt 2006), and a complete list of the studied sources can be found in (Simonsson 2005). Therefore, the methodology employed will only be briefly addressed in this paper. In short, 60 different articles from forums such as the MIS Quarterly, Information Systems Control Journal, Information Systems Research, International Journal of Information Management, International Journal of Accounting Information Systems, and the Hawaii International Conference on System Sciences were chosen, e.g. (Roepke et al. 2004; Hamaker and Hutton 2004; Sambamurthy and Zmud 2000; Trites 2004; Ridley 2004). Within these 60 sources, about 150 statements implicitly or explicitly defining IT governance, or indicating the relation between IT governance and its underlying constituents were selected and analytically classified to create dimensions. The results of the analysis can be found in Sect. 4.

In order to find the dimensions of the IT systems, 105 quality attributes were collected from over 20 sources. Section 5 describes the quality attributes that are used in the model. Thorough investigations throughout this well delimited field of IT system representation have already been carried out by other authors. Sources are well established, and explicitly state exhaustive lists of quality attributes that can be used to describe IT systems, see e.g. (Bass 1999). This source and others found within the field of software architecture can be divided in glossaries and compilations of attributes, e.g. (IEEE 1990; Kazman 1994; SEI 2004; Barbacci 1995; Johnson 2002), references focusing on a few attributes, e.g. (Bass 1999; Kazman 1994) and references on a specific attribute, e.g. (Brackett 1994; US Government 2002). The remaining sources belong to enterprise architecture and strategic information system planning, e.g. (The Open Group 2002; Spewak 1992). In order to reduce redundancy among the identified attributes, similar attributes were clustered and grouped together. The list was narrowed further by excluding attributes that are only of interest in a system development situation and thereby of little interest for IT management. As an example, code understandability and testability were excluded. These attributes were considered too highly detailed and thereby not explicitly relevant on an IT management level. The 105 attributes were thereby decreased to the nine, c.f. Fig. 2. The methodology is detailed in (Lindström et al. 2005).

Given that the perspective of the framework is taken from IT management’s decision-making stance in relation to IT management responsibilities, it is not relevant to include the business organization per se in the framework since it is not controllable by IT management. Nonetheless, it is relevant to include the way IT management decisions influence the business organization. As stated in Sect. 2, the concept of IT systems and the IT organization as providers of value to the business organization is used in the framework. In order to find the different dimensions in which IT delivers value, literature on the benefits and business value of IT were extracted from the fields of enterprise architecture, systems engineering, business administration, economics, organizational studies, and behavioral sciences. In total, several hundred papers, articles and books were screened and more than 625 individual benefits and examples of business value were colleted from over 80 sources, see e.g. (Love et al. 2004; Mirani and Lederer 1994; Hares and Royale 1994; Kearns and Lederer 2004; Bannister and Remenyi 2000; Farbey et al. 1993) The complete set of references can be found in (Gammelgård 2006). The selected articles came from publications such as International Journal of Information Management, Journal of Information Technology, Journal of Strategic Information Systems, Information & Management, Strategic Management Journal, Information & Management, Decision Support Systems, Communications of the ACM, International Journal of Accounting Information Systems, and proceedings of the Hawaii International Conference on System Sciences. The individual benefits and examples of business values were then grouped into 25 sub dimensions that were subsequently aggregated into three main dimensions as presented in Sect. 6. The grouping was performed so that the dimensions should cover all aspects of business value contribution from IT, as well as being of a comparable level of abstraction. The result of the literature study, including the detailed Business value dimensions, have been published previously, c.f. (Gammelgård et al. 2006a).

4 The IT organization

The presentation of the framework starts by describing the IT organization, whose main purpose is to ensure the quality of services delivered to the business, either directly or through the IT systems that the IT organization manages. These services include, e.g. functionality, support, and automation of processes. Several existing frameworks provide guidance in delivering these services (ITGI 2005; OGC 2002). On a higher level, the quality of these services relies on the quality of the decisions made by IT management. The IT organization, in regard to the usage of the assessment framework, focuses on decision making. This is also the case in academic discipline of IT governance (Hendersen and Venkatraman 1993; Loh and Venkatraman 1992), wherefore literature support was taken from that field. In previous studies (Simonsson 2005, 2006a, b), the concept of IT governance was explored in order to clarify who should make the decisions, what the decisions should consider, and how the decisions should be carried out and followed-up. In Simonsson’s studies, suitable dimensions representing the IT organization were determined from a broad range of literature sources, see e.g. (Weill and Ross 2004; Roepke et al. 2004; Hamaker and Hutton 2004; Peterson 2001; Ridley 2004). In the aforementioned studies by Simonsson, a description of IT governance as the preparation for, the making of, and the implementation of IT-related decisions regarding goals, processes, people and technology on a tactical or strategic level was used. IT governance, and therefore also the IT organization, is represented as three dimensions, namely Scope, Process and Domain.

The Scope dimension is employed to differentiate between the low level, detailed, rapidly carried out, IT-focused Tactic decisions on the one hand, and top management, low granularity, business oriented Strategic decisions on the other hand.

Three phases of the Decision making process are highlighted. In short, this dimension deals with the relation between the real world IT inferno, and the models used for decision making. Before making any decision, facts have to be collected and transformed into an abstract model. This phase is named Understanding. Then, the actual decision can be made by the right individuals, according to corporate architectural principles, etc. The decision is based on the content of the model of the reality, and is represented by the Decide phase. Planning how to make the decision is also included in this phase. Finally, a decision is not worth much unless its implementation is assured and Monitored. This is typically accomplished by applying control objects on the organization. The decision-makers then compare the state of the reality with the should-be values obtained from the models. Finally, the Domain dimension denotes what the decisions should consider. Goals include strategic decisions, development and refinement of IT policies and guidelines, and control objectives. Processes include the implementation and management of all IT processes such as acquisition, service level management, and incident management. People include the relational architecture, and the roles and responsibilities of different actors within the organization. Technology represents e.g. hardware, applications and infrastructure.

Control Objectives for Information and related Technology (COBIT) lists a number of so-called Critical Success Factors (CSF), and the IT organization dimensions can be used to classify them. For instance, the CSF “All assumptions of the IT strategic plan have been challenged and tested” can be categorized as Strategic-Goal-Monitor in the dimensions of the IT organization (ITGI 2005), c.f. Fig. 3. The above CSF is taken from the COBIT process PO1: Define a strategic IT plan. Another CSF statement, “A formal process for hand-over from development to operations is defined” can be categorized as Strategy-Process-Monitor. The latter example is taken from the process AI6: Manage Changes.

Fig. 3
figure 3

An example of the use of the IT organization dimensions. Critical Success Factors (CSF) from COBIT are categorized (ITGI 2005 )

Full size image

5 The IT systems

The IT systems refer to the infrastructure, applications, and information within the enterprise. The IT systems can be categorized in several ways. One approach is to use the description of what they consist of, such as technology, applications, and information. However, even though this is a fairly common division within Enterprise architecture (The Open Group 2002; Armour and Kaisler 2001), it is not applicable in this context where focus is on IT’s accumulated contribution to the business organization. An example of this is modifiability where infrastructure, applications and organization are important aspects to consider upon obtaining a modifiable system. The IT systems can also be divided by their functionality. The problem with this approach is of course that it will not be possible to make the dimensions general; every system has its unique set of functions and to list all possible of them in a generic framework would be an overwhelming and impossible task. Instead, quality attributes are used to describe the capabilities of the IT system (Bass 1999; IEEE 1990; Kazman and Bass 1994; SEI 2004). The fit between the functional requirements, and what the IT systems actually provide has been made a quality attribute of its own; Functional fit. The literature review resulted in the following list of quality attributes for IT systems:

Availability and Reliability. Availability is the time that systems and components are operational and accessible when required for use. Reliability is the ability of systems and components to perform their required functions under stated conditions for a specified period of time.

Data quality. The data employed by the business organization must be of good quality, i.e. data should be accurate, complete, appropriate, available, etc. However, when deploying a data quality program, aspects like tools for enhancing data quality, processes and activities for gaining data quality, organizational roles and responsibilities, and rules for designing information systems should be taken into consideration.

Functional fit. The goal is to minimize the gap between the functions of the IT system, and the business requirements, and to consolidate the application and infrastructure portfolios. In order to find the gaps and the potential consolidations a functional map can be drawn. Such map lists the functions needed in order to execute the business principles and compares it with the actual functions found in the applications.

Information security. The issues of protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction in order to provide confidentiality, integrity, and availability are considered here. The scope of information security can be divided in technical systems, products, documentation, routines, and surrounding factors like burglar alarms or entry controls. These can serve as preventive measures such as firewalls, detective measures like a burglar alarm, or responsive measures such as incident management.

Interoperability. Interoperability is the ability to integrate systems and components so that they can exchange information and use the information that has been exchanged in a proper way. As with modifiability below, the organizational issues play an important role. Other aspects to consider are software architecture, the use of integration products and standards, use of models that describes how the systems should interoperate and modeling and documentation tools.

Modifiability. Modifiability is the ability of systems and components to facilitate the incorporation of changes, for example due to changed functionality in other applications or environments, or growing/shrinking the system. In order to measure modifiability or make systems modifiable, many aspects must be taken into consideration. These include organizational quality such as project planning, user acceptance of the change and the competence of the developers, system quality (for instance if standard platforms are used), source code complexity and degree of commenting. The existence of well defined and well executed processes for change management, testing and documentation quality and updating also affect the modifiability of an IT system.

Performance. Performance can be assessed as the time needed for an application to respond to input, or the amount of resources used in order to perform its functions.

Safety. The IT systems should not perform any actions that cause death, injury, occupational illness, damage/loss of equipment or property, or damage to the environment.

Usability and user productivity. An IT system that is easy to use will affect the user productivity positively, including his or her ability to operate, prepare inputs for, and interpret outputs from a system. Usability can be assessed by user satisfaction and performance, the design of the information system and its interface, and to what extent the development process of the system considers usability and user productivity.

6 The business organization

The Business organization top dimension of the framework is seen as the interface to the rest of the business organization for IT management. IT management employs the IT organization to manage the IT systems, and both are providing value to the Business organization. This implies that the Business organization top dimension should only represent the business values from IT that it is reasonable to assume that IT management can influence. Based on this assumption, the scope of the literature has been narrowed. Effects of IT on the (macro) economic level, such as increased productivity in an industry as a result of IT, have been left out.

From the literature, the following three main dimensions of the Business organization, or Business value dimensions as they also are referred to, have been identified. Please notice that within each dimension, several sub dimensions are found. These are only exemplified in the descriptions below, along with examples of concrete business benefits in each sub dimension. The business value sub dimensions along with a more detailed description of how they have been identified from the literature are found in (Gammelgård et al. 2006a; Gammelgård 2006).

Inputs and outputs of the organization. In this dimension, the business is viewed as a black box where products and services from suppliers go into the business. The outputs from the black box are products or services the business produces and delivers to customers. These products/services are defined by their quality and diversity. Examples of sub dimensions are improved Supplier relations and improved Customer relations. Other sub dimensions are improvements in Deliveries of products or services, e.g. new ways to distribute products, and Differentiation in products/services such as adaptations of existing products to other parts of the market.

Organizational resources. The business value related to the resources of a business is divided into two general types: human related resources and non-human resources. The human resources are: business value related to improved Decision making, Learning and knowledge, and Organizational culture. While the two first sub dimensions relate to individuals in the organization, the latter concerns business value associated with how humans act in business organization. Under the non-human sub dimensions, the separation is made between business value associated to the improvement of Information, e.g. availability or accuracy of information, and value related to other Technology and tools in the organization. This latter is, e.g. improvement in a factory’s production technology through IT.

Organizational structure. This is a relatively large group of sub dimensions concerning business value from IT in relation to how the business organization is structured. It is related to the departments and processes of the organization and how these organizational components are interlinked. IT can deliver value to the structure as such, e.g. in terms of increased Flexibility. IT can also deliver value in relation to the characteristics of the individual components, e.g. making processes more Efficient. IT may also impact how the different parts of the organization are interlinked, e.g. improvements in Communication or better Control and follow-up.

7 EA scenario evaluation case study

Although the focus of the case study presented in this section is on evaluation of EA scenarios (application area b in Sect. 2.1), it is illustrated how the IT systems dimensions are broken down into concrete questions/measure that can be used to perform measures (application area a). Here, the questions were used to make an aggregated evaluation of the EA scenarios in the case study. The questions can of course be used to measure the individual dimensions as well. In this section, it is also presented how IT management can demonstrate business value (application area c) by relating the IT systems’ quality attributes to the dimensions of the Business organization. In the case study, this is done in order to assess the business value of the EA scenarios.

The presented case encompasses two parts of the assessment framework: the IT systems and the Business organization. As discussed in section two, both the IT organization and the IT systems serve as value enablers and service providers to the Business organization. The principles for IT organization value delivery and IT system value delivery are analogous. The dimensions of the IT organizations are broken down into concrete questions in the same way as demonstrated for IT system dimensions in this section. Given the limited space available, a reasonable limitation for a detailed case study is therefore to present only the IT systems when illustrating how the framework can be used for EA scenario evaluation.

7.1 Background on the case study

The case study was conducted at a large European power company. Due to a number of recent mergers and acquisitions, the power company’s system portfolio related to the asset management process, had become very heterogeneous. Some 70 different IT systems were distributed unevenly within 16 business units. Given the vast amount of systems, the company felt a need to investigate the possibilities of system portfolio consolidation. In this case, three different EA scenarios for the asset management process in the company’s electricity distribution unit were evaluated.

7.2 Breaking down the IT system quality attributes to questions

Before breaking down the general IT system quality attributes to concrete questions, an important difference between the quality attributes must be addressed. In total, there are nine quality attributes as presented in Fig. 2. The Functional fit quality attribute relates to the kind of business that the IT systems should support. The larger the conformity between the functionality of the IT system and the activities of the business, the higher the score would be with respect to Functional fit of the systems. So, what functions do the particular business consider relevant to have IT support for? In order to answer this question, a functional reference model was developed. This is described in further detail in the next subsection. The eight remaining quality attributes, also referred to as non-functional quality attributes, do not vary with business domains. The difficulty here lies in the fact that they are complex topics in themselves. In order for the breakdown to be relevant, the eight non-functional quality attributes have to be reduced into unambiguous definitions that reflect the state of the art for each dimension. The second subsection details the breakdown of non-functional quality attributes.

7.2.1 Functional fit: the functional quality attribute

To derive the functional reference model, i.e. a suitable breakdown of the Functional fit quality attribute, a refinement of a standard functional reference model was done. In this case, the functional reference model was based on the industry standard IEC 61968-1 (IEC 2003). This standard was developed to facilitate integration of Distribution Management Systems of which asset management systems can be seen as a subset. In the case, the IEC 61968-1 was adapted to the current business and complemented with information from system vendors as described in (Gammelgård et al. 2006b). The functional reference model was refined and adapted to the situation at hand by interviewing business area experts within the company. In total, 26 persons were interviewed in this part of the study. The result was a hierarchically designed functional reference model with 13 groups of functions which were broken down to 85 individual functions. For each function, a number of concrete questions were derived. Table 1 below is an example of this breakdown. A more detailed description of the model is found in (Gammelgård et al. 2006b).

Table 1 Example of the breakdown of the functional fit quality attribute
Full size table

7.2.2 Non-functional quality attributes

The eight non-functional quality attributes are in themselves complex. Most of them are not only quite extensive and feature many aspects, but also ambiguous: their definition depending on the consulted source. In order to break down the attributes to a level allowing for real-world assessments, a literature review was performed and a consistent evaluation framework was created. This literature review consisted of evaluation of between 5 and 50 renowned sources per quality attribute (Gammelgård et al. 2007a). The literature review was conducted partly within this case study and partly within separate projects, e.g. Information security (Johansson and Johnson 2005). Table 2 below exemplifies this breakdown for the non-functional quality attribute Interoperability.

Table 2 The breakdown of the non-functional quality attribute Interoperability
Full size table

7.3 Using the breakdown for the evaluation of EA scenarios

Once the quality attributes were detailed, a set of EA scenarios describing possible system setups were created. The (alternative) scenarios for asset management systems support at the power company’s distribution unit were summarized according to the list below.

EA Scenario 1. The present situation is based on a best-of-breed asset management system: System A combined with different support systems.

EA Scenario 2. A scenario in which an upgraded version of System A is introduced and a number of the supporting systems are abolished.

EA Scenario 3. The present asset management system is replaced by an asset management module: System B. System B originates from the same suite as the ERP system presently used within the company.

The broken down quality attributes presented in the Sect. 7.2 were utilized in order to assess the Functional fit and the non-functional quality attributes of each scenario. The assessments were mostly made through interviews with questions as exemplified in Tables 1 and 2. The data collection was performed by means of 27 interviews, lasting altogether approximately 44 h. About 30 h were spent on the functional attributes and 14 on the non-functional. The respondents were system experts and users mainly working within the power company. A more elaborate description of the data collection method, as well as the full range of case study results are found in (Gammelgård et al. 2007a; Johnson 2006).

The three EA scenarios demonstrated differences regarding Functional fit. EA scenarios 2 and 3 were slightly better than EA scenario 1. This is consistent with what was to be expected, since both EA scenarios 2 and 3 are future scenarios with emphasis on more modern asset management applications than the current EA scenario 1. Looking at the non-functional quality attributes, no major differences were identified between the scenarios. However, when looking at the individual IT systems in the scenarios, differences appeared. System A, a central component of both scenario 1 and 2, demonstrated higher values in the quality attribute Usability than system B.

The example shows that it is indeed possible to break down IT system attributes in order to evaluate EA scenarios. The EA scenario evaluations can be presented as comprehensive decision-support for IT management. See (Gammelgård et al. 2007a) for more detailed results of this part of the case study.

7.4 Linking IT system quality attributes to the business organization dimensions

This section demonstrates how the business value of EA scenarios can be assessed. The purpose here is not to be fully exhaustive on the process and the results, but rather to exemplify and demonstrate the applicability of the assessment framework for business value assessments of EA scenarios through outlining how it was done in the case study.

7.4.1 Relating IT systems quality attributes to the Business value dimensions

Detailing the Business value dimensions. One objective when defining the Business value dimensions described in Sect. 6 is to enable mapping between IT system quality attributes and the ways IT delivers business value. The three dimensions of the Business organization consist of altogether 25 different sub dimensions. As presented in Sect. 3, an extensive literature study was carried out to identify these sub dimensions. For example, for the organizational resources in Sect. 6, a separation is first made between human and non-human resources. These are then further separated into three sub dimensions related to human resources and two related to non-human resources. To illustrate the bottom level sub dimensions, the three dimensions related to human resources are presented: Decision making in terms of better decision support, more well-informed decisions, decisions taken closer to operations, shorter time to decisions, increased reliability in decisions, less uncertainty and complexity in decision making, etc. Learning and knowledge, i.e. improved learning and/or increased knowledge of persons in the organization, e.g. the promotion of competence development, and the retention of knowledge. Organizational culture, i.e. improved organizational culture, e.g. increased involvement/interest from management, higher job satisfaction and stress reduction. The process for identifying the business value sub dimensions and the full list of the 25 sub dimensions with explanations are found in (Gammelgård et al. 2006a). The sub dimensions are also listed in Table 3 below.

Table 3 Examples of how links between the Functional fit IT system quality attribute, and the business value dimensions were established
Full size table

Relating IT system quality attributes to the Business value dimensions. The linking between IT systems attributes and the 25 business value sub dimensions was done by interviewing respondents at the power company and asking them to express the value of the IT systems quality attributes in terms of the business value sub dimensions. In order to provide meaningful results, the breakdown of the IT system’s quality attributes and the detailing of the Business value dimensions were necessary first steps.

The respondents consisted of people working in the business processes, this case being the asset management process at the distribution unit. In total, 15 persons were interviewed. The respondents were not confronted with the actual EA scenarios, but were instead confronted with the breakdown of the quality attributes described above. One important reason for this that it makes it possible to add, remove or change EA scenarios without having to redo the whole study. As demonstrated further down, only the assessments of the IT systems quality attributes has to be updated then. The respondents were first asked to indicate on a five grade scale (0–4) how much value a particular part of the functional reference model (the breakdown of the functional fit quality attribute) generated in each of the 25 business value sub dimensions above, c.f. example in Table 3.

The non-functional quality attributes were treated in a somewhat different manner. For these attributes the respondents were not asked to directly relate each attribute (e.g. Usability) to the business value sub dimensions. Instead, the respondents were asked to identify each relation between a function in the functional reference model, and a business value sub dimension. On a scale (0–4), the respondents stated how much the relation was influenced by each of the eight non-functional quality attributes. It was assumed that business value stems from the functionality provided by a scenario, i.e. the ability to perform and assist certain activities. However, the amount of generated value depends on the non-functional quality attributes. A highly simplified example for an e-mail function illustrates the reasoning above: Having the ability to send e-mails generates business value because it makes it easier to communicate with colleagues (value in the business value sub dimension Communication), and makes it easier to distribute information to a large number of persons (value in the business value sub dimension Information). However, if the e-mail function has a non-intuitive user interface (low Usability) and is frequently out of order due to an instable server environment (low Availability) the business value is reduced. It is then more difficult to communicate using the e-mail function, and less information can be distributed when using the systems.

A more detailed description of the method, as well as the full case study results can be found in (Gammelgård et al. 2007b; Andersson 2006).

Results from relating IT system quality attributes to Business value dimensions. In Fig. 4 the results are exemplified for the relations between the breakdown of the Functional fit quality attribute and the business value sub dimensions. The figure indicates the strongest connections between parts of the functional reference model (the x-axis) and the business value sub dimensions (y-axis). For example, the function Asset investment planning (AIP) did provide value in terms of better Decision making whereas there was little value in terms of improvements in Efficiency. In the case study, the result for the non-functional quality attributes were that Usability and Data quality were considered the most important attributes in order to achieve the most business value. Modifiability was the least important. However, the relative differences between the attributes were relatively little. More detailed results are found in (Gammelgård et al. 2007b; Andersson 2006).

Fig. 4
figure 4

An example of relations between the broken-down Functional fit quality attribute, and the business value sub dimensions

Full size image

Demonstrating the business value of EA scenarios. To estimate the business value, the relations between IT system quality attributes and the Business value sub dimensions above are combined with the result of the assessment of IT system quality attributes for the EA scenario in Sect. 7.3. This is further described in (Gammelgård et al. 2007b). The principle behind establishing the business value is that if an EA scenario receives high scores for the IT system quality attributes, it will produce a higher business value compared to a scenario with low functional fit and low non-functional quality, i.e. low scores in the EA assessment in Sect. 7.3. However, some parts of Functional fit and some non-functional quality attributes are more important than other in order to generate business value, as illustrated in Fig. 4. Hence, a scenario should only be assigned a higher business value if it has high scores regarding the important (sub) dimensions. Consider the following example based on the results of the case study above: If a scenario has higher functional fit in the functions Work dispatch, Work scheduling, Switch action scheduling, and Trouble Call Management (cf. Fig. 4 above) compared to another scenario it results in a higher business.

7.5 Industrial feedback

IT management at the power company considered both the assessment framework applied in the case and the results helpful. Not only the aspect of pointing out differences between scenarios and assisting in the decision making was received positively, but also that the framework was facilitating communication between the business management and IT management. It was also appreciated the way the framework contributed with structure to the evaluation. The IT management pointed out that the assessment framework seamed to eliminate much of the gut-feeling normally involved in EA scenario evaluations.

8 Summary

The paper presents an assessment framework for scooping questions for IT management decision-making. This is achieved through guiding IT management in confining relevant questions for evaluation of EA and EA scenarios. The three top dimensions, IT organization, IT systems, and Business organization, are presented and detailed. The case in Sect. 7 illustrates how the assessment framework has been further broken down in order to assess EA scenarios both in technical terms, i.e. the IT systems quality attributes, but also to assess how the scenarios contribute to business value. Demonstrating how value is generated to the business is perhaps one of the main challenges for IT management.