1 Introduction

Corporations are demanding quantifiable proof that proposed technology will drive corporate benefit, and proof that current projects have actually had an actual impact. And yet, distinguishing between good and bad technology investments is often quite difficult. This is problematic, or at least it should be, because information technology is the central nerve system of most organizations, providing the tools to act rapidly to changes in the business environment.

For large Korean enterprises, the level of implementation and operation has already surpassed the average level whereby IS evaluation is directly connected to benefit. Recently, these large Korean enterprises have shown great interest in the quantification of IS investment, the so-called “IT ROI”, but they consider operation-centered issues such as IT cost management or IT service management (ITSM) as their starting point of interest.

On the other hand, advanced enterprises in the EU and US consider IS investment evaluation and benefit management as an essential factor of IS management. They consider it to be the tool that could be beneficial to the business as long as business and IS strategies are properly aligned. IS investment guarantees its benefits by transmitting its value and by improving the relationship between business strategy and IS strategy. Therefore, large Korean firms should not only acknowledge IS investment evaluation and benefit management, but also set up fundamental measures regarding IS benefit management (instead of only focusing on adopting new technology itself).

Efficiently and effectively managed IS investments that meet business and mission needs can create a new value-revenue generation, build important competitive advantages and barriers to entry, improve productivity and performance, and decrease costs. However, many executives question whether they are receiving full value from their IS spending and also whether this spending is being properly directed. In the 1980s, a series of studies found that despite the improvements made by technology, the correlation between how much a company spends on IS and the accompanying productivity generated as a result of IS investments is minimal. This conundrum of spending without visible results has now been tagged as the ‘computer paradox’. Moreover, Strassmann [49] defined these phenomena as the ‘IT/IS paradox’ in the 1990s and offered a negative opinion about productivity and performance improvement obtained from the application of information systems, as suggested further in other researches. Differences between existing and new processes, inappropriate application of a new system, inefficient processes, organizational changes, inaccurate information, and an inappropriate response of an IS department can generate a negative result with surplus investment.

The computer or IT/IS paradox has recently been examined in numerous studies including one by Dedrick et al. [16], who concluded that the productivity paradox as first formulated has been effectively refuted, saying that greater investment in IT is associated with greater productivity growth. Thus, a relationship between computer investments and any measure of productivity persists to this day. Many attempts have been conducted to resolve these IT/IS paradoxes and to enhance IS investment performance. The contingency theory, which deals with the application methodology of organizational effectiveness in a changing business environment from the late 1960s or early 1970s, was applied to the practical analysis to verify the effectiveness of IS performance in the 1990s. Among the researchers, Weill and Olson [58] especially suggested the application of an integrated model to the contingency theory in the MIS field; the model has served as a conceptual basis in numerous researches [34, 46]. However, these research models have some limitations as they simply listed the contingency variables; the studies therefore did not reflect the strategic alignment regarding the internal and external organization environment. In addition, in order to improve IS performance, a process oriented approach is required. From these perspectives, their researches do not handle the concept of a sequential approach regarding investment and management. Besides, IS performance success factors are unable to prove a mutual relationship in practice, which was an unsolved problem in the previous IS success model.

Even though there have been many attempts to implement IS investment performance measurements, distorted IS investment decisions will continue to be committed, negatively affecting the information system management department, unless a concrete examination on the cause of performance is established. Furthermore, this will only increase the work burden and decrease efficiency, reducing system quality and information to IS users, becoming the main cause of IS investment decline.

The remainder of this paper is organized as follows. The next section describes the literature review on strategic alignment, IS success, and IS investment. Section 3 presents our hypotheses, which focus on the mediation role of IS investment between strategic alignment and IS success. Sections 4 and 5 explain the research design and the results of the empirical survey. The final section presents a research summary and discussion, as well as implications for future research.

2 Theoretical development

2.1 Strategic alignment

Strategic Alignment was conceptualized so that IS could be managed in a way that mirrors the managerial directions of the business [45]. Other studies define it as the degree to which the mission, objectives (goals), and plans contained in the business strategy are shared and supported by the IS strategy [21, 41, 47]. Thus, we adhere to the conception of strategic alignment [6]: business and IS must work side by side.

The link between strategic alignment and IS business value has been studied, demonstrating that an absence of strategic alignment induces insufficient IS investments [19]. Also, strategic alignment has been found to positively affect organizational agility [51]. Hence, strategic alignment is not only crucial to organizational effectiveness and efficient resource utilization, but it must also be present before IS can be chosen and diffused to achieve maximum IS effectiveness [9].

As a measurement of the strategic alignment manner, this study intends to use the ‘level of alignment between various business and IS strategies’ method used by Sabherwal and Chan [44]. For example, when four business strategies and four IS strategies are matched and their relations are in agreement also.

2.2 IS success

Many organization executives wonder whether they obtain adequate value from IS investments and if the investments perform well in the right directions. Many researchers have endeavored to discover the correlation between IS investment and its productivity with the advancement of technology [4]. As such, researches on the evaluation of IS performance are very important in both business practices and academic studies, so as to manage IS successfully. However, because the role of IS extends from cost reduction or business support to strategic use, measuring IS effects can be a very difficult task for the board of directors and CIOs [58]. In order to evaluate IS performance more effectively, we should understand the definition of IS performance. Even though IS performance can be defined in diverse ways, it must be different from business performance, which has been dealt with in a number of studies. Business performance is the total management outcome of all management activities which a corporation executes, but management outcome is indirectly influenced by IS performance.

According to Norton, the difficulty of IS performance classification arises from the three way relationship between IS investment and corporate financial performance, which implies management outcome [36]. In other words, IS enhances the in-between value such as customer service, improves customer confidence, and finally, increases corporate sales. Thus, diverse measures have been suggested by several studies. This is because one can measure the performance differently at the various stages of information flow. That is, information system performance can be the output of an information system, or it can be the performance of the information system itself, or even the influence of the information (system) on a user or organization.

DeLone and McLean [14] provided an expanded view of IS success based on Shannon and Weaver’s three levels of information flow (e.g., technical level, semantic level, effectiveness or influence level). Saunder and Jones [46] executed Delphi studies aimed at determining the important dimensions and their measures for evaluating the functional performance of information systems. Their findings reflect an information system’s functional performance dimension rankings, measurement usage, and so forth. The most important measure dimension was ‘the impact of an information system on strategic direction,’ which seems to match Mason’s ‘influence on system’ stage. DeLone and McLean [14] also claim that IS succeeds if the process includes interdependent factors of determinants.

Their model synthesized a six-dimensional model of IS success based on a review of 180 published conceptual and empirical studies. The dimensions were system quality, information quality, use, user satisfaction, individual impact, and organizational impact. In their model, system quality and information quality affect both use and user satisfaction. The use and user satisfaction affect each other, but the direction of the affection has not yet been identified. Additionally, they jointly affect individual impact, which eventually evokes organizational impact.

The individual impact (personal productivity or decision-making) and organizational impact (organizational performance) are measured at the benefit level. However, in their updated model, they amalgamated individual and organizational impact into one dimension, referred to as net benefit, so as to broaden the impact of IS also on groups, industries and nations, depending on the context. In addition, service quality was added to the quality dimension [16]. In this way, they updated their original success model based on changes in the role and management of information systems.

2.3 IT investment

Although ‘IS investment’ has been dealt with by many researches regarding information systems, the definition of IS investment is ambiguous. This is because the perceptions of researchers differ both in terms of the elements and characteristics of IS investment, and further, IS investment can be classified as a capital, asset, or expenditure account by a corporate accounting policy. Among researchers, Weill and Olson [58] defined IS investment as all expenditures on hardware, software, human resources, communication, indirect cost, education, or maintaining IS costs. However, Bacon [3] defined IS investment as expenditures on information system development so as to acquire IS assets such as computer hardware, network infrastructures, and software, as well as to improve organizational information capability. Moreover, to expound IS investment as an asset, Earl [17] described IS investment as not differing with capital investment in other corporate areas, so it must be a capital account.

Academic research on IS investment tends to separate IS investment into expenditure and capital. However, this research tends to define IS investment objectives as an IS investment-concentrated system or it pursues the information system from an IS strategies direction, rather than trying to understand it as a separate entity.

2.3.1 Type of IS investment

First of all, in order to examine the relation between an IS investment objective (system) and business performance, which has to be closely related to business strategy, a method capable of systemically classifying IS investment is needed.

The founding research that classified IS investment types was written by Weill and Olson [58]. They sorted IS investment types into (1) threshold, (2) transactional, (3) informational, and (4) strategic. Weill presented three types of IS investment, excluding threshold, focusing on the basic ability of an information system rather than on analyzing its cost efficiency [55]. Additionally, in order to manage IS investment as a portfolio, Weill and Aral added infrastructure type to the existing types [56]. The strategic type is related to the long-term objective (goal) like competitive advantage, the informational aspect is related to the mid-term objective (goal) like improvement of the management decision making process. The transactional type is related to the short-term objective (goal) like human cost reduction or productivity improvement. And infrastructure is shared by IS service uses or applications.

Johnston and Carrico analyzed a number of corporations in 11 industries that had strategically applied IS in response to competitive pressure [26]. They classified IS investment in business strategy and the degree of IS system integration into (1) traditional, (2) evolving, and (3) integrated. Traditional strategy focuses on the improvement of management processes like accounting processes or decision making support rather than corporate strategy, concentrating on process integration or adjustment in internal perspectives. An evolving strategy concentrates IS investment on corporate or business strategy rather than on strategic potential. Integrated strategy completely integrates business strategy and IS strategic potential, applying IS more aggressively and positively. It concentrates IS investment on the development of new products or services, supplier or customer relation enhancement, innovation of organizational structure or business process, and the guarantee of industrial competitiveness.

Explaining the strategic values of IS through various case studies, Earl [17] classified it according to the purpose of IS investment: (1) competitive advantage, (2) productivity performance, (3) new way of managing, and (4) new businesses. This classification especially presents, from an IS infrastructure point of view, the different types related to system maintenance. During a 3-year research project analyzing 60 managers who had participated in IS investment in 34 British corporations, Hochstrasser [22] arranged the IS investment types of management purposes and business domains into 8 groups: (1) infrastructure projects, (2) cost replacement projects, (3) economy of scale projects, (4) economy of scope projects, (5) customer support projects, (6) quality support projects, (7) information sharing and manipulation projects, and (8) new technology projects. Mooney et al. [32], classified IS projects or investment types in terms of effectiveness of management and operation process into (1) automation, (2) information, and (3) transformation.

The aforementioned IS investment typologies as generalized by many researchers can be summarized into internal and external systems. Table 1 presents the IS investment types and characteristics.

Table 1 IS investment typologies and characteristics
Full size table

The internal perspective is related to process efficiency and production improvement on an organizational value chain. This implies that IS investment is concentrated within decision making, process adjustment and cooperation, document processing of organizational information, data analysis, and process standardization and simplification. The external perspective improves business efficiency and the effectiveness of external relations such as customers, suppliers, distribution channels. From this perspective, IS investment is concentrated within supply chain management and the informatization of distribution, offering a differentiated customer relationship management services, cost reduction of distribution and sales, business-to-business (B2B), and online sales/purchases.

Taking these two IS investment characteristics into account, this research will classify the purposes of IS investment, which correspond to the business of IS strategic types, into four groups: internal system investment, external system investment, both systems investment, and ambiguous system investment.

3 Research model and hypotheses

It is valuable to pursue strategic alignment because there is increasing interest in the creation of value from IS investment and business even though it sometimes seems impossible to reach a perfect level of strategic alignment, which occasionally requires somewhat complicated and multi-sided consideration [24, 57]. The study of the effect of strategic alignment between business and IS can be divided into IS investment and business-IS performances [50]. That is, one is the study concerning the effect of business-IS strategic alignment on IS investment and its direction, and another is the study of the effect of business-IS strategic alignment on business performance or IS performance.

Johnston and Carrico [26] claimed that the direction of IS investment can be setup according to the level of integration of IS and business strategy. Also, Earl [17], explaining the strategic value of IS after going through diverse case studies and previous research, argues that the focused target of business strategy and IS can be classified into four points according to the goal of the strategy: (1) strengthening competitiveness, (2) improving productiveness and internal performances, (3) developing business management methods, and (4) creating new business models. Cash et al. [7] also claimed that the goal of IS operation, contents of investment and method of administration could differ according to the strategic role of IS, and the scale of IS investment and its contents tend to be more strategic when the IS role is strategic.

As shown above, the direction of strategic alignment between business and IS can have a significant impact on IS investment. Henderson and Venkatraman [21] suggested conceptual fundamentals regarding business-IS strategic alignment, and since then many researchers have studied strategic alignment and its performance. Especially Chan et al. [10, 11] and Sabherwal and Chan [44] demonstrated from their studies that business-IS strategic alignment can affect the performance of business and/or IS. In addition, Tallon et al. [52] studied it from the executives’ perspective and claimed that the alignment between business strategy and the strategic intention of IS can be achieved in the form of IS performance; furthermore, it can make an appearance in the form of business performance.

Meanwhile, Weill [55] proved that strategic IS investment is achieved by a firm’s IS strategy based on focus on the market, competitiveness, and strategic environment. He also proved that the result of this can affect strategic performances such as acquiring competitiveness. This means that IS investment, as a matter of process, is directly affected by a firm’s strategies, and this IS investment is also directly affected to IS performance. That is, Weill’s [55] study suggests that IS investment plays an intermediary role between business strategies and IS benefits.

However, most previous studies implicitly have taken a comprehensive approach to the IS investment target in the relationship between performance and the strategic alignment of business and IS. The reason why the examination of strategic alignment and the IS investment goal is important is because performance is highly affected by the result of the interaction between strategic alignment and the goal of IS investment rather than from strategic alignment itself. Namely, this suggests that only when business strategy is aligned to IS strategy and only when the IS investment goal is aligned to this IS strategy, can IS effectiveness be maximized.

This study investigated IS investment as a mediator variable to discover whether higher strategic alignment accordance creates higher IS success. So, we categorized comprehensive strategic alignment into the following four types and tried to understand the relationship among them: (1) integration of business strategy and IS strategy, (2) fit of business strategy and organization structure, (3) integration of organization structure and IS architecture, and (4) fit of IS strategy and IS architecture.

3.1 Integration of business strategy and IS strategy

As seen above, initial business-IT alignment was considered as the degree of integration between the business plan and ISP (Information Strategic Plan) [40]. However, as the strategic role of IS expands, there is a clear trend whereby IS is treated as a separate strategy (goal).

Even though Chan et al. [10] ignored IS investment in their research, they found that a greater degree of integration between business strategy and IS strategy induces greater IS effectiveness and business performance. In subsequent studies, Sabherwal and Chan [44] divided such performance by matching the IS strategy types handled in Miles and Snow’s competitiveness strategy with the four types of IS characteristics. Tallon et al. [52] set up four types of IS strategies that correspond to Porter’s competitiveness strategy and examined the IS performance relationship with the role of IS administration such as strategic alignment and IS investment evaluation. They suggested that if the four IS strategy steps are required—unfocused, operation focused, market focused, dual focused—the six IS performance categories increase and the IS administration effort becomes effective. Therefore, based on these previous studies, we propose and test the relationship between strategy integration (business strategy and IS strategy alignment) and the benefit to IS with the mediating effect of IS investment.

H1

Strategy integration is positively related to IS investment, which in turn is positively related to IS benefit.

3.2 Fit of business strategy and organization structure

According to the latest studies on the mutual relationship between business strategy and organization structure, these two are interdependent and both must properly correspond for performance to be better [33]. However, there are different views as to which variable, strategy or structure, has an effect on another variable. In an earlier study, Chandler [12] examined the development of 100 US firms, and noted that the performance of an organization can be increased when the type of business strategy corresponds properly to the type of organization structure. Miller, in his study regarding the influence of the relationship between Porter’s strategy types and organization structure on the performance of an organization, also came to the conclusion that the performance of an organization can be increased when each of these three variables—(1) the environment of the firm, (2) business strategy and (3) structure—correspond to one another [31].

Meanwhile, Peterson [37], in research on European firms’ IS organization structure, found that the choice of an organization type depends on the variables set up, such as business strategy, business governance, the scale of an organization, the degree of information centralization, the safety of the environment and business competitiveness. Furthermore, many firms take on a federal type, which manages IS infrastructure using a centralized type and the application uses a decentralized type.

On the basis of the previous studies, we take four types of organization structure—bureaucratic, divisionalized, matrix, and network—into consideration. Also, we propose and test the relationship between business fit (business strategy and organization structure alignment) and IS benefit with the mediation effect of IS investment.

H2

Business fit is positively related to IS investment, which in turn is positively related to IS benefit.

3.3 Integration of organization structure and IS architecture

According to recent studies, organization structure and IS architecture can mutually affect each other, causing a difference in IS performance [43]. However, there are different views about which variable, between organization structure and IS architecture, has an effect on another variable. Initial studies described that the introduction of IS could improve the centralization of a decision making structure or accelerate the decentralization of it. In the 1980s, a series of studies argued that organization could have an impact on technology rather than vice versa as IS architecture moved from main frame to a server/client environment. Crowston et al. [13] claimed that the key factors determining the structure of IS are strategy, the degree of decentralization of decision making and the scale. Ahituv et al. [1] showed from their research that the degree of IS centralization is dependent on the degree of centralization of an organization’s decision making structure. That is, firms with the most centralized decision making structure avoid concentrating the IS structure but the most decentralized firms have a dispersed IS structure. However, Huber [23] claimed instead, that the same IS structure encourages centralization in firms with a decentralized structure.

On the basis of the prior studies reviewed above, we take three types of IS architecture into consideration: centralized, decentralized, and distributed. Also, we propose and test the relationship between operation integration (organization structure—IS architecture alignment) and IS benefit with the mediation effect of IS investment.

H3

Operational integration is positively related to IS investment, which in turn is positively related to IS benefit.

3.4 Fit of IS strategy and IS architecture

To successfully realize IS, Earl [17], when explaining the management of IS strategy, emphasized organic alignment between IM, which is the operational strategy in the management and ITA, which in turn is the basis of realization in technology. Henderson et al. [19–21] also came to the conclusion that the effectiveness of IS can be maximized only when IS strategy fits strategically into the IS infrastructure. Maizlish and Handler also explained the relationship between business strategy, IS plan and IS infrastructure [4]. They described this relationship in view of a portfolio and as the embodiment system. Furthermore, they claimed that the accordance of this vertical alignment is important.

The terminology “IS infrastructure” here could be understood as the practical IS architecture, which Henderson et al. [21] explained as the embodiment system in the stage of technology. In light of this, their claims suggest that the effectiveness of IS can be maximized only when business strategy is aligned to IS strategy and IS architecture. The implementation system must conform to this IS strategy through IS investment whereby the goal is clearly defined. Thus, the conclusion from previous studies is that an increase in IS performance and proper IS investment can be achieved only when the strategic goal of an organization’s IS is aligned to the IS architecture, which is the foundation of the investment of practical IS resources.

In view of the available research and the above logic, we propose and test the relationship between IS fit (IS strategy—IS architecture alignment) and IS benefit through the mediation effect of IS investment. The following hypothesis and the overall research model have been included in Fig. 1:

Fig. 1
figure 1

Research model of the relationship between strategic alignment and IS success through the mediation effect of IS investment

Full size image

H4

IS fit is positively related to IS investment, which in turn is positively related to IS benefit.

4 Research design and methodology

4.1 Research methodology

This research aims to develop an assessment model for IS success by identifying general factors that affect such success depending on the organization situation and by proving the influences of and the correlation among these variables affecting IS success.

We set up the research model by linking business-IS strategies that reflect the internal and external environment in firms and the infrastructure, such as organization structure or IS architecture, adding IS investment as an intermediate progress factor to attain IS success and a comprehensive IS management concept within the IS life cycle. To test this research model, 4 detailed hypotheses were set.

In order to test the foregoing hypotheses, we conducted group surveys from mid 2008 to late 2009, among a random sample of business executives in approximately 300 companies. This number represented both listed South Korean companies and other EU & US foreign companies based in Korea of similar size and operating characteristics. The survey targeted a range of business executives in these firms including, but not limited to, the CEO, CFO, COO and CIO. Responses were received from 260 executives (one per firm) yielding an overall response rate of 86.6 %. We conducted both fax and e-mail surveys to cover data insufficiency from the group interviews and to obtain a high response rate.

4.2 Operational definition of variables

4.2.1 Business strategy

Business strategies are defined as major decision making guidelines that generate and maintain corporate competitive advantages, distributing limited managerial resources. Miles and Snow [35], and Porter [38, 39] differ from other researchers. They focused on competitive strategy, which studies how to fight in market, rather than on market entry processes, which is related to managerial dimensions.

The major strategic attributes dealt by their researches are operational excellent and strategic positioning. Porter’s [38, 39] modified original strategy, which classifies business strategies more intuitively, defined the following:

  1. (1)

    Cost leadership: emphasizes producing standardized productions/services at very low per unit costs for many buyers who are price sensitive.

  2. (2)

    Differentiations: refers to outputs that are considered unique industry-wide and are addressed by many buyers who are relatively price insensitive.

  3. (3)

    Simultaneous strategy: normally offers lower costs wherever possible, while providing high value added output.

  4. (4)

    Stuck in the middle: does not develop one of these strategies, relegating itself to low profitability.

To measure business strategy, Snow and Hrebiniak suggested at least four alternative approaches: (1) investigator inference, (2) self-typing, (3) external assessment, and (4) objective indicators [48]. In this study, we use the objective indicators approach, whereby the firm’s executives assessed their strategy character dimensions (objective indicator) using descriptions of operational excellent and strategic positioning.

The objective (goal) of operating effectiveness has two kinds of sub-properties. The first kind emphasizes reducing operation costs and increasing quality and speed (efficiency) and the other one concentrates on enhancing the effectiveness of the firm’s overall performance (effectiveness). The strategic positioning object also has two kinds of sub-properties. One aims to extend their market and geographic reach and create or enhance the value proposition for their customers (reach), and the other tries to change industry and market practices (structure). The above four business strategies are characterized by a combination of the two business objectives (purpose or goal) and are illustrated by using their four sub-properties (efficiency, effectiveness, reach, and structure).

These two indicators are typical 7-Likert measures often used to operationalize business strategies. These descriptions and measurements have been adopted and are presented in Table 2.

Table 2 Business strategy variables: descriptions and measurements
Full size table

4.2.2 IS strategy

IS strategy is the strategic objective (goal) of an information system section to support corporate or business strategy. While IS strategic types have been dealt with by several researchers, STROIS (Strategic orientation of IS), suggested by Chan and Huff [9], classified IS strategic types, lifting IS strategies to a business strategic dimension. However, in application, STROIS generates too many questions, with 8 kinds of dimensions, to gather accurate data from respondents. Also the 8 strategic characteristics occasionally overlap or are ambiguous, incurring incorrect answers or confusion.

Therefore, in order to measure the IS strategic objectives (goals) precisely and easily in an actual situation, simpler IS strategic classifications are required. In this perspective, the Tallon et al. model [52] with 4 types of IS strategies is more useful. As mentioned before, if IS strategy is defined as an IS strategic objective (goal) and is related to business strategy rather than generated independently, the Tallon et al. [52] model provides a precise and intuitive IS strategy linked to the business strategy. By applying the Tallon et al. model [52] to this study, the IS strategy items can be defined as follows:

  1. (1)

    Operations focus: Focus on operational excellence required by the business strategy including cost reduction, quality-speed-productivity improvement, and a general increase in corporate effectiveness.

  2. (2)

    Market focus: Apply IS to reinforce corporate strategic positioning in industry and the market, extend market-geographical scopes, and change industry and market business practices.

  3. (3)

    Dual focus: Promote both operations and market focus, apply IS to extend the market and to generate new business as a new market capture method

  4. (4)

    Unfocused: No precise IS goal, or unconcerned about IS

To measure IS strategy in this study, we used the objective indicators approach [48, 52], whereby the firm’s executives assessed their firm’s IS strategy character dimensions (objective indicator) by describing the supporting role of operational excellence and strategic positioning. As such, the purpose of operational excellence positioning of a business strategy can be achieved by using IS to reduce operating costs, improve productivity and enhance overall firm effectiveness (operation focus). The strategic positioning goal of a business strategy can be achieved using IS to extend existing market or regional customer access and to transform the traditional transaction process of an industry or market (market focus). The aforementioned four IS strategies are characterized by a combination of the two IS goals and can be illustrated by using their four sub-properties (efficiency, effectiveness, access, and transformation).

These two indicators are the typical 7-Likert measures often used to operationalize IS strategies. These description and measurements have been adopted and are presented in Table 3.

Table 3 IS strategy variables: descriptions and measurements
Full size table

4.2.3 IS investment

IS investment can be defined as the investment that focuses on the target system or pursues the goal of the system by applying the IS strategy. According to Weill and Olson [58], Johnston and Carrico [26], Earl [17], Weill [55], Remenyi et al. [42], and Weill and Aral [56], the IS investment characteristics can be summarized into internal operating efficiency and external competitiveness advantage guarantee. In other words, it can be divided into an internal system and an external system. Based on these two IS investment characteristics, this study intends to classify IS investment objectives (aimed or planned systems) into four corresponding IS strategic types: internal system investment, external system investment, both systems investment, and ambiguous system investment. These are defined below:

  1. (1)

    Internal System Investment: IS investments related to efficiency and productivity improvement of core or supporting business activities in an organizational value chain process such as ERP, MES, PDM, KM, EP, MIS, etc.

  2. (2)

    External System Investment: IS investments related to the improvement of business efficiency and the effectiveness of external relations or the expansion of customer relations and sales entities such as CRM, SCM, CLAS/EC, EDI, etc.

  3. (3)

    Both Systems Investment: IS investments that focus on both internal and external systems.

  4. (4)

    Ambiguous System Investment: No IS investments or low IS investments.

To measure IS investment in this research, we used the objective indicators approach [48], whereby the firm’s executives assessed their IS investment character dimensions (objective indicator) using the descriptions of the internal and external system investments. The above four IS investments are characterized by a combination of the two IS investment goals and are illustrated by using their four sub-properties systems (operation system, management system, strategic relation system and sales system). These two indicators are the typical 7-Likert measures often used to operationalize IS investment. These descriptions and measurements have been adopted and are presented in Table 4.

Table 4 IS investment variables: descriptions and measurements
Full size table

4.2.4 IS success

A wide variety of measure factors have been suggested for information systems by many researches because one can measure the benefit differently at different stages of information flow. That is, the IS benefit can be the output of the information system or could be the information system itself. The benefit could even be the influence of the information on the user or organization. IS benefit can be defined as being the direct or indirect, strategic and operating-process effects on business benefits as realized by IS investment, the objective of which is to support the business goal and business processor. From a corporate point of view, IS benefit can be separated into both tangible and intangible benefits. Moreover, quantitative performance is composed of financial and metric performance. IS benefit can be divided further by means of another classification method: those that are econometrics oriented and those that are business process oriented.

However, in this research, the effects of an information system on business performance was generally measured by metric or quantitative values, but in the valuation process, we had to focus on executives’ recognized perceptions because that information is usually hard to obtain. The validity of this method has been proved by several researchers [5, 14, 15, 25, 54]. Based on Tallon et al.’s IS benefit typologies [52], we could provide a richer assessment of IS benefit through multiple perspectives rather than using a single, firm wide measure (see Appendix). The following lists are very significant in terms of actually proving the study as it requires precision and simplicity in the evaluation measure.

This study will deal with IS benefit at the process level and the survey executives’ perception. The benefit dimensions of the IS success model have been adopted and are presented in Table 5. The indicators are the 7-Likert scales.

Table 5 IS success (benefit) variables: descriptions and measurements
Full size table

4.3 Data collection

4.3.1 Survey objectives (Subjectives)

The study targeted 213 South Korean companies and 60 EU & US companies based in Korea. Data for this study was collected using interviews and questionnaire surveys administered in South Korea. As the research scope was comprehensive (handling the firm’s business strategy in relation to IT performance), the business executives in these companies, including the CEO, CFO, COO and CIO, were canvassed.

A summary of the firms characteristics in our sample are presented in Table 6. Since our sample represents a wide range of companies, we used a one-way analysis of variance to determine whether the responses varied by geographic location, industry, and firm size. Firm size was measured using two variables: the number of employees and annual sales [28]. Common operationalization of firm size includes gross sales or gross value of assets [29]. In this study, we categorized SME (small medium enterprise) firms as those with 500 or fewer total employees and/or annual sales of $1 billion or less; large firms are those with more than 500 total employees and/or annual sales of over $1 billion.

Table 6 Characteristics of the sample (N = 273)
Full size table

4.3.2 Data collection methods

The data collection method was used for both group interviews and e-mail and fax questionnaire surveys in this research. The gang survey method was used for the group interviews whereby responses were taken immediately after explaining the purpose of the research to the respondents.

A group interview was conducted on the following groups: (1) firm executives who had participated in the CIO academy courses at the Federation of Korean Information Industries (FKII) in 2008–2009, (2) firm executives who had attended the Advanced Management Program (DAMP) courses at Seoul National University in 2008–2009, and (3) the firm CFO & CIO who had partaken in the breakfast IT forum at the Korea Foreign Company Association (FORCA) in 2009. For the information to be consistent with the goals of this research, we also included a number of companies in the target interview groups that are listed on the Korean stock market as well as EU & US foreign companies based in Korea, as they are secure and solid bases and environments of information system-oriented business.

4.3.3 Pilot survey

A preliminary survey was conducted to check and inspect the form and content, as well as the number of questions and the validity of the survey method. To conduct this pilot survey, we held three discussion meetings with two university professors and six business executives. (2 CEOs & 2 CFOs from Korean companies and 1 COO & 1 CIO from foreign companies).

We first completed the questionnaire with the professional help of a professor, asked the executives to review it, and finally reformed the content and the number of questions after reflecting on their comments and suggestions. Some of their guidelines and suggestions were: (1) minimize the number of questions because the executives of a firm do not have a great deal of time to respond, (2) change the terminology to make the business words understandable (3) the questions should be brief enough to give an intuitive answer within 5 min. As they were business executives, we followed their advice and trimmed down the number of questions and revised the survey instruments to minimize the respondent’s inconvenience by avoiding overlapping questions.

4.3.4 Description of research variables

To extract research variables that could be applied to the actual analysis of the measurement items in the research model, the following averages of the research model measurement items were used. Table 7 shows a summary of the descriptive statistics of the research variables.

Table 7 Descriptive statistics of research variables
Full size table

5 Data analysis and hypotheses test

5.1 Assessment of validity and reliability

Firstly, the survey of the business context consists of 4 questions designed to assess business strategy (2 for operational excellence and 2 for strategic positioning). Secondly, the survey of IS context consists of 8 questions. The first four were designed to assess IS strategy (2 operation focused and 2 market focused), and the second four assessed IS investment (2 on the internal system, 2 on the external system). Lastly, the IS benefit survey consisted of 29 questions. The first half was designed to assess operational excellence benefits (5 questions each on planning and management support, production and operation and product and service enhancement, respectively) and the second half was done to evaluate strategic positioning benefit (5 questions each on supplier relations and sales and marketing support and 4 questions on customer relations).

Table 8 shows a summary of the results of the factor analysis of the convergent validity and discriminant validity of the survey questions tests. The KMO (Kaise-Meyer-Olkin) values, which indicate the adequacy of the sampling and explain the extent of the interrelation between variables, are shown on the table as only 0.500 and 0.500, respectively. This could be considered as not high enough but the selection of CFA variables makes it adequate. Also, it verifies the existence of a common factor in the values of Bartlett’s test of sphericity, which indicates the consistency of factorial analyses. Moreover, all of the p values are 0.000. The Eigen values of the 12 factors are also much higher than the recommended level, 60 %, in Social Science. All the variables demonstrate an acceptable reliability.

Table 8 CFA and reliability of measurement
Full size table

5.2 Correlation analysis of research variables

It is necessary to examine the correlation of the research variables before verifying them. Table 9 shows a summary of the results of the correlation analysis. It indicates that all of the correlations among the research variables are meaningful at the level of 0.01 and the coefficient of correlation ranges from a minimum of −0.58 to a maximum of 0.97. The coefficient of correlation is the average value of each group (It is not an individual indicator).

Table 9 Correlations among research variables
Full size table

Consistent with our hypotheses, all the variables are correlated significantly with other variables. These results provide preliminary evidence supporting the hypotheses.

5.3 Hypotheses test

We used the structure analysis of covariance to test whether the research variables correlate. There are many statistical packages for structure analysis of covariance such as LISREL, AMOS and EQS etc. We used AMOS 7.0. The advantage of AMOS over LISREL is that it easily displays the path model in the form of an illustration instead of a matrix equation.

An overall model fit assessment of the research model is needed before the hypotheses can be tested, using SEM’s path analysis or structure model fit. The overall model fit evaluates the actual correspondence or observes the input matrix with the predicted one from the proposed model [18]. Generally, model-fit measures are used to assess the model’s overall goodness of the fit. There are many overall goodness-of-fit measures which can be categorized into three types: absolute fit measures, incremental fit measures, or parsimonious fit measures.

5.3.1 Analytical approach

This research deals with the goal of IS investment as a mediator variable to discover whether higher accordance of strategic alignment creates higher IS benefits. To measure the degree of integration between business strategies and IS strategies, the fit between business strategies and organization structures, the integration between organization structures and IS architectures, and the fit between IS strategies and IS architectures in the research model, the following calculation was used. Alignment (integration or fit) has to evaluate the agreement of strength and direction between two elements at the same time, and so it was calculated with Max (element 1 evaluated value, element 2 evaluated value)—ABS (element 1 evaluated value—element 2 evaluated value).

To verify this statistically, we followed Baron and Kenny’s mediation analysis [2]. We applied ‘Business—IS Alignment (X)’, ‘IS Investment (M)’, and ‘IS Benefit (Y)’ to measure the data given above with the parameter model by using the structural equation; to verify the significance, the Sobel test should be used. The basic parameter model to be verified in the research model is shown below in Fig. 2.

Fig. 2
figure 2

Path diagram and parameters of the hypothesized and tested model

Full size image

In the illustration above, Business—IS alignment directly affects IS benefit but has an indirect effect (carrying effect) when it goes from Business—IS alignment to IS benefit (path b) through IS investment (path a). The indirect effect can be represented as a*b, and the Sobel test can be used to judge the significance. According to Baron and Kenny, when the indirect effect is significant and the direct effect is insignificant, it can be said to have a perfect mediation effect [2]. If both the direct and indirect effects are significant, it can be said that there is a partial mediation effect.

5.3.2 Overall model fit

The overall model fit evaluates the correspondence of the actual or observed input matrix with that predicted in the proposed model [18]. As recommended by Kline [30], we summarized the indices used in this research: the χ2 statistics divided by its degrees of freedom (CMIN/DF), Goodness-of-Fit Index (GFI), and Root Mean Square Residual (RMR).

Table 10 shows a summary of the goodness-of-fit results for the proposed model to identify whether the overall structure of the causal relationship is valid. Absolute fit measures such as χ2 (Chi-square), the degree of freedom, the p value, root mean square residual (RMR), and the goodness-of-fit index (GFI) have been calculated and presented in the Table. Notably, the sample size (n = 273) is big enough, the GFI is >0.8, and the RMR is <0.8 [27], and thus it can be considered in general that the suitability of the model approached the optimized model even though it failed to form the optimized congruence model.

Table 10 Overall goodness of fit measures
Full size table

5.3.3 Hypotheses testing

In Hypothesis H1, we examined the mediation effect of IS investment between Strategy Integration (alignment of business strategy—IS strategy) and IS benefit. First, we tested the mediation effect of internal system investment between operational strategy integration (operational excellence business strategy—operation focus IS strategy) and internal benefit. In the case of this causal relationship, the path coefficients a and b are significant (p < 0.05), the path coefficients of c are not significant (p > 0.05). The statistical significance of the model was checked using the Sobel test (Z = 8.573, p < 0.05). It shows that internal system investment has a perfect mediation effect on the relationship between operational strategy integration and operational excellence benefit.

Next, we tested the mediation effect of external system investment between strategic integration (strategic positioning business strategy—Market focus IS strategy) and external benefit. In the case of this causal relationship, the path coefficients a and b are significant (p < 0.05), the path coefficients of c are not significant (p > 0.05). The statistical significance of the model was checked using the Sobel test (Z = 10.332, p < 0.05). It shows that external system investment has a perfect mediation effect on the relationship between strategic integration and strategic positioning benefit. Table 11 shows the results of Hypothesis 1.

Table 11 Results of the mediation effect of IS investment, between strategy integration and IS benefit (H1)
Full size table

In Hypothesis H2, we examined the mediation effect of IS investment between business fit (alignment of business strategy—organization structure) and IS benefit. As noted earlier, we tested four types of organization structures: bureaucratic, divisionalized, matrix, and network.

First, we looked into the relationship between the business fit of a bureaucratic organization and internal benefit via internal system investment. In the case of this causal relationship, the path coefficients of a and b are significant (p < 0.05), while the path coefficients of c are not significant (p > 0.05). It means that internal system investment has a perfect mediation effect on the relationship between the business fit of a bureaucratic organization and operational excellence benefit. The statistical significance of the model was checked using the Sobel test (Z = 6.837, p < 0.05). Second, we tested the mediation effect of external system investment between business fit of divisionalized organization and external benefit. In this causal relationship, the path coefficients of a and b are significant (p < 0.05), the path coefficients of c are not significant (p > 0.05). This means that external system investment has a perfect mediation effect on the relationship between business fit of divisionalized organization and strategic positioning benefit. The statistical significance of the model was checked using the Sobel test (Z = 9.855, p < 0.05). Third, we tested the mediation effect of both internal and external system investments between business fit of matrix organization and overall IS success. In this case, the path coefficients of a and b are significant (p < 0.05), while the path coefficients of c are not significant (p > 0.05). This means that both internal-external system investments have a perfect mediation effect on the relationship between business fit of matrix organization and both operational excellence benefit and strategic positioning benefit. The statistical significance of the model was checked using the Sobel test (Z = 10.882, p < 0.05). Finally, we tested the mediation effect of both internal and external system investments between business fit of network organization and overall IS success. In this case, the path coefficients of a and b are significant (p < 0.05), the path coefficients of c are not significant (p > 0.05). This means that overall system investments have a perfect mediation effect on the relationship between business fit of network organization and both operational excellence benefit and strategic positioning benefit. The statistical significance of the model was checked using the Sobel test (Z = 10.323, p < 0.05). Table 12 shows the results of Hypothesis 2.

Table 12 Results of the mediation effect of IS investment between business fit and IS benefit
Full size table

In Hypothesis H3, we examined the mediation effect of IS investment between operational integration (alignment of organization structure—IS architecture) and IS benefit. As noted earlier, we tested three types of IS architecture: centralized, decentralized, and distributed.

First, we tested the mediation effect of internal system investment between operational integration of bureaucratic organization—centralized IS architecture and internal benefit. In the case of this causal relationship, the path coefficients a and b are significant (both p < 0.05), while the path coefficients of c are not significant (p > 0.05). This means that internal system investment has a perfect mediation effect on the relationship between integration of bureaucratic organization—centralized IS architecture and operational excellence benefit. The statistical significance of the model was checked using the Sobel test (Z = 1.969, p < 0.05). Second, we tested the mediation effect of external system investment between operational integration of divisionalized organization—decentralized IS architecture and external benefit. In this case, the a, b and c path coefficients are significant (all p < 0.05). This means that external system investment has a partial mediation effect on the relationship between integration of divisionalized organization—decentralized IS architecture and strategic positioning benefit. The statistical significance of the model was checked using the Sobel test (Z = 7.879, p < 0.05). Finally, we tested the mediation effect of both internal and external system investments between operational integration of matrix organization—distributed IS architecture and overall IS success. In this case, the a, b and c path coefficients are significant (all p < 0.05). This means that both internal-external system investments have a partial mediation effect on the relationship between matrix organization—distributed architecture and both operational excellence benefit and strategy positioning benefit. The statistical significance of the model was checked using the Sobel test (Z = 7.478, p < 0.05). Table 13 shows the results of Hypothesis 3.

Table 13 Results of the mediation effect of IS investment, between operational integration and IS benefit (H3)
Full size table

In Hypothesis H4, we examined the mediation effect of IS investment between IS fit (alignment of IS strategy—IS architecture) and IS benefit.

First, we tested the mediation effect of internal system investment between IS fit of operation-focused IS strategy—centralized IS architecture and internal benefit. In the case of this causal relationship, the path coefficients of a and b are significant (p < 0.05), while the path coefficients of c are not significant (p > 0.05). This means that internal system investment has a perfect mediation effect on the relationship between IS fit of operation-focused IS strategy—centralized IS architecture and operational excellence benefit. The statistical significance of the model was checked using the Sobel test (Z = 5.962, p < 0.05). Second, we tested the mediation effect of external system investment between IS fit of market-focused IS strategy—decentralized IS architecture and external benefit. In this case, the a, b and c path coefficients are significant (all p < 0.05). This means that external system investment has a partial mediation effect on the relationship between IS fit of market-focused IS strategy—decentralized architecture and strategic positioning benefit. The statistical significance of the model was checked using the Sobel test (Z = 9.331, p < 0.05). Table 14 shows the results of Hypothesis 4.

Table 14 Results of the mediation effect of IS investment between IS fit and IS benefit (H4)
Full size table

As illustrated above, we proved that IS investment has a mediation effect between strategic alignment and IS benefit. In other words, we can conclude from this research that strategic alignments affect the goal of IS investment and this IS investment affects IS benefit, in this order.

6 Discussion and concluding remarks

The aim of this research was to develop a comprehensive IS evaluation model for IS success linked to organizational performance. A theoretically-based, comprehensive set of IS evaluation measures was presented as along with a contingency theory for selecting appropriate measures such as for a business-IS alignment practice. By complementing previous studies, this research offered a contribution by proving the contingency relationships between business strategy, IS strategy, IS investment, and IS success. By understanding the differences in IS performances through such research processes, the intention was to provide an empirically proven evaluation model. Four detailed hypotheses were presented to test such a research model. In order to test the foregoing hypotheses, we conducted group surveys on a random sample of business executives from approximately 300 companies. The number listed came from both South Korean and Korean based EU & US foreign companies of similar size and operation characteristics. Responses were received from 273 individual executives (one person per firm).

The results of the mediation analysis mostly supported our hypotheses. By confirming the crucial role of IT investment, we captured clues as to the indirect effects of Korea’s economic development, accompanied with appropriate strategic alignments of successful IS reputations in the global market. With this overall perspective, the following can be contributed to this research.

First, this research empirically verified the relationship (matching type) of mutual strategy alignment between business strategy and IS strategy based on the contingency theory. That is, using the suggested ‘matching as fit (adjusted deviation score analysis)’ method, a business-IS alignment method presented by Venkatraman [53] and tested by Chan [8, 9], this research demonstrated the corresponding ‘business strategy and IS strategy’ types. We showed several respective matching patterns between business strategy and IS strategy, such as ‘cost leadership and operation focus’, ‘differentiation and market focus’, ‘simultaneous and dual focus’ and ‘stuck in the middle and unfocused’. This result practically verified Tallon’s hypothesis [52].

Second, with regard to the mediator effect of IS investment between IS business-IS strategic connection, this research found that this strategic alignment affects the IS investment goal whereby it affects the benefits of IS. This is within the same context as the research by Weill [55] or Johnston and Carrico [26], pointing to IS investment as a mediator variable of business strategy and IS benefit. However, previous studies were limited to strategic alignment at the plan level, so this research expanded its scope to IS architecture and organization structure at operational level. The result shows that the higher the respective alignments are between ‘cost leadership business strategy and operation focus IS strategy’, ‘cost leadership business strategy and bureaucratic structure,’ ‘bureaucratic structure and centralized architecture,’ ‘operation focus IS strategy and centralized architecture’, the higher the operational internal outcome with internal system investment.

The higher the respective alignments are between ‘differentiation business strategy and market focus IS strategy’, ‘differentiation business strategy and divisionalized structure,’ ‘divisionalized structure and decentralized architecture,’ ‘market focus strategy and decentralized architecture’, the higher the competitive advantage of external system investment. Furthermore, the higher the respective alignments are between ‘simultaneous business strategy and dual focus IS strategy,’ ‘simultaneous business strategy and network structure (or matrix structure),’ ‘matrix structure and distributed architecture,’ the more both internal and external system investments lead to both operational excellence and strategic positioning benefits.

6.1 Recommendations

Considering the research analysis results presented here, it can be summarized that the goal of IS investment is dependent on business strategy and IS strategy. That is, if the strategy is focused on the strategic goals of internal operational excellence, IS investment is likely to focus heavily on the internal system, which is oriented to operational affairs, and the benefit is also founded mostly in internal operations benefits. On the other hand, if the strategy is focused toward enhancing differentiation, IS investment is likely to concentrate on areas that can enhance the relationship between clients and partner companies with market differentiation, and can thus achieve a relatively better competitive advantage. Only when both business and IS strategies are aligned to each other, can firms achieve the desired level of strategic IS goals. This results in proper IS investment designed for the goal of the firm’s strategy, and this will lead to a corresponding benefit in IS. This research verified this strategic alignment empirically for Korea. Strategic alignment demonstrates how IS and business are harmonized and how they should be properly harmonized.

In other words, strategic alignment focuses on the activities that management performs to achieve cohesive goals across the IS and other functional organizations (e.g., finance, sales, and manufacturing). Alignment evolves into a relationship where IS function and business function adapt their strategies together. Achieving alignment is evolutionary and dynamic. It requires strong support from senior management, good working relationships, strong leadership, appropriate prioritization, trust, and effective communication, as well as a thorough understanding of the business and technical environments.

This research also closely examined the fact that IS benefit is a natural result of investment corresponding to the strategy. However, an even better IS benefit can be achieved by managing IS systematically after IS investment. That is, setting up goals to realize efficient and high IS management maturity is very important. Thus, to achieve those IS management goals, it is necessary to set up the following steps in the IS life cycle, from planning to the disposal of IS; the necessary pursue strategy, an organization to implement this strategy, the right order of working, the necessary methodology (or mechanism) and a well-organized system for all the above. Hence, this research provides practical signification to (by applying contingency theory) setting up an appropriate IS evaluation system in accordance with a firm’s IS investment.

6.2 Limitation and further research

The research survey was conducted on listed, local Korean firms and Korean based European & US branch firms. In the case of the foreign companies, there might be non-negligible differences between the global headquarters and the Korean branches. Furthermore, the instrument was applied in group interviews with executives, so it was not used as part of a postal questionnaire or a web-based survey, which may present different results due to interviewer effects. Each respondent represented an executive per firm and we tried to solve the common method bias problem in the research design level by asking the IS department or his or her secretary office to help get a reply. However, progress was not made easy by just classifying the independent variable and dependent variable in the survey (of course we eliminated CMB might-have-been by means of the statistical method, just as with the ex post approaches). It was observed that it is important to have more than one executive level respondent per firm for an in-depth analysis.

In addition, this research delivers an extended model, adding strategy alignment, IS investment, and IS maturity to the existing IS Success model. It divides it into four parts to test the models but the whole model connection verification could not be made. In the future, if the R 2 value of the whole model is obtained, it will be possible to grasp the completeness of the integration model. However, this study created and used a new instrument to measure IS maturity. While many sound research practices were followed in the development of the instrument, the instrument has only been tested on one sample of 273 firms.

It will be necessary to perform a more practical investigation by not only limiting this to a specific industry (or simple industry comparison, e.g. manufacturing industry vs. service industry) but also applying it to the firm’s intense stages of globalization, by carrying out the case study on differences, targeting the firm’s headquarters in Asia, EU and US. In addition, it is regarded as necessary to supplement strategic alignment research by considering special circumstances such as a firm’s long active M&A.

Notwithstanding the limitations stated above, we feel that the contribution of our research, which is integrated and has extended the IS success model, is significant. As mentioned, an attempt will be made to continue these meaningful studies to overcome the limitations of the research.