Keywords

1 Motivation

The term ‘framework’ is frequently used in the field of information systems. In fact, there are many different kinds of frameworks to find in articles published in journals or conference proceedings. However, the term ‘framework’ itself is quite inconsistently used across the domain and interpreted very differently in these publications. For example, there are frameworks used to evaluate information systems, to describe information systems or to develop information systems. Despite being used so frequently, there is no clear and absolute definition what a framework actually is. There are no guidelines, rules or a consistency on how the term is used or should be used. Moreover, there are no basic studies that have dealt with this issue.

Johnson and Foote [6] describe a framework as a set of classes that contains an abstract design for solutions for a cluster of related problems which is reusable.

Johnson [5] described a framework in three different ways. First, a framework is the sum of components and patterns. Second, it is described as a reusable design of a whole system or parts of a system that is represented by a set of abstract classes and the interaction between the classes. Third, it is defined as a skeleton of an application that can be customized by the developer.

The Oxford Dictionary [4] describes a framework as “an essential supporting structure of a building, vehicle, or object” or the “basic structure underlying a system, concept, or text”.

These are only some existing approaches demonstrating that there is no accepted definition on the subject and explaining why the term is used in so many different ways.

The scope of this work is to give an overview about the usage and structure of frameworks in the scientific literature of the last 10 years. A systematic literature review helps to gain deep insight in the field. The structure of this work will mainly follow the systematic literature proposed by Kitchenham [8]. This procedure ensures traceability and a clear focus.

The systematic review is carried out to answer the given research question:

Research Question: How can frameworks be categorized in the field of information systems?

The contributions of this work are: (1) a general overview about the usage of the term ‘framework’ in the field of information systems, (2) a proposed categorization of identified frameworks and (3) a basic definition of the term ‘framework’.

This work is structured as follows: Sect. 2 provides a general overview about our methodological approach in general. Section 3 describes the procedure used to conduct the systematic literature research. Section 4 discusses the results of this work. The last Sect. 5 summarizes and gives an outlook on further research.

2 Methodological Approach – Systematic Literature Review

This chapter describes the systematic literature review by Kitchenham. Furthermore, advantages and disadvantages of this method are discussed as well as the decision to use it.

Several discrete activities are part of a systematic review. According to Kitchenham, a systematic literature review consists of three parts [8]:

  1. 1.

    Planning the review.

  2. 2.

    Conducting the review.

  3. 3.

    Reporting the review.

However, the parts are not sequential. Most of them involve iteration. That means that predefined methods and guidelines in the review protocol can be refined and optimized when the actual review takes place. If the actual review process differs from the review protocol, it has to be documented. The instructions for each part are described in the following:

Planning the review.

In this part, a check is performed to see if a systematic review is necessary to solve and look at a certain problem. It is necessary to gather all information about a problem in an unbiased, objective and rigorous manner. Also, a review protocol is required. This protocol is predefined before the research starts to avoid any bias on the part of the researcher. It serves as a guideline describing how the review is performed, what methods are used etc. In detail, the review protocol provides the following information:

  • Reasons for performing the survey.

  • Research questions the review wants to answer.

  • The search strategy used to find primary studies (Search terms, resources).

  • Inclusion and exclusion criteria for studies.

  • Study quality assessment.

  • Strategy on how relevant data is extracted from primary studies and what data should be extracted.

  • How the data synthesis is performed?

Conducting the review.

After the creation of the review protocol, the review process begins. However, the pre-defined steps in the protocol can still be refined and optimized when performing the review. The actual review consists of the following steps:

  • Research identification: The goal of this step is to identify as many primary studies as possible which might be helpful to answer the research questions by using an unbiased search strategy. This process is then documented. Individual studies considered in the review are called primary studies while the systematic review is a secondary study.

  • Study selection: After identification of the primary studies, the actual relevance of them is assessed by applying study selection criteria. Inclusion criteria are the criteria a study needs to meet in order to be relevant while exclusion criteria make a study irrelevant. Two or more researchers discuss whether or not a study is relevant.

  • Quality assessment of studies: Not only the relevance is important, but also the quality of the included studies. Quality instruments are used to evaluate the quality of the studies.

  • Extraction of data and progress monitoring: Relevant data to answer review questions is extracted from the primary studies. Data extraction forms are created to capture the relevant information. The forms should be defined in the review protocol. Two or more researchers should carry out the extraction of data independently.

  • Synthesis of data: During this step, the results of the primary studies are summarized and collated in a way consistent with the research questions.

Reporting the review.

In this part, the results of the systematic review are illustrated.

The systematic literature review offers a meticulous and fair approach in order to increase its scientific value and eliminate the risk of unbiased results. By creating a precise and detailed search strategy, it is possible to cover all necessary sources to gain the relevant information and thus to answer the research questions comprehensively. It also allows to summarize existing evidence concerning the technology and to find new areas for further investigation, which is a part of this work, too. Another advantage is that the method is transparent and repeatable. If another researcher performs the same task again according to protocol, similar results should be obtained.

Nevertheless, the systematic literature review has also some disadvantages. It is possible that some relevant information is not found by using the search strategy because it does not cover the source, or the identified studies do not provide sufficient information.

3 Conducting the Research Search Process

In this chapter, the search process of the systematic literature review for relevant studies is documented and the execution of the systematic literature review will be described. The execution includes the presentation of the review protocol, the initial search for the primary study and the final selection of relevant primary studies considered in the review.

First, the time frame of the research has to be defined. We agreed on the past 10 years to cover recent trends and developments and to consider older works as well. Second, we had to agree on the sources for the research. Urbach et al. [15] already identified the relevant journals like MISQ, CACAM or BISE and conferences like ICIS, ECIS or AMCIS in the field of information systems in their work. Their choice will be adopted in the following.

The search is performed mainly by hand using the existing search engines. If possible, the search function offered by electronic libraries is used to search for search strings described below. Otherwise, all the titles and abstracts are read. Studies from certain journals and conferences in the time frame from 2005–2014 are searched through. During the initial search for primary studies, the abstracts are checked with one question in mind:

For English literature:

Does the abstract or title contain a description or any information about frameworks used in the field of information systems? If yes, the study is selected and later checked thoroughly applying the inclusion and exclusion criteria. If not, the study is not relevant. Also, the synonym ‘structure’ is considered. The search term is formalized in a search string to increase traceability. The search string used to find studies is:

Title: ‘Framework’ or Abstract: ‘Framework’ in publication ‘[Name of source]’/Title: ‘Structure’ or Abstract: ‘Structure’ in publication ‘[Name of source]’ between 2005–2014.

For German literature:

Same procedure with the exception that the abstracts and titles are searched through additionally with the German translation ‘Rahmenwerk’ and the German synonyms ‘Plan’ and ‘Struktur’. The search string used to find studies is:
 Title: ‘Framework’ or Abstract: ‘Framework’ in publication ‘[Name of source]’/Title: ‘Rahmenwerk’ or Abstract: ‘Rahmenwerk’ in publication ‘[Name of source]’/Title: ‘Plan’ or Abstract: ‘Plan’ in publication ‘[Name of source]’/Title: ‘Struktur’ or Abstract: ‘Struktur’ in publication ‘[Name of source]’ between 2005–2014.

The search strings are very general and not very specific on the purpose (e.g., it would have been an option to include ‘IS’, ‘Information System’ etc. in the search string to make it more specific). However, the goal is to capture as many primary studies as possible. The advantage of this decision is that more potentially relevant studies are detected and the whole research area is covered more extensively. Furthermore, when reading the title or abstract, the slightest hint that the study could contain information about frameworks in the field of information systems is sufficient to check the study later by using the inclusion/exclusion criteria. Only those studies which title or abstract are completely off topic are not considered. The overall goal is to reduce the risk of missing potentially relevant studies.

Conference proceedings and journals that are not open access or do not deal with information systems are not considered in this review. The exception is the BIS conference, where the years 2005–2006 were not accessible. These journals or conference proceedings are: MISQ, JMIS, JCIS ASQ, SMR, Omega, HBR, DSS, DSI, HCI, ISR, IJEC, AMJ, AMR, EJIS, IBMSJ, IEEESW, JACM, OS, and IEEETrans.

After a first study selection based on the title and the abstract, the studies are checked again in a more detailed manner. This means that the complete studies are read and then a decision is made whether or not a study is relevant. The following criteria are applied to make a study relevant or irrelevant:

Inclusion criteria:

  • A study deals with a framework (or synonym of the word used during the initial search) related to information systems. The word information system or its abbreviation IS does not have to appear in the study if a software or system is described that fits the definition of information system given in the background chapter.

Exclusion criteria:

  • The paper does not contain any relevant information about frameworks in the field of information systems.

Data Extraction/Data Synthesis:

In this step, relevant data required to answer the research questions and general information about the study is extracted from the studies. The following data is identified as relevant and extracted from the studies: Title; Author; Source; Purpose of the framework; Structure of the framework; Development approach; Framework implementation described in the study; Updates on existing Framework.

The purpose, structure and development approach of the frameworks are used later in order to categorize them.

The extracted data is gathered in a table to show which study provides data to answer the research question. The table contains the above-mentioned data and is based on the data extracted from each study.

353 primary studies were identified during the first broad search for relevant studies using the search strings by checking the abstracts and titles whether or not they contain relevant information about frameworks in the field of information systems.

Selection of relevant studies:

After selecting 353 primary studies based on their title and the content of their abstract, the inclusion and exclusion criteria were applied to identify primary studies to be included and evaluated in the final review. Through this procedure, it was possible to reduce 353 primary studies to 71 studies to be included in the final review. The following charts show the number of studies before and after the application of the inclusion and exclusion criteria sorted by source and publication year.

Figure 1 shows that most of the potentially relevant and the relevant studies originate from the conferences (ECIS, ICIS, HICSS, AMCIS, BIS, BIR). Exactly 226 out of 353 (64 %) studies before the application of the inclusion/exclusion criteria and 50 out of 71 (70 %) after the application make up the majority of the relevant studies. The journals only make up 36 % (127 out of 353) of the relevant studies before the criteria application and 30 % (21 out of 71) after it.

Fig. 1.
figure 1

Studies sorted by sources

Full size image

Figure 2 shows that the occurrence of studies before and after the application of the criteria is not evenly distributed. For 2014, there are only two studies left after the criteria application compared to the years 2005 and 2011 with12 studies each.

Fig. 2.
figure 2

Studies sorted by date

Full size image

Another point to mention is that eight studies provide information about more than one framework. This explains why 93 frameworks were identified in the literature although there were only 71 studies in the field of information systems.

In this chapter, the relevant literature was identified through the systematic literature review. The next chapter outlines the results of the review based on the literature identified during the review.

4 Discussion of the Results

In this chapter, the research questions are considered in relation to the literature review. In addition, a definition of the term framework is given. This definition should fit all the identified frameworks in the studies. This chapter reports the review step of the systematic literature review. In this step, the results of the review are communicated.

4.1 Definition of the Term Framework

As already demonstrated, there is a variety of different definitions of the term ‘framework’ in the field of information systems. The focus is now on extracting information about the frameworks purpose, structure, development process, existing updates and implementations out of the systematic literature analysis.

It can be stated that the identified frameworks do not have many things in common. Out of the 71 identified studies, only one study defines the term ‘framework’. The other studies employ the term without any given definition. According to the study by Kajan and Stoimenow [7], a framework is a middleware used to bridge problems with heterogeneity by putting in a generic template providing the desired functionality. However, this definition is very specific and does not cover the other identified frameworks. Based on all identified frameworks and the extracted information (purpose, structure, development process, implementations, and updates) about them and the definitions of framework proposed before, the following definition includes all identified frameworks in the field of information systems:

A framework is a structure underlying ‘something’ serving a specific purpose.

This definition is the lowest common denominator describing what all identified frameworks have in common. It is still very general but due to the fact that the identified frameworks vary strongly in all their aspects, finding a more specific definition is not possible and does not make sense. The definition is very close to the definitions given by the Oxford dictionary [4] describing a framework as ‘basic structure underlying a system, concept, or text’. Basically, the notion of a framework as a structure of something is taken from these definitions and is then extended with the notion of solving a problem as the goal of a framework. It should be emphasized that only the structure part and not the basic structure part is taken from these definitions because a framework does not have to be necessarily only basic. The degree of detail describing the framework structure varies a lot between the identified frameworks. Some framework structures are described only generally while other framework structures are described very detailed. A framework with a very simple structure is, e.g., the Clic framework [2]. In comparison to the Clic framework, the Zachman framework and its extension has a very complex structure [17]. Therefore, a definite degree of detail in frameworks cannot be included in the definition. Another point to mention is that a framework can provide the structure for anything. In the identified frameworks, the categorisation provided supports implementation processes, research agendas, green information systems etc. The fact that a framework has a purpose is added, implying that a framework is used to fulfil a purpose. These purposes can be very different. They can consist in describing an implementation, summing up existing research results on a topic, describing factors influencing the outcome of an activity and many more.

To sum up, a framework gives a structure to something. The goal is to fulfil a purpose by using the framework.

4.2 Answering the Research Question

RQ: How can frameworks be categorized in the field of information systems?

It seems useful to categorize the identified frameworks according to the extracted meta-data about them. These characteristics are the framework purpose, the development process of the framework and the structure of the framework. Figure 3 gives an overview by using percentages.

Fig. 3.
figure 3

Categories in percentage

Full size image

Categories based on framework purpose:

  • Green Information System Framework: Green information systems frameworks focus on the environmental aspect of information system, e.g., the lifecycle assessment framework [13] for sustainable IS Management used to measure IS related environmental impact. Four frameworks are part of this category.

  • Test Framework: The purpose of test frameworks is to examine the implementation of information systems, e.g., the distributed systems monitoring framework used to support unit component testing in distributed component-based systems [10]. Six frameworks are part of this category.

  • Development Framework: These frameworks support the development of information systems or new information system features from two perspectives, the technical and general perspective. The technical perspective includes concrete parts of the implementation, e.g., the Gulliver framework [9] with the purpose to build smart speech based applications. The general perspective focuses on instructions for the development process, e.g., the Ethnorelative framework [11] which provides information system designers with information to understand their own cultural values relative to users of other national cultures. There are 27 development frameworks, 11 take the technical perspective and 16 the general perspective.

  • Research Framework: Research frameworks focus on theoretical topics with little practical application, e.g., the Computer Security Research Framework [12] with the purpose to synthesize and summarize research done in the area of Information System Security. In total, 18 research frameworks belong to this category.

  • Evaluation Framework: The purpose of evaluation frameworks is to evaluate information systems or certain aspects of information systems. Evaluated aspects are for example the systems compliance [3]. There are 16 evaluation frameworks.

  • Mixed Purpose Framework: Frameworks in this class do not fit into only one of the categories. They are a combination of two or more framework categories based on the frameworks purpose. An example of a mixed purpose framework is the environmental impact framework that combines the green information system and development category. The purpose of this framework is to support IT system design based on the system environmental impact. It belongs to the green information system category because it deals with the environmental aspect of the information system. Also, it belongs to the development framework category because it supports the development of information systems [18]. Five frameworks are a combination of framework categories.

  • Not Categorizable Frameworks/ITSM: The ITSM frameworks ITIL, the ITIL related HP ITSM, the Microsoft Operations Framework, and ISO/IEC 15504 are not categorizable and form a separate category.

Categories based on Framework Development Process:

Only 50 frameworks contain a description of how they were developed. The development process of these frameworks can be divided into four categories. Also, there are four additional methods used to support the development of the frameworks. They are not used as the only method to create a framework but they support its development combined with one of the four framework development categories. The development of 17 out of the 50 frameworks is supported by these methods. These methods are:

  • Interviews: Interviews with experts or practitioners (three frameworks).

  • Authors’ experience: Authors’ personal experience (six frameworks).

  • Case Studies: Research on a social phenomena (nine frameworks).

  • Field Studies: This method supports the development of one framework.

The identified frameworks can be divided based on their development process into the following four categories:

  • Literature Review Developed Frameworks: Frameworks in this category are developed through a literature review. 18 frameworks belong to this category. Case studies are used together with a literature review five times, field studies one time, interviews one time and the authors’ own experience four times.

  • Research Developed Frameworks: Frameworks of this category are based on existing research like models, theories, frameworks and so on. The research background of these frameworks varies a lot. 25 frameworks belong to this category. Case studies are used together with the existing research three times and interviews two times.

  • Requirements Developed Frameworks: Frameworks of this category are developed based on the identified requirements the frameworks needs to fulfil. The framework is built on these requirements. Four frameworks belong to this category. In one case, a case study is combined with the framework requirements during the development process.

  • Mixed Developed Frameworks: Frameworks of this category are developed based on multiple categories. There are three mixed developed frameworks. The three mixed developed frameworks combine the literature review approach and the use of the existing research for the development of a framework. Two times, the process is supported by the authors’ personal experiences.

The following Fig. 4 shows the distribution of categories based on the development process in percentages.

Fig. 4.
figure 4

Development process in percentage

Full size image

Half of the frameworks are research-based frameworks presenting the knowledge used to develop the framework. The process of obtaining the knowledge is not shown like in the literature review developed framework which is also a method often used to develop frameworks (36 %). Frameworks developed based on their requirements and mixed developed frameworks are a minority.

It is noticeable that case studies are used to verify, improve and test the developed frameworks five times. An example for this is the IS flexibility framework. The initial framework is developed through a literature review. After that, the framework is tested with a case study. The case study uses data collected from interviews with managers [1]. Another observation is that there are only six frameworks where the authors’ personal experience with the topic plays a significant role to develop the framework. In most cases (44 frameworks), the author relies on knowledge of other people.

Categorization based on framework structure:

In 69 cases, the structure of the framework is described in detail. There exist eight framework structures building the categories. A combination of two categories is also possible. The following categories for the structures of frameworks were identified:

  • Layered Structured Frameworks: Frameworks in this category have a layered structure, e.g., the Architect framework which consists of four layers [16]. A layer describes system features on different abstraction levels. Every layer is implemented based on the next lower layer. Six frameworks are part of this category.

  • Technical Structured Frameworks: Frameworks in this category contain a description of the technical components they consist of. Technical components are e.g. include used protocols like HTTP and SOAP in the B2B Ontology-Driven Framework [7]. 12 frameworks are part of this category.

  • Sequence Structured Frameworks: Frameworks in this category consist of activities performed in a sequential order and partly at the same time. There are frameworks which sequence has defined beginning and end cycle [2]. Four frameworks are part of this category.

  • Category Structured Frameworks: Frameworks in this category structure a study phenomenon into different categories which can have different characteristics in each category, e.g., the cultural dimensions from Hofstede and their characteristics in different countries to help information system developers understand the people they deal with [11]. There are 10 frameworks in this category.

  • Factors-outcome Structured Frameworks: Frameworks in this category take relevant factors into account and determine how these factors influence the outcome of a phenomenon, e.g., the IS flexibility framework shows which factors influence IS flexibility and how they influence this phenomenon [1]. 15 frameworks belong to this category.

  • Component Structured Frameworks: Frameworks in this category have a component-based structure. The components describe the framework and the relationships between the components are shown, e.g., the conceptual framework linking Enterprise Systems to organizational agility which consists of enterprise systems related components and how they are connected to organizational agility [14]. However, the difference to sequenced and technical structured frameworks is that the components are not technical and there is no sequence of activities between the components. Six frameworks belong to this category.

  • Mixed Structured Frameworks: Frameworks in this category have structural attributes from multiple categories. Seven frameworks belong to this category.

  • Not Categorizable Frameworks: These frameworks do not fit into any of the above-mentioned categories. Thus, nine frameworks are not categorizable by their structure. Each of the nine framework has its own structure category because their structure is unique compared to the frameworks being categorized.

The following Fig. 5 shows the distribution of categories based on the framework structure in percentages. The frameworks are relatively even distributed compared to the other two categories. Most frameworks are factors-outcome frameworks that make up the majority of identified frameworks (22 %).

Fig. 5.
figure 5

Structure in percentage

Full size image

5 Summary

The term ‘framework’ is frequently used in the field of information research. We could show that there are many different definitions of the term focusing mainly on the purpose of the framework and that many different frameworks were published.

The purpose of the performed research was to find out how frameworks in the field of information systems are handled in the scientific literature, i.e., in the most important journals and conferences of the field of information systems of the last ten years. To get an objective overview, a systematic literature review according to Kitchenham was used. First, 353 studies appeared relevant for this analysis. After a refinement, 71 papers with 93 frameworks were identified. Afterwards, relevant information about the frameworks was extracted and analysed to answer the research question. In the introduction, we stated that there is no consistency or any rules whatsoever in the usage of the term framework. There is also no accepted definition of what a framework is. Therefore, we proposed a definition of the term ‘framework’ extending the Oxford Dictionary version. This definition of a framework has to be very broad due to the many different kinds of identified frameworks.

The identified frameworks vary considerably in all their aspects as shown by the categorization of frameworks. It seems to be useful to categorize framework according to purpose, development process and structure.

It has to be noticed that only a few frameworks were implemented. In total, 10 frameworks were implemented in the studies. However, most of the implementations are prototypes. Only one framework was fully implemented in a real world application. It can be observed that only frameworks classified as test frameworks and development framework with the focus on technical aspects have been implemented.

It seems that frameworks are rarely updated. Only nine studies dealt with frameworks being updated and released in a new version. It should also be noted that no framework category is more likely to be updated than other classes.

Unfortunately, some sources were not open access. This might be the main shortcoming of this research. Nevertheless, all of the high-ranked journals and conference proceedings were freely available and included in our research.