Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Since the widespread diffusion of personal computers and internet access, researchers and practitioners have developed an in-depth understanding on how information and communication technology (ICT) in particular the internet can be used to improve the relationship between government and society [1]. While e-government research during its early days was primarily concerned with improving the efficiency and effectiveness of government service provisioning (“managerial approach”), the advent of social media has opened up unexpected new opportunities of engaging the public in government work (“participatory approach”) [2]. Within the last years, governments around the world have started several open government initiatives which aim to transform the relationship between government and society by “establish[ing] a system of transparency, public participation, and collaboration” [3].

At its core, the Open Government initiative is manifested under three integrating themes [4]. First, Open Government aims at increasing information transparency. By providing citizens with access to government information, the public may better understand the workings of their government [5]. At the same time, having access to Open Government data empowers the public to hold government accountable for its policy and service delivery performance. Thus, increasing accountability is another key theme of Open Government. Third, Open Government aims at increasing public engagement [6]. Social media-based citizen-sourcing strategies can be utilized to engage the public in governmental policy processes and service delivery programs [7]. Through participation and collaboration governments can give the public the opportunity to exert influence on the outcomes of government processes [8]. Research on Open Government has produced conceptually interesting models on how Open Government initiatives will or should evolve using mostly anecdotal evidence from best practice cases. E.g., Lee and Kwak [8] prescribe government organization to conduct “ubiquitous and continuous engagement” with the public. In this mode, the public engages in various government activities through multiple social media channels. Thereby, “Open Government becomes the norm for government culture” and the benefits of transparency, participation and collaboration are fully realized [8]. Similarly, Kalampokis et al. [5] propose that government organization should integrate their data with non-government formal and social data to improve the overall transparency of government activities (more examples are provided in the next section).

While this guidance might be to an extent useful for practice, most research on Open Government assumes that more Open Government is always better without reflecting the assumptions and implications underlying this new “system of transparency, public participation and collaboration” and the consequences of the IT artifacts themselves [9]. We argue here that in order to empirically evaluate Open Government initiatives—which we believe should be a central concern of research in the future—we should rethink the assumptions underlying our models in order to produce more rigorous and relevant Open Government research. The goal and expected contribution of this paper is threefold. First, we aim to critically reflect previous research and uncover biases in favor of implementing Open Government. Second, by using empirical evidence from the field we show that Open Government portals have an impact on the perceptions of people and that this effect may be also negative. Third, we conclude that we need more research that evaluates the impact of Open Government for the authorizing environment (in particular minorities), government organizations, politicians and the political system as a whole and come up with a research agenda for evaluating open government initiatives.

The remainder of this paper is structured as follows. First, we introduce two distinct and opposing political perspectives on how government can retain legitimacy and support which we assume is the central goal of public managers. Based on these perspectives, we analyse previous research with respect to the perspectives employed. In section three, we introduce the case of an Open Government initiative at a German university to illustrate why we need more research on the consequences of implementing Open Government and need to provide more rigorous and relevant knowledge for overcoming practitioners’ concerns. The last section introduces our ideas for a research agenda on how the identified gap in the literature could be successfully addressed.

2 Evaluating Open Government Initiatives

2.1 The Kantian Perspectives

In his book “Groundwork of the Metaphysic of Morals”, the renowned philosopher Emanuel Kant (1724-1804) provides major insights into the relationship between morality and politics [10]. Kant describes two different strategies for retaining legitimacy and power—two pertinent goals of governments. On the one hand, he describes the character of the “political moralist” who focuses on getting people to do things with specific ends in view. This view allows a power holder to manipulate, lie or deceive people in order to achieve certain outcomes (such as peace or political stability). On the other hand, the “moral politician” is a character assuming that moral actions will necessary lead to moral ends. Kant argues that the position of the moral (or honest) politician is superior to the “political moralist” since straying from morality requires that “empirical conditions which permit the proposed end to be realized can be assumed to exist” [10, p. 122]. Thus, because of the unpredictability of achieving beneficial outcomes through immoral means (e.g. concealment of government data to deceive the authorizing environment (citizens)), straying from morality (i.e. openness and transparency) during political activity cannot be justified [11].

The normative implication of Kant’s view on politics and morality is that transparency, accountability and public engagement in government work should be categorical imperatives for a public administration if legitimacy and retention of power are central goals of decision-makers as we assume in the following.

2.2 The Machiavellian Perspective

A political philosopher who would not agree with Kant is Niccolo Machiavelli. In his famous book “The Prince” he acknowledged that a power holder who is honest and faithful to his word will be admired by the society but that this admired authority is not always the most successful [12]. Compared to Kant, Machiavelli has an opposing view on the prospect for the “political moralist” and clearly prioritizes the individual benefit of the power holder above the collective benefit of society. If we take this self-serving perspective, we have to consider the individual advantage that public manager have when adopting Open Government initiatives.

Compared to their private sector counterparts, public managers have to cope with the misalignment of financial performance, organizational survival and social value [13]. Private companies can focus solely on their financial performance and in doing so be sure that they guarantee their survival and the production of social value. In contrast, a focus on financial performance is not sufficient for public managers. Moore’s [13] strategy framework for the public sector suggests that managers should focus on three different calculations. First, the value propositions should be clearly formulated and disseminated, that is the social mission plan of the public administration. Second, public managers have to ensure that they have sufficient legitimacy and support in the authorizing environment for their value proposition and how the organization operates. Third, governments have to ensure that sufficient know-how and capability (sourced from inside or outside the organization) are existent to achieve the desired results.

The implication of the Machiavellian perspective is that Open Government initiatives aiming at transparency, accountability and public engagement by the means of open data or citizen-sourcing strategies are not good by default but have to be evaluated based on their consequences for government (cf. Fig. 1).

Fig. 1
figure 1

Open government strategy framework (inspired by [14])

Full size image

2.3 Previous Research

We conducted a structured literature review to create a full picture of previous literature on the evaluation of Open Government initiatives (cf. Table 1). Our search was conducted within the core outlets of e-government research [15]: Electronic Government—An International Journal; Electronic Journal of E-Government; Government Information Quarterly; Information Policy; International Journal of Electronic Government Research; Journal of Information Technology and Politics; Transforming Government: Process, People, and Policy; E-Government track at Hawaii International Conference on System Sciences; DEXA EGOV; and DBSNA’s dg.o conference. Overall, only five papers have been found that directly address the evaluation of Open Government initiatives. Papers on the adoption (why is Open Government adopted?) were excluded from the analysis. We used a concept matrix [16] to structure our findings.

Table 1 Current research on the evaluation of open government initiatives
Full size table

Our results suggest that both open data and citizen-sourcing have been in focus of previous research. Access to public data (Open Data) is valuable for citizens, organizations, and businesses for both democratic as well as economic reasons. On the one hand, open data is expected to improve the public’s ability to hold government responsible. Accountability portals such as Recovery.gov report how public funds are being spent by recipients [5]. This allows citizens to monitor how their money is spent and may better legitimize the collection of taxes at the end. On the other hand, open data is expected to foster the creation of innovative products and services. Data portals such as data.gov provide raw data about unemployment statistics, traffic, job offers or geographical data [17]. By inviting private companies to access and use government data, society can benefit from new value-added services and products provided to citizens for marginal costs. In the light of these expected benefits, Kalampokis et al. [5] provides guidance on how to aggregate and integrate government data and increase the value for the public. A cross-country comparison of the accessibility of public data is presented by Alanazi and Chatfield [18].

The term citizen-sourcing was introduced by Nam [7] but principally captures all participatory and collaborative elements of Open Government initiatives. Citizen-sourcing refers to the process operated by governments that involves out-sourcing certain tasks to a distributed group of citizens. According to Nam [7], there are two main purposes for government to outsource small tasks using ICT. On the one hand, citizen-sourcing initiatives can be used to improve the image of government as an adopter of modern ICT such as wikis, forums or social networks. On the other hand, government can benefits from citizens’ enthusiasm to participate in mass collaboration projects [7]. Thereby, government can crowd-source their problem to citizens and benefit from the wisdom of the crowd.

Apart from Nam [7], all studies at least implicitly assume that more Open Government is better (Kantian perspective) indicated by the fact that the frameworks assessing the initiatives using metrics such as number of functionalities (Alanazi and Chatfield [18]) or number of search engine results (Bertot et al. [17]) or propose maturity models (Kalampokis et al. [5]; Lee and Kwak [8]). Only Nam’s [7] evaluation framework emphasizes that citizen-sourcing should be evaluated with respect to the outcome. However, the paper is only conceptual with only a short sketch on how the outcome should be precisely evaluated. In a nutshell, no empirical study exists that evaluates the consequences of Open Government initiatives. Our assertion is that public managers will tend to resist implementing Open Government if the impact on relevant outcomes such as legitimacy and support as well as operational capacity is widely unpredictable (Machiavellian perspective).

3 Limitations of the Kantian Perspective: The Case of Open Government at a German University

3.1 Methodology

Case Background

The introduction of tuition fees in Germany has been a controversial political issue. Particularly controversial is the question of whether tuition fees are socially acceptable and to what extent students should share in the funding of research and teaching. In 2007, the state of Baden-Württemberg introduced tuition fees of 500€ per semester for all students with few exceptions. The primary objective of the funds is to improve conditions for studying and teaching. In 2010, the University of Mannheim—a public university—received around six million euros from tuition fees. The primary objective of the funds is to improve conditions for studying and teaching. A focus of the measures financed by tuition fees is to increase the range of courses and supporting social events. At the Faculty of Social Sciences, for example, additional teaching assistants were hired to reduce the size of the group seminars. Students of the Department of Law were able to benefit from free scripts and economics students benefited from extended opening times to work in their computer lab.

With the introduction of tuition fees, the university administration has gathered all the information about how the funds from the tuition fees were used. Similar to activities by the US federal government (cf. recovery.org), the information about the use of tuition fees for improving studying and teaching conditions has been made available to the public through an Open Data portal. Through the website, students were able to monitor how their tuition fees were spend and whether they were spend for the intended purpose, i.e. the improvement of studying and teaching conditions at Mannheim University. The goal of the initiative was to increase transparency, accountability and public engagement in the tuition allocation process. Thus, the insights gained in this context can also be informative for future open government initiatives with similar objectives.

Survey Development and Deployment

In order to evaluate the impact of the Open data portal, an online questionnaire among students who paid tuition fees at the university was admitted. Inspired by Bertot et al. [17], we asked students about their awareness and degree of use of the Open Data portal but also their evaluation of the tuition fee allocation process. Awareness was operationalized as the degree to which users were conscious that the information about the allocation was provided online. Use was measured as the degree to which users have visited and used the platform. Users’ evaluation of the tuition allocation process was assessed based on the extent to which students belief that the administration has attributes that are beneficial to them (trusting beliefs, cf. Pavlou et al. [19]) and the extent to which they belief that the process of allocating tuition fees was fairly governed (procedural fairness, cf. Herian et al. [20]). Thus, the survey intends to explore the impact of the awareness and use of the open data platform on relevant relational outcomes and on the overall evaluation of the allocation process (cf. Fig. 2).

Fig. 2
figure 2

Relational impact of open data initiative (Note: ** = p < 0.001)

Full size image

The measurement scales were developed based on previous literature and pilot tested with 20 participants (cf. Appendix → Table 4). After the pre-test, respondents were asked for their feedback regarding comprehensibility of instructions and questions, overall time, and other issues they experienced. Based on the results of the pre-test, the measurement instruments were shortened, revised, and validated for its statistical properties.

The final survey was conducted online in October 2012 and was sent out to members of a university Facebook group composed of Bachelor, Master and PhD students from the University of Mannheim. An Amazon voucher was set as an incentive to participate in the survey. Overall 83 responses were collected. Furthermore, students which had never paid tuition fees at the University of Mannheim were removed. In the end, 70 responses were assessed as usable for further analysis. Respondents were 43 % female (67 % male) between the ages of 19 and 34 (mean: 24.5).

Data Analysis and Results

The descriptive statistics, correlations and average variance extracted for all constructs are depicted in Table 2. Our statistical tests suggest that the measurement models are valid and reliable [21–23]. For all constructs, composite reliability is above 0.9 which indicates a satisfactory level of internal consistency reliability [23]. The validity at the construct level is assured when the latent constructs account for the majority of the variance in its indicators on average. Accordingly, average variance exceeds even 0.7 for all constructs [22].

Table 2 Descriptive statistics and average variance extracted
Full size table

Discriminant validity of the constructs was confirmed by two methods. Fornell and Larcker [21] suggest that the square root of the average variance extracted for each construct is higher than the variance that the construct shares with every other construct in the model. This criterion was fulfilled for every construct (cf. Table 2). Furthermore, we conducted the between constructs test recommended by Anderson and Gerbing [24]. The differences between the two chi-square statistics for each pair of constructs were significant (α = 0.01), implying that the constructs are empirically distinct.

Bootstrapping with 200 subsamples was conducted to estimate the significance of the PLS path coefficients and to compare the path estimates statistically. The results of the structural model testing are presented in Fig. 2. Apart from the relationship between use and trusting beliefs in administration (−0.210; p > 0.01), all assumed relationships were highly significant.

3.2 Discussion

Our results suggest that the deployment of an open data platform has a significant impact on relevant relational outcomes in the context of a German university administration. While this finding might be not surprising, we try to show how we can differently interpret the results depending on the political perspective adopted.

The Kantian Perspective

The positive influence of awareness on trusting beliefs and procedural fairness shows that students reward the deployment of an Open Data portal. Students who know about the possibility to monitor public spending have a better trust and fairness perception than those students who are not aware of this possibility. Society’s trust and confidence in government is important for many reasons. Trust and fairness perceptions are crucial for the legitimacy and stability of the political system [2]. If people’s distrust or feel treated unfairly, their willingness to accept the government authority will also decline. Trust in government also encourages people to comply with laws and regulations. Overall, if we believe in the integrity, competence and benevolence of processes, we are much more willing to accept the outcome.

The negative influence of use on trusting beliefs and procedural fairness (although the latter is not significant) suggests that students who were actually engaging with the allocation of tuition fees were very dissatisfied with how the university administration operates. However, from a Kantian perspective the inappropriate allocation of funds from tuition fees would have come out anyway, at least in the long run. Recent phenomena such as Wikileaks—organization publishing classified media from anonymous news sources and whistleblowers—show that the widespread use of ICT in government make it harder for government to keep information secret. In fact, the lesson of Wikileaks might have made power holders aware that no secret is safe and that the only efficient defense against leaks is transparency and openness. Seeing it from this Kantian perspective, we should see the negative influence of use on trust and fairness perceptions as an alarm signal for government to improve the allocation processes and include students directly in decision-making in order to retain their legitimacy. Public manager following this logic would view Open Government as a tool for receiving direct feedback from the public on their work.

The Machiavellian Perspective

The Machiavellian evaluation and interpretation of our result is quite different. The positive influence of awareness on trusting beliefs and procedural fairness suggests that Open Data can be used as a tool that improves the legitimacy and support of government. However, public managers have to be aware that the implementation of Open Government may very well backfire. The introduction of tuition fees in Germany was highly controversial. Therefore, the allocation of these resources is subject to critical public scrutiny. A potential threat for Open Government initiatives is that these projects can also get out of control. The challenge lies in finding the right balance between government control and public autonomy. As our case shows, public engagement (use of open data portal) does not always lead to intended results (increase of legitimacy and support). One potential explanation is that students are misinterpreting the information provided by the administration or have only limited capacity to understand the legal or practical context of the resource allocation process. In that case, the administration could oppose the problem by improving the design and content of the open data portal. Another much simpler explanation is that the resource allocation is in fact unfair. The administration might misspend the public funds for useless activities. In that case, the government should stop or change back to the mode of secrecy before the students withdraw their confidence in the university administration and their bodies.

4 Towards a Research Agenda for Evaluating Open Government Initiatives

Our structured literature analysis reveals that research on the evaluation of Open Government initiatives in still at an emerging state. Moreover, the frameworks developed for evaluating Open Government initiatives assume that more openness in government is always better. Seeing this as a major limitation (cf. also the arguments provided by Coursey and Norris [9]), we argue here that previously developed evaluation frameworks (adopting a Kantian perspective: cf. Kalampokis et al. [5]; Lee and Kwak [8]) are only of limited practical relevance. Rather, practitioners strive for understanding the individual short- and long-term consequences of openness and transparency in government enabled by IT. As a research community, we should not solely focus on proposing new IT artefacts or solutions how to open up government but should be very explicit about the consequences for the authorizing environment (in particular minorities), government organizations, politicians and the political system as a whole. A similar perspective on the goal of IS research is proposed by Agarwal and Lucas [25] who request IS researchers to “focus on the impact of the IT artifact rather than the artifact itself” [25].

Based on our insights from previous Open Government research, we derive a research agenda that is twofold (cf. Table 3). First, we focus on who has actually access to Open Government services (who is affected?). Thereby, we can build on digital divide research which might provide the conceptual and theoretical basis for this avenue [26]. Second, we focus on direct consequences of offering Open Government services (how is the impact on those who are affected?). As our literature analysis shows, there is a lack of empirical research on the ultimate consequences of Open Government services on the behaviour and perceptions of society towards government. We believe that the understanding will provide a more accurate and useful guidance for practice in starting and developing Open Government initiatives than the guidance that maturity models provide which assume that transparency, collaboration and participation are categorical imperatives.

Table 3 Towards a research agenda for evaluating open government initiatives
Full size table

In order to estimate the impact of Open Government initiatives we first need to understand who has actually access to Open Government services and is thus, affected. Thereby access to Open Government services can be seen a “process with many social, mental and technological causes and not […] a single event […]” of using a particular Open Government service [27, p. 224]. Before people get access, they have to first recognize that it is available to them and then determine whether Open Government services are relevant to their interests or purposes. In von Dijk’s [27] framework, motivational access is placed in front of physical access (which can be neglected in our context) followed by skills access and usage access. Thus, users need also sufficient skills to make use of Open Government services. Taking this view, researchers should analyse the social, mental and technological causes of access to Open Government services. Moreover, we have only limited knowledge about who are the active users of Open Government services. Having elaborated the meaning of access to Open Government, research should also investigate the factors that determine why some groups engage more successfully with Open Government, whereas others do not. Informed by this understanding, guidelines should be developed on how Open Government initiatives can be better democratized so that all groups of society have the same capabilities to benefit from and participate in government work.

Having examined who is affected, we need to focus on the consequences of Open Government initiatives on those who are aware and participate in Open Government. Therefore, we need a more explicit and elaborated discussion on the desired outcomes of Open Government initiatives. On the individual level, trust in government or procedural fairness might come into question as proposed in our study [2]. Moreover, one might have to distinguish between different types of Open Government initiatives. While data portals might aim at fostering innovative information products and services to boost the information economy, responsibility portals might aim to improve trust in government. A more elaborated taxonomy of Open Government initiatives could be a first step to systemize evaluation frameworks. Moreover, once the quest for the dependent variable is resolved and assumptions are reflected and made explicitly, research should focus on the different contexts that Open Government initiatives operate (e.g. low or high public scrutiny) and the variations of success with respect to the pre-defined outcomes in these different contexts.

5 Limitations and Concluding Remarks

The goal of this paper was to propose a research agenda for evaluating Open Government initiatives. Therefore, we critically analyse previous Open Government research with respect to two distinct and opposing political perspectives. Our argument is that we need to conduct more research on the impact of open government initiatives in order to provide more rigorous and relevant insights for practitioners. The proposed ideas aim to be the basis for a fruitful discussion at the conference in order to develop a more comprehensive agenda for future research on the evaluation of Open Government initiatives.