Introduction

Abstract

This book presents the results of a 3-year international research cooperation on concepts and methods to improve the evaluation of e-participation in the area of sustainable development and climate protection.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1.1 How It Began

In November 2007, the European Science Foundation (ESF) invited scholars from all over Europe to a research conference entitled “Electronic Democracy—Achievements and Challenges” held in a former monastery in Vadstena, Sweden. Under the moderation of Herbert Kubicek about 40 researchers, well known seniors and young PhD students, tried for 3 days to identify the theoretical and methodological challenges for the future research agenda in different subareas of e-democracy, for example, e-consultation, e-petitioning, e-movements, e-voting, and more. They agreed that the biggest challenge in all of these fields is the evaluation of the deployment, use, and impact of the new electronic tools in their respective context.¹

The biggest barrier to valid assessment is the lack of comparability in existing research, which is mostly case oriented, providing a set of highly heterogeneous cases. There is a need for international and interdisciplinary comparative empirical research. As the effects of electronic tools are highly dependent on their context, it is necessary to compare similar tools in a similar context in order to detect success factors. Success can only be assessed and success factors can only be identified by comparing a number of cases with the same kind of participation on the same subject and by the same target group of participants.

The ESF offers the format of European Cooperation Research Projects (ECRP) for this kind of research under a two-step review process and with particular support for the cooperation between research teams from at least three different countries. It was during the conference that four researchers from Austria, Denmark, Germany, and the UK sat down together and started a discussion on what such a research cooperation might look like. We agreed that we were most interested in evaluating government-initiated processes of citizen consultation and collaboration. From previous research on this kind of participation, we knew that a salient issue and personal concern are the most important success factors for reaching a large number of participants. As this discussion coincided with the preparation of the UN Climate Change Conference in Copenhagen, we agreed that climate protection is an appropriate subject of individual concern and, at the same time, of European significance. In addition, the subject of fighting climate change has a methodological advantage because it allows the comparison of the impact of participation in a quantitative way via the measurement of the CO₂ reduction achieved.²

We also felt that the four of us only covered the northwest of Europe and that we should get a partner from Southern Europe on board. We invited the research team at the University of Saragossa, Spain, to join us and they accepted. The group of six senior researchers, Georg Aichholzer (AT), Jens Hoff (DK), Herbert Kubicek (DE), Ann Macintosh (UK), and Lourdes Torres and Vicente Pina (ES) not only represented different countries but also different disciplines, and thereby permitted a comprehensive and interdisciplinary concept of evaluation. We met several times in order to elaborate a coordinated research plan for an ECRP, which was submitted to the ESF in April 2008. The official title is “Comparative Assessment of E-Participation in the Context of Sustainable Development and Climate Change.” For outside communication we chose “e2democracy,” meaning “electronic environmental democracy.”

1.2 The Set Up

In order to assess the impact of electronic tools a quasi-experimental design seemed necessary, in which one group of citizens participated by traditional means (face-to-face, telephone, and mail) and a second group via the Internet. There should be more than only one project of this kind in each participating country. We agreed to find three local communities of different sizes in each of the participating countries. For the acquisition of cooperating local communities, the signatories of the Aalborg Commitments³ were considered to be the most promising, as they had committed themselves to having their citizens participate in the efforts of CO₂ reduction and to providing a regular monitoring. However, several of the signatories did not welcome the offer of a free evaluation of their activities and a complementary contribution to their monitoring, and it took some time for letters of intent to be included in an application for funding.

The joint application successfully passed the review process by the ESF, but an additional review by the national research funding organizations is necessary for the funding of each team. While the Austrian, German, and Spanish agencies; the Austrian Science Fund (Der Wissenschaftsfonds—FWF); The Deutsche Forschungsgemeinschaft—DFG; and the Ministry of Science and Innovation (Ministerio de Ciencia e Innovación) in Spain approved the funding, the Danish and the British agencies did not. Therefore, the remaining three teams had to take over the tasks assigned to the two excluded teams.

When the national research teams were established and the local communities which had provided letters of intent were informed, some were not willing or able to provide the necessary personal support for carrying out the project. In Germany, in particular, it turned out that the three cities which had signed the Aalborg Commitments and letters of intent to participate in the project had not conducted any kind of citizen participation in their initiatives against climate change, or any systematic monitoring including private households. Consequently, they were not able to raise the resources for setting up participation processes with their existing budgets or were afraid that they would not succeed in recruiting the desired number of 400 panelists. So, the search for participating communities had to start all over again. In Spain, the cities of Saragossa and Pamplona immediately upheld their commitment to the project. The third local administration which had signed a letter of intent, the Provincial Government of Biscay, was more interested in issues of water supply rather than CO₂ reduction when planning the details. The research team was willing to follow this priority but the results would not have been comparable. In Austria, an active involvement materialized for two of the contacts which had signed letters of intent to participate in the e2democracy (e2d) project. In the case of Bregenz, the definite agreement came about immediately, and in the case of Styria three regions had been considered for participation by the Regional Development Agency Styria⁴ (southeastern Styria, Schladming, and Mariazeller Land); the decision was finally made in favor of the latter within a rather short time span. Negotiations with our original third candidate, the city of Vienna, extended over several weeks, starting with the Local Agenda 21 Office which finally found it would be more appropriate the project be hosted by the city’s Climate Protection Coordination Office. After several meetings of scoping the fit with the Coordination Office’s strategies and activities it finally became clear that they preferred to pursue alternative formats of citizen participation and advice in energy efficiency as opposed to joining the e2d project. Initial attempts to continue the search for interested alternative candidate cities were finally stopped before too many resources were absorbed and delays incurred in view of lengthy negotiation processes already experienced.

So, finally the cooperation of seven local communities had been won. Three local communities joined the project in Germany (Bremen, Bremerhaven, and Wennigsen), while two could be won in Austria (Bregenz and Mariazeller Land), and another two in Spain (Pamplona and Saragossa). The two Spanish cities are the only signatories of the Aalborg Commitments in this sample.

1.3 The Basic Research Design

In each local community a cooperation agreement had to be achieved, including a commitment to CO₂ reduction between the administration, local businesses, and panels of citizens, as well as an additional consultation project on a subject in this area. The local administration was to take responsibility for the recruitment of citizens and business partners and for organizing and moderating the collaboration and consultation processes. The national research teams provided the electronic tools and user support and conducted the evaluations through surveys, interviews with local organizers, and observations.

As measures of CO₂ reduction include heating and electricity, they are season dependent. In order to assess any improvement, at least a 2-year period of measurement is necessary. Therefore, the field study was scheduled for 24 months. However, it was not possible to start with all the panels in the three countries at the same time. The first panel started in Bremen, Germany in December 2009, and the last one, because of delays in acquisition, in March 2011 in Wennigsen.

1.4 Data Collection and Analysis

Each research team took responsibility for different aspects of the evaluation and the respective research instruments.

The German team was responsible for the CO₂ calculator, which transforms energy consumption and other areas of consumption into CO₂ emissions. While it was appropriate to use the same calculator in Austria and Germany, it turned out not to be the case for Spain. Because of differences in the national energy mix and emission factors, the Spanish team elaborated their own CO₂ calculator based on the Austro-German one.

The Austrian team took responsibility for three rounds of surveys of participants of citizen panels, the German team for the participants in the consultation processes and the Spanish team for two rounds of interviews with local organizers and government managers involved in citizen collaboration. All the teams made proposals about the instruments proposed by the others, which were discussed and finally agreed upon in cooperation. Proposals for questionnaires and interview guides were submitted in English, discussed and finally agreed upon in cooperation meetings and then translated into the respective national language. For comparative analysis, the data collected in each country had to be integrated into a common database. Each team collected and coded data, which was to be analyzed by another team. It is no surprise that the consolidation, particularly of the CO₂ data, raised a lot of questions which had to be clarified between the collecting and the analyzing teams. The biggest practical problem was that comparative data analysis could only be started after the last panel had delivered its last measurement. Thus, some inconsistencies of the data from different panels became apparent when drafting the chapters of this book and clearing these inconsistencies led to delays in finishing the respective chapters.

1.5 The Chapters

Due to how the responsibilities were distributed among the three teams for different aspects of the research design and the evaluation, this book is organized by authored chapters. Each chapter has gone through an internal review by the partner teams and an additional external peer review.

Chapter  2 starts with a state of the art report on the evaluation of e-participation. Herbert Kubicek and Georg Aichholzer summarize the expected benefits of citizen participation by traditional means as well as electronic tools, introduce a distinction between information, consultation, and collaboration, present different approaches to evaluation, introduce the generic Input–Activities–Output–Outcome–Impact model as the conceptual framework for the evaluation exercises in the e2d project and adapt it for the evaluation of consultation processes. The chapter ends with the basic hypothesis underlying the whole project and this book, which is called a twofold relativity theory. Thereby, we mean that it is not appropriate to evaluate different kinds of participation processes with the same success criteria and measurement tools, but rather one has to take into account the differences between information provision, consultation, collaboration, petitions, and other forms, and develop tailored evaluation concepts and measurement tools. Even for a single kind of participation, there should be no universal evaluation, but a multi-perspective approach, taking into account that, for example, organizers and participants have different expectations and apply different success criteria when assessing the same process.

(e-)Participation in local climate governance is a key subject of several chapters in this book. For this reason, Chap.  3 by Georg Aichholzer introduces the development of public participation in climate governance and the theoretical background of various approaches of public engagement with climate change, energy conservation, and transition to a low-carbon society. A special focus is made here on behavior change interventions and rationales behind the participation format of the local climate initiatives studied in the e2d field study. The ongoing discourse on the role and limitations of behavior change in policy approaches is also touched upon (cf. Kurz et al. 2015).

Chapter  4 by Basilio Acerete, Ana Yetano, and Sonia Royo analyzes the websites of the environment departments of European local government signatories of the Aalborg+10 Commitments. It evaluates a first category of e-participation, that is, electronic access to information. The aim is to establish the extent to which the signatories make use of the Internet to promote e-participation and environmentally friendly behaviors among their citizens. The results of this chapter show that the developments in e-participation are higher in areas that only give information than in areas of interactive communication. This chapter shows that the Internet, as a tool to revitalize the public sphere, is still limited to countries with higher levels of transparency and a culture of citizen engagement.

In Chap.  5 , Herbert Kubicek applies the basic model for evaluating consultation processes which has been introduced in Chap. 2 to six different consultation processes and presents the respective research tools. Following the general idea of a twofold relativity theory of evaluating (e-)participation, as outlined in Chap. 2, three pairs of similar cases are subject to evaluation from different views, caught with different instruments:

• For comparing two one-stage consultation processes, which have been carried out online only in Pamplona and Saragossa, Spain, a template is applied for assessing success criteria and success factors by external observers, in these cases the research team, which based their evaluation partly on data collected from managers and partly on their own observations.

• Data collected by questionnaires for assessing aims and expectations of organizers at the beginning and their assessment at the end of the consultation are compared for two consultations on political documents in Vienna and Bremen.

• Finally, two consultations on local development and planning in Bremerhaven and Wennigsen, Germany, are compared, which have been carried out in two phases: one for idea collection and one for priority building. Participants were asked about their expectations at the beginning of the process as well as their assessments of the process and its results at the end. The final section of this chapter provides some methodological conclusions on the research instruments. An interesting finding in this respect is that the judgment of managers and participants varies with the point in time it is made, that is, before, during, or after the consultation process.

In Chap.  6 , Georg Aichholzer and Stefan Strauß introduce the special form of participation which is the focus of this research; it is generally labeled as cooperation, collaboration, or coproduction. The essence of such relationships between public agencies and citizens is to collaborate in policy-making and implementing policy decisions on shared goals. Prominent collaborative (e-)participation models are participatory budgeting, citizen assemblies, citizen panels, community councils, round tables, and similar procedures especially in domains such as spatial planning and local governance. Electronic communication and an expanding repertoire of Internet-based applications play an essential role in facilitating collaborative participation.

Chapter  7 by Georg Aichholzer, Doris Allhutter, Herbert Kubicek, and Stefan Strauß presents the approach and the empirical setting for the evaluation of a collaborative type of (e-)participation in local climate governance. The focus of the quasi-experimental field study is on assessing outcome and impacts. Common core elements of a set of similar participation processes characterized by a combination of individual and collective activities are explained and their potential effects and impacts are outlined. Checks for possible alternative explanations of impacts and potential biases caused by Hawthorne effects are addressed as well.

In Chap.  8 , Ralf Cimander, Ana Yetano, and Sonia Royo review the criteria used to select the most appropriate CO₂ calculator for the e2d project and explain the criteria applied to adjust the calculator for continuous measurement with various feedback functions. The chapter goes into the details of the different categories of CO₂ measurement analyzed during the 2-year period and also describes the actual functioning of the calculator, how participants interacted with it and the feedback provided to them. Finally, some challenges, such as lack of data about emission factors, interpolation, validation, and comparability, are also discussed.

The integrative framework tailored to a longitudinal evaluation of the citizen–government collaboration on local climate targets (see Chap. 2 and 7) distinguishes between process, output, outcome, and impacts. Chapter  9 by Georg Aichholzer, Doris Allhutter, and Stefan Strauß analyzes the relationship between process outputs, that is, the supply side of the e-participation processes such as the quantity and quality of devices and products offered to citizens for information, communication, and engagement, and the process outcomes, that is, the immediate effects of the output such as the number and activities of participants and their contributions. The chapter’s first part outlines the evaluation framework, specifies the inputs and activities that provided the basic setup for seven local (e-)participation processes, and describes their output. The second part presents the outcome of the citizen–government collaboration studied.

Chapter  10 by Georg Aichholzer, Dieter Feierabend, and Doris Allhutter is the first of three chapters on the impacts of collaborative (e-)participation exercises studied in the e2d project. The contribution investigates attitudinal and behavioral impacts of (e-)participation in the citizen panels collaborating with local governments in joint efforts to reduce CO₂ emissions. Attitudinal changes turned out to be greater than behavioral changes, which can partly be explained by the difficulties of changing social practices (e.g., nutritional habits) and local context conditions (e.g., transport options) as argued, for example, by Shove (2010). An investigation of the causal mechanisms and mediating factors suggests moderate “gentle nudge” type effects (cf. Thaler and Sunstein 2008) from CO₂ footprint monitoring among panelists. A number of community-related factors, such as social learning and reported removal of personal barriers through community support clearly showed a positive relationship to behavior change and underline the importance of community-based initiatives (Heiskanen et al. 2010).

In Chap.  11 , Ralf Cimander presents the results of the analysis on the ecological impact at individual level of the seven citizen panels in the local climate initiatives. The criterion used is the number of panelists who improved their own CO₂ balance by at least 2 % p.a. during up to 2 years of monitoring. A conceptual frame of reference developed by Wilber (2000) serves to explain different models and theories of individual action and behavior, and to compare results among the seven citizen panels which are explained against the background of relevant context factors.

Chapter  12 by Ralf Cimander, Sonia Royo, and Ana Yetano analyzes whether the cooperation of citizens as participants of citizen panels has had a positive ecological impact at a collective level, by contributing to the achievement of a 2 % annual reduction in the CO₂ emissions in their city or region. The authors suggest that a combination of different methodological approaches is the best option to assess the ecological impact at this level. Depending on the kind of calculation, some panels met the reduction targets completely, others only partially and one failed. It is an interesting finding that learning results are obtained after 1 year and that longer participation beyond this period does not seem to yield further savings but serves to prevent relapse.

In Chapter  13 , Ralf Cimander analyzes the extent of dropout in the seven citizen panels during the monitoring processes in order to understand the reasons why panelists stopped their participation and/or did not enter their data. The author identifies mainly two kinds of dropout: one group of participants who only registered themselves and withdrew before entering any data for the CO₂ monitoring process and one who dropped out during one of the actual monitoring periods. Explanations for both types of dropout are sought with reference to local context factors.

Chapter  14 by Vicente Pina and Lourdes Torres evaluates the effectiveness of citizen participation from the organizers’ point of view. It analyzes the experience of local managers in Germany, Austria, and Spain and their expectations about citizen participation in local government programs through an empirical survey focused on citizen participation in climate change programs. It seeks to compare the opinion of these managers, experts in climate change initiatives, about the impact of e-participation. This research contributes to better understand the opinion of managers about the success and failure factors of citizen participation in environmental programs.

In Chap.  15 , Herbert Kubicek examines the consultation and collaboration processes described in the previous chapters with a special focus on the communication channels and tries to answer the question, what difference the “e” made, that is, whether there are any differences in satisfaction and/or impact between participants who communicated face-to-face and those who used the Internet. After a short review of the relevant literature on media choice and effects, the assessment of participants and of organizers of the six consultation processes with regard to costs, effort, outreach, and effects of the two modes of communication is presented. Furthermore, perceptions and observations on the effects of both modes are compared for one consultation case in Bremen where online and offline modes were offered in parallel. Finally, the impact of the online and offline panels is compared with regard to the CO₂ emissions, the accuracy of the consumption data delivered by the participants and the dropout rates of the respective panels.

Chapter  16 by Herbert Kubicek and Georg Aichholzer concludes the volume. It provides a summary of major results and lessons from the empirical evaluation of examples of three different types of (e-)participation processes: access to e-information, e-consultation processes, and collaborative forms of e-participation. The outlook addresses the methodological contribution as well as the policy field-specific contribution of the results (the extent to which collaborative forms of citizen participation can enhance climate protection) and points to important qualifications under which the participation format studied in local climate initiatives can lead to positive impacts.