Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

16.1 Introduction

The previous chapters present the results of research in the e2democracy project (e2d) aiming to close the evaluation gap in the field of e-participation by focusing on the policy field of climate protection. The reason for choosing this subject area was the assumption that this subject is a salient issue and that there is a possibility to assess the impact of participation processes quantitatively in terms of CO2 reduction achieved.Footnote 1 Hence, our research allows for insights into two different scientific fields:

  • The methodological issue of exploring and testing an appropriate concept and effective methods for the comparative evaluation of different kinds of (e-)participation processes, i.e. information, consultation and cooperation processes.

  • The policy-field-related issue of whether and to what extent collaborative forms of citizen participation can contribute to climate protection.

In this chapter, we summarize the results achieved and the lessons learned for both issues.

16.2 Applying a Generic Evaluation Framework

Following the OECD classification, three different kinds of citizen participation may be distinguished (OECD 2001, 15 ff.):

  • Access to information

  • Communication or consultation

  • Active participation or cooperation

For each of these categories a variety of methods is available (Table 2.2 in Chap. 2) and applied in participation processes initiated by governments on all levels dealing with different objectives, a wide range of subjects and addressing different groups of citizens. But little is known about the degree to which the respective objectives have been achieved and what led to success or failure.

16.2.1 A Twofold Relativity Theory of Evaluation

We assume that it is not promising to aim at a general standardized yardstick to assess the success of each and every participation process and that there are at least two different purposes of an evaluation, which call for different approaches.

Governments and/or politicians as organizers of exercises pursue certain objectives when they initiate a participation process, and citizens who participate have their own expectations of what can be achieved and gained. If at the end both parties want to know whether their objectives have been achieved and their expectations have been met, an assessment should be made against exactly these individual objectives and expectations. There is no practical use in imposing other success criteria on such an individual process. However, in order to learn from these individual exercises, to gain some general insights and to draw some consequences for future participation processes, it is necessary and possible to classify success criteria and success factors.

After looking at several classifications of success criteria and characteristics of participation processes, we have selected the Input-Activities-Output-Outcome-Impact model (see Chap. 2) as the most promising generic framework which allows the range of individual objectives to be classified by an important but often neglected differentiation. Quite frequently, the success of a participation process is measured by the number of people who come to a meeting or visit a website. But this is not the main objective. Rather the contributions they make in this process, the change that is initiated and the effects on the organizers, the participants and in the policy field count. Therefore, it is important to distinguish between output, outcome and impact.

We maintain that any participation process, which is initiated by a government or by politicians, providing information to citizens, consulting them or asking for cooperation, employs some kind of input, runs through certain stages of a process, produces an output, which finds some kinds of resonance among or is more or less used by the participants (outcome) and hopefully has an impact on the organizers, the participants and the policy field. Therefore, these categories can be used to collect and map the success criteria and success factors of individual participation processes and to make a comparison between different processes. Such a collection in turn provides a pool of indicators for tailoring an assessment of new individual processes or for a comparative assessment of a number of similar processes.

However, when selecting indicators we have proposed a twofold relativity theory. By this, we mean that account has to be taken of the differences between information provision, consultation and cooperation processes, and that within each category there is no absolute, quasi-objective result. Rather a multi-perspective approach is recommended, taking into account the fact that, for example, organizers and participants have different expectations and apply different success criteria when assessing the same process.

16.2.2 Appropriate Research Methods

The evaluation of (e-)participation processes uses a variety of research methods. An evaluation of information offered to citizens by print or electronic media is frequently done by content analysis of documents and websites and less often by surveys of recipients or users. Consultation processes are mainly evaluated by surveys of participants and interviews with organizers about objectives, expectations and achievements. Similarly, cooperation processes are assessed by interviews, surveys, protocols, diaries and participant observation.

In evaluating the different kinds of participation processes initiated and studied in the e2d project, it became obvious that for the assessment of output, outcome and impact not all of these methods are equally appropriate and that the point of time when an assessment is done plays an important role.

  • Output in terms of the information provided or the number of meetings and events offered can easily be counted.

  • Outcome can be measured by the number of participants, their representativeness and/or the number of contributions and the content and quality of these contributions.

  • Impacts can occur at different levels and different points in time. Changes in attitudes of organizers and participants may be assessed most easily, but changes in behavior some time later and the assessment of any impact on the issue at stake, for example the quality of a decision or as in our case climate protection, needs much more effort, may occur with a time-lag and most often is influenced by other factors as well.

If we take a multi-stakeholder approach as recommended for evaluating a consultation process, ideally there should be interviews with organizers and a survey of participants about objectives and expectations at the start. At the end of the exercise, there should be an analysis of input and output, interviews with organizers about the number and composition of participants and the quality of their contributions, as well as surveys of participants about the extent to which their expectations have been met and how far their attitudes towards the organizer’s institutions or issues at stake have changed (e.g. trust in government agencies and/or politicians; appropriateness of policy measures) and their competence has improved. Wherever possible the subjective assessment of impacts by participants and organizers should be matched by assessments and data from third parties or other sources. However, in practice most often such a comprehensive approach is not possible because of limited resources or other constraints, and the process for assessing e-consultation exercises itself usually does not allow for an ex ante survey of participants. Therefore, some kind of compromise has to be made for each kind of participation.

16.3 Findings on e-Information

The evaluation of e-information is such a case of limited possibilities for a comprehensive evaluation. From users and citizens, respectively, it is only possible to collect and analyze login data (outcome) and to offer an online questionnaire at the end of the session with questions about expectations and satisfaction. Organizers may well be interviewed about their objectives and perceived achievements, but their perception can only partially be matched with the assessments by the users as the response rate of online surveys is usually very low and not representative.

However, our analysis of websites providing e-information in Chap. 4 was not directed at evaluating a participation tool as such; it rather aimed at assessing European local governments’ environment departments’ compliance with the obligations under the Aalborg + 10 Commitments which they had deliberately signed. The results show that more signatories are only providing information rather than offering interactive communication. The websites were rated on the four dimensions of transparency, interactivity, usability and sophistication. If the provision of information or interactivity becomes more developed and requires greater effort by the local governments, the level of adoption decreases. Thus, the creation of an interactive e-dialog still seems to be a pending issue for European local governments fighting against climate change.

The public administration style seems to condition the level of development of environment-related e-participation initiatives among European local governments; Anglo-Saxon, Nordic and Germanic cities are among the leaders in this regard. Local government using the Internet as a tool to revitalize the public sphere is still limited to those countries with higher levels of transparency and a culture of citizen engagement.

16.4 Findings on Consultation Processes

For the evaluation of consultation processes, we have adapted the basic model in Chap. 2 and applied it to the assessment of six different consultation processes employing different methodological approaches and research designs.

16.4.1 Informal and Voluntary Consultations

The six consultation processes under investigation can be called informal or voluntary. There was no legal obligation on the organizers to conduct such a consultation and there was no legal right of the citizens to be consulted before a decision was taken. The results of the consultation were in no way binding on decisions taken by the organizers. The main differences between the consultation processes are their objective and the number of stages. In general, a consultation by a government agency is undertaken either to collect ideas for solving a problem at stake or to get feedback on predefined alternative options for solving a problem. Depending on the objective, different process designs are appropriate. For example, for collecting ideas, open questions should be formulated, comments should be possible on ideas by other participants, and registration should not be necessary. In contrast, getting feedback for setting priorities is a kind of voting. Alternatives should be defined exactly, registration is necessary to avoid double voting etc. In both cases an assessment by organizers and participants can be made before and after the exercise or only afterwards.

Among our own six consultation processes the ideal two-stage multi-stakeholder before and after assessment was only possible in two cases. For two one-stage consultation processes (in Pamplona and Saragossa carried out online only), a template has been applied for assessing success criteria and success factors. This was done by external observers, in these cases the national research team, partly based on data collected and partly based on its observations. The final assessment by the researchers was chosen because interviewees from two different departments involved in the consultation did not agree whether it was a success or not. The lesson learned from this experience: As the result of an evaluation depends on who is asked and as within a government agency most often different departments and levels are involved, assessments should be made by a representative sample of organizers.

In two other one-stage consultation processes (dealing with political documents in Vienna and Bremen), it was possible to assess the organizer’s aims and expectations at the beginning and at the end of the consultation as well as the participants’ views at the end. Again we found different assessments by different organizers, for example diverging views between political leaders and staff in the case of the consultation on the Bremen SPD government program. As this is in line with other reports, the recommendation is to include in any case political leaders and administrative heads of units and operative staff in an evaluation exercise.

In both cases, there was a combination of face-to-face meetings and online consultation and dialog. As regards citizens’ expectations, it was not possible to get both ex ante assessments and reviews at the end. For online dialogs, it is in practice not possible to get a questionnaire answered before people have read the webpages and entered the dialog. In the case of physical meetings it seems possible to ask for expectations at the beginning by distributing questionnaires on-site, collecting the replies and providing another questionnaire for assessments at the end of a meeting. But we were skeptical of the response rate and used only one questionnaire to be completed at the end of the meeting, asking ex post for the initial expectations and the extent to which these were met. Of course there is a risk of a bias when the original expectations are reported in the light of the experience made in the meantime. A more realistic method for a before and after assessment could be an electronic assessment tool used by the moderator (interactive response software or polling app). At the start of a meeting, he or she can put a few questions about expectations on a screen, and participants are given small handheld keypads and enter numbers for predefined answers or can download an open source app and use their mobile phones.Footnote 2

Finally, in two cases, in the German cities of Bremerhaven and Wennigsen, there was a two-stage consultation with a first phase for collecting ideas and a second phase assessing the results in order to set priorities for implementation. Organizers were interviewed at the start of phase 1 and at the end of phase 2, and participating citizens completed questionnaires before and after. Therefore, a comparison between expectations ex ante and a separate ex post assessment of how far they have been met was possible, but not on an individual level, as there was no identifier on the questionnaires distributed in the meetings. A more serious problem is posed by the fact that the composition of participants between meetings and even more between the two stages usually changes. Moreover, in the two cases investigated, some people who reported their expectations at the start did not show up later, while others took part in the evaluation at the end who had not been present at the beginning.

This can raise questions as to the validity of the final survey. However, with regard to the impact of participation and future behavior, it is not relevant what citizens expected before the consultation, but only what they think at the end, whether their expectations and aspirations have been met, regardless of what they said months earlier. Therefore, in view of the cost of conducting an evaluation, for a final assessment it is sufficient to conduct only an ex post survey and to ask about the extent to which expectations have been met or missed. Ex ante surveys, however, are relevant as input for organizers in order to design and adapt an ongoing process to the expectations of the participants.

There remains an almost unsolvable limitation to the assessment of the impact of such processes. Participants are asked when the results of the second consultation phase are presented in a meeting and/or online. However, at this point in time it is not clear whether the decision-making entities and boards will adopt these priorities. There may be legal, financial or political reasons for ignoring the options with the highest number of votes. The implementation may take months, and most often there is neither a final report to the public nor a justification of why high-ranking proposals have not been realized. This has an influence on the participants’ satisfaction with the process, their trust in the organizing bodies and their future engagement. However, participants in earlier meetings can usually not be reached any more. Participants in an online survey might be asked again after some time, if they have left an e-mail address, but only a small number of participants are ready to do so. Another option may be a telephone survey after the implementation of the decisions taken. But this only seems appropriate if the participating people can be located and if their share of the population in the respective area is significant.

16.4.2 Transfer to Formal Participation Structures

All the consultation processes initiated in the e2d project were voluntary and informal in the sense that there was no legal obligation and no specification or regulation on how to do it. More relevant for sustainable environmental development and also with regard to climate change are mandatory participation procedures defined by law which lead to binding decisions and can be contested before a court. At present, the legal provisions in Germany and Austria require the participation of organizations representing the public interest, but increasingly they are being supplemented by aspects of the involvement of interested citizens as well. The federal state of Baden-Württemberg recently issued a directive recommending its public authorities to conduct citizen participation before and alongside the formal participation procedures (Erler and Arndt 2014). In the case of new supply networks for the transport of electric power from wind energy parks in the North Sea to the industrial plants in the southern parts of Germany with 28 regional subnets, the Ministry of Economics and Energy and leading politicians are encouraging the regional network providers to involve local citizens as early as possible, and the coordinating Federal Network Agency (Bundesnetzagentur) has also started a dialog.Footnote 3 A set of laws regulates the whole process in four stages, the planning of demand, the national network development plan with a broad north–south alignment and the 28 regional projects, and the routing decisions to be made in two stages. First a corridor of about 10 km is specified, for which a broad environmental assessment is made. If the latter is positive, the exact routing is planned on the basis of many detailed expertises and surveys. The two stages may each take several years. Politicians and the Federal Network Agency expect there to be a greater acceptance of power pylons by local citizens if they are involved in the planning procedures, but there is no proof of this so far, and some serious doubts can be raised (Kubicek 2014a; Schweizer et al. 2014).

There is also hope for greater acceptance of infrastructure projects through citizen participation in the field of road and motorway planning. The German Federal Ministry of Transport has issued recommendations for citizen participation (BMVI 2012) The German team has accompanied and evaluated two projects in this field for 2 years, with formal stakeholder consultation having been amended by an informal citizens’ dialog with citizens’ initiatives and action groups and in one case with randomly selected citizens. Based on the generic Input-Activities-Output-Outcome-Impact model, organizers and the participants in a steering group or, as the case may be, citizens board including local action groups, were interviewed at the beginning and the end, and surveys were conducted with citizens participating in town hall meetings, public road shows etc. In both cases, the planning agencies and the responsible state ministries hoped most of all for broad acceptance of their planning through transparency, and in one case even asked for a binding citizens’ ballot on whether to proceed or not. From the two detailed reports (Kubicek 2014b, 2015) three important lessons can be learned which are also relevant to the planning of the energy supply networks:

  • Planning bodies at each stage are obliged to build on the decision taken in the previous stage. In contrast, most action groups and citizens in the later stages do not accept previous decisions and in each phase raise the question of demand and the “whether at all” again.

  • In each case several action groups opposing the highway project left the dialog after a few meetings. They had gained up-to-date information on the state of the planning process. Having failed to attract the support of other members for a vote to stop the project, they continued their fight against the project outside the dialog and in one case raised money to file a lawsuit.Footnote 4

  • An official citizens’ ballot on different options is legally impossible or has no relevance, as this decision can only be taken according to the environmental and health requirements laid down in the relevant laws. A citizens’ vote is not among the criteria that have to be considered and weighed in the final decision. Moreover, its validity could be questioned because there is a fundamental asymmetry between those citizens who may gain and those who will probably lose something. It is typical of infrastructure projects that they provide benefits at supra-regional level at the expense of local residents. While it is easy to collect the votes of local residents, it is almost impossible to identify a representative sample of voters at supra-regional level. Therefore if only locals are allowed to vote, there is a negative bias, while the admission of citizens at supra-regional or national level might lead to a positive bias instead.

To summarize the results within the e2d project including these additional insights, the most important lesson to be learned is that there is no use striving for a standard set of objectives and expectations against which all kinds of consultations could be evaluated appropriately. There were good reasons why the three research teams in the e2d project selected tools differently and adapted them to their respective situation. Accordingly, it would not make sense to suggest the different variants of tools employed as standard instruments. They rather serve as examples, and it seems more reasonable to document the major components of the generic tool from which selections and adaptations can be made, tailored to each individual project, as each evaluation will need a unique design and instruments.

16.5 Findings on Cooperation Processes

There are many different forms of collaborative participation, cooperation and co-production (see Chap. 6). While in consultation processes people exchange opinion about what the consulting party should do, in cooperation processes participants are expected to actively co-produce something, for example an agenda for action on a public issue, or the implementation of policies such as measures to achieve climate targets which also demand a pro-climate behavior change. This is of particular relevance as the behavior of private households contributes significantly to pollution and CO2 emissions. Therefore environmental democracy is not so much about discussing and voting but much more about collective action and changes of behavior, life styles, consumption patterns and values (see Chaps. 3, 11 and 12).

The extent to which the collaborative participation in the e2d project has contributed to climate protection will be discussed in Sect. 16.7. Here we will deal with the methods of evaluation employed, the experiences made with the adapted Input-Activities-Output-Outcome-Impact model (see Chap. 7), different methods of assessing various output and impact indicators, the validity of measurements, dropout rates etc.

As already mentioned we chose the field of climate protection for our evaluation exercises not only because it is a salient issue of global relevance but also because we had assumed that an impact here can be measured immediately and quantitatively by a CO2 reduction achieved on an individual level for each participant and on a collective level for larger panels of citizens at each of the seven sites.

16.5.1 Sample Size: Recruitment of Panelists and Dropout

While consultation processes most often do not take longer than a few weeks or up to 3 months, citizens’ cooperation in fighting climate change by reducing CO2 emissions had to last much longer. Because of seasonal differences in energy consumption, it had to cover 2 yearly seasonal cycles in order to measure impacts. Therefore, according to the comparative research design, at each of the seven sites a citizen panel with about 400 people was to participate over a timespan of 2 years, collaborating with local governments, reporting bimonthly consumption data and continuously receiving feedback about their CO2 emissions produced in relation to the previous period and to average scores of the panel. However, only in Saragossa was the target size reached (398 participants), followed by Pamplona (260), Bremen (213), Wennigsen (114), Bregenz (64), Mariazell region (62) and Bremerhaven (48). It is well known that over 2 years of bimonthly measurements it is inevitable that there will be what is known as panel attrition, that is some people will drop out over time. In our case it was surprising that a high percentage of people who had registered never started really participating by reporting basic data and/or their first consumption figures after 2 months. Between registration and the first periodic report, the exercises lost between 33 % of the participants (Bremerhaven) and 58 % (Pamplona and Bremen), or 47 % on average. By comparison, the dropout rate over the much longer timespan of the subsequent periodic measurements up to the final round was much smaller (on average 30 % of the first round participants or 16 % of the total at registration). While in Saragossa, Pamplona and Bremen 181, 73 and 60 participants respectively continued until the end and provided for a sufficient sample size, in Wennigsen, Bremerhaven, Mariazell region and Bregenz the final sample sizes of 43, 29, 22 and 21 were not satisfactory, in particular for the concluding assessment since the response rates to the final survey was below 100 % (see Chap. 7). As dropouts do not occur by chance, account must be taken of the fact that the assessment of those who remained will have a positive bias. Therefore, it was important to conduct a separate survey among the dropouts (see Chap. 13).

16.5.2 Impact Measurement via Carbon Calculators

As described in Chap. 8, carbon calculators transform consumption data on electricity, gas, water, use of traffic means, food, and more general statements on consumption patterns into CO2 emissions. This tool had an important function in our research design: A reduction of energy consumption by x percent was to produce feedback on the reduction of CO2 emissions and visually show the contribution to the fight against climate change. For this transformation the calculator has to consider the national electricity mix, as electricity produced by coal power plants is associated with higher emissions than solar or wind energy. As there are different energy mixes in Austria, Germany and Spain, some adaptations were necessary in order to produce correct figures. While this could be achieved by a one-off adaptation, some problems arose from mistakes the panelists made when entering data. There was a need for regular check for plausibility of the data reported and for clarification via e-mail or telephone. Panelists had the choice to report their data online or by telephone; the error rate of online entries was much bigger, as some obvious mistakes in reports by phone were immediately detected and cleared. This was time consuming but in the end led to data which are well comparable.

16.5.3 Outcome and Social Impact Measurement

Project output comprised not only the bimonthly feedback of CO2 data but also included regular newsletters, various events and meetings with expert inputs, the provision of advice and opportunities for exchange between the panelists. For each of these measures, it was possible to measure outcome in terms of the number and distribution of participants (see Chap. 9). However, we did not trace the individual usage over the entire time and therefore were not able to look for relations between the intensity of engagement and the impact achieved individually. As impact was conceived in a broader sense than actual CO2 reduction, three surveys were conducted among the participants to try to assess intermediate effects (e.g. individual and social learning, community building, etc.), and behavior changes (see Chap. 10). Here again it was not possible to link survey responses to the CO2 emission data on an individual level. For the evaluation of the collaborative participation process this was not essential; the data from the third survey provided a sufficient basis. The main methodological shortcoming as already mentioned was response rates below target to the participation offer at the start in most cities and a high dropout rate in the initial phase. This was mainly due to the specific participation format demanding above average levels of commitment and endurance. It is therefore almost tautology to report that those who stayed on and responded were well or very much satisfied with the participation process.

16.6 Comparing Online and Face-to-Face Participation

It was one of the objectives of the e2d project to find out whether there are systematic differences between online and face-to-face participation. In most of our consultation case studies there was no choice of channels but rather a sequential combination. But for the collaboration panels on CO2 reduction, participants had the choice of reporting their data by telephone and receiving feedback by mail or doing this online.

The literature on e-participation mostly maintains that online participation can reach larger numbers of participants and a broader range of contributions compared to face-to-face meetings on the same subject. But this has seldom been proved. In the e2democracy project, we used different research methods and designs to test these claims, that is surveys on preferences and comparative assessments, observations of discussions, document and content analysis as well as quantitative measurements of tangible impacts. Based on evidence from six cases of three different kinds of consultations and seven collaborative citizen panels we found no single qualitative advantage of online communication compared to traditional modes of communication that would have occurred in all processes. Rather, the differences between the outcome and impact of online communication in two or more similar participation processes were most often greater than those between online and traditional modes. This points to a great influence of person and role related characteristics in the conceptual framework presented in Chap. 15 (Sect. 15.3). While this is already known from the literature, we found that the measurement design itself, in particular the point in time and the size and composition of the sample, contributes to additional variance. One should not generalize research findings without reflecting the methods by which they have been generated.

Surveys of preferences for different communication channels are most frequent but deliver the least valid and least reliable results as they are highly volatile.

  • Reported preferences and perceptions of advantages are biased when respondents have practiced only one of the two modes which they are asked to compare.

  • Before-and-after comparisons show that the assessment changes as a result of the actual experience.

  • Reported preferences do not allow for conclusions on future action. For example, in the Wennigsen and Bremerhaven cases respondents who reported the advantages of on-site voting did not choose this option and voted online instead.

Comparing the quality of online and face-to-face discussions by observation and content analysis delivers more valid but still no clear-cut results. There are too many influencing factors, for example the rationality, the concreteness, the discursivity or the length of contributions that cannot be controlled to isolate the influence of the communication channel.

The same is true with regard to the influence on the impact in terms of achieving CO2 savings. The only general effect in all of the seven climate protection panels is the lower dropout rate when panelists are contacted by telephone instead of having to take their own initiative to report their bimonthly consumption data online. However, only a minority was ready to participate in this more traditional way; 74 % of the participants registered at the start had preferred e-participation.

Against this background, we cannot provide clear evidence for the general claim that online communication is superior to traditional ways and helps to overcome barriers in political engagement and participation. Rather, we would not encourage any organizer to substitute traditional modes of communication completely by an online channel only. However, an additional online channel is necessary in order to make full use of the participation potential.

Accordingly, the vast majority of the organizers interviewed in the e2d project said that in future they would offer a combination of communication channels, a so-called media mix or blended participation. This preference is not based on any well-founded cost-benefit analysis, but only on the hope of getting more people involved and achieving a higher degree of inclusiveness and representativeness of participants and, thereby, a higher legitimacy of the results. Offering a media mix is undoubtedly more expensive than offering an online channel only and there may be a demand for justifying additional expenses.

We cannot encourage the authorities that are deciding on budgets for participation to expect well-founded empirical evidence which tells them exactly in which cases which kinds of channels should be offered. Rather, we recommend blended participation as the rule. When organizers are to decide which communication channels they should offer in a particular consultation or cooperation process, they have to make assumptions about the channel choice of the members of their target group. The model presented in Sect. 15.3 supposes a high degree of complexity of the task-related and person-related factors influencing this choice. It is unlikely that further research in the future will discover stable relationships between these factors which will allow for a good prediction of the channel choices of a heterogeneous group of people addressed. On the contrary, all the data presented in Chap. 15 show that there are always some participants who prefer one channel and others who prefer the other one. So the simple conclusion is that if both groups are to be involved, both channels have to be offered.

16.7 Do Monitoring Information and Feedback Reduce CO2 Emissions?

Besides contributing to methodological issues of evaluation the e2d project also allows for some insights into the policy field of climate protection and the contribution individual citizens can make via monitoring of consumption behavior and feedback of comparative information, as for example claimed by Thaler and Sunstein (2008). So far there has been no empirical assessment of the extent to which and the conditions under which what kind of participation procedure would serve this function. In the e2d project seven largely identical participation processes with citizen panels were set up and studied in a quasi-experimental field study which allows for some answers to these questions.

The first finding was that the gap between declared assurance and actual participation in these initiatives demanding long-term commitments and continuous input turned out to be huge. Actual participation was much lower than could be expected from declarations of intent in population surveys conducted in the relevant cities and regions before the start of the processes (see Aichholzer et al. 2013). Those who participated show a special profile: They are characterized by significantly higher levels of interest in fighting climate change, of sensitization, issue knowledge and belief in the efficacy of targeted action. However, to some extent these participation processes have also reached out beyond typical “environmentalists”. Thus, there is still potential for CO2 reduction among the panels; however, demanding requirements of CO2 calculation (providing consumption data, entering it into a complex tool, etc.) and limited scope for sustainability improvements among participants with advanced sustainable practices reduce their interest in continuing their participation over the projected 2-year period.

The second most important finding is that monitoring information and providing feedback, as proposed by Thaler and Sunstein, alone do not lead to sustainable changes of behavior. A social environment allowing for issue-specific community building, mutual exchange and social learning is important for enhancing “carbon capability” (Whitmarsh et al. 2011) and stimulating changes in everyday behavior. In such a context the regular provision of information and feedback to citizens over a longer time, based on their individual consumption data, encourages and reinforces responsible behavior in favor of reduced CO2 emissions. This tends to induce informed choices among the participants in some relevant areas. When it comes to impacts in terms of an increased awareness of climate effects, changes of behavior and CO2 balance, a more differentiated picture emerges. A substantial percentage of the participants shows an increased sensitization and reports behavioral changes in certain areas of consumption, induced by the participation process. However, some activities causing higher CO2 emissions, including high impact cases such as flights, largely persist. On the individual level, the majority of participants in Germany and Austria achieved a CO2 reduction of at least 2 % per year and a lower percentage also in Spain, although this does not imply linear improvements across all sub-areas. The collective level, that is the overall CO2 balance of each local panel, shows a less positive picture. Five of the seven panels reduced their collective emissions by at least 2 % in the first year and two of these (in Bregenz and Saragossa) also did so in the second year. However, the collective emission level in Bremen—despite a slight improvement during the first year—was almost the same after 2 years, and in Pamplona it even deteriorated over the 2 years. It is only when flights are excluded from the CO2 balance that the Bremen panel as a whole also achieved the target of a reduction by 2 % p.a. This underlines the decisive role of high impact activities like flights in particular and at the same time shows the limited influence on changing social practices as exemplified by (long distance) travelling and holidaymaking. Moreover, the “perplexity of environmental information” together with the limited functionality and user-friendliness of carbon calculators makes it difficult to choose pro-climate travel alternatives (Juvan and Dolnicar 2014).

Some options for CO2 reduction are one-off activities such as changing the electricity provider and switching to green electricity or installing new heating equipment, while others require changes of long-established consumption patterns that are hardened by habits and often constrained by external barriers. Information provided on the basis of a rational choice model obviously does not provide an effective framework for an answer to the question of how to change such patterns and institutional constraints. Hence, the Thaler-Sunstein hypothesis of “Information saves energy” seems of limited validity. As an alternative or in addition, social marketing approaches have been suggested that promise incremental increases of climate-friendly behavior from developing and employing tailor-made strategies for identified segments of the population (cf. Barr 2008). But changing individual behavior has to come to terms with the fact that this behavior is deeply embedded in social, institutional and material contexts and occurs as part of social practices (cf. Shove et al. 2012). How these can be influenced, how they can be accounted for by different participation formats and how the methodological constraints and validity problems of CO2 calculation can be overcome are issues which require further research.

The further perspectives of the climate initiatives studied in the seven cities and regions suggest that the momentum created will be carried on at varying intensities and in different forms. The provisions for the feedback of CO2 balances were financed by the funded research projects in the three countries. This kind of support ended with the end of the funding period. City or regional governments were not ready to maintain this kind of support. But they sustained the collaboration on climate protection with their citizens in different ways: In Austria, the environment department of the city of Bregenz continues to work with its established citizen panel, local companies and schools, organizing excursions and other events, trying to engage wider sections of the population. In the Mariazell region, municipal governments have joined a climate alliance which promises to reinforce the grassroots level activities of local panel members for climate protection. In the case of Germany, in Wennigsen a group of panelists have founded a round table on their own initiative in order to encourage their behavioral changes. In Bremen and Bremerhaven, the regional climate protection agency established similar round tables; interested panelists still receive regular newsletters and are invited to discussions, excursions and relevant events. The two Spanish cities are building on their established tradition of citizen participation and climate protection and have also several follow-up activities in the pursuit of their climate targets.

16.8 Lessons for Environmental Democracy

It is obvious that citizens’ consumption behavior contributes to climate change and that they have the potential to fight it by changing their behavior. Therefore, citizen participation and regular monitoring are essential elements of the Aalborg Commitments.Footnote 5 More than 700 local government authorities have signed the Aalborg Commitments and committed themselves to annual, regular monitoring and citizen participation. So far no assessment has been made of the extent to which the signatories have offered what kind of citizen participation and what outcomes and impacts have been achieved. The panels in the e2d project do not prove that there is a general and significant positive impact on CO2 reduction. What does this mean for the Aalborg commitments and the broader ideas of environmental democracy?

The first relevant evidence is that several signatories of the Aalborg commitments were not ready to initiate such a participation process, most of all because they could or would not provide the personnel resources to support such an exercise. But they were also afraid of high dropout rates or little impact compared to measures to reduce the emissions of industrial plants in their region. This reluctance of local governments as regards citizen participation became visible in the analysis of the websites of the Aalborg + 10 signatories (Chap. 4). The results show that the degree of information provided is much higher than the interactivity, participation and cooperation.

We have not only found this gap between declarations signed and actual behavior amongst local governments but also a gap between verbal intentions and actual behavior amongst citizens. With some variation between the three countries and between urban and rural communities, the overall result is that there is a big gap between citizens’ verbal commitment to CO2 reduction and their actual behavior.

According to Eurobarometer surveys, climate change is among the top priorities particularly when global problems are asked for (EC 2009, 2011, 2014). In the representative telephone surveys, which were conducted in the e2d project in each region, residents were asked to rank the topicality and salience of climate change in relation to unemployment and the local provision of childcare facilities (Aichholzer et al. 2013). In Austria and Germany, climate change was named as the second biggest concern after unemployment. Values for climate change as the biggest problem vary from 20 % of all respondents in the Mariazell region (AT) to 40 % in the region of Wennigsen (DE). However, the survey in Wennigsen took place several months after the other surveys and just 2 weeks after the nuclear disaster in Fukushima, which certainly had some impact on the ranking. The majority of participants in all surveys also said that local governments would not meet the CO2 reduction objectives by themselves but that industry and citizens would also have to contribute and that they were ready to commit themselves. The percentage of respondents ready to take part in a regular CO2 monitoring project ranged between 69 and 92 %. From these results, a rather high rate of citizens could be expected to register for the collaborative participation project when invited locally. But as reported, this was not the case and the dropout rate was higher than expected. And the Internet, the second “e” in e2democrcy, does not change this reluctance. Rather on the contrary, dropout rates among panelists who had chosen the online channel for CO2 footprint monitoring was higher than for those communicating by telephone and mail.

As regards CO2 reduction via consumption monitoring and information feedback in particular, we have to add that even in a competitive arrangement it does not lead to long-lasting environment-friendly behavior under all circumstances. Therefore, often heard hopes on the impact of smart meters are not justified to the full extent. Although CO2 emissions seem to be well measurable, appropriate feedback is only meaningful for comparable households. But this needs classification systems with regard to household size, heating system, infrastructure, etc. While energy consumption is most easily measurable, it is probably not the most influential area of consumption with regard to CO2 emissions. CO2 reduction also needs changes of behavior in areas such as mobility and travel, nutrition and purchasing consumer goods, with different established patterns of behavior and different barriers for changes.

Against this background, the reluctance of some local government authorities to engage in citizen participation for fighting climate change bears some rationality. To be effective, such a participation format aimed at sustained pro-climate awareness and behavior change needs substantial accompanying investments in process support, opportunities for exchange, community building and social learning plus infrastructural measures to enable and facilitate alternative options in all areas of climate-relevant behavior. The hope that environmental democracy as a bottom-up movement could become a big step forward has to be put into perspective. It seems that climate change still has to be fought by thousands of different steps on all levels and in all areas.

16.9 Tools for Evaluating (e-)Participation

When starting the project we had hoped that the tools used for evaluation, that is checklists, interview guides for organizers and online or postal questionnaires for surveys of participants, could be validated at the end and serve as some kind of standard instruments. In the course of the project, we have learned that each participation project is unique and that the tools for evaluation have to be tailored to each case. They can still serve as a starting point for future evaluation exercises and can be downloaded in English and German from the e2d project website www.e2democracy.eu.