Introduction

This article describes the application of a sustainability assessment (SA) process and methodology developed as part of a wider European Union’s Fifth Framework funded research project (BioScene) which investigated the implications of agricultural restructuring and decline for biodiversity conservation in Europe’s mountain areas. The target was to provide practical outputs enhancing implementation of Natura 2000 (EU priority conservation sites) strategy through integration of agri-environmental, conservation, and rural development policy.

The project took a case study approach to the analysis of the biodiversity processes and outcomes of different scenarios of agri-environmental change in six countries (France, Greece, Norway, Slovakia, Switzerland, and the United Kingdom) covering the major biogeographical regions of Europe. Study area stakeholder panels, reflecting the range of perspectives (rather than representing the interests of specific organizations), were constructed with around twelve members in each. These stakeholder groups worked with the project team in each area to guide the research and to test the outputs. There were three major work strands: ecological, socio-economic, and sustainability assessment, each of which engaged with the stakeholder panels at various stages. In each study area the various scenarios of future agriculture change were characterized and ecological modeling was used to explore the biodiversity consequences in a range of agri-environmental settings. The ecological modeling comprised: the analysis of past landscape change using air-photographs and geographical information systems (GIS); the generation of hypothetical landscape mosaics of each of the scenarios using statistical tools; and assessing the impacts of scenarios on priority habitats and species using habitat suitability models and expert knowledge (Mitchley and others 2006). The socio-economic analysis included the development of study area narratives through stakeholder interviews, visualizations of landscape change under the different scenarios, and cost effectiveness analysis of management options.

The purpose of this article is to describe an innovative methodology developed and used for SA in the BioScene project. This article first reviews the relevant literature around SA, followed by the agricultural and biodiversity scenario context in which the project was based. It then describes and discusses the key steps in the SA process and the methodological rules and options for dealing with scenario assessment and engagement with stakeholders. Finally, this article draws conclusions for the application of SA more widely, and reflects on working with partners and their different expertise, and the challenges of inter-disciplinary working.

Background to Assessment

Environmental impact assessment (EIA) — “the process of identifying, predicting, evaluating and mitigating the biophysical, social, and other relevant effects of development proposals prior to major decisions being taken and commitments made” (IAIA 1999) — has been one of the major instruments of environmental policy since the late 1960s, and has been primarily applied at project level (e.g., for infrastructure developments). In the European Union, EIA has been legislated through the Council Directive of June 27, 1985 on the assessment of the effects of certain public and private projects on the environment (85/337/EEC), subsequently amended by the 1997 Amendment Directive (97/11/EC) and the Public Participation Directive (2003/35/EC).

More recently, there has been an increasing interest in applying the principles and methods of EIA to more strategic initiatives, such as policies, plans, and programs (i.e., strategic environmental assessment — SEA), and in broadening out the scope of assessment to embrace all three dimensions of sustainability: social, economic, and environmental (i.e., sustainability assessment — SA) (e.g., Partidário 2000). Strategic environmental assessment draws heavily on the principles of EIA, but particularly places an emphasis on evaluating strategic alternative options, with the option or mix of options having the least environmental impact being preferred. In the European context, four major initiatives illustrate these trends: (1) the European Union has adopted a Directive on the environmental assessment of Member State’s plans and programs, known as the Strategic Environmental Assessment (SEA) Directive (CEC 2001); (2) the European Commission has established a new Impact Assessment tool which looks at all three elements of sustainability for its major policies (CEC 2002); (3) the European Spatial Development Perspective has promoted Territorial Impact Assessment as a method of evaluating whether development policies (such as agriculture) contribute to the European Union’s objectives of social and economic cohesion, environmental sustainability, and polycentric development (ESPON 2004); and (4) the United Nations Economic Commission for Europe launched a protocol on SEA (UNECE 2003).

Sustainability assessment (or sustainability appraisal, SA) aims to inform and improve strategic decision making. It has been defined as “a form of strategic assessment that integrates environment, social and economic parameters, compared with [Strategic Environmental Assessment] which deals primarily with environment” (Sheate and others 2001: glossary). Devuyst (1999) also highlights the importance of the integration of these parameters as a priority of sustainable development and that SA methodologies should reflect this. Emphasis in SA is on the integration (and analysis) of social, environmental, and economic considerations rather than just listing and considering these parameters separately (Buselich 2002), with SA seen as going beyond more traditional forms of environmental assessment such as EIA and SEA, towards a more “integrated assessment” (Dalal-Clayton and Sadler 2005). Such integrated assessment should not only look at the environmental, social, and economic implications of proposals but also the interrelations between these parameters (Pope and others 2004). However, Buselich (2002) points out that there are in fact very few examples of truly integrated SAs, however “integrated” might be interpreted (Scrase and Sheate 2002). Increasingly, the literature also reflects attempts to combine environmental assessment and management tools and make better linkages between these tools (e.g., Baumann and Cowell 1999; Burström 1999; van der Vorst and others 1999; Sheate 2002; Emilsson and others 2004).

There are numerous approaches to, and forms of, SA worldwide (Becker 1997; Harridge and others 2002; Noble 2002; Buselich 2002; Jenkins and others 2003; Pope and others 2004; Marsden and De Mulder 2005). Equally the definitions/terminologies used to describe the various different processes are wide-ranging. Dalal-Clayton and Sadler (2005) provide examples of at least 27 different approaches/applications of sustainability assessment, highlighting that SA “lies at the most demanding and testing end of a wide spectrum of integrative approaches.” The complexity and difficulty of assessing likely economic, social, and ecological effects is also recognized (Jacob and others 2004). However, integration in itself is not necessarily inherently a “good thing” (Scrase and Sheate 2002), since it comes in many guises and can create problems of “trade-off” between factors, and the inevitable simplification required can result in rather superficial or mechanical “tick-box” assessments.

Despite its rapid progress, especially in Anglo-Saxon countries such as the United Kingdom, Australia, and Canada (Short and others 2004; Smith and Sheate 2001a, b; Pope and others 2004; Gibson and others 2005; Gibson 2006) and in the international arena, where it has been at the forefront of evaluations of trade agreements (George and others 2001; Abaza and Hamwey 2001; Schramm 2000; Lee and Kirkpatrick 2001; Zerbe and Dedeurwaerdere 2003), SA has also been the subject of much criticism. Most notably, the UK Royal Commission on environmental pollution raised concerns on two fronts: for SA’s lack of quantification and for the “poor science” involved in its environmental analyses; also for potentially marginalizing the very environmental and social appraisals that it is supposed to bolster as a counterpoint to dominant financial and economic assessments (RCEP 2002). Similar concerns have been raised by other authors (e.g., Wood 2003; Owens and Cowell 2002) and SA in the UK has often been characterized as being “environment light” (Sheate 2005). Other potential weaknesses of the sustainability assessment/appraisal process include the lack of input from relevant stakeholders and the absence of any baseline data (CAG 2003). The SA process, however, can provide an ideal opportunity for stakeholder engagement. Planning and learning for sustainable development increasingly need transdisciplinary approaches (Meppem and Gill 1998) to help deliver “sustainability learning” (Scholtz and others 2005). Transdisciplinary approaches are those that produce, integrate, and manage knowledge in technological, scientific, and social areas and as the prefix “trans” suggests, goes beyond traditional disciplines, so that society (the public) is fully engaged in decision making (Scholtz and others 2005; Scholtz and others 2006). However, prior to the research reported here (BioScene), the lessons from transdisciplinary research (e.g., Wiek and Binder 2005; Scholtz and others 2006), had not been widely applied explicitly in the context of sustainability assessment.

The lack of or insufficient provision of baseline data is considered a particular constraint with regard to the consideration of environmental issues (Dalal-Clayton and Sadler 2005) and can lead to “inherently contentious value judgment…where knowledge about impacts is limited or uncertain” (Jacob and others 2004). Quite often SAs have been undertaken quite late in the planning process, i.e., when a plan or strategy has already been written, therefore reducing the iterative nature of the process and lessening the opportunities for effective use of the SA process towards improved sustainability performance (CAG 2003; Harridge and others 2002). However, it would appear that rather than the actual product of SA, it is the process by which the SA is developed that increases its effectiveness (Cash and Clarke 2001 cited in Zerbe and Dedeurwaerdere 2003; Sheate and others 2003; Pope and others 2004; Gibson and others 2005; Arbter 2005).

The BioScene Project

For centuries, agriculture has played a multifunctional role in sustaining mountain biodiversity in Europe through management of habitats, species, and landscapes. With significant agricultural adjustment and even contraction now in prospect, there is potential for major impacts on mountain biodiversity. Some of these changes may be deleterious, e.g., loss of locally adapted species and semi-natural habitats; others may be beneficial, e.g., new successive pathways providing opportunities for restoration of some of the large predators that occurred in the pre-agricultural landscape, e.g., raptors, wolves, and bears. For decades policymakers have tried to address the problems in mountain areas through Less Favored Area (LFA) policies designed to prevent land abandonment, preserve the farming population, and conserve the countryside by linking biodiversity objectives directly to the viability of farming. Less Favored Area policies have had mixed effects, slowing down structural change, but also encouraging intensification of production on better land. Rural development assistance has recently been improved through the “second pillar” of the Common Agricultural Policy (CAP), but the transition towards more sustainable LFA policies is at an early stage. Different policy approaches make different assumptions about the limits of acceptable change, from those designed to maintain the status quo to other more laissez faire approaches. Objectives of the European Union Biodiversity Strategy (CEC 1998) include the integrating of biodiversity objectives into the CAP and regional rural development policies and spatial planning. This trend is paralleled in the recent agreement (and implementation in 2004) in the European Union of the Strategic Environmental Assessment (SEA) Directive (2001/42/EC) and its extension into broad policy areas described above as SA. Mountain biodiversity, including the human communities, face unprecedented threats from social, economic, and environmental forces of change, e.g., climate change. These same forces also bring exciting opportunities for the integration of knowledge and expertise to achieve sustainable solutions across the mountains of Europe.

The use of scenarios is increasingly being used in environmental research at the regional scale to evaluate future trajectories of land-use change (Hawkins and Selman 2002; Peterson and others 2003; Nassauer and Corry 2004; Penker and Wyrtzens 2005; Verburg and others 2006; Audsley and others 2006). Scenario studies may be used to facilitate policy optimization, vision building, or strategy development (Westhoek and others 2006). The BioScene project used scenarios strategically and developed stories of future change to describe the possible results of specific assumptions about policy trends and drivers of change. The BioScene scenarios included a Business As Usual Scenario, based on an extrapolation of current market and policy trends, an Agricultural Liberalization Scenario, based on the effects of withdrawal of agricultural support in the transition to free market conditions, and a Managed Change for Biodiversity Scenario, based on a liberalization scenario, but geared to maximizing biodiversity conservation (Mitchley and others 2006). Two variations on this last scenario were possible, depending on the case study location: managed change based on current biodiversity priorities, and managed change based on a natural succession or “wilding” approach, i.e., changed biodiversity priorities. The scenarios were characterized by the partners according to their country and study area context, i.e., what would be the key policy drivers influencing each scenario? The scenarios were characterized over a 25 year timescale, from the present time to 2030, so that Business as Usual was a future scenario based on current trends. The public perceptions of these different scenarios were first evaluated through landscape visualizations (using computer-manipulated photographs and stakeholder preferences). The scenarios were then subjected to sustainability assessment, incorporating consultations and feedback from stakeholders, to evaluate the scenarios in relation to wider environmental, economic, and social sustainability. Dalal-Clayton and Sadler (2005) emphasize the importance of the use of scenarios, particularly when working outside formalized plan-making processes, in presenting “plausible, pertinent, alternative ‘stories’ that are very much concerned with strategic thinking (as opposed to strategic planning) and particularly with quality thinking.” The final project synthesis provided a cross-country comparative assessment and recommendations for new strategies, plans, and policies for integrating biodiversity conservation and sustainable rural development in the mountains of Europe (not reported here, but see Mitchley and others [2006] and Partidário and others [submitted]).

BioScene used a form of sustainability assessment, so-called to distinguish it from sustainability appraisal that in the past has too often been based on poor or infrequent use of baseline knowledge (Smith and Sheate 2001b; Sheate and others 2001, 2003; Harridge and others 2002). Fundamentally, the SA methodology developed for BioScene differs from most previous applications of the tool by being much more informed by detailed baseline information (Dalal-Clayton and Sadler 2005). Indeed, the project offered an unique opportunity to both apply SA in a sector (agriculture) rarely subject to systematic forms of environmental or other assessment, and to develop an SA methodology that was unusually well informed by ecological baseline data and socio-economic data from the two other streams of the project. Furthermore, it was also to be used in combination with scenario analysis and through a process of periodic stakeholder engagement. BioScene therefore presented the opportunity to undertake SA in both an interdisciplinary and a transdisciplinary context. Although developed in the agricultural context, there is no reason why the methodology developed could not be applied in other sectors.

Sustainability assessment in BioScene was a systematic process for the assessment of the likely economic, social, and environmental consequences of each of the scenarios and the combinations of policy and land management measures contained within them. The aim of the assessment was to understand the potential impacts of each of the scenarios on wider sustainability objectives and identify changes that would increase desirable consequences and reduce undesirable consequences, i.e., identifying the most sustainable policy interventions and management measures for the future. The SA work was coordinated jointly by UK and Portuguese SA teams, working with nonassessment specialist partners (i.e., the socio-economic and ecological teams in the partner countries). This posed a particular interdisciplinary challenge. Issues raised are discussed later in this article. The coordination role was shared, with the UK SA team working with Norway, France, and Slovakia, and the Portuguese SA team working with Greece, Switzerland, and the UK. This also provided an essential independent overview role to the SA process.

The concept of sustainability in BioScene was fundamentally rooted in the need to maximize economic and social benefit alongside biodiversity (and other environmental) benefits, i.e., to seek win-win situations (Keough and Blahna 2006). Therefore a scenario that was good for economic and social dimensions, but not for biodiversity, was unlikely to be an acceptable scenario. However, a scenario that was not viable from an economic and a social perspective was unlikely to be realistic. Figure 1 summarizes how the scenarios, which were at the heart of BioScene, were assessed from different interrelated perspectives, including biodiversity consequences, cost-effectiveness, and sustainability. The ecological and socio-economic analyses are not described in detail here, although a summary is reported in Mitchley and others (2006).

Fig. 1
figure 1

Assessments of BioScene’s scenarios

Full size image

Study Methodology — the SA Process

The purpose of the SA in BioScene was twofold: (1) to facilitate the interaction, mutual understanding, and communication between the ecological and socio-economic teams; and (2) to understand the potential implications of each of the scenarios (positive, neutral, or negative) for the study area sustainability objectives and identify changes that will increase desirable consequences and reduce undesirable consequences. It is important to clarify that BioScene was first and foremost a research project that aimed to produce scientific and policy recommendations to the European Commission for the medium and long-term development of agriculture and rural policies. Therefore, the SAs discussed here were not intended to influence a concrete plan or program in each case study area, instead they were being applied to expected trends defined within BioScene as alternative scenarios for agriculture in mountain areas. This was important since it provided a relatively “neutral” space (accepting that there is no such thing as a truly neutral space) in which the wide range of stakeholder interests could engage in the stakeholder panels without concern about immediate outcomes for the area concerned. This allowed the opportunity for better mutual understanding of different perspectives rather than attempting to “win” arguments through conflict. This also presented an interesting departure in the application of SA, which is most often applied to specific plans or programs, e.g., in the context of land use planning. Of course, the lack of immediate impact may reduce the incentive to participate, but in all study areas there was interest in participating. The recruitment of stakeholders was undertaken following preliminary meetings with around 60 individuals in order to ensure the participation of people across all perspectives, including farmers, conservationists, and people who could speak for recreational and rural development interests. Recruitment of the panels of approximately a dozen individuals was undertaken by the socio-economic teams in each study area and each panel met three times during the course of the project (Mitchley and others 2006).

Common to many strategic-level assessments centered on sustainability objectives, the SA concept involved the following stages:

  • Establishing a framework for sustainability assessment: Identifying broad sustainability objectives for the project as a whole and at country level, developing a baseline reference document, defining a framework of sustainability assessment objectives, and associated indicators for each study area against which the scenarios could be assessed.

  • Evaluating the sustainability of proposed scenarios: Assessing the scenarios against the agreed objectives/indicators to see how well they fulfilled the objectives and where improvement was possible (the common “objectives-led approach” to assessment).

  • Reporting on the sustainability assessment of scenarios: Writing up the results of the SA into a country sustainability report that documented the sustainability of each of the scenarios and possible improvements.

These stages were integrated within the overall project, illustrated in Fig. 2. Four main outputs for each case area were therefore required for the SA process: (1) sustainability objectives and indicators, (2) sustainability baseline reference document, (3) sustainability assessments of all scenarios, (4) sustainability reports.

Fig. 2
figure 2

Inputs and feedback between BioScene’s components

Full size image

The purpose of involving stakeholders in the SA was to make sure that the expert views of the partners were complemented by the knowledge, opinions, and values of members of the stakeholder panel and the wider public. Constraints on the depth of participation were inevitably imposed by the complexity of the overall project and the resources available. Ideally, the stakeholders would have been involved more actively throughout the SA process, rather than being engaged at various points throughout. However, the stakeholder panels were contributing to other parts of the BioScene project as well as the SA process, and to have required them to be more involved, even if the resources had been available, may have been expecting too much. Nevertheless, stakeholder involvement was an integral part of the SA process.

Consultation focused on: eliciting the views of the panel on what matters for sustainability in the area (during Stakeholder Meeting 1); obtaining feedback from panel members on the proposed study area objectives (during and after Stakeholder Meeting 2); discussing the results of the assessment and identifying key issues and priorities (during Stakeholder Meeting 3); obtaining feedback from the panel members and the public on the sustainability reports (after Stakeholder Meeting 3).

More detailed accounts of the contributions received from stakeholders and partners to the SA process are described elsewhere (Mitchley 2005; Mitchley and others 2006; Partidário and others [submitted]). The key steps in the SA process — sustainability objectives, stakeholder engagement, assessing the scenarios, and sustainability reports —are described in detail below.

Sustainability Objectives

The study area sustainability assessment objectives were drawn down from two sets of more generic sustainability objectives (overall objectives, derived from international and European priorities (Table 1), and country specific objectives, derived from national priorities, but related to the overall objectives), which were collated by the SA Team in conjunction with the partners during Year 1 of the BioScene project. Based on these overall and country objectives, initial versions of country specific study area sustainability assessment objectives, intended to describe what sustainability meant for the study areas, were produced during Year 1 and then revisited when the main SA work packages began after 18 months (half way through the project), after more data on ecological, social, and economic issues in each study area had been gathered and analyzed by the partners. The final list of the study area assessment objectives drew on the work of the baseline reference document to reflect the main priorities that this document identified (see Table 2), reflecting what partners and stakeholders thought were the key priorities in order for land use/development in the study area to be more sustainable. It is important to note the distinction between study area assessment objectives and overall sustainability objectives derived from international and national priorities. Those objectives used for the assessment have to be tailored to the study area in order to be meaningful, and the relative balance of these assessment objectives may vary across the study areas.

Table 1 Overall sustainability objectives for BioScene
Full size table
Table 2 Purpose and contents of the baseline reference document
Full size table

To ensure a degree of consistency and comparability across the study areas, study area objectives were finalized with a minimum of one objective for each of the six main themes and a maximum of 20 assessment objectives in total. Proposed study area objectives were also evaluated to ensure they were internally consistent and did not inherently conflict with each other, e.g., two opposing biodiversity objectives.

Finalizing the list of study area objectives involved partners revising a draft list of study area objectives and selecting indicators prior to the second stakeholder meeting. The draft version of the study area objectives (and the background overall and country objectives) was produced by the relevant SA team in the light of new material from the other work packages together with a first draft of the baseline reference document. Partners revisited these amended objectives, making sure they reflected the main priorities in the baseline reference document (while also finalizing the latter document). While finalizing the study area objectives, partners also defined one or more indicators to be used in the assessment to help understand the likely effect on a particular objective (Table 3). During the second stakeholder meeting, partners elicited views of stakeholders on the six themes of sustainability.

Table 3 Examples of study area assessment objectives and indicators
Full size table

Stakeholder Engagement

At the second stakeholder meeting (held late in 2004) about 30 minutes at the end of a 3–4 hour meeting were devoted to introducing the SA process to stakeholders, following a very short briefing document that had been sent to panel members previously. Where possible, in order to get an impression of the debate and dynamics of the participation process, and also to provide background support for the SA process, at least one member of the SA team was present during the meeting as an observer. Partners facilitated a short exercise intended to gain stakeholder ideas on what sustainability meant for their area. This was a simple brainstorming session, where stakeholders were asked to think about sustainability, starting from the six themes, and then individually to write about their priorities on post-it notes (maximum six); the post-it notes were then re-arranged and clustered by the facilitators in discussion with the stakeholders, helping to reveal the frequency of certain themes/priorities.

Following the second stakeholder meeting, the study area objectives and indicators (to be used as the framework for the SA) were then finalized. Partners first revised the study area objectives and indicators in light of the outputs from the exercise at the second stakeholder meeting. The revised set of objectives was then sent to stakeholders for final feedback (with a short questionnaire asking stakeholders to rank the objectives according to their own individual priorities). The feedback was then used to produce a final list of objectives/indicators by the end of January 2005.

Assessing the Scenarios

Disaggregating the Scenarios

The scenarios, and more specifically their drivers, formed the object of the assessment, and in order to assess, causal chain flow diagrams were developed for each scenario in each study area, showing the relationship between drivers, impacts, and land use consequences (Fig. 3).

Fig. 3
figure 3

The sequence of causal chain analysis

Full size image

The aim was to establish how each scenario contributes to, or conflicts with, the study area sustainability assessment objectives and indicators. The drivers, or their degree of influence, differed across the scenarios. The nature of the flow diagrams developed, therefore, for each scenario was quite complex, showing interactions between drivers and impacts. In order to undertake the assessment, each scenario flowchart had to be first disaggregated by individual driver categories, so that their consequences for sustainability could be assessed, including the consequences of the combination of different types of impacts triggered by each driver. Some examples are shown in Fig. 4. Having done this, those driver categories appearing to have the greatest influence, within each individual scenario, were identified, i.e., some drivers may be relatively minor and, therefore. of low significance in terms of potential influence. Scenarios could then be compared to establish whether a particular driver is consistently present and has a significant influence.

Fig. 4
figure 4

Examples of individual disaggregated causal chains

Full size image

The Assessment Matrix

For each scenario, the assessment matrix recorded the agreed evaluations made by the researchers of the drivers identified as likely to have the greatest influence and their associated causal chains against the study area assessment objectives. Table 4 illustrates the template used for the matrix.

Table 4 Initial assessment matrix
Full size table

The last column (shaded in grey) was used to record the aggregated results of individual drivers against each objective. The assessment followed a common approach used in SA (e.g., Lee and Kirkpatrick 2001; Gibson and others 2005; Dalal-Clayton and Sadler 2005; Hacking and Guthrie 2006;) using expert judgment to decide how each driver (and/or causal links) relates to an objective and the related indicators: whether it would contribute positively to a particular sustainability objective or whether it would be likely to conflict, compared to the trends outlined in the baseline reference document. The robustness and validity of this type of assessment depended on open discussion between the members of the BioScene socio-economic and ecological teams (and, for subsequently finalizing the assessments, on discussions with stakeholders during the third meeting). Any disagreements between the socio-economic and ecological teams were recorded in the “comment” cells.

To promote consistency, the assessment was made using a five point scale:

  1. ++

    (green) — driver makes a major positive contribution to the sustainability objective

  2. +

    (light green) — driver makes a positive contribution to the sustainability objective

  3. 0

    (white/blank) — driver has no significant contribution to the sustainability objective

  4. -

    (pink) — driver conflicts with the sustainability objective

  5. - -

    (red) — driver in major conflict with the sustainability objective

  6. ?

    uncertainty about the likelihood of an impact (not the significance of impact) was highlighted in yellow.

From the point of view of transparency, it was important to explain the rationale behind a particular scale (e.g., where different conclusions are possible). From a practical point of view partners were encouraged to carry out the assessment in a group meeting involving representatives of both the socio-economic and ecological teams. The SA team recommended that partners consider each disaggregated driver and associated causal chain in turn and discuss how these would affect each objective compared to the current situation and the major trends as described in the relevant sections of the baseline reference document. They also needed to consider the likely effects on the indicators, which helped to illustrate theoretically or quantitatively the judgment made in the assessment matrix with regard to a particular objective. Each driver and associated causal chain was likely to give rise to a range of effects some of which would be direct and others increasingly indirect. In completing the matrix, the focus was on assessing the direct and reasonably foreseeable indirect impacts; it was not considered necessary or practical to consider very remote impacts. The time period to consider for the assessment was the timeframe for the scenarios, i.e., to 2030. Therefore, the focus was on assessing what the extent/level of the effects of each driver/causal chain will be over the next 25 years.

The final column in Table 4 provides the starting point for the scenario comparison matrix (Table 5).

Table 5 Scenario comparison matrix
Full size table

The Scenario Comparison Matrix

This second matrix (Table 5) compares the effects of the different scenarios against the same set of study area objectives. The aggregation of the effects of different scenarios is strictly a qualitative description, rather than an attempt to express impacts quantitatively. It expresses the overall implications of the drivers on each objective, according to the judgment of experts. The discussion and aggregation was conducted according to two rules: (1) the most adverse category and (2) likelihood of cumulative effects.

A scenario as a whole should generally be judged according to the most adverse assessment of all the drivers on a particular objective, although the relative importance of each of the drivers needs to be taken into account. For example, if driver 1 has a “- -” assessment on a particular objective, driver 2 has a “-” assessment on the objective, and the remaining drivers have a “0” assessment on the objective, unless driver 1 is a minor driver for the scenario relative to the other drivers, the overall assessment score should be “- -”. The rationale for this approach is that highly adverse impacts should not be diluted or masked by less adverse impacts (this also encourages the development of alternative options which avoid such adverse outcomes).

A cumulative effect can take place where a range of drivers (and impact chains) affect a particular objective, leading to negative or positive consequences which can be more than the sum of individual effects. For example, several drivers may negatively affect a particular objective, each of which is assessed as “-”. Where it is clear that there is an adverse cumulative effect across the drivers, the scenario as a whole would be assessed as “- -”. Therefore, the scenario as a whole should be scored in a higher category than the drivers in isolation.

Where there are both positive and negative effects on a particular objective, experts have to make a judgment about what the overall level of effect will be, always ensuring the same logic prevails.

Combining Matrices

SA and SEA literature discusses the relative benefits/drawbacks of SA/SEA being carried out primarily by independent experts or by people directly involved in the study area (e.g., James and Donaldson 2001; Smith and Sheate 2001b; ODPM 2005a, b; Therivel and Walsh 2006). To reduce the risk of bias by one or other group, the matrices in BioScene were initially completed separately by the local partners and by the SA Team. The results were then compared and discussed by partners and the SA Team (over the telephone or via email correspondence) to enable partners to produce agreed combined versions of the two matrices, which were presented and discussed at the third stakeholder meeting. The iteration between partners and the SA team was valuable since it provided for a more reasoned assessment than if undertaken by only one or other group. At the end of this process the outputs were one draft assessment matrix per scenario and one draft scenario comparison matrix.

Finalizing Matrices with Stakeholders

At the third stakeholder meeting, the results of the draft SA comparison matrix (see the example from France in Table 6), as well as extracts from the individual scenarios draft assessment matrices where appropriate, were presented to the stakeholders to enable them to discuss key aspects. As the assessment produces different matrices it was important to select the most appropriate amount of information to be presented to specific study area stakeholders. Partners decided how best to present the information given their knowledge of the stakeholder panel (one size does not fit all). Again, one or more members of the SA Team were generally present at the meeting to help support the partners in facilitating the meeting, as well as learning about the group dynamics. The meetings generally began with a brief introduction and reminder about the nature of the scenarios and the purpose and basis of the SA process. The following detail was tailored to the needs and capacities of each stakeholder panel and required a significant degree of flexibility in facilitation. Generally the debate was focused around four aspects: (1) the uncertainty of some of the more significant consequences (positive or negative) and scrutiny of the overall draft assessment; (2) the general “acceptability of” or preference for particular scenarios: possible trade offs between one set of consequences and another, using the six sustainability themes; (3) approaches to trade offs, e.g., definition of a minimum set of trade-off rules (Gibson and others 2005; Gibson 2006; Keough and Blahna 2006) based on the expectations and preferences of stakeholders; (4) the possibility of changing or mitigating some of the problems that have been “traded off,” suggesting how the policy interventions and management activities could be made more sustainable.

Table 6 Example overall comparison matrix results from France (excluding comments)
Full size table

As a general rule, it was agreed that in considering trade-offs, it would be preferable to follow this order: First, decide which impacts (social, environmental, and/or economic) are simply unacceptable according to experts and stakeholders; second, seek ways to avoid unacceptable impacts; third, reduce or mitigate impacts; fourth, compensate for impacts that cannot be avoided or mitigated. Following the third stakeholder meeting, the assessment matrices were revised in light of the discussion with stakeholders.

Sustainability Reports

Production of Sustainability Reports

The results of the SA process fed into the production of a Sustainability Report for each study area, discussing the sustainability of each of the three scenarios and making recommendations as to how policy measures and management activities might be made more sustainable. For some study areas, shorter summary pamphlets were also produced to aid consultation with the wider community (though this was at the discretion of the partners, tailored to local circumstances). These reports (and pamphlets) were completed in October 2005. Reports outlined the BioScene project (for context), summarized the SA process undertaken, explained the purpose of the wider consultation process and the key issues on which consultation comments were sought, e.g., the assessments of sustainability of each of the scenarios, suggestions as to how particular scenarios might be made more sustainable, preferred scenarios. These reports also described the scenarios which were assessed, summarized the baseline information, presented summary forms of the SA matrices, summarized and discussed the opinions of stakeholders as expressed during the third stakeholder meeting, summarized and discussed the sustainability of each of the three scenarios and recommendations for making policy interventions and management activities more sustainable, and explained the final outputs of the SA process and how comments from consultation would be used.

Consultation on Sustainability Reports/Pamphlets

The sustainability report/pamphlet (as appropriate) was distributed for wider consultation requesting comments/feedback (for example, to stakeholders and more widely to experts, organizations, and the public with an interest in the study area). A consultation period of at least three weeks was allowed to give people enough time to submit comments. The consultation feedback, perhaps not surprisingly, was not extensive, but the SA process was generally well received by the stakeholders. For example, in Norway, the stakeholders saw the landscape as an expression of the overall situation in the study area, linked to livelihoods, social, cultural, and aesthetic/visual aspects, and did not consider biodiversity as the major issue, although they recognized aspects of biodiversity, such as certain plants, that were significant culturally to them. As such biodiversity was seen more as a function of the wider landscape (Rønningen and others 2005). In this respect, stakeholders often had a more intuitive understanding of sustainability and SA than some experts, especially those more quantitatively or theoretically inclined who were more suspicious of the approach, e.g., agricultural economists consulted in Switzerland (Soliva 2005). A consultation feedback summary document was produced for each study area and appended to the Sustainability Report as an Addendum. The Sustainability Report and consultation feedback summary provided key inputs into the final conclusions and recommendations work package of the BioScene project.

Discussion

BioScene was a challenging project, not least because of its interdisciplinary nature, six country case studies and stakeholder panels, ten institutional partners, and three “disciplinary” streams: ecological, socio-economic, and sustainability assessment. The degree to which partners in each country worked closely and efficiently together, e.g., between ecological and socio-economic teams varied considerably, some being very used to interdisciplinary/transdisciplinary working, some located in the same institutions (though interestingly this did not necessarily make for better working), and some finding it difficult to shed traditional disciplinary perspectives and boundaries. However, the working relationship overall was extremely constructive and partners generally delivered in a timely manner. Importantly, the SA process provided an important mechanism through which ecological and socio-economic teams collaborated, e.g., by being in the same room!

From the SA point of view, this was a very new subject for many of the partners. Although the SA process was begun right from the start of the project (setting of objectives), specifically to ensure partners engaged in the SA process from the start, some individuals were inevitably suspicious of SA. Some quantitative ecologists saw it as “unscientific” as it appeared highly qualitative (even subjective) rather than quantitative (even though “quantitative” ecological models contain qualitative assumptions). On the other hand, some socio-economists were equally skeptical, believing SA to lack real theoretical grounding in social science. The challenge for the SA teams was to work closely with nonassessment specialist partners, building their willingness and capacity to undertake SA, so that they could also facilitate their own stakeholders to engage in the SA process. This was an exciting challenge for the SA teams and had varying degrees of success. Limitations to success were often due to resource constraints (i.e., an inevitable tendency for some partners to prioritize time and money to their primary areas of activity), even though specific resources had been earmarked for SA. This manifested itself especially in reluctance among some to prioritize interdisciplinary academic papers when individual research profiles are most often best enhanced by single disciplinary publications in high ranking journals (the perpetual bane of interdisciplinary studies). However, there were also notable successes in turning initial skepticism about SA into real acceptance (e.g., Norway; see Olsson 2005) and even positive enthusiasm for the techniques (e.g., in France), where subsequent joint research activities in the SA area have been developed. Others (e.g., Slovakia) were already open to the SA process from the start. Furthermore, there was widespread recognition among all partners of the value of the sustainability reports as useful documents for communicating the overall issues addressed by BioScene.

Perhaps most interesting was the extent to which stakeholders were often much more open to the concepts of SA than some individual members of partner teams. Indeed, in some case study areas the stakeholders often had an intuitive understanding of sustainability in their area and had no difficulty understanding and engaging with the SA process. This may be because their concerns were often rooted in consideration of livelihoods (i.e., the social and economic dimensions of sustainability), which in mountain areas are likely to be inextricably linked to the natural environment. This finding does, perhaps, add support to the argument for taking a transdisciplinary approach to sustainability, where the participation of stakeholders is central to the study and to decision making.

The benefit of SA team members attending the relevant stakeholder meetings, even where unable to understand the language, was seen primarily in the extent to which they were able to support the partners in understanding, appreciating and undertaking the SA process, and to develop a first hand understanding of the dynamics of the stakeholder deliberations. The specific sustainability lessons for the study areas from the BioScene SA application are reported elsewhere (Mitchley and others 2006; Partidario and others submitted), although it is important to note that the Managed Change for Biodiversity scenario was found to be consistently the most sustainable of the scenarios for all the study areas. It was also preferred by stakeholders, not least because it was seen to promote win-win situations in terms of livelihoods and biodiversity (Keough and Blahna 2006).

Finally, the SA methodology developed for BioScene, although a research project has resonance in a very practical way, for example in undertaking strategic environmental assessment (SEA) of the new round (2007–2013) of Member State Rural Development Plans under the EC Rural Development Regulation No. 1698/2005 (EC, 2005). The techniques developed in BioScene addressed directly financial support to the rural economy and agri-environment schemes, now playing such a significant role under the reformed CAP. These rural development plans are very strategic plans and the use, for example, of causal chain analysis as developed in BioScene can be helpful in making the connection between high level funding schemes to support certain activities and the assessment of their environmental consequences, both positive and negative (Sheate and Kiely 2007). The causal chains also, therefore, help to link knowledge of the baseline data more closely to the assessment itself by providing transparency to the process of tracing the pathways of effects from broad drivers down to land use change and impacts upon agricultural and biodiversity management on the ground.

Conclusions

Generally, for SA methodology there are a number of key conclusions that can be drawn from the BioScene experience. Comparing the BioScene SA process to many other SA processes in the literature (e.g., Dalal-Clayton and Sadler 2005; Gibson and others 2005), a significant development can be seen, most notably in the use of extensive ecological and socio-economic data in creating a robust baseline against which to reference the assessments. The creation of a baseline reference document in this case was important to pull together data from other elements of the BioScene project and provide a basis for understanding through causal chain analysis the effects of the drivers on land use, landscape, and biodiversity. The implementation of the SEA Directive in Europe now requires the collation of appropriate baseline data and, where implemented in conjunction with SA (e.g., in the UK), is now helping to strengthen the role of baseline data in informing strategic assessments. While time consuming, it is essential to provide sufficient grounding for strategic thinking. The appropriateness of the data, though, is critical, i.e., in terms of the scale and relevance to the level of decision making.

On the other hand, the use of scenarios presents a particular problem for assessment. For real plans and programs the object of assessment will be the individual policies or proposals contained in the plan or program, such as land use plans. For scenarios, the scenarios themselves are the object of assessment, or rather the drivers which characterize them. But drivers in themselves are rather nebulous and remote from physical impact on the ground, hence the need for causal chain analysis. The causal chain analysis therefore provided the critical link between the baseline data and the assessment against objectives. This proved to be a valuable technique and one which ecologists and socio-economists could use together through brainstorming. It therefore served the dual purpose of connecting the baseline to the assessment and providing a key mechanism for facilitating interdisciplinary working. Causal chains (or network analysis), of course, are nothing new (e.g., Sorensen 1971). However, in practice such diagrams are a rarity in environmental impact statements and only recently have increasingly been used in the context of SEA, and especially with respect to cumulative effects assessment. So the use of causal chains should not be restricted to working with scenarios, and indeed the more explicit use of causal chain analysis would be useful elsewhere, e.g., for assessing land use plans.

While stakeholder involvement in BioScene was in practice perhaps more consultative than truly participative, SA nevertheless proved to be a technique which stakeholders generally found appropriate, understandable, and well able to engage with in active debate. The central importance of the stakeholder panels throughout the three years of the project ensured the project at least had an important element of transdisciplinarity. However, a more participative approach would be likely to engender a greater sense of ownership of the SA process within the community concerned (Keough and Blahna 2006), which would be particularly valuable in a formal planning and decision making process (rather than a research context). The use of nonSA experts for facilitating consultation and undertaking the assessments also provided valuable lessons into the techniques best suited to building SA capacity among experts and stakeholders (e.g., particular types of workshop exercises). Furthermore, the importance of making a research project relevant to stakeholders, the creation of opportunities for different disciplinary teams physically to work together, and the importance of ongoing communication between teams (within a country and across countries) were all important lessons resulting from the SA process.

BioScene has also provided important experience of applying SA in the agricultural and biodiversity sector and how SA can be used in conjunction with scenarios and stakeholder engagement. While compromises were inevitable in such an ambitious and complex project — e.g., in terms of the extent and effectiveness of public participation, the nature and variety of the scenarios chosen, and the selection of sustainability objectives — the SA methodology proved to be workable and one with which both partners and stakeholders were able to engage and find some value. It also provides further evidence that win-win situations are possible (Keough and Blahna 2006), particularly where stakeholders’ livelihoods are closely connected to the natural environment.

The practical application of BioScene SA techniques to Rural Development Plans under the EC RDP Regulation No. 1698/2005 is just one example of how the methodology can be applied in practice, but there is no reason why it could not be applied equally at the level of area and regional plans as well as country-wide plans, and across a range of different sectors. The causal chain approach, though, does lend itself particularly well to helping understand the possible consequences of very strategic actions, where the level of detail is relatively low. In this way, causal chains create an important link between the baseline data and the assessment itself. The development of appropriate and practical methodologies for high level strategic plans in particular is needed urgently, in order to avoid an overly mechanical approach to assessment becoming the norm and, importantly, to inject a renewed sense of innovation and creativity into strategic assessment thinking.