1 Introduction

Life cycle assessment (LCA) is a standardized methodological approach (ISO 2006a) aimed at assessing the potential environmental impacts associated with a product, service, or system. LCA is a key tool in pursuing sustainable production and consumption patterns which has been increasingly integrated into the policymaking process, either at the stage of policy design and impact assessment or directly into legislative documents (Sala et al. 2016).

The ISO 14040 standards series (ISO 2006a, b) define the principles and provide a framework for carrying out an LCA study. The framework consists of four phases, namely (1) the goal and scope definition, (2) the inventory analysis, (3) the impact assessment phase, and (4) the interpretation phase (see Fig. 1).

Fig. 1
figure 1

Life cycle assessment framework, adapted from (ISO 2006a)

Full size image

In the first phase, the intended application and the reasons for carrying out the study are set (i.e. the question addressed in the study is stated). In the second phase, an inventory of input/output data related to the system under study is compiled. The purpose of the third phase is to provide additional information to help assess a product system’s LCI results, and to better understand their environmental significance.

While the first three phases are deeply interconnected; in particular, the second and the third are determined by the first, and the fourth phase is the key step in which the whole LCA study is scrutinized for its quality and its capability to fulfil the question set in the goal and scope definition phase. According to the ISO 14044 standards (ISO 2006b), the interpretation phase should deliver results that are consistent with the defined goal and scope, which reach conclusions, explain limitations, and provide recommendations. The interpretation phase, thus, is the key step that guarantees quality and consistency and gives meaning to the work carried out.

However, in previous reviews and synthesis of LCA studies on bioenergy (Marelli et al. 2016; Agostini et al. 2013; Rocca et al. 2015), we have observed a scattered implementation of the ISO recommendations on the interpretation phase. We have also noticed that, although there is broad agreement in the scientific community that LCA is one of the most effective methodologies for the evaluation of the environmental burdens associated with bioenergy production (Cherubini et al. 2009), there is still a heated debate in the scientific community on the actual environmental sustainability of bioenergy production, and in particular on its potential to mitigate climate change (examples of exchanges in this debate (Chatham House 2017; International Energy Agency - Bioenergy 2017)).

We have hypothesized that the debate on this divisive topic is, at least in part, fuelled by flaws in the interpretation phase of influential bioenergy LCA studies rather than by the lack of a shared methodology, as suggested by other authors who have critically reviewed the available literature on LCA of bioenergy trying to compare and synthetize the results (Cherubini et al. 2009; Gnansounou et al. 2009; Cherubini and Strømman 2011). Their findings suggest that large differences in greenhouse gases (GHGs) balances of biofuels stem from modelling choices about system definition and boundaries, functional unit, reference systems, and allocation methods. However, we think that such exercise is not pertinent as the LCA studies considered have different goals and, rightfully, they set different scopes and provide different conclusions.

As far as the goal is properly set and described, and the methodological approach is consistent with the goal set, LCA analyses provide meaningful insights on the environmental performances of products and systems, but results of studies with different goal and scope should not be compared. On the contrary, we think that flaws in the interpretation phase of LCA studies of bioenergy may seriously mislead readers and lead to conflicting and divisive results. To verify our hypothesis, we have performed a critical review of the interpretation phase of some of the most cited and influential LCA studies of bioenergy; in a second stage, we draw recommendations for LCA practitioners on how to improve the quality of their studies, and to policymakers, stakeholders, and the general audience on how to read and properly interpret the results of LCA studies of bioenergy.

2 Methods

To test our hypothesis, we have selected the 100 most cited articles found in SCOPUS utilizing the following query: KEY ((“lca” OR “life cycle assessment”) AND (“bioenergy” OR “biogas” OR “biofuel” OR “bioethanol” OR “biodiesel” OR “wood pellet”)). These research parameters should include most of the relevant works on LCA of bioenergy published before the 3rd of May 2018. The rationale underpinning the choice of the most cited articles is that these are presumably the most influential studies up to now. Our query method is applied consistently, so that certain influential papers may not appear in the search results because of the lack of bioenergy-related keywords or because more novel concepts may not have reached the number of citations to appear in our top 100. We reiterate that our review provides a snapshot of the status of the most cited LCA bioenergy research up to 2018; these results may look different in a few years as new concepts and methodologies become mainstream. Nonetheless, we provide an up-to-date methodological discussion in this paper which captures concepts and approaches well beyond what appears in the search results. A further screening identified off-topic articles, reviews, and methodological papers, which were discarded. We have also checked whether the articles analysed referred to the ISO standards.

The work is organized as a reasoned and parametrized review. We refrain from assessing the quality of the articles in terms of systems chosen, input data, system boundaries, inventory modelling, allocation, etc., which are part of the goal and scope definition; we have rather focused on the consistency between goal, scope, and conclusions reached and on how the results are presented and interpreted. The parameters assessed are described in the following sections. The list of reviewed articles is reported in the Electronic Supplementary Material (ESM) together with their categorisation and extensive justification for our limitation and consistency assessment. The reasons for the exclusions are also reported in the ESM.

2.1 Modelled bioenergy systems

We have carried out a statistical parameterized analysis of the following aspects describing the type of bioenergy system analysed:

  1. A.

    Energy carrier produced

  2. B.

    Feedstock used

  3. C.

    Main processing technology modelled

2.2 Geographical scope and timing

The categories defining the geographical scope were determined as follows:

  1. A.

    Local: a specific plant or an area with homogeneous characteristics

  2. B.

    Regional: large inhomogeneous areas, countries, or set of countries

  3. C.

    Global: the whole world, or all the areas producing a specific product

  4. D.

    Not applicable as there is no reference to a location or the location is irrelevant

The timing of pollutants’ emissions may be relevant, especially if climate change is considered. Identifying whether the emissions of GHGs (greenhouse gases) take place at the beginning or at the end of a project, or the temporal imbalance between emissions and absorption by plant growth may be of fundamental importance to properly grasp the climate change mitigation potential of a project or policy. We therefore assessed whether the analysed articles considered time-dependent inventory as well as time-dependent impact assessment, e.g. through absolute climate metrics (Giuntoli et al. 2015).

2.3 Socio-economic impacts

In previous works, we observed that often LCA studies of bioenergy claimed to reach conclusions on the sustainability of the bioenergy produced, while only focusing on environmental impact assessment. We argue that sustainability assessments should include environmental, social, and economic considerations (Sala et al. 2015). We therefore assess also whether these two aspects (social and economic) were considered in the analysed articles to verify whether the authors assessed the sustainability of a bioenergy system consistently.

2.4 Life cycle impact assessment

The first distinction in the evaluation of the life cycle impact assessment (LCIA) was made between studies considering endpoint and midpoint impacts. Endpoint methods (or damage-oriented approaches) are models that provide indicators at or close to the level of Areas of Protection (natural environment’s ecosystems, human health, resource availability) (ISO 2006b; European Commission 2011). They provide more aggregated results, simplifying the interpretation; however, much of the information is lost. For example, endpoint methods may aggregate all the emissions impacting human health into one indicator, e.g. disability-adjusted life years (DALY), losing the information on whether the impact comes from emissions to air or water.

A midpoint method is a characterisation method that provides indicators to compare environmental interventions at a level of cause-effect chain between emissions/resource consumption and the endpoint level (European Commission 2011). It aggregates the emissions relevant for a specific area of environmental concern without considering the damage. For example, GHG emissions are aggregated in one indicator, measured in kilograms of CO2 equivalent, without assessing the impact on human health or to the environment.

Both methods, at endpoint or midpoint, are valid and accepted as far they can fulfil the needs of the goal and scope defined in the study. Specifically, endpoint methods are helpful in the comparison of alternative products, keeping in mind that some information is lost, while for eco-design purposes the endpoint methods are to be avoided, as it would be impossible to identify where the impacts are generated along the production chain.

Though several impact assessment methods at midpoint are available in literature for each environmental impact category, we have categorized all the studies in the 15 impact categories reported in Table 1, irrespectively of the methods used.

Table 1 Environmental impact categories used for the classification of the reviewed studies
Full size table

As most of the studies on bioenergy are focused on climate impacts, we have further parametrized the impact assessment method for climate change to clarify to what extent the assessment is comprehensive of all the climate forcers and, if not, whether this limitation is identified and reported.

Bioenergy products and system can impact the climate through well-mixed GHG (WMGHG) but also by near-term climate forcers (NTCF, aerosols, and ozone precursors), and other biogeophysical forcers such as evapotranspiration and albedo. Several metrics can be used to assess the potential impact on climate change. Global warming potential is the most used one; it is defined as the cumulative increase in radiative forcing of the emission of 1 kg of a gas, relative to the increase in radiative forcing from release of 1 kg of CO2. Global warming potential (GWP) is integrated over a specific time horizon (e.g. in international treaties, such as the Kyoto Protocol, it is mostly used in the 100-year time frame). The global temperature change potential (GTP) goes one step further in the cause-effect chain and is defined as the change in global mean surface temperature at a chosen point in time in response to an emission pulse, relative to that of CO2 (IPCC 2014; Cherubini et al. 2013). These metrics answer different questions. GWP 100 can be used to understand the average long-term impact of GHG emissions, while GTP 20 is suitable to characterize the short impact on global surface temperature 20 years after the emission has taken place (Frischknecht and Jolliet 2016). Absolute metrics, which assess impacts not relative to CO2, can be used as well, and as shown in Giuntoli et al. (2015), they can better show the evolution in time of the earth surface temperature response not only to WMGHG emissions but also to NTCF, and other biogeophysical climate forcers (see also Cherubini et al. 2012; Brandão et al. 2013; Kirschbaum 2014).

Other environmental impact assessment methods can be interpreted using different metrics to answer different questions. For example, as regards eutrophication, the emission of nutrients can be analysed as total amount emitted, or amount per hectare (see Battini et al. 2016).

2.5 Modelling approach

One of the main causes of inconsistency, and therefore of misleading results, lays on the choice of the inventory modelling approach which may not be appropriate for the goal of the study (Plevin et al. 2014). Modelling approaches can be consequential, attributional, or a combination of them (hybrid approaches). Consequential modelling is defined as a “System modelling approach in which activities in a product system are linked so that activities are included in the product system to the extent that they are expected to change as a consequence of a change in demand for the functional unit”, and attributional modelling is defined as “System modelling approach in which inputs and outputs are attributed to the functional unit of a product system by linking and/or partitioning the unit processes of the system according to a normative rule” (Sonnemann and Vigon 2011).

In short, consequential modelling refers to an inventory modelling approach that considers also the scale effects. This type of modelling aims at internalizing the market-mediated impacts caused by a change in the installed capacities of a system on the rest of the economic system. This modelling approach is suitable for capturing the impact of macroscale choices, such as policies aimed at changing the installed capacities. By contrast, the attributional approach models the impacts of a specific amount of product without considering impacts on other sectors of the economy, and it is therefore valid when installed capacities are not impacted, either because the study aims at supporting a microscale decision, or because it is performed for accounting purposes.

The inventory modelling categories considered are as follows:

  1. A.

    Consequential

  2. B.

    Attributional

  3. C.

    Attributional with elements of consequential

Category C has been added because some of the reviewed articles have added impacts measured with a consequential approach (e.g. indirect land use change factors) to attributional results.

2.6 Multifunctionality

Bioenergy systems are often multifunctional systems providing more than one product or service along the supply chain (e.g. soybeans produce vegetable oil for biodiesel and protein meal as feed). There are two main ways to solve multifunctionality (ISO 2006b): by allocation (i.e. allocating shares of the impacts to the different products or services) or by system expansion.

System expansion is defined by (ISO 2006b) as “Expanding the product system to include the additional functions related to the co-products”; in practice, in comparative studies, a system providing two products is compared with two, or more, systems providing the same products or level of services. Allocation can be carried out considering the economic value, or some physical properties of the products.

The approach to multifunctionality in the papers was therefore placed into one, or more, of the following categories:

  1. A.

    System expansion

  2. B.

    Economic

  3. C.

    Exergy

  4. D.

    Mass

  5. E.

    Carbon content

  6. F.

    energy

The choice of the approach to solve multifunctionality has large impacts on the results of the study. ISO14044 standards (ISO 2006b) recommend system expansion; however, the choice of the product replaced is a subjective choice which should be addressed with a sensitivity analysis and reported in the limitations (e.g. assuming that glycerol from biodiesel replaces glycerol used in nutraceuticals should take into account the different scales of production). The economic allocation represents well the drivers for any activity to take place; however, given the volatility of the markets, results may not be consistent and comparable over time. The allocation procedures based on physical properties, on the other hand, allocate the impacts to the different products based on fixed properties which, though stable over time, may not reflect properly the drivers for a specific production (e.g. allocation by mass in the case of meal and oil in algae biodiesel production would allocate most of the impacts to the meal, while it is not the main driver for algae cultivation, and especially if wet biomass is considered, it would be disproportionate (Rocca et al. 2015). It should be noted that allocation causes imbalances in the allocated datasets. Allocation breaks up the original system into two or more artificial systems according to an allocation key, and the only balance that remains intact in the resulting systems is that given by the allocation key. In fact, with mass allocation, the mass balance remains intact, but energy and elemental balances are skewed; with economic allocation, none of the physical balances remains intact, unless, by chance, a physical parameter follows the price of the products (Weidema and Schmidt 2010).

2.7 Main assumptions

Few methodological assumptions, which may affect the consistency of the results and/or would require a particular attention in describing and interpreting the results, were identified.

The biogenic carbon neutrality is a very common assumption through which the LCA practitioner avoids accounting for the biogenic carbon cycle by assuming that the carbon emitted from biomass combustion or decomposition will be reabsorbed by the growing plants on a time scale significantly shorter than the relevant scale of the analysis. In some cases, e.g. for assessing the impacts of processing alternatives (eco-design), this assumption would not jeopardize the results; however, when it is applied when performing a strategic assessment of the climate change mitigation potential of bioenergy options compared to other sources of energy, it may lead to erroneous conclusions (Agostini et al. 2013; European Commission 2016; Camia et al. 2018). Therefore, we have assessed whether the articles analysed have included all the most relevant carbon pools for the specific case.

Another common assumption that was tested in this review is that of perfect substitution. With this assumption, the LCA practitioner assumes that bioenergy replaces a given amount of another source of energy or that a co-product replaces an alternative product. While the use of this assumption may be appropriate, and the results properly interpreted with the limitations clearly presented, the use of this assumption based on subjective choices, without a proper analysis of the impact on markets, both in terms of type of products and amount replaced may lead to misleading results.

The assumption that the impacts deriving from the construction of the infrastructures needed to produce the bioenergy are negligible may have an impact on the results and conclusions of the LCA studies. We have therefore also analysed how often this assumption was made.

2.8 Uncertainty and sensitivity

Uncertainty in bioenergy LCA studies, like in any other modelling exercises and any scientific enquiry, is an aspect that shall be thoroughly investigated before drawing conclusions and recommendations. In all the studies reviewed, most or all of the inventory data used are secondary data, taken from models or literature; therefore, LCA practitioners should provide a clear description of what is the level of uncertainty in the results. This could either follow an uncertainty propagation assessment (e.g. Monte Carlo simulation) or, at minimum, a sensitivity analysis of the most important parameters, as recommended by ISO14044 standards (ISO 2006b). Hence, we assessed whether the reviewed articles presented sensitivity or uncertainty analyses.

2.9 Limitations, conclusions, and recommendations

From our previous studies, we have also noticed that another frequent cause of potential misinterpretation is the lack of limitations reporting. The ISO standards, as well as any other manual or guidelines (e.g. JRC 2010), recommends to present LCA results together with their limitations, to offer the reader the tools for understanding the applicability and significance of the results. We have therefore checked the presence and completeness of limitations reporting, which were parameterized in three categories:

  1. A.

    Properly identified and reported

  2. B.

    Incomplete

  3. C.

    Missing

The conclusions and recommendations drawn in the reviewed studies are classified according to 4 main categories of LCA outcomes to check whether they correspond to the goal and scope set for the studies. The 4 categories, with the definition used in this study, are as follows:

  1. A.

    Eco-design: studies which draw conclusions and recommendation on the best way to reduce the impacts of processing (e.g. which source of energy for distillation or drying is to be preferred to reduce the environmental burden of a given bioenergy pathway, what are the trade-offs due to increased use of inputs and increased yield of farming intensification)

  2. B.

    Accounting: studies which provide quantification of the environmental burden of a specific bioenergy system

  3. C.

    Benchmarking: conclusions are drawn comparing the environmental performances of several different bioenergy pathways among them and/or with other sources of energy

  4. D.

    Policy-relevant (installed capacities): conclusions and recommendations are drawn on the desirable scale and/or impacts of bioenergy expansion

2.10 Consistency check

From our experience, the most frequent cause of misinterpretation of LCA results is the lack of consistency. In particular, we noticed that way too often LCA practitioners draw conclusions on aspects or scales not addressed at all in the study. We have therefore assessed the studies to verify their consistency according to the definition provided by the International Organisation for Standardisation (ISO) (ISO 2006b). In the ISO standard 14044 “Environmental management - Life cycle assessment - Requirements and guidelines” (ISO 2006b) consistency check is defined as process of verifying that the assumptions, methods and data are consistently applied throughout the study and are in accordance with the goal and scope definition performed before conclusions are reached.

We have identified the following four typical consistency categories which describe all the consistency performances of the analysed articles:

  1. A.

    CONSISTENT: the conclusions and recommendations are robust and well presented.

  2. B.

    MISLEADING: the results are robust, but the conclusions and recommendations may be misleading because drawn on aspects or scales not analysed, because the modelling approach or main assumption would allow for different conclusions or because of the lack (or partial lack) of limitations’ reporting.

  3. C.

    INCONSISTENT: inconsistencies between the goal and/or conclusions and the modelling approach, impacts analysed, metrics, and assumptions make the study inconsistent.

  4. D.

    POOR: lack of transparency, poor data quality, inconsistent system boundaries, incomplete inventories, and speculative assumptions make the study misleading.

While categories A and D are clearly identifiable, the border between B and C may be subjective. We have positioned in B the studies which can be interpreted correctly (e.g. the practitioner has built a good inventory and impact assessment), but in the conclusions, the limitations are not considered (e.g. biogenic carbon neutrality, perfect substitution, allocation). Instead, in category C are those studies which are misleading because the methodological approach does not allow the reader to draw consistent conclusions (e.g. methodological choices throughout the study do not enable proper comparisons among bioenergy pathways and other sources of energy).

3 Results and discussion

As the objective of this review was to analyse whether the interpretation phase was performed properly, all articles which could not be assessed with the method described in Sect. 2 were discarded. The 100 articles chosen were selected from a total number of 171 papers. Among the excluded articles, 26 were off-topic, 23 were reviews, and 22 were methodological articles. The final list of articles considered is reported in the ESM. Out of the 100 papers selected, 52 make explicit reference to the ISO standards.

3.1 Modelled bioenergy systems

The most common feedstocks assessed in the studies are residues and corn (see Fig. 2a). The oldest articles mainly dealt with first-generation biofuels, such as corn ethanol or biodiesel, but the focus progressively moved towards bioenergy from biomass residues. The trend in LCA studies mirrors the actual trend in industry and policy where support was shifted towards advanced conversion technologies relying on biomass residues (European Union 2018), and that was partly driven by the increasing awareness of the unsustainable consequences of food-based biofuels unearthed by LCA studies (Searchinger et al. 2008; Broch et al. 2013; Valin et al. 2013).

Fig. 2
figure 2

Statistical data on the bioenergy systems modelled: a feedstocks used; b energy carriers; c processing technologies

Full size image

Surprisingly, algae are one of the most studied feedstocks, even though currently there are no commercial-scale plants available. This is very likely due to the attempt of identifying an economic and environmentally sound biorefinery concept; in fact, most of the studies on algal biofuels fit into the eco-design category.

The most studied energy carrier is biodiesel, produced both from oilseeds and algae, followed by other transportation fuels, ethanol, biomethane, and biogasoline (see Fig. 2b). This trend highlights the research focus on finding climate change mitigation solutions for the transport sector, the most difficult to be decoupled from fossil fuels. We observe the same trend in processing technologies, where extraction and esterification, again for both algae and oilseeds, are the most frequently studied, followed by fermentation, digestion, and thermochemical processing (see Fig. 2c).

3.2 Geographical scope and timing

About one-third of the studies in our sample (30/100) are performed at the local level, meaning at the level of a specific plant or of a small homogeneous geographic area (see Fig. 3a). These studies normally rely on site-specific data or on commercial technologies, and thus may have a low level of uncertainty but may lack in representativeness. About two-thirds (62/100) of the studies analysed regional case studies, where by regional we mean a region or country with inhomogeneous characteristics. In general, we have observed that at the regional level the uncertainty increases, because data averaged over the entire region or data produced from models are used. A few cases were not located geographically, mainly because just the processing was modelled and there was no need to specify the location. Only two studies address bioenergy production at the global level.

Fig. 3
figure 3

a Geographical scope of the articles. b Time modelling approach

Full size image

Regarding temporal modelling, practically all the papers reviewed but one (McKechnie et al. 2011) have modelled static inventories; therefore, the emissions trends are not captured (see Fig. 3b). Some articles have annualized emissions from land use change or SOC changes; we have considered these studies as static since the information on when the emissions take place is lost. While for most impact categories this approach is reasonable, for climate change impacts the specific emission profiles can greatly influence the final result (see Giuntoli et al. 2015).

3.3 Socio-economics

As previously stated, sustainability is more than environmental sustainability; sustainability assessment should include environmental, social, and economic aspects (Sala et al. 2015). Although some of the articles claim to analyse the sustainability of bioenergy production, or even draw conclusions on the sustainability of bioenergy, only in 3 cases were the impacts on labour analysed, and only in one paper were aspects related to rural development considered (see Fig. 4a). The economics of bioenergy production were analysed in 18 cases as direct costs and in 2 cases as indirect costs (cost of externalities) (see Fig. 4b).

Fig. 4
figure 4

Statistical data on the aspects analysed and methods adopted: a social aspects; b economic aspects; c impact assessment methods; d climate change forcers

Full size image

3.4 Life cycle impact assessment

Most of the studies (84/100) have performed their analyses at midpoint, i.e. the emissions or consumptions were calculated and grouped using characterisation factors (see Fig. 4c). Six studies limited their analysis to the provision of inventories of emissions or consumption. In 4 cases, the endpoint methods were used.

Climate change impacts are the most investigated (90 papers out of 100, see Fig. 5). Bioenergy, or at least modern bioenergy, is often seen as a tool to fight climate change, and in most developed countries its use is promoted for that reason; therefore, most studies aim to verify whether various bioenergy pathways can be a valid climate change mitigation option.

Fig. 5
figure 5

Environmental impact categories

Full size image

According to the International Panel on Climate (IPCC), the climate is impacted by several climate forcers: WMGHG (well-mixed GHG), including CO2, CH4, and N2O, and NTCF (near-term climate forcers), such as aerosols and ozone precursors, and biogeophysical forcers such as surface albedo and evapotranspiration (IPCC2013). As reported by Agostini et al. (2013), bioenergy production and consumption impact the climate via all these climate forcers. However, most studies, 82 out of the 90 which analyse climate change impacts, have considered solely WMGHG. Seven studies have analysed only carbon dioxide emission. Only in one case were other biogeophysical forcers other than WMGHG considered (see Fig. 4d).

Energy consumption is the second most analysed impact (64/100). This result is not surprising, with bioenergy being produced to provide energy services; the analysis of its energetic performances, for comparative or eco-design purposes, was to be expected. Eutrophication and acidification, which impact heavily the environment during cultivation, but also combustion processes, are analysed in about half of the studies. The impact categories most relevant for human health, POCP (photochemical ozone creation potential) and toxicity were analysed by 27 out of 100 studies. Most of the studies assess a very limited number of environmental impact categories (see Fig. 5).

3.5 Inventory modelling approach

As stated in Sect. 2.5, we have categorized the choice of a modelling approach according to JRC (2010). Practically, almost the totality of the studies has adopted an attributional approach (97/100, see Fig. 6a). This means that one functional unit of the product, either a kilogram, a petajoule, or 1 ha, was modelled, in a static way, without considering scale-dependant impacts on other sectors of the economy, the so-called market-mediated impacts. In 2 cases, elements of consequential modelling, namely indirect land use change (ILUC) factors, were mixed within an attributional approach. We refrain from questioning whether the use of elements of consequential thinking into attributional modelling may be appropriate. However, we believe that when it comes to providing results that will influence strategic decisions, it is better to be roughly right than precisely wrong and include all the elements necessary for a more complete assessment even at the expenses of precision (Brandão et al. 2014). However, it is clear from the results that almost all the articles are not suitable to support policies aimed at changing the installed capacities. Attributional studies are used for eco-design (i.e. identifying ways of reducing the environmental burden of a process or product) or for identifying criticalities and hot spots in a supply chain, or for accounting purposes. Consequential modelling, on the other hand, is focused on modelling scale effects and therefore is appropriate for supporting policies and strategic decisions aimed at promoting bioenergy by analysing most of the expected impacts of bioenergy expansion (e.g. competition for feedstocks, competition for land, competition with other renewables or energy sources, rebound effects). One of the most frequent causes of inconsistency in bioenergy LCA studies is related to the use of the attributional approach to draw policy relevant conclusions with recommendations on the installed capacities.

Fig. 6
figure 6

Statistical data on the methodological approach adopted by the articles analysed: a inventory modelling approach; b approach to solve multifunctionality

Full size image

3.6 Multifunctionality

Multifunctionality was solved in most cases (48) by system expansion, i.e. by subtracting the emissions to produce an amount of product delivering the same function as the co-product considered (see Fig. 6b). The choice of what is replaced has large impacts on the results, and it is a subjective and uncertain choice, both for what is assumed to be replaced and in which amount. These “credits”, in fact, can be large enough to shift the impact of a product from negative to positive; given the large uncertainty and the great influence on the final assessment of this choice, a sensitivity analysis should be carried out to understand the robustness of the results. For example, glycerol resulting from the esterification of biodiesel is often assumed to replace fossil glycerol in nutraceuticals, without considering that the scales of the productions are not comparable, or the need for refining and purification (Bernesson et al. 2004; Halleux et al. 2008; Hou et al. 2009; Achten et al. 2010; Talens Peiró et al. 2010; Brentner et al. 2011; Tonini and Astrup 2012).

The physical allocation methods, based on energy, mass, exergy, or carbon content, are the second most used method for solving multifunctionality, followed by economic allocation. All these approaches present limitations, and the ISO standards (ISO 2006c) state that “whenever several alternative allocation procedures seem applicable, a sensitivity analysis shall be conducted to illustrate the consequences of the departure from the selected approach”. For instance, economic allocation properly represents the drivers for any economic activity, but prices are variable in time and space and thus a sensitivity analysis which assesses the historical range of variability of the prices considered is required. Allocation by physical quantities does not represent effectively the drivers of economic activities (e.g. some studies have allocated the upstream emissions by mass to residues), and its use must be justified and checked with sensitivity analyses and thoroughly discussed and presented in the Sect. 2.9.

3.7 Main assumptions

We have identified three main common assumptions recurring in the studies. The first common assumption is that environmental impacts associated to energy infrastructures are negligible and therefore are usually not accounted for (78/100 see Fig. 7a). The few studies who account for those impacts, though, found that infrastructure emissions can account for up to or more than 10% of total GHG emissions of bioenergy systems (Hartmann and Kaltschmitt 1999; Agostini et al. 2015). Crucially, the studies that excluded those emissions also did not report this limitation (see Fig. 7a).

Fig. 7
figure 7

Statistical data on the methodological approach adopted by the articles analysed: a main assumptions; b uncertainty

Full size image

Another common assumption is that biogenic CO2 does not impact climate change, and it is therefore not accounted explicitly (71/100) (see Fig. 7a). However, once in the atmosphere, there is no difference between fossil and biogenic carbon, and only an explicit inclusion of all the carbon pools (SOC, aboveground and belowground biomass, carbon stored in products, their end of life including landfilling in case) can provide a comprehensive understanding of the climate impacts of biomass. This assumption is not always incorrect: for instance, studies which investigate the impact of different processing technologies may justify the exclusion of biogenic C from the analysis. In many other cases, especially if the study is of comparative and strategic type, the alternative fate of land or the alternative fate of biomass must be accounted for.

The third assumption, made in 79 studies (see Fig. 7a), is that the bioenergy produced replaces perfectly, i.e. 1:1, another energy source, normally a fossil alternative chosen among a basket of possible energy sources. While comparing the environmental performances to a similar product may provide a useful reference for interpreting the results, the assumption that a specific product is replaced in a specific amount, without considering neither current nor future market developments, leads to erroneous interpretation of the results. For example, the assumption that wood pellets will replace coal in a coal-powered plant is not appropriate. The electricity market is very complex, with high variability in time and space; the coal-pellet plant competes with all the other sources of electricity, including renewables, both in the operation phase and in the construction/refurbishing phase. Moreover, the co-firing of pellets in coal plants may cause a delay in the dismission of the coal plant and a barrier to the penetration of other renewables. Regarding biofuels, the same applies: even biodiesel or bioethanol does not compete only with their fossil alternatives but also with other fossil/renewable fuels and with electricity in electric vehicles. For example, diesel cars are being phased out in Europe while electric cars are expected to grow substantially; therefore, assuming that biodiesel can only replace fossil diesel is not appropriate.

Another phenomenon to be accounted for when assuming perfect substitution are rebound effects and behavioural aspects (Vivanco et al. 2018). Many recent studies have found that the substitution effect of renewable energy to fossil energy may be significantly less than 1 (Hochman et al. 2010; Rajagopal et al. 2011; Thompson et al. 2011; York 2012; Chen and Khanna 2013). For example, York (2012) showed that on average, across most nations of the world over the past 50 years, 1 MJ of total energy use from non-fossil sources (hydropower; nuclear, geothermal, solar, wind, tidal, and wave energy; combustible renewables; and waste) displaced less than 0.25 MJ of fossil fuel. Focusing specifically on electricity, each MJ of electricity generated by non-fossil fuel sources, York (2012) found that it displaced less than 0.1 MJ of fossil fuel electricity.

These three assumptions are not wrong “per se”, and in some cases, depending on the goal of the study, they are acceptable and the simplifications appropriate. However, their widespread use, without any sensitivity analysis and proper reporting in the limitations, makes most of the studies that rely on them seriously misleading.

3.8 Uncertainty and sensitivity

Modelling environmental impacts of bioenergy systems is complex due to the linkages between these systems and many sectors of the economy (agriculture, forestry, food and feed supply, transport, etc.) and due to the multiple environmental pressures associated to bioenergy. As any complex modelling exercise, bioenergy sustainability modelling is very challenging and subject to great uncertainty due to the enormous amount of input data required, and to the use of several numerical and conceptual models needed to grasp the environmental implications of biomass production and processing and the fate of the compounds involved along the supply chain. On top of these factors, most of the studies have actually analysed hypothetical bioenergy pathways, using data extrapolated from other studies, other contexts, or roughly modelled, with plenty of methodological aspects and input values chosen at the discretion of the practitioners.

According to ISO14044 standards (ISO 2006b), uncertainty analysis is the procedure to determine how uncertainties in data and assumptions progress along the calculations and how they affect the reliability of the results of the LCA; sensitivity analysis is a procedure to determine how changes in data and methodological choices affect the results of the LCA.

Sensitivity analysis was performed by about half of the articles reviewed (49/100, see Fig. 7b); most of the times, the sensitivity was carried out on key input data (e.g. yields from cultivation) and allocation procedure.

Uncertainty analysis was usually performed with a Monte Carlo technique (14/100, see Fig. 7b), to provide ranges of likely results rather than single score values.

As explained above, the nexus between bioenergy and the environment is very complex and far from being fully understood; for this reason, we expected a wider use of sensitivity and uncertainty analyses to properly interpret the robustness of the results of the studies performed.

3.9 Limitations, conclusions, and recommendations

The identification of limitations and their explicit communication, together with the results, is a fundamental step in LCA, which is required to avoid misleading messages and misinterpretation of the results (ISO 2006b). However, we find that in most of the studies analysed (65/100, see Fig. 8a), limitations were not reported at all. In 24 cases, we found an incomplete set of limitations, while only 11% of the studies properly identified and reported the limitations of their approach and data inventory. Surely, the most frequent gap was the limitations reporting on the 3 main assumptions described in Sect. 3.7. Reporting of limitations was also missing for the limited set of impact categories analysed, the poor quality of input data, the completeness of the system boundaries used, the inventory modelling approach, the multifunctionality, and, in general, all the choices made by the practitioners. In most cases, the results were presented as a single score number without any reference to the representativeness, uncertainty, robustness, intended use, and audience.

Fig. 8
figure 8

Statistical data on the methodological approach adopted by the articles analysed: a limitations; b conclusions and recommendations

Full size image

The most common conclusions and recommendations are on the comparison among different bioenergy pathways or with fossil fuels; therefore, they are comparative studies, or benchmarking (83/100, see Fig. 8b). In this case, the ISO standards (ISO 2006b) provide stricter recommendations on all the phases of the LCA, in particular strict sensitivity checks and identification and reporting of limitations. However, most of the studies did not follow the standards and just reported single score results without limitations. An extensive justification for the assessment of the limitations’ reporting for each study is provided in the ESM.

Conclusions and recommendations on eco-design are drawn in 59 articles out of 100. These studies are the most likely to be consistent, as they consider and compare different processing technologies for the same bioenergy pathway; therefore, they can draw robust conclusions even with partial modelling of the full life cycle.

The same number of studies (59/100) draw conclusions relative to the accounting of the environmental impacts of bioenergy systems. Such conclusions are normally presented with other types of conclusions, benchmarking, policy relevance, or eco-design. Only in two cases do the conclusions present only the accounting results without any comparison with other systems or alternative production processes.

Policy-relevant conclusions (15/100) are conclusions drawn on the potential impacts of policy choices affecting installed capacities. Practically all the studies that presented policy relevant conclusions recommend the expansion of bioenergy production.

3.10 Consistency check

The results of our consistency check are reported in Table 2. Extensive justification for the placement of each article in a consistency category is reported in the ESM.

Table 2 Results of the consistency check
Full size table

We are aware that there is a margin of subjectivity in the assessment of consistency, as many studies are on the border between two categories. However, following the methodology described in Sect. 2.7, we have assessed that only 29 articles out of 100 are actually consistent, which means that the assessment framework used by the LCA practitioners was suitable to answer the question set in the goal, though sometimes the lack of limitations may still generate a certain degree of misinterpretation.

In category B, misleading, is the most populated with 45 articles. Most of the studies falling in this category have a comprehensive and robust model, both in the inventory and in the impact assessment phase; however, the use of the results is misleading mostly because they draw conclusions and recommendations on aspects or scales not analysed, or because of the lack of proper reporting of limitations on assumptions, input data, or modelling approach.

In category C, inconsistent, we have placed 22 articles. For these studies, a correct interpretation of the results is difficult because the modelling approach, main assumptions, gaps, system definition, and subjective choices do not enable the drawing of meaningful conclusions.

In category D, there are 4 studies with serious flaws in the approach and/or data used. These studies are profoundly misleading.

4 Conclusions

This study shows that it is a rather common practice in LCA studies of bioenergy to overlook some of the fundamental steps in the interpretation phase. Although most of the studies referred to the ISO standards (52/100), the identification of issues, their framing with sensitivity analyses, and the identification and reporting of limitations, which are all fundamental components for high-quality standards, and recommended by ISO14044, are still not seen as mandatory by the majority of practitioners. However, the most problematic part of the interpretation phase of these studies is clearly the consistency check. In most cases (71/100), the assessment framework built is not even apt at answering the question set in the goal. In practice, it was found that either the conclusions or recommendations drawn are not robust because the inventory and the impact assessment phases are not consistent with the goal of the study, or the conclusions and recommendations go well beyond what the limitations of the study would allow. In addition, limitations are properly identified and reported only in 11 studies, 65 studies do not report limitations at all, and 24 studies report an incomplete set of limitations. The lack of limitations reporting makes most of the studies potentially misleading, as the key issues and subjective choices are neither identified nor properly framed through sensitivity analyses, and, most importantly, not communicated to the readers.

In our opinion, these flaws in the interpretation phase play a significant role in fuelling the debate around the governance of sustainability of bioenergy.

5 Recommendations

Given the result of this assessment, two sets of recommendations can be drawn, one directed to LCA practitioners carrying out LCA studies of bioenergy, and another directed to regulators and decision-makers who are the potential users of these studies, to provide them with the tools needed to assess the significance of LCA results.

5.1 Recommendations to LCA practitioners

  1. 1.

    A clear identification of the goal and scope of the analysis is essential to properly perform the interpretation phase. State clearly which question is at stake and how you plan to answer it.

  2. 2.

    Limitations shall be clearly identified and reported. Care should be placed in identifying the limitations relative to the modelling approach chosen and all the subjective choices taken (allocation or system expansion, system boundaries, sustainability aspects and environmental impacts analysed, systems definition).

  3. 3.

    If only 1 MJ (or other units) is modelled, the conclusions can only be drawn on that amount of bioenergy, hence in contexts of microscale decision, eco-design, or accounting, such as in environmental product declarations or national (or other) inventories. To draw policy-relevant conclusions, thus affecting the installed capacities, a strategic assessment shall be carried out, and the potential consequences of the policy together with all the related current and expected market-mediated effects shall be modelled.

  4. 4.

    Conclusions must be drawn only on the impact categories analysed. In a descending order of completeness, sustainability assessment includes social, environmental, and economic aspects; environmental sustainability assessment encompasses all the relevant areas of environmental concern; climate change impact assessment includes all the climate forcers; and GHG emissions assessment includes all WMGHG.

  5. 5.

    In the atmosphere, there is no difference between biogenic and fossil carbon, it is the overall carbon cycle between emissions and sequestration which matters. Biogenic carbon neutrality assumption is often erroneous or misleading; hence, all carbon pools must be included in the analysis (including soil organic carbon, aboveground and belowground biomass, carbon stored in products, and landfills) as they may be more significant for climate change impact assessment than the direct processing and transport emissions.

  6. 6.

    Different impact assessment methods have different meanings; thus, the methods used for the impact assessment shall be understood and explained.

  7. 7.

    Different metrics answer different questions (see Sect. 2.4 on GTP vs GWP, for climate change, kg N or kg N ha−1 for eutrophication).

  8. 8.

    If system expansion is used, the type and amount of product replaced is a critical subjective assumption which should be thoroughly assessed via a sensitivity analysis.

  9. 9.

    Uncertainties are always present in complex modelling exercises; if uncertainty assessment is not performed, this choice should be clearly mentioned and justified.

  10. 10.

    Including phenomena for which site-specific data are scarce (e.g. SOC dynamics) or that are only predictable and not empirically measurable (e.g. ILUC) increases the uncertainties, but improves the accuracy. It is better to be approximately right than precisely wrong.

  11. 11.

    As current decisions can only affect actions from now on, in strategic assessments supporting policy decisions, the temporal scope should be forward looking.

  12. 12.

    If allocation is used, the rationale supporting the chosen allocation methods should be explained and assessed via a sensitivity analysis.

  13. 13.

    In comparative attributional bioenergy LCAs, alternative sources of energy (not only fossil) may be used to present the results, but the assumption of perfect substitution should be avoided, unless this is properly justified and interpreted. For example, for transportation services biodiesel may be compared to any other system providing the same function (fossil diesel, but also other modes, or electric vehicles) with a reasoned range of substitution factors.

  14. 14.

    The alternative use of land (e.g. natural regeneration or other uses; for further insights, check Soimakallio et al. (2015) and Koponen et al. (2018)), or fate of residues (e.g. left to decay or other uses; Giuntoli et al. (2016)), must be included in the analysis and properly interpreted.

  15. 15.

    The intended use and audience of the study should be clearly identified and reported.

5.2 Recommendations to the readers

  1. 1

    Avoid extrapolating the results from microscale to macroscale. If one unit is modelled, the results are only valid at that scale. To grasp the impacts of policies affecting the installed capacities, the policy must be modelled with all the relative scale effects and market-mediated impacts.

  2. 2

    Check carefully which aspects of sustainability are addressed keeping in mind that several possible trade-offs may arise in areas of environmental concern not modelled in the study, e.g. if only climate change is modelled, keep in mind that bioenergy very likely impacts land use, air quality, water, eutrophication, biodiversity, acidification, or other sustainability pillars.

  3. 3

    There is no free lunch. All the biomass used for bioenergy is either cultivated or is a residue. If the goal of the analysis is to evaluate the impacts of a change-oriented decision, then the alternative use of land (natural regeneration or management for other purposes) or alternative fate of the biomass (in other sectors of the economy or left to decay or landfilled) should be accounted for in the analysis.

  4. 4

    Perfect substitution does not exist in the real world. When a product is placed on the market, it is the market and the consumers who decide what is replaced and in which amount. If perfect substitution is used, it should be justified and accompanied by a sensitivity analyses and a proper interpretation.