1 Introduction

Over the last decade, dozens of developing countries have launched national processes to reduce emissions from deforestation and forest degradation (REDD+). Such efforts have translated into the development of REDD+ country programmes capable of receiving results-based finance from the Green Climate Fund, the Forest Carbon Partnership Facility (FCPF) Carbon Fund, bilateral arrangements with donor countries and other sources (FAO 2019).

Carbon finance requires strong assurance that greenhouse gas emission reductions actually occurred. Quantification of emissions can draw on accepted international standards designed to yield estimates with low uncertainty, where the risk of overestimating emission reductions is small. Establishing the amount of emission reductions needs to meet three requirements related to uncertainties that are described as follows.

First, appropriate measurement methods and high-quality data are needed to measure the results of mitigation programmes in forests and land-use change. Over the last 10 years, much work has concentrated on developing country capacities, resulting in impressive progress (Neeff and Piazza 2019; Romijn et al. 2015). Some authors have concluded that capabilities exist now in developing countries to meet operational needs for measuring REDD+ results (Goetz et al. 2015). However, detailed case studies have shown that the remaining uncertainties are still substantial (Morton et al. 2011; Pelletier et al. 2011, 2013; Watson et al. 2013). The fact that uncertainties could be high makes a careful analysis important.

Second, the uncertainties involved in estimating emissions and emission reductions need to be analysed. The Intergovernmental Panel on Climate Change (IPCC) looks at the uncertainty analysis as an essential element of a greenhouse gas inventory, and carbon standards for REDD+ dedicate much attention to approaches for assessing uncertainties (FCPF 2016; IPCC 2019; Verra 2019). Nonetheless, reporting on uncertainties is uneven in countries’ REDD+ submissions (FAO 2019), and a recent review points to obvious deficiencies (Yanai et al. 2020).

Third, the risk of overestimating emission reductions needs to be reduced through conservative estimation approaches. Those providing carbon finance for forest mitigation will wish to avoid compensating emission reductions that did not truly occur. In the context of the Kyoto Protocol of the United Nations Framework Convention on Climate Change (UNFCCC), a conservative accounting approach is applied that greatly reduces the likelihood of overestimating emission reductions (UNFCCC 2001, 2003, 2008). Conservativeness has also been discussed in the context of REDD+ (Grassi et al. 2008, 2013; Köhl et al. 2009; Plugge et al. 2013). In practice, conservative estimation could mean, among other things, that estimates are discounted (Pelletier et al. 2015). The conservativeness principle is embedded into the estimation approaches for several leading REDD+ carbon standards (ART Secretariat 2020; FCPF 2016; Verra 2019).

These three requirements demand much effort to address uncertainties. However, the analysis of uncertainties in country reports is, at best, variable, and gauging the quality of emission reduction estimates and any associated risks of overestimation is often difficult (Yanai et al. 2020). Difficulties in compiling representative datasets on uncertainties in countries’ emissions reporting have limited previous studies of conservative estimation approaches in a REDD+ context to conceptual discussions, simulation, and case studies (Grassi et al. 2008; Köhl et al. 2009; Lusiana et al. 2014; Plugge et al. 2013).

To illustrate what is achievable, this paper reports from a context where, despite such difficulties, a group of countries has estimated greenhouse gas emissions from forests and land-use change and quantified associated uncertainties in a broadly comparable manner. The 18 countries that have proposed jurisdictional REDD+ programmes to the Carbon Fund use a common methodological standard, and a review process ensures their compliance. This group of countries provides an unusually consistent dataset that can be the basis for examining achievable uncertainty in estimating emissions and emission reductions from forests and land-use change.

A reliable measurement is both precise and accurate, i.e. has only small random and systematic uncertainties (IPCC 2019). This paper focuses on random uncertainties (“precision”), commonly described by confidence intervals. Precise estimates have only a narrow spread around a mean estimate (although they may still be inaccurate), which depends on the variability of the underlying parameter to be measured (e.g. forest carbon density), as well as on the sampling applied. Imprecise estimates, in contrast, can have wide confidence intervals, and the mean estimate could therefore be far away from the truth. This paper is less concerned with systematic uncertainties (“accuracy”). An estimation is accurate when measurements and classification are free from bias (although it may still be imprecise). Accuracy is hard to quantify and not usually represented through confidence intervals, but it can be assured through the use of high-quality data, appropriate definitions, and sound measurement protocols (IPCC 2019).

This paper describes emissions estimation in practice: what uncertainties developing countries, with due technical assistance, can operationally achieve when estimating emissions in jurisdictional REDD+ programmes – and what can be done to ensure conservative estimation. The paper starts out by providing some background on the Carbon Fund and the approaches to quantifying emissions and uncertainties. Then, I assess achievable uncertainty in estimating emission reductions by looking at what countries report on forest area, forest carbon density, and emissions. Finally, the paper discusses how to generate conservative emission reduction estimates.

2 Background

To explore what uncertainties are achievable when countries estimate emissions and emission reductions, this paper examines a set of jurisdictional REDD+ programmes that countries have proposed to the Carbon Fund. These programmes follow a common approach to quantifying emissions and to analysing uncertainties.

2.1 The Carbon Fund portfolio

The Carbon Fund pilots results-based payments to countries that have achieved verifiable emission reductions from their forests or other land-use sectors. By early 2020, 18 countries had programmes accepted into the portfolio of the Carbon Fund: Chile, Costa Rica, Dominican Republic, DRC, Fiji, Ghana, Guatemala, Indonesia, Ivory Coast, Lao PDR, Madagascar, Mexico, Mozambique, Nepal, Nicaragua, Peru, Republic of Congo, and Vietnam (World Bank n.d.).

Reporting to the Carbon Fund represents a best case in terms of estimation uncertainties because the prospect of results-based funding motivates technical effort, and countries can access technical assistance (Neeff et al. 2020). The Carbon Fund portfolio is also a good case to investigate emissions estimation because reporting is guided by a methodological framework and a review process (FCPF 2016) that together ensure a degree of comparability across countries.

The FCPF methodological framework provides detailed guidance through a set of dedicated criteria and indicators on managing uncertainties when estimating mitigation results (FCPF 2016). Reporting occurs initially through publicly available emission reduction programme documents that include estimates of historical emissions for setting the reference level. These are the sources for the information provided in this paper. The country reports initially undergo a technical assessment and later also verification audits to establish emission reduction estimates.

With regard to the technical requirements for estimating emissions and analysing associated uncertainties, the Carbon Fund jurisdictional programmes differ markedly from countries’ REDD+ submissions to the UNFCCC. When preparing forest reference levels submissions or reporting REDD+ results to the UNFCCC, countries are guided merely by a set of general principles, the Warsaw Framework for REDD+ (UNFCCC 2013), which does not have the same level of detail as the FCPF methodological framework. Many countries report partially or not at all on uncertainties (FAO 2019). Rather than verification audits, UNFCCC submissions undergo a technical assessment/analysis that is designed to be “nonintrusive, non-punitive and respectful of national sovereignty” (UNFCCC 2010). They have been found to be less effective than audit-style reviews when it comes to ensuring the soundness of data and methods (Neeff and Lee 2018). Sometimes, glaringly obvious mistakes are made in analysing uncertainties (Yanai et al. 2020).

2.2 Countries’ approaches to quantifying emissions

There are four parameters of primary interest for quantifying REDD+ results: forest area and its changes, forest carbon density, emissions, and emission reductions.

These parameters correspond to the main terms of the basic equations for quantifying the reference level (RL), actual emissions (E), and emission reductions (ER) (GFOI 2020; IPCC 2019). Activity data (AD) are essentially forest areas and their changes (e.g., through reforestation or deforestation). Activity data are relevant both for a reference period where they underlie the reference level (ADRL) and to measure performance when they underlie actual emissions (ADE). Emission factors (EF) describe the amounts of emissions per unit of activity data and are related to carbon densities and differences in densities upon land-use change.

$$ \mathsf{ER}=\mathsf{RL}-\mathsf{E} $$
(1)
$$ \mathsf{E}={\mathsf{AD}}_{\mathsf{E}}\cdotp \mathsf{E}\mathsf{F}\kern0.5em \mathrm{and}\kern0.5em \mathsf{RL}={\mathsf{AD}}_{\mathsf{RL}}\cdotp \mathsf{E}\mathsf{F} $$
(2)
$$ \mathsf{EF}=\mathsf{Carbon}\ \mathsf{density}\ \mathsf{before}\kern0.5em \mathsf{conversion}-\mathsf{Carbon}\ \mathsf{density}\ \mathsf{after}\ \mathsf{conversion} $$
(3)

Countries employ a range of approaches for estimating those parameters, which the FCPF recently reviewed in detail (Neeff et al. 2020). There are two main approaches for measuring forest area and its changes (i.e. the activity data): using wall-to-wall maps or sampling. In practice both approaches are often combined: sampling draws on wall-to-wall maps for stratification; wall-to-wall maps use sampling for accuracy assessment and bias correction (GFOI 2020).

About one-third of country programmes are using map-based approaches for area estimation (Neeff et al. 2020). Typically, the maps are based on an analysis of medium-resolution satellite imagery, delivering a robust representation of forest versus non-forest area, as well as any land-use changes. The maps are usually thematically poor and distinguish only few types of land use, which help increase their accuracy. Some countries have used existing map-based cadastres rather than generating new maps specifically for REDD+. When using map-based approaches, the area estimates are derived from pixel counts.

Most country programmes, however, have opted to measure area of forest and of forest changes using sample-based methods (Neeff et al. 2020). Here, a variety of available reference datasets support a visual identification of land-use change, where high-resolution data are increasingly becoming available. Ancillary datasets can include vegetation indices, distance to boundaries and infrastructure, elevation, results of automated high-frequency time series analysis, and others. Especially for identifying forest disturbance, such ancillary datasets are widely used. Often, maps are used for stratification to help increase sampling efficiency. In other cases, samples are allocated by systematic grids.

Other than area measurements, estimates of the emissions per area unit (i.e. the emission factors) are needed, which are related to forest carbon density (Neeff et al. 2020). Some countries have access to data from national forest inventories: large-scale campaigns where hundreds, if not thousands, of sample plots are measured in the field. Where such data do not exist, any other field inventories can still provide a basis for estimating carbon density. Only a few countries have had access to advanced remote sensing approaches for estimating carbon density through laser and radar.

2.3 Approaches to quantifying uncertainties

In the context of estimating emissions from forests, uncertainties are defined following guidance from the IPCC, from the Global Forest Observations Initiative (GFOI), and other sources as follows (GFOI 2020; IPCC 2019):

$$ {\mathsf{U}}_{\mathsf{E}}={\mathsf{t}}_{\alpha /2}\cdotp {\mathsf{std}}_{\mathsf{E}}/{\mathsf{E}}_{\mathsf{est}}, $$
(4)
$$ \mathsf{P}\left\{{\mathsf{E}}_{\mathsf{est}}-{\mathsf{t}}_{\alpha /2}\cdotp {\mathsf{std}}_{\mathsf{E}}\le {\mathsf{E}}_{\mathsf{t}\mathsf{rue}}\le {\mathsf{E}}_{\mathsf{est}}+{\mathsf{t}}_{\alpha /2}\cdotp {\mathsf{std}}_{\mathsf{E}}\right\}=1-\alpha, $$
(5)

where UE is the uncertainty in estimating emissions at confidence level t-α, relative to the emissions estimate, Eest is the emissions estimate, Etrue represents the (unknown) true emissions that the confidence interval contains, tα/2 represents the t distribution for percentile α/2 and stdE is the estimated standard deviation of the emissions estimate (not of the emissions themselves).

Albeit estimation approaches differ, Carbon Fund countries are required to use a common standard to address uncertainties. This common standard is described in the FCPF methodological framework that includes guidance on 38 criteria and related indicators on major aspects of jurisdictional REDD+ programme design (FCPF 2016). The criteria 7–9, with seven indicators, lay out how countries first identify and assess sources of uncertainty, second minimize uncertainty where feasible and cost effective, and third quantify remaining uncertainty. This guidance ensures a common methodological foundation in terms of a common set of error sources and variables reported in countries’ uncertainty analysis.

In the following, this paper includes summary information about reported uncertainties in the Carbon Fund portfolio. To arrive at comparable values, some adjustments were made. For example, countries report uncertainties as absolutes or relative to the mean, as standard errors or according to 90% or 95% confidence levels. Countries report areas and carbon densities according to their own breakdown of forest and forest change classes, which required aggregation and error propagation to arrive at comparable aggregate classes.

Some aspects remain not fully comparable: most importantly, different methods were used for estimating forest areas or forest carbon densities (see above). Moreover, not all error sources were always accounted for. Notably, regarding area measurements, some countries include classification errors, others do not. Despite such discrepancies, it provides confidence in the dataset that the uncertainty analyses all had to follow a detailed set of indicators and were subject to detailed scrutiny during reviews.

2.4 Conservativeness

A conservative emission reduction estimate is the product of a calculation chain that has been set up to reduce the risk of overestimation. Conservativeness is not a principle in the IPCC guidance, the basic guidance on estimating greenhouse gas emissions, which posits that national greenhouse gas inventories should contain estimates that “are neither systematically over- nor under-estimating the true emissions or removals, so far as can be judged” (IPCC 2019). But conservative estimation is a core principle when accounting for emissions against a baseline and towards emission reduction targets.

In this context, the UNFCCC Marrakesh Accords lay out that methodologies for the Clean Development Mechanism “should be conservative in order to prevent any overestimation of reductions in anthropogenic emissions” and that project baselines “shall be established … in a transparent and conservative manner regarding the choice of approaches, assumptions, methodologies, parameters, data sources, key factors and additionality, and taking into account uncertainty” (UNFCCC 2001). Later draft guidance developed “conservativeness factors” for discounting emission reductions according to their “random uncertainties” (UNFCCC 2008). Similarly, the Verified Carbon Standard, referring to ISO 14064-2, requires using “conservative assumptions, values and procedures to ensure that net GHG emission reductions or removals are not overestimated” (Verra 2019). In a REDD+ context, the FCPF methodological framework requires using “conservative assumptions” and “conservative approaches to setting reference levels” and includes a set of “conservativeness factors” for discounting emission reductions (FCPF 2016).

In line with the above references, in this paper, I think of conservative estimation as a means to derive a conservative estimate (ERconserv) with a low risk of overestimating (α) true emission reductions (ERtrue), such that:

$$ \mathsf{P}\left\{{\mathsf{ER}}_{\mathsf{conserv}}\le {\mathsf{ER}}_{\mathsf{true}}\right\}=1-\alpha . $$
(6)

There are several ways to modify the calculation chain to ensure conservativeness. Where several candidate approaches are available for setting a reference level (e.g. simple average, linear projection, more complex modelling), choosing the lowest reference level among the candidates is conservative. Where emissions are reduced in several carbon pools and gases, excluding some from the scope would conservatively underestimate emission reductions.

Also, discounts are frequently used to achieve conservativeness (Pelletier et al. 2015). The mean estimate of emission reductions (ERest) is reduced using a discount factor (discountER), which can be established using Eqs. 5 and 6 (while referring to emission reductions instead of emissions, and considering the case of a one-sided confidence interval), as follows:

$$ \mathsf{P}\left\{{\mathsf{ER}}_{\mathsf{est}}\cdotp \left(1-{\mathsf{discount}}_{\mathsf{ER}}\right)\le {\mathsf{ER}}_{\mathsf{true}}\right\}=1-\alpha, $$
(7)
$$ {\mathsf{discount}}_{\mathsf{ER}}={\mathsf{t}}_{\alpha}\cdotp {\mathsf{std}}_{\mathsf{ER}}/{\mathsf{ER}}_{\mathsf{est}}. $$
(8)

For example, applying the provision in Verra’s Verified Carbon Standard, methodologies need to be designed to include uncertainty discounts when the uncertainty of estimates exceeds 10% at the 90% confidence level (Verra 2019). If in a hypothetical programme, emission reductions were 50% uncertain at a 90% confidence level, then they would need to be reduced downwards.

When considering the risk of overestimation, a one-sided confidence interval must be used. Two-sided confidence intervals are used for basic reporting on uncertainties (Fig. 1). The upper bound of a two-sided 90% confidence interval corresponds to the bound of a one-sided 95% confidence interval. Because of this, relying on a 90% confidence interval as basis for conservative estimation entails a 5% overestimation risk.

Fig. 1
figure 1

Confidence intervals (a) two-sided for uncertainty reporting and (b) one-sided for conservativeness discounting.

Full size image

The tolerable risk of overestimation differs between contexts. For example, Verra’s Verified Carbon Standard relies on 90% confidence intervals as basis for conservative estimation, which entails a 5% overestimation risk. In contrast, the adjustment procedures for countries’ greenhouse gas inventories under the Kyoto Protocol (in case of bias from incorrect application of guidance) are based on the 25th and 75th percentiles of recalculated estimates (UNFCCC 2003), which entails a 25% overestimation risk. Comparative analysis of several standards has identified stark discrepancies between applicable discounts (Pelletier et al. 2015; Yanai et al. 2020) and thus of tolerable overestimation risk.

3 Uncertainties in estimating areas, carbon densities, and emissions

3.1 Measuring areas

When measuring forest areas, uncertainties depend on the types of areas to be measured (Fig. 2). In reporting to the Carbon Fund, uncertainties associated with forest loss are mostly below 20%. Uncertainties associated with forest-area gain are around 10–40%. Uncertainties in measuring area of changes within forests (e.g. as a result of degradation or forest recovery) are also below 20% in most countries, although some countries report uncertainties higher than 40%. (Other countries have chosen not to report this type of uncertainty and instead used a maximum conservativeness discount.)

Fig. 2
figure 2

Uncertainty in area estimation (Neeff et al. 2020). Note: Countries report on diverse forest categories, requiring aggregation to enable comparison

Full size image

The fact that uncertainties are, in many cases, substantial, shows how difficult forest area estimation is with available methods and datasets (FAO 2019). Datasets are not always fully adequate: for example, medium-resolution satellite imagery provides an imperfect basis for reliably identifying the results of granular land-use decisions by small farmers. Furthermore, vegetation changes can be rather gradual; for example, when vegetation regrows on abandoned pasturelands, the threshold of when the “pastureland with shrubs” turns into “forest” is not obvious. The enormous underlying variability of forests certainly does not make reducing uncertainties any easier.

The estimates reported here only represent a partial view on uncertainty (in Fig. 2, as well as in Figs. 3 and 4). Only random uncertainties are reported, but bias in estimation could be significant. For example, measuring area of forest degradation is notoriously difficult, and disturbance does not necessarily reflect in lasting change in spectral properties that could be observed in satellite imagery (GFOI 2020). In their Carbon Fund reporting, many countries instead rely on observing crown cover changes – and assume that these reflect degradation with a concomitant loss in forest carbon (Neeff et al. 2020). While Fig. 2 shows that the area of forest with reduced crown cover can be measured with acceptable precision, other questions would relate to the link between reduced crown cover and the occurrence of degraded forest, as well as the resulting long-term loss in forest carbon. Calibrating such relationships is difficult and bias hard to exclude, but the associated systematic uncertainties do not impact the random uncertainties reported here.

Fig. 3
figure 3

Uncertainty in forest carbon density estimation (Neeff et al. 2020). Note: Countries report on diverse forest categories, requiring aggregation to enable comparison

Full size image
Fig. 4
figure 4

Uncertainty in estimating emissions (Neeff et al. 2020)

Full size image

3.2 Measuring forest carbon density

In estimating forest carbon density, uncertainties associated with the country average carbon density are between 5 and 15% in most cases (Fig. 3). Some countries report being able to estimate forest carbon density with uncertainty below 5% of the mean. As for the estimates reported to the Carbon Fund, in most cases, carbon density estimates are more precise than area estimates. (This better precision does not imply a higher accuracy.)

3.3 Measuring emissions

Uncertainties in quantifying emissions reflect the uncertainties in underlying forest area and carbon density estimates – i.e., the input for emissions estimation. The uncertainties reported here for estimates of forest area and its changes, forest carbon density, and emissions all refer to sums or averages at the scale of jurisdictional programmes. Uncertainties for estimation at a smaller scale would differ (Lusiana et al. 2014).

Uncertainties in estimating historical emissions amounted to 10–30% in more than half of the countries. Uncertainties differ among the REDD+ activities (Fig. 4). Almost all historical deforestation emissions estimates are 0–40% uncertain, while most emissions estimates from forest degradation are 40–60% uncertain. Removals from enhancement of carbon stocks or conservation are only 20–40% uncertain in some cases, while in other cases, uncertainties exceed 80%.

Different uncertainties in estimating emissions for the REDD+ activities reflect different uncertainties in the underlying activity data. Deforestation is easier to see in satellite imagery than forest degradation because land use changes during deforestation, often to agriculture. While both deforestation and forest degradation can occur through specific events with a short-time scale that can be identified in image interpretation, enhancements are especially hard to detect because trees grow gradually.

3.4 Correlation of uncertainties

Taking into account the correlation of uncertainties in estimating the reference level and the actual emissions is important in the case of REDD+. In earlier work, such an approach has been referred to as analysis of the “uncertainty of the trend”, and it has been distinguished from a “reliable minimum estimate” approach that treats uncertainties as independent (Grassi et al. 2008, 2013).

Uncertainties in estimating emissions from forests are often correlated over time because of correlation in underlying activity data and emission factors. Emission factors may be seen to be fully correlated, as often the same emission factors are used for calculating the reference level and actual emissions. Depending on how activity data are measured, uncertainties may also be correlated, although usually only to a small extent (e.g. when using permanent sample plots for estimating activity data, then a deforestation observation at time point one might exclude the possibility of a deforestation observation at time point two). Introducing a simplification, the IPCC guidance proposes treating emission factors as fully correlated, and uncertainties from estimating activity data as fully independent (IPCC 2019).

This slight simplification helps in estimating the correlation coefficient (ρ) for uncertainties in emission estimates. The correlation coefficient depends on the covariance between reference level and actual emissions. If one makes the assumption that activity data are independent and that emission factors are fully correlated, then the following is true:

$$ \operatorname{cov}\left(\mathsf{RL},\mathsf{E}\right)=\operatorname{cov}\left({\mathsf{AD}}_{\mathsf{RL}}\cdotp \mathsf{EF},{\mathsf{AD}}_{\mathsf{E}}\cdotp \mathsf{EF}\right)={\mathsf{AD}}_{\mathsf{RL}}\cdotp {\mathsf{AD}}_{\mathsf{E}}\cdotp \operatorname{cov}\left(\mathsf{EF},\mathsf{E}\mathsf{F}\right)={\mathsf{AD}}_{\mathsf{RL}}\cdotp {\mathsf{AD}}_{\mathsf{E}}\cdotp {\mathsf{std}}_{\mathsf{E}\mathsf{F}}^2 $$
(9)

And since ρ = cov(RL,E) / (stdRL • stdE), the correlation coefficient can be estimated, as follows:

$$ \rho ={\mathsf{AD}}_{\mathsf{RL}}\cdotp {\mathsf{AD}}_{\mathsf{E}}\cdotp {\mathsf{std}}_{\mathsf{E}\mathsf{F}}^2/\left({\mathsf{std}}_{\mathsf{RL}}\cdotp {\mathsf{std}}_{\mathsf{E}}\right). $$
(10)

Substituting ADRL = RL / EF and ADE = E / EF, one can also write:

$$ \rho ={\left({\mathsf{std}}_{\mathsf{E}\mathsf{F}}/\mathsf{E}\mathsf{F}\right)}^2/\left({\mathsf{std}}_{\mathsf{RL}}/\mathsf{RL}\right)/\left({\mathsf{std}}_{\mathsf{E}}/\mathsf{E}\right). $$
(11)

The correlation of emissions lowers the uncertainty of emission reduction estimates. General assumptions have sometimes been made on its magnitude. For example, a recent comparative analysis of uncertainty discounts in several carbon standards has assumed that the correlation coefficient amounts to 0.5 (Yanai et al. 2020). For scenario calculations of emission reductions uncertainty, FAO has implied that the correlation coefficient may amount to 0.7 (FAO 2019). Using Eqs. 10 and 11 one can estimate correlation of emission estimates as a function of emission factor uncertainties and activity data uncertainties. These calculations indicate that the correlation coefficient will mostly fall into the range 0.3–0.7 if stdEF/EF and stdAD/AD are around 20–40% (Fig. 5).

Fig. 5
figure 5

Correlation of emission estimates over time as a function of (a) emission factor uncertainties and (b) activity data uncertainties

Full size image

4 Outlook on uncertainties in estimating emission reductions

4.1 For individual programmes

High-quality information on uncertainties in estimating emission reductions is even harder to obtain than for estimating emissions. In the Carbon Fund, countries have thus far submitted historical emission estimates for setting reference levels, estimating emission reductions by subtracting current emissions from the reference level will become a focus in the coming years. Submissions to the UNFCCC are more advanced (FAO 2019), but lack the methodological rigour that this paper’s analysis needs. Nonetheless, this paper attempts an outlook on expected uncertainties in estimating emission reductions.

Invariably, emission reduction estimates will be more uncertain than emission estimates. This is because the “signal-to-noise ratio” shrinks when subtracting current emissions from the reference level. This ratio describes how the “signal” (i.e. the emission reduction) relates to the “noise” (i.e. the uncertainty in estimating emissions – reference level and actual emissions). Where the uncertainty is high in relation to the emission reductions, these emission reductions may not be detectable.

A simplified example illustrates this point (Fig. 6): if a programme’s reference level ranged anywhere between 15 MtCO2e and 25 MtCO2e, and emission reductions were measured at 15 MtCO2e, then these emission reductions could be clearly detected despite the uncertainty. If, however, emission reductions were measured at only 2 MtCO2e, i.e. much smaller than the uncertainty in estimating the reference level, then there would be a significant probability that the detected emission reductions did not truly occur. The “signal” (i.e. the emission reduction) would be buried in “noise” (the uncertainty). In the same vein, it has also been concluded in several other studies that “signals” from countries with large deforestation reduction relative to their reference level are much clearer (Grassi et al. 2008; Köhl et al. 2009; Plugge et al. 2013).

Fig. 6
figure 6

Signal-to-noise ratio in estimating emission reductions (modified from Chagas et al. 2020)

Full size image

Once countries begin reporting emission reductions with methodological rigour, whether to the Carbon Fund or in other contexts, it remains to be seen what the signal-to-noise ratio will be and how uncertain emission reduction estimates will be. Some scenarios can now already be built that depend on two factors: the underlying uncertainty in emissions estimation and the amount of expected emission reductions. Uncertainties of emission estimates amount to 10–30% in most cases (see above). Planning for the Carbon Fund indicates that most programmes expect emission reductions of 10–40% of the reference level. This is consistent with results actually submitted to the UNFCCC where countries on average reduced emissions by 32% below the reference level (FAO 2019).

Basic rules of error propagation (IPCC 2000) allow us to estimate the uncertainty of emission reductions from the difference between actual emissions and a previously agreed reference level while fully taking into account their correlation. By doing this, I adopt an approach that analyses the “uncertainty of the trend” (Grassi et al. 2008, 2013). I estimate the expected uncertainty of emission reductions (UER) from emission reductions, reference level and actual emissions (ER, RL, and E), their uncertainties (URL and UE), and the covariance (cov(RL,E) = ρ • stdRL • stdE) as follows:

$$ {\mathsf{U}}_{\mathsf{E}\mathsf{R}}=\frac{1}{\mathsf{E}\mathsf{R}}\cdotp \sqrt{{\left({\mathsf{U}}_{\mathsf{RL}}\cdotp \mathsf{RL}\right)}^2+{\left({\mathsf{U}}_{\mathsf{E}}\cdotp \mathsf{E}\right)}^2-2\cdotp \operatorname{cov}\left(\mathsf{RL},\mathsf{E}\right)}. $$
(12)

Scenario calculations suggest that uncertainties in estimating emission reductions could be substantial (Fig. 7). For example, if emissions were reduced by 30% of the reference level, the emission estimate was 20% uncertain, and reference level and actual emissions were correlated by ρ = 0.4, then, the emission reduction estimate would have an uncertainty of 64%. But where emissions estimates have higher uncertainties or where the relative effectiveness of mitigation programmes is lower, uncertainties could be orders of magnitude larger.

Fig. 7
figure 7

Uncertainty in estimating emission reductions by scenarios of effectiveness in reducing emissions below the reference level and by uncertainty in measuring emissions (modified from FAO 2019)

Full size image

4.2 For a portfolio

Where emission reductions of a portfolio of programmes are of interest, rather than of individual programmes, random uncertainties shrink greatly. A portfolio’s expected emission reductions are simply the sum of the individual programmes’ emission reductions: ERPortfolio = ∑ ERProgramme. But the uncertainties sum in quadrature, according to standard error propagation rules: stdER-portfolio = √ ∑ (stdER-programme)2. Because of this, the relative uncertainties shrink when adding emission reductions of programmes, proportionally to the square root of the number of programmes in the portfolio (√n). (This is a simplification because for uncertainties that are shared among projects, e.g. because of common calculation factors, covariances would increase uncertainties.)

That relative uncertainties shrink when adding projects to a portfolio can easily be illustrated using dummy numbers (Fig. 8). In the hypothetical example four projects, all have uncertainties between 10 and 50%. Their sum has an uncertainty of only 10%.

Fig. 8
figure 8

Random uncertainties of four hypothetical programmes compared to the uncertainty of the portfolio including these four programmes

Full size image

For example, the Clean Development Mechanism includes thousands of projects, which all must be assumed to overestimate or underestimate credited emission reductions to varying and potentially significant degrees. Much of the overestimation and underestimation cancel each other out such that the portfolio’s total emission reductions (i.e. of all projects) would have only a negligible (random) uncertainty. Because of this, some documentation surrounding the Clean Development Mechanism points to the “law of large numbers” to justify its leniency in establishing conservativeness (UNFCCC 2008).

5 Ensuring conservativeness

5.1 For individual programmes

As described above, several carbon standards and methodologies include discounting provisions, whereby emission reduction estimates are decreased, and the likelihood of overestimation is low. Discount factors depend on the uncertainty of emission reduction estimates and on the tolerable risk of overestimation.

Emission reduction discount factors can be calculated for varying degrees of tolerable uncertainty (Table 1). These factors relate to Eqs. 7 and 8 and can be used to derive a conservative emission reduction estimate (ERconserv):

$$ {\mathsf{ER}}_{\mathsf{conserv}}={\mathsf{ER}}_{\mathsf{est}}\bullet {\mathsf{discount}}_{\mathsf{ER}}. $$
(13)
Table 1 Conservativeness discounts at several levels of tolerable risk of overestimating emission reductions
Full size table

In some contexts, it has been found practical to discount the reference level and the actual emissions separately. For example, this is being done for The REDD+ Environmental Excellence Standard (TREES) (ART Secretariat 2020), and this is also being undertaken during Kyoto Protocol reviews and ensuing adjustments (UNFCCC 2003). Separately discounting the reference level and the actual emissions will yield a rather stringent combined emission reduction discount. For the common case where uncertainties in estimating the reference level and the actual emissions are of a similar magnitude, their combined discounts could be by a factor of √2 higher than an emission reductions discount at the same percentile. If stdRL = stdE, then √2 • stdRL = √2 • stdE = stdER, and therefore,

$$ {\mathsf{std}}_{\mathsf{RL}}+{\mathsf{std}}_{\mathsf{E}}=\sqrt{2}\cdotp {\mathsf{std}}_{\mathsf{E}\mathsf{R}}. $$
(14)

Typically, the reference level discount alone will be close in its magnitude to an emission reduction discount. This is because reference level emissions and actual emissions are often correlated, which reduce uncertainty in estimating emission reductions (Grassi et al. 2013). The basic rules of error propagation that Eq. 12 is based on also imply that stdER2 = stdRL2 + stdE2–2 • ρ • stdRL • stdE. If the correlation coefficient was around ρ = 0.5, and if one were to look at a simple case of stdRL = stdE, then the reference level discount would equal the emission reduction discount.

5.2 For a portfolio

Overestimation risks are much lower for a portfolio than they are for individual programmes, if applying the conservativeness discounts that were described above. This is because errors of the individual programmes in a portfolio partly cancel each other out, but the conservativeness discounts of the individual programmes simply add up.

For example, a conservativeness discount can be chosen such that the individual programme has a 1/3 probability of overestimation and therefore a 2/3 probability of underestimation. Should a portfolio include several such programmes, then overestimation may occur if more than half of the programmes in the portfolio overestimate (if the programmes deliver similar amounts of emission reductions). The corresponding probabilities are given with a binomial distribution with probability parameter 1/3 and number of trials equals the number of programmes in the portfolio. For example, for a portfolio of ten programmes, the overestimation risk would amount to about 7% and for a portfolio of twenty programmes to about 4% (Fig. 9).

Fig. 9
figure 9

Overestimation probabilities when a programme has 1:2 odds of overestimation and underestimation: (a) for the individual programme, (b) for a portfolio of ten programmes (binomial distribution with n = 10 and p = 1/3), and (c) for a portfolio of twenty programmes (binomial distribution with n = 20 and p = 1/3).

Full size image

Clearly, in some cases, the overestimation risks for the individual programme are of chief concern and not of the portfolio. Also, the portfolio effect would be a less effective way of reducing uncertainties if the portfolio has only a few projects or programmes in it – or if the magnitudes of the individual programmes and their uncertainties vary greatly.

6 Conclusions

This paper explores what (random) uncertainties are achievable in estimating countries’ forest mitigation results. The principal dataset comes from the Carbon Fund where 18 countries have developed emission reduction programmes using largely consistent approaches to estimating emissions and emission reductions from forests and to quantifying uncertainties. The country reports were the basis for identifying achievable precision in measuring forest area and area change, forest carbon density, and emissions from forests. With this input, I investigated achievable uncertainties when estimating emission reductions.

Accordingly, uncertainties in estimating historical emissions amount to 10–30% in most cases. The principal components of uncertainty stem from the estimation of forest area and area changes and from the estimation of forest carbon density. Uncertainties in estimating deforestation areas amount to below 20% in most countries. Uncertainties in estimating forest carbon density are usually below 15%. (All uncertainties are reported as percentages of the mean at a 90% confidence level.)

For emission reductions, uncertainties must be expected to be substantial too. Although methodologically rigorous reporting on uncertainties in emission reductions is hard to come by, available country reports provide grounds for scenario calculations. Such scenarios indicate that emission reduction uncertainties greatly depend on relative effectiveness in reducing emissions, as the magnitude of emission reductions and of uncertainties need to be compared against each other.

A key question is what risk of overestimation needs to be excluded. In some contexts, addressing systematic uncertainties is prioritized, while random uncertainties are only a smaller concern. Notably, emissions accounting under the Kyoto Protocol relies on mean estimates (and adjustments are made only in case of bias due to deviation from guidance). The mean emission reductions estimate carries a 50% probability of overestimation (and also of underestimation), but the amount of such overestimation shrinks greatly when considering a growing number of countries. The overestimation risk is negligible for a large portfolio – as long as systematic uncertainties can be excluded.

In other contexts, addressing systematic uncertainties is not sufficient, and a risk of overestimating emission reductions for the individual project or programme due to random errors may not be tolerable. Discounting the emission reduction estimate is then required. Carbon standards, such as the Clean Development Mechanism, Verra’s Verified Carbon Standard, and the rules of the Carbon Fund, which generate carbon credits potentially for offsetting, include provisions not only for reducing systematic uncertainty but also conservatively discounting emission reduction estimates for random uncertainty. For example, if a mean emission reduction estimate was 60% uncertain (at a 90% confidence level), then it would need to be reduced by 19.1% in order to reduce the risk of overestimation to 30%.

This paper analyses a best case of achievable uncertainty when countries estimate emissions and emission reductions from forests for the Carbon Fund. Data collection follows strong guidance, addressing the uncertainty analysis itself in detail, and a verification audit ensures that guidance is fully implemented. The context is rather different for country reporting to the UNFCCC under its Warsaw Framework where guidance is broad, implementation is inconsistent, and uncertainty analyses often have great weaknesses. Any conclusions from uncertainty analyses presented in those reports would have to be drawn with great caution.

The use of discount factors that this paper lays out is a practical way of reducing overestimation risks that investors into forest mitigation deem intolerable. The discounts can enhance trust that emission reductions actually occurred and thereby help unlock funding. Since discounts penalize programmes with high uncertainties, they may also provide an incentive to improve data collection. Such discounting approaches are widely applied in carbon standards for forest mitigation. Nonetheless, discounting of emission reduction estimates should be seen as only a partial solution, while work towards improving estimates continues. There are two principal limitations to the discounting approach.

First, no matter how precisely one estimates emission reductions, the estimate might still be inaccurate. Addressing random uncertainty (as per the thinking contained in this paper) does not obviate for the need to also address systematic uncertainty through the use of clear guidance, high-quality data, and sound measurement protocols.

Second, using discount factors is disadvantageous because this systematically reduces the amount of emission reductions that are recognized and remunerated. As a whole, such an approach will lead to understating the mitigation contribution that forests deliver. When forest mitigation programmes depend on remuneration for emission reductions achieved, discounts may curtail their viability. For example, should discounts amount on average to 20%, then funding will also be 20% lower. Whether this discounting is indeed necessary is debateable because much of the random uncertainties cancel each other out, if one is able to look at a portfolio, rather than at individual programmes.

To conclude, I consider again the question that this paper’s title asked on the risk of overestimating emission reductions from forests (“…what can be done about it?”). Discounting is a widely used solution to address overestimation risks, but it can be no more than a partial solution. For a more comprehensive solution, I hope that data quality continues to improve, and that trust into data and methods will also grow, along with increasing transparency around the uncertainties involved.