1 Introduction

In the 11th Five-Year-Plan (FYP) period between 2006 and 2010, the Chinese Government had planned to reduce its energy intensity by 20%. Several energy policy measures were implemented to achieve this goal, some of which were implemented in the residential sector. In 2011, residential energy consumption consumed 414.54 Mtce of energy, which accounted for 11% of total energy consumption in China (LNBL 2012).Footnote 1 Bao et al. (2012) show that, in the northern part of China, a very high proportion of the urban residential buildings have low levels of energy efficiency. Furthermore, the share of total building energy consumption used for heating purposes is high in these provinces (Cai et al. 2009). Tsinghua (2007) reports that heating energy consumption in terms of per square meter of building space in China is about 1–1.5 times more than that in Northern Europe. This difference is mainly due to poor insulation, inefficient heating systems, and lack of heat metering facilities.

To reduce energy intensity in the residential sector, in 2007, the Chinese Government introduced a policy measure to promote heat metering and energy efficiency retrofitting (HMEER) in the northern areas of China. The program was directed and supervised by the joint efforts of the Ministry of Housing and Urban–Rural Development (MOHURD) and the Ministry of Finance (MOF). The performance of the provinces covered in the policy was evaluated by the investment assessing centers and energy efficiency testing organizations, which reported to the MOHURD and MOF, respectively. The main aim of the HMEER policy was to promote the installation and retrofitting of heat meters for temperature regulation of heating systems and to promote energy efficiency retrofitting of building envelopes (MOHURD and MOF 2008). The HMEER policy was implemented in the northern part of China from 2007 until 2010.Footnote 2

Heating systems are legally required for buildings located in northern China, with both district heating and decentralized heating systems. However, in southern China, there are generally no district heating systems and heating systems are not legally required. In these regions, space heating in winter is generally obtained using electrical radiators or air conditioners, which are also used for cooling during the summer. Therefore, the energy service of “space heating” is generally produced with different heating technologies in the north and south of China. However, this difference does not affect our policy analysis, because, as discussed in more detail later in the paper, we are interested in analyzing the impact of the HMEER policy on energy consumption.

According to Richerzhagen et al. (2008), the Chinese Government utilized multiple channels to raise funds for improving energy efficiency in buildings. Overall, in the period of the 11th FYP, the government offered a total of 24.4 billion RMB for application of the HMEER policy directly or indirectly through central financing, local leveraging, and other social sources. Although there are papers that discuss the organizational issues, challenges, and difficulties (Zhao et al. 2009a, b) and calculate the implementation effects based on technical numbers (Ding et al. 2011; Bao et al. 2012), there is still a lack of an economic approach to investigate the impacts of the HMEER policy.

We are interested in the HMEER policy for several reasons. First, China relies heavily on command-and-control measures for policy design and the HMEER policy is one typical example of this type. Second, the HMEER policy was designed and supervised by the central government but implemented by local governments. A study of the policy effects can determine the level of efficiency of the top–down system for achieving the designed policy target. Third, as the residential sector is contributing to an increasing share of total energy consumption, a clear understanding of the policy implemented in the residential sector is helpful for future management of energy conservation.

The goal of this paper is to perform an empirical analysis to evaluate the impacts of the introduction of the HMEER policy on residential energy consumption in Chinese provinces. For this purpose, we employ the difference-in-differences (DID) approach proposed by Ashenfelter and Card (1985) using panel data.Footnote 3 Although the policy is now relatively outdated, it is important to rigorously study the effectiveness of the policy to assist policymakers to design successful energy efficiency policy in the future. For the empirical analysis, we obtained data for 29 provinces between 2003 and 2011 on residential energy use, the presence of the HMEER policy and various economic, social, and climate variables.

The main contribution of this paper to the literature is to provide a post-policy evaluation of the effectiveness of the HMEER policy using a rigorous evaluation method such as the DID approach. To our knowledge, this is one of the few studies applying a causal identification method in the evaluation of an energy policy measure adopted in China, and the first analysis related to HMEER. Furthermore, this paper contributes to the literature on the analysis of the impact of the introduction of energy policy measures in emerging countries.

The paper is organized as follows. Section 2 reviews the previous studies in the literature and Sect. 3 introduces the model specifications, and provides a descriptive summary of the data used for this analysis, and Sect. 4 presents and discusses the estimation results. Section 5 concludes the paper.

2 Literature review

This paper contributes to the literature on policy evaluation. During the last few years, several studies on the impact of energy policy measures on the adoption of energy-efficient technologies or renewable energy sources have been published. In this section, we review some of the studies that have a direct or indirect relation to the policy measure analyzed in this paper. The literature contains both studies that have used randomized-controlled trials and studies that have used a DID approach based on disaggregate as well as aggregate data sets. For instance, recently, some researchers have performed randomized control trials to evaluate energy-related programs.Footnote 4 Fowlie et al. (2015) evaluated the impact of a large energy-saving program, the Weatherization Assistance Program, which was implemented in the US residential sector. The results of this analysis showed that the economic benefits obtained with this program were lower than the costs. In addition, Allcott and Kessler (2015) presented a social welfare evaluation of home energy reports through a randomized-controlled trial and suggested larger energy savings from survey respondents. In another US study, Boomhower and Davis (2017) estimated the change in electricity consumption due to the residential air conditioner program in Southern California. They found that electricity savings occurred disproportionately during hours when the value of electricity was high. Gillingham et al. (2012) explored the heating/cooling incentives and insulation incentives between owners and occupiers of residential dwellings, indicating that overall energy savings may be small for correcting the split incentive issues, while Houde et al. (2013) conducted a field experiment to estimate the impacts of real-time feedback technologies, which showed that access to feedback leads to a reduction of 5.7% in energy use. In Japan, using survey data, Tanaka et al. (2017) analyzed the factors that affected the purchasing decision time for solar photovoltaics (PV), and highlighted the importance of the availability of information in the process of decision-making. They also noted that, on average, consumers spend 4 months making their purchasing decisions.

Studies that have investigated the impact of energy policy measures using a DID approach include that of Horowitz (2007), who used a DID approach to study the impacts of the demand side management programs on electricity demand and electricity intensity using aggregate data for the US states from the 1970s to 2003, which confirmed that the energy-efficient programs dramatically reduced state electricity intensity. Datta and Filippini (2016) also used a DID approach to investigate the impacts of ENERGY STAR rebate policies in the US using aggregate data from 2001 to 2006, and concluded that the rebate policies increased the uptake of energy-efficient appliances. Another study using DID was that of Alberini and Bareit (2017), who used DID to analyze the effect of the introduction of a bonus-malus system on the adoption of energy-efficient cars in some Swiss cantons using aggregate panel data. The basic idea of this bonus-malus program is to differentiate the annual car registration tax depending on the energy efficiency of a car, where efficient cars receive a discount and inefficient cars have a surcharge imposed. The empirical analysis confirmed a positive effect on the adoption of energy-efficient cars, although the effect was observed to be rather small.

Further studies on the evaluation of energy-efficient policies include that of Sheer et al. (2013), who quantified the energy saving from Ireland’s home energy-saving scheme using data from 210 households. They found that all dwellings in the study underwent energy-efficient improvements. Likewise, Adan and Fuerst (2015) confirmed that energy-efficient measures in UK homes decreased energy consumption significantly. In the US, Ameli et al. (2017) conducted a natural experiment in northern California based on DID and regression discontinuity to test how the Property Assessed Clean Energy (PACE) program helps to promote solar PV installations. The results from this study show that PACE has been effective in promoting residential solar PV installations. Sekitou et al. (2018) evaluated how the installation of a solar PV system affects the electricity use in Japanese households, and showed that, in monetary terms, Japanese households can save 334 Japanese yen annually for each additional 1 kW increase in battery capacity.

To the best of our knowledge, this paper is one of the first studies that uses an evaluation method such as the DID method to estimate the ex-post effects of the HMEER policy in the Chinese residential sector.Footnote 5 The residential sector is an important component of total energy consumption in China; therefore, several studies have analyzed China’s residential energy consumption using other methods. Chen et al. (2008) introduced a data aggregation method to investigate national energy consumption in the residential building sector of China, while Zhao et al. (2012) implemented a decomposition analysis of China’s urban residential energy consumption for the period 1998–2007, Zheng et al. (2014) developed a comprehensive survey of 1450 households in 26 provinces in 2012 to study residential energy consumption in China, and Xu et al. (2016) introduced a set of six criteria to evaluate through a scorecard method the four types of policies implemented in urban residential buildings in China.

Our findings shall attract a broader readership as both national and provincial governments have responsibility for saving energy and reducing emissions from both domestic and international perspectives.

3 Empirical strategy and data

As discussed previously, this study employs the difference-in-differences approach to estimate the impacts of the HMEER policy on provincial energy consumption of the residential sector.

Since publication of the work authored by Ashenfelter and Card (1985), the use of DID methods has been widespread. The general setup of DID is that the outcome variables are observed for at least two groups and for at least two periods (before and after the policy intervention). One group, the “treated group”, is exposed to a treatment (policy shock) in the second period but not in the first period. The other group, the “control group”, is not exposed to any treatment during either period. In this paper, we investigate the average causal impact of the HMEER policy on provincial energy consumption. In our case, we consider the HMEER policy as a natural experiment, with the northern provinces receiving a treatment and the southern provinces receiving no treatment. As already anticipated, the HMERR policy had the goal of supporting the installation of heat meters and promoting energy efficiency retrofitting of building envelopes.

By adopting a DID approach, the policy impact can be estimated using the following regression:

$$E_{{it}} = \alpha _{i} + \alpha _{{{\text{d}}T}} {\text{d}}T_{t} + \alpha _{{{\text{dPOL}}}} {\text{dPOL}}_{{it}} + X_{{it}} \delta + \mu _{{it}},$$
(1)

where Eit is the energy consumption for province i in time t, dPOL is a dichotomous variable equal to 1 if the HMEER policy has been adopted by province i in time t, and uit is the error term. The province-specific fixed effects αi allow us to control for time-invariant unobserved heterogeneity.Footnote 6 Finally, dTt is time fixed effects and Xit is a set of socioeconomic and climatic variables that influence the level of residential energy consumption. The residential energy consumption and the continuous variables included in Xit are expressed as logs.

Levinson (2016) shows how the decline in energy use in California, which is purported to be a result of energy efficiency policies, is, in fact, driven by other factors, such as demographic factors. Those potential confounding factors need to be taken into account when attempting to identify the true policy effects. Including both time and province-specific fixed effects and other independent variables, Xit, such as energy price, income, heating and cooling degree days, population, and household size in the model enables us to disentangle the impact of the HMEER policy from socioeconomic and climatic determinants, time-invariant provincial characteristics, and time effects.

In a DID approach, the coefficient of interest is αPOL, which, in our case, provides an estimate of the average effects of introducing the HMEER policy on provincial energy consumption. We expect a negative impact of the HMEER variable, as the HMEER policy promotes installation and retrofit of heat meters in the heating systems and energy-efficient retrofitting of buildings, which are measures that should reduce the demand for energy services.

This article utilizes a balanced Chinese panel data set for a sample of 29 provincesFootnote 7 observed over the period 2003–2010 (t = 2003–2010). We limit our analysis to provinces (including provinces, autonomous regions, and municipalities according to China’s administrative classification) in mainland China, and exclude Tibet and Hainan from this study due to incomplete statistical information. The main data source is the China National Bureau of Statistics reports “China Statistical Yearbook” (2004–2011),Footnote 8 which records most of the provincial macro data, including income, population, households, temperature, urbanization, etc. The final energy consumption data in the residential sector are obtained from China Energy Databook V8 (LNBL 2012). The price index information is taken from “China Urban Life and Price Yearbook” (2004–2011). Table 1 presents descriptive statistics of the dependent and independent variables used for the empirical analysis.

Table 1 Descriptive statistics of the dependent and independent variables
Full size table

As discussed previously, the HMEER policy has been implemented in the northern provinces of China. The classification of the groups of provinces that were in the treated and control groups are listed in Table 2. All treated and untreated provinces considered in the main empirical analysis are listed in the upper part of Table 2, and the treated and untreated provinces located along the border between northern and southern China are listed in the lower part of Table 2. The provinces listed in the lower part of the table make up a subgroup of provinces that were used in a robustness analysis to verify the results obtained using all the provinces.Footnote 9 The idea is that the provinces located along the border between northern and southern China are relatively similar with respect to several time variant unobserved characteristics. Of course, from an econometric point of view, the problem with this subsample is that it is relatively small.

Table 2 Provinces and the HMEER policy
Full size table

Two important identification assumptions need to be fulfilled to use the DID method. The first is that, in the absence of treatment, there is a parallel trend in the outcome variable for both the treatment and control groups. If this assumption is violated, the estimated effects of policy intervention could be biased. The second assumption is that the assignment of treatment has to be exogenous. This may be violated if the selection is based on unobserved characteristics of the units.

3.1 Parallel trend

To verify the parallel trend assumption, we estimated Eq. (1) using only the data for the pre-policy period and introduced a new variable given by the interaction between a time trend and a dummy variable that indicates whether a province belongs to the treated group or not. By doing this, it is possible to test if the coefficient of the interactions variable is equal to zero and, therefore, verify if the parallel trend assumption is satisfied. In our case, the coefficient of the interaction variable is not statistically significant (p value 0.934). This illustrates that the common trend is similar for the two groups prior to the policy period. In addition, we verify the common trend assumption by substituting the time trend with time dummies and create a series of interaction variables obtained by multiplying the time dummies with the policy dummy. The reported value of the F statistic is 2.39, with p value 0.076. This value confirms that the common trend is similar for the two groups prior to the policy period.

The north (treated) and south (non-treated) regions may present different technological patterns due to the differences in the technologies (heating and building systems) used in the production of space heating services. To verify this, we estimate a model that includes an interaction variable between the time trend and a dummy variable that indicates whether a province belongs to the treated group or not. The estimation results show that the coefficient of the interaction variable is not statistically significant (p value 0.30). Therefore, we are confident that there exists no different technology trend between the treated and control groups. In Fig. 1, we illustrate the average energy consumption over time for both the treated and control groups.Footnote 10

Fig. 1
figure 1

Average residential energy consumption of treated and control groups over time

Full size image

3.2 Exogenous choice of treated group

Another assumption that should be satisfied in a DID approach is that the assignment to the treatment group is exogenous. This assumption can be violated if there are unobserved characteristics of the provinces that affect both the outcome variables and the policy decisions. We believe that this should not be an issue in our case as the treated group includes all the provinces within the official heating regions. There are two reasons there being no endogeneity issue related to the policy variable. First, the decision to introduce the HMEER policy into a province was taken by the central government and included in the national development plan of the 11th FYP as a mandatory regulation for regional governments. Second, as China sets ambitious targets for reducing emissions over time, there is considerable political pressure behind the application of the HMEER policy and its goals of energy saving and emission mitigation in these provinces.

4 Empirical results

The empirical analysis of this paper is composed of two parts. The first part shows a simple difference-in-differences calculation of the energy consumption data through a descriptive analysis and the second part, the most important part of this paper, involves the econometric estimation of Eq. (1).

The DID estimate of the effects of the HMEER policy on energy consumption can be computed using the following formula:

$${\text{DID}} = \frac{1}{{n_{\text{T}} }}\mathop \sum \nolimits \left( {E_{2}^{\text{T}} - E_{1}^{\text{T}} } \right) - \frac{1}{{n_{\text{C}} }}\mathop \sum \nolimits \left( {E_{2}^{\text{C}} - E_{1}^{\text{C}} } \right) = \overline{{\Delta E^{\text{T}} }} - \overline{{\Delta E^{\text{C}} }} ,$$
(2)

where E is the outcome variable, the subscript 1 denotes the pre-policy period and 2 denotes the post-policy period, T denotes the treated group, and C denotes the control group. The value of DID is obtained using group average values for treated and control provinces of the two outcome variables.

We can now compute the effect of the implementation of the HMEER policy on energy consumption based on these mean values and using expression (2). Of course, we are aware that this approach does not control for other variables that may influence the outcome variable and does not allow verification if the treatment effect is statistically significant. However, we believe that this simple approach is informative and provides a first-hand potential outcome of the policy. The results of calculations based on expression (2) are displayed in Table 3.

Table 3 The results of simple DID calculation of Model A
Full size table

The highlighted number in the bottom-right corner provides the difference-in-differences value. A negative value indicates that the policy reduces the energy consumption of the treated group compared to the control group. It shows that the HMEER policy induces, on average, a 0.48 Mtce reduction in energy consumption.

In Table 4, we present the empirical results obtained by estimating Eq. (1). Column (A) includes the results obtained using all the provinces, while column (B) shows the results obtained by considering only the border provinces. Overall, the values of the coefficients of the models have the expected signs and are statistically significant at the 10% level. Both province and year fixed effects are considered in the estimation.

Table 4 Estimation results of DID
Full size table

The estimation results reported in column (A) indicate that the HMEER policy has a statistically significant negative impact on the residential energy demand. By transforming the log-linear form of the coefficient for the HMEER policy, we find that the implementation of the HMEER policy contributes, on average, an approximate 10% reduction in residential energy demand.Footnote 11 The estimation results reported in column (B) are similar.

The coefficients reported in Table 4 provide information on the impact of different variables on residential energy demand. To note, that in a log–log functional form, the coefficients can be interpreted as elasticities. As expected, the price effect is negative statistically significant. However, the value of the price elasticity is very small. This result may be due to the fact that the level of aggregation of the price index used in this study is relatively high or due to the low variation of the energy prices due to the price regulation in China, or a bit of both.

The income elasticity is around 1.39–1.51 in the models, and both are highly significant at the 1% level. This indicates that a 1% increase in household income will lead to more than 1% increase in energy demand for households. Therefore, the income elasticity of energy demand is quite high, as is expected for emerging countries.

The coefficients of the three demographic variables, household size, population, and urbanization rate, provide an interesting picture for the emerging economy. As expected, population has positive and significant effect on energy demand, whereas the average household size has negative and significant impact on energy demand due to economies of scale. The coefficient of urbanization rate is positive and insignificant.

Both heating degree days (HDD) and cooling degree days (CDD) affect the demand positively. The effects of CDD are significant at the 5% level. This may imply that, as global warming continues, households demand more energy for cooling services, while the variation of HDD becomes less significant. This is in line with Zhang (2013) that energy used for cooling purpose is becoming a significant part of China's energy consumption. 

The econometric results reported in Table 4 can be used to calculate the reduction of CO2 emissions attributed to the introduction of the HMEER policy. To obtain this reduction, we multiplied the average energy saving obtained in each province after the introduction of the policy by the national CO2 emission coefficient.Footnote 12 The direct impact of the HMEER policy on energy saving for the policy period (2007–2010) is about 80 Mtce, whereas the reduction of CO2 emissions (2007–2010) is around 200 Mt CO2 equivalent.Footnote 13

As discussed previously, the main goal of China’s 11th Five-Year Plan was to reduce the level of energy intensity. Using the results of the empirical analysis, it is also possible to provide a rough approximation of the impact of the HMEER policy on decreasing the energy intensity of the residential sector during the 5-year plan period. In fact, between 2005 and 2010, the level of energy intensity measured as the level of energy consumption divided by GDP in the Chinese residential sector decreased by 13%.Footnote 14 Part of this reduction, approximately 43%, was due to the introduction of the HMEER policy.

5 Conclusion

Since energy efficiency has received much attention and a series of measures have been implemented for China’s development strategy, rigorous ex-post evaluation of these policies is vital for understanding and, hence, improving the contributions of such policies to improvement in energy efficiency. In this study, we integrate the energy demand model with the difference-in-differences approach for an ex-post evaluation of the impacts of the HMEER policy on residential energy consumption at the provincial level.

Our analysis confirms that the HMEER policy contributes to a reduction in residential energy consumption. We find quantitatively that the average impact of the policy is a 10% reduction in energy demand for the treated provinces. This result provides empirical support for the continuation of the HMEER policy in the future. As discussed in Bao et al. (2012), the HMEER policy in the 11th FYP promoted the retrofit of only 4.6% of the total building stock that needs retrofitting. The energy-saving potential can be very promising if the HMEER policy could be implemented on a large scale.

Of course, we should recognize that a limitation of our analysis is that we considered only the impact of the policy on energy consumption and did not account for the costs of implementing the HMEER policy. Therefore, we were not able to estimate the cost of reducing CO2 emissions through the HMEER policy. Moreover, a complete cost–benefit analysis of the HMEER policy would also require estimating the benefits of the improved environment and air quality. Furthermore, a limitation of this study stems from the data used. The sample considered in the empirical analysis was relatively small and based on aggregate data at the provincial level. For future research, it would be interesting to perform a similar analysis using energy consumption data at the building level.