Introduction

In recent years, urban energy consumption has increased dramatically with the rapid development of the economy and society (Grimm et al. 2008). According to statistics, global energy consumption increased from 11,588.4 million tons of standard coal in 2007 to 13,511.2 million tons of standard coal in 2017 (BP 2018), of which urban activity consumed 67% (Chen and Chen 2015). Energy is not only an important input element of economic growth, but also the heart and the lifeblood of urban industrial development and the material basis for human survival. Energy has played an irreplaceable role in social and economic development (Raggad 2018). However, the huge growth in urban energy consumption has also posed great challenges to the urban environment. Urban energy consumption has generated 70% of global greenhouse gas emissions (Chen and Chen 2015). At present, global greenhouse gas emissions are about 14% higher than the international annual standard level, and environmental pollution, global climate change, extreme weather, and other problems are becoming more and more serious.

Because it is difficult to increase energy supply, improving urban energy performance is one of the most important ways to solve these problems and has therefore become a major concern of researchers, urban managers, and residents. Urban energy performance has become a major sustainable development issue that is related to urban resources, environment, economy, society, management, and well-being. At the theoretical level, studies on urban energy performance evaluation could help to define scientifically the connotation of terms, expand the theoretical basis, and enrich the content framework for urban energy performance. At the practical level, they could effectively improve urban energy performance, propose ways and key points for improvement, and provide policy recommendations. However, the existing concept of urban energy performance has become divorced from the full scope of the concept because of changes in the social economy and the urban environment. Against this new background, there is a lack of indicators and influencing factors to evaluate urban energy performance. Existing evaluation methods need to be improved, evaluation systems need to be built, and the key points and difficulties of studies in this field need to be further sorted out and summarized.

Therefore, based on relevant studies on urban energy performance, this paper aims to review systematically current research trends, conceptual connotations, evaluation indicators, influencing factors, evaluation methods, and evaluation systems, after which future research directions will be proposed. Its innovations include: first, redefining the concept of urban energy performance from economic, environmental, and well-being perspectives. Second, a new framework of urban energy performance evaluation indicators and influencing factors is constructed. The framework reveals the content structure of urban energy performance evaluation indicators and influencing factors and defines the relationships between the various indicators and factors. Third, directions for improving urban energy performance evaluation and factor analysis methods are pointed out from three aspects: variable settings, the requirement for performance evaluation and factor analysis at the same stage, and the embodiment of policy effects. Fourth, the construction thought, system design, and realization approach for an urban energy performance evaluation system are proposed.

This paper is structured as follows: the second section presents the research trends. The third section systematically reviews the concepts, evaluation indicators, influencing factors, evaluation methods, and evaluation systems related to urban energy performance. The fourth section presents results and discussions. In this section, the limitations of existing studies are summarized from various aspects. The concept of urban energy performance is redefined, and a new framework of urban energy performance evaluation indicators and influencing factors is constructed. Improvement directions for evaluation and factor analysis methods are proposed, and the construction thought for an evaluation system is also presented. The fifth section is a summary of the main conclusions and the research prospects.

Research dynamics

The research theme of this paper is urban energy performance evaluation, and therefore “urban energy performance evaluation” was set as the literature retrieval search phrase. The “Web of Science” database has a convenient and customizable search function and was therefore chosen as the database source for literatures. A total of 412 papers were retrieved from 1986 to 2017 on April 3, 2018. The journals with the most publications were Energy and Buildings, Energy Procedia, Applied Energy, Renewable & Sustainable Energy Reviews, and Energy, which are devoted to studies of energy performance.

Table 1 summarizes the most published authors in the research field of urban energy performance evaluation. Grimmond CSB had the largest number of papers and a wide range of collaborations with authors from Singapore, Finland, the UK, the USA, Sweden, and other countries. The other authors had four or five papers each and came from Belgium, China, Spain, France, Italy, the USA, and South Korea. According to the authors in Table 1, China has the largest number of researchers active in the field, followed by Belgium and the USA. It is also apparent that China, Belgium, and the USA have the most training and support for researchers in this field. In addition, these authors often collaborate with each other and pay attention to developments in the energy and resource disciplines.

Table 1 The most published authors in the research field of urban energy performance evaluation
Full size table

As shown in Fig. 1, the number of papers related to urban energy performance evaluation has increased year by year since 2007. Especially since 2015, the number of papers has increased significantly and is expected to continue to increase in the future. Urban energy performance evaluation has become a focus and a challenge for current research work.

Fig. 1
figure 1

Number of published papers related to urban energy performance evaluation from 2007 to 2017

Full size image

The analysis results presented in Fig. 2 show that developed countries have conducted more extensive studies on urban energy performance evaluation and have paid more attention to the effects of urban energy use. The largest numbers of publications are from Italy and the USA, with these publications accounting for 33% of the total. As the largest developing country in the world, China not only has a huge land area and abundant energy, but also needs to consume a large amount of energy. It is therefore crucial to improve urban energy performance in China. Therefore, Chinese researchers have also carried out many studies on this topic.

Fig. 2
figure 2

Distribution of the countries with the most publications

Full size image

According to the titles of 412 papers, 18 high-frequency words were extracted, as shown in Fig. 3. “City,” “energy performance,” and “performance evaluation” were the literature retrieval search terms and are shown in Fig. 3. These high-frequency words reflect the research emphases of urban energy performance evaluation from the aspects of research objectives, perspectives, contents, and means. “Green,” “low carbon,” and “sustainability” are core themes directing urban development and have therefore become the research objectives of urban energy performance evaluation. “Energy system,” “energy consumption and saving,” “energy efficiency,” “environmental performance,” “economic performance,” “social effect,” and similar phrases show that the evaluation perspective of urban energy performance is not simply to reduce energy consumption or to achieve economic growth unilaterally, but to integrate economic performance, environmental performance, and social impact effectively. These high-frequency words expand the research perspectives of urban energy performance evaluation. The phrases “evaluation model and system,” “concept model,” “evaluation method,” “energy policy and strategy,” “energy management,” and “indicator framework” explain the research contents and research means of urban energy performance evaluation.

Fig. 3
figure 3

High-frequency words in the titles of studies related to urban energy performance evaluation

Full size image

A comprehensive analysis of existing studies reveals that the number of studies on urban energy performance evaluation is relatively small. More studies focus on energy performance evaluation in other objects such as countries, regions, industries, enterprises, and buildings. Different kinds of energy performance include energy well-being performance and carbon emissions performance, and studies closely related to urban energy performance include various themes such as urban energy efficiency evaluation, urban sustainable performance, urban environmental efficiency evaluation, and low-carbon ecological city evaluation. Therefore, this study uses these earlier studies related to urban energy performance as literature analysis materials to analyze the conceptual connotations, evaluation indicators, influencing factors, evaluation methods, and evaluation systems involved in these studies in detail, and provide references for future studies on urban energy performance evaluation.

Literature review

Connotations of the urban energy performance concept

The core of urban energy performance is “performance,” a term that has mostly been used in management. The concept of performance has been understood from three main points of view. The first view was that performance was the result of work. The second view regarded performance as a process of work behavior. The third view was a quality-oriented view (Yan 2014). As a multi-dimensional concept, performance included not only task performance, but also contextual performance. In other words, performance measurement involved not only measuring the various activities that convert raw materials into products or services, but also quantifying other behaviors and results that help to achieve good task performance (Borman et al. 2001). Therefore, urban energy performance mainly measures the behavior and results of urban energy use. In addition, energy “contextual performance” must be considered. This is determined based on the comprehensive urban goals of economic development, social progress, environmental quality, and social well-being.

The definition of energy performance originated from energy efficiency. Patterson believed that energy efficiency referred to producing the same number of services or useful outputs using less energy (Patterson 1996). Hence, the primary problem became how to define effective output and energy input. Four kinds of energy efficiency indices were produced, including the thermodynamic index, the physical-thermodynamic index, the economic-thermodynamic index, and the economic index. Among these, the economic-thermodynamic index was most commonly used. In other words, the concept of energy efficiency was expressed through energy intensity. However, this definition still needed to be expanded to measure heterogeneous energy, to sum energy consumption, and to define the scope of energy inputs and the types of outputs. In addition, energy intensity included a large number of industrial structural factors, the substitutions between elements and the changes in energy input structure, and it was not sufficient to explain changes in the technical efficiency of energy production. Therefore, it was inadequate to use the ratio of energy to economic output to represent energy efficiency. On this basis, researchers started from the basic meaning of efficiency and transformed the measurement of energy efficiency into the distances from samples to the production frontier. Accordingly, researchers further suggested that the concept of energy performance was influenced by many input variables, output variables, and factors. As part of this trend, the content of energy performance also changed and expanded. Energy well-being performance, carbon emissions performance, energy saving and emissions reduction performance, and environmental performance emerged. Energy well-being performance took human well-being as the output (Dietz et al. 2012; Knight and Rosa 2011); carbon emissions performance and energy saving and emissions reduction performance viewed the carbon dioxide emissions produced by energy consumption as an undesirable output (Wang et al. 2015a; Zhou et al. 2016); and environmental performance comprehensively considered the economic, social, resource, and environmental benefits of the energy consumption (Wu et al. 2017). The concept of energy performance has been used to define national, regional, industrial, enterprise, and building energy performance. However, a concept of urban energy performance has not been proposed.

The above analyses show that the definition of urban energy performance should be combined with the substitution and influence of energy and other inputs, rather than taking only economic output as an output indicator. Therefore, whether from the performance standpoints or from the research trends in energy performance, the concept of urban energy performance has subverted the view that energy performance is equivalent to energy efficiency. Energy performance has been transformed into a comprehensive concept that measures energy consumption processes and results, energy economic performance, and energy contextual performance and is influenced by variable factors.

Urban energy performance evaluation indicators and influencing factors

Existing studies on urban energy performance are relatively few. Table 2 summarizes the evaluation indicators and influencing factors related to urban energy performance from three existing perspective studies related to urban energy performance.

Table 2 Evaluation indicators and influencing factors summarized from existing studies related to urban energy performance
Full size table

As shown in Table 2, the evaluation indicators and influencing factors in different research contents are different. National energy performance is mainly evaluated based on a country’s overall energy consumption and economic output. To consider the roles of other input-output elements, variables such as capital, labor, and carbon dioxide emissions are included, which fully reflect the sustainability of national environmental development. The influencing factors are focused on industrial structure, energy price, territorial area, population, and energy imports and exports at the national macro level. The variables of evaluating regional energy performance are similar to those at the national level. Their points of difference are that the undesirable outputs of regional energy performance evaluation take regional characteristics and data availability into account and add sulfur dioxide emissions, exhaust emissions, and other variables. The choice of influencing factors for regional energy performance is more targeted, and the influencing factors are chosen based on the research emphasis. For example, to study regional economic, energy, and carbon emissions performance in China under market-oriented reform, Lin and Du (2015) selected the degree of marketization in different regions as the main influencing factor and set energy price, energy consumption structure, industrial structure, foreign trade openness, policy, and province as control variables. Within the scope of regional energy performance evaluation, energy price, energy consumption structure, the degree of openness, and environmental regulation were common influencing factors. Industrial energy performance can also be measured by energy, capital, labor, and industrial outputs. Its factors include the level of economic development, industrial structure, population density, and energy price. The energy performance of enterprises and buildings is evaluated from a more micro perspective. The energy consumed by enterprises is mainly used in production activities, and therefore enterprise energy performance is closely related to the level of energy use, production activity level, product mix, non-energy inputs, and the external environment. Building energy performance is measured by the energy consumption and utility of the building, which are related to building area, energy-consuming facilities, location, price, energy consumption behavior, policies, and other variables. By analyzing the energy performance of different subjects, it can be concluded that the energy performance of countries, regions, industries, enterprises, and buildings is mostly analyzed by the utility level of their outputs from the economic perspective.

As for other categories of energy performance, energy well-being performance pays more attention to the improvements in well-being obtained from energy consumption, and therefore energy well-being performance is measured by the levels of energy consumption and social well-being. The level of energy consumption is mainly determined by the per capita ecological footprint or actual energy consumption, whereas measurements of the level of social well-being can be derived from an objective or a subjective perspective. Objective well-being indicators include the index of sustainable economic welfare (ISEW) (Menegaki et al. 2017), the genuine progress indicator (GPI) (O'Mahony et al. 2018), the average life expectancy (Dietz et al. 2012; Jorgenson et al. 2014), and the human development index (HDI) (Koohi et al. 2017). Subjective well-being indicator refers mainly to life satisfaction (Knight and Rosa 2011). The influencing factors of energy well-being performance include the degrees of economic development, political democracy, and income equity, the level of industrial structure, and the health level. The evaluation of energy security performance focuses on energy availability, diversity, affordability, technology and efficiency, environmental sustainability, regulation and governance, and other aspects. In the context of increasing energy consumption and environmental problems, studies on carbon emissions performance are increasing in number. The core point of carbon emissions performance measurement is to incorporate carbon dioxide and other pollutants into the outputs and to take energy, labor, and capital as input variables. Economic scale, industrial structure, and income level have important impacts on carbon emissions performance.

Some researchers have studied urban energy efficiency and urban environmental efficiency from the perspective of total-factor inputs and outputs, but some researchers studied these questions from a broader perspective including urban area, population, family income, family refrigeration and heating days, the number of employees in the home, the number of families with two cars, and life expectancy. Urban sustainable performance evaluation and low-carbon ecological city evaluation comprehensively consider resources, environment, economy, society, lifestyle, technology, and urban accessibility and focus on renewable energy use, urban planning, social well-being, transportation convenience, research development and innovation policy, and waste disposal.

Through a detailed analysis and summary of these evaluation indicators and influencing factors, it can be concluded that the research perspective, purpose, objective, and content are the important bases for setting up evaluation indicators and influencing factors. The content and structure of framework for urban energy performance evaluation indicator and influencing factor needs further analysis and determination.

Methods of evaluating urban energy performance

Single-factor evaluation method

The energy performance evaluation method is derived from the single-factor evaluation method, that is, the ratio of energy consumption to GDP. This method reflects the interaction and constraint relationships between energy and economy and is easy to understand and use. In view of the impacts of climate, energy price, and population on energy performance, Nagata (1997) proposed the concept of real and single energy intensity, which effectively removed the impacts of the above factors on measuring energy performance.

Total-factor evaluation method

The single-factor evaluation method considers only a single input factor. It is obvious that this method has some defects, and therefore the total-factor evaluation method was proposed. Besides energy input, the total-factor evaluation method can also deal with other input variables, output variables, and various influencing factors. The total-factor evaluation method is not confined to any specific evaluation method, leaving the selection free to be based on actual needs and different variables. Table 3 summarizes the evaluation methods used in existing studies related to urban energy performance evaluation.

Table 3 Evaluation methods used in existing studies related to urban energy performance evaluation
Full size table

Through the above analysis, it could be concluded that SFA, fuzzy comprehensive evaluation method, DEA, regression analysis, and the Malmquist-Luenberger index are commonly used in studies related to urban energy performance evaluation. The fuzzy comprehensive evaluation method includes many specific methods. The core of these methods is to set up reasonable weights for different evaluation indicators, where the indicator weights are set to reflect decision preferences according to the actual situation and experts’ opinions. Similarly, the specific regression analysis method should be selected according to the research purpose, independent variable attributes, dependent variable attributes, and data sample conditions. SFA and DEA aim to reflect performance values by measuring the gap between target and actual energy inputs or the gap between expected and actual outputs. The fundamental difference between these two methods is whether the parameters of the variables can be estimated. DEA is a nonparametric analysis method, but SFA is a parametric method (Moutinho et al. 2018).

DEA was first proposed by Charnes et al. (1978) on the basis of Färe and Lovell’s (1978) study. After developments and improvements, CCR model, BCC model, Russell model, quasi-fixed inputs model, and SBM model were gradually developed into Shephard input distance function model, the directional distance function (DDF) model, the DDF + SBM model, multi-period data envelopment analysis, meta-frontier data envelopment analysis, and the Malmquist-Luenberger index. These models developed from models that included only desirable outputs to ones that included both desirable and undesirable outputs, and then to ones that considered different regions, periods, and technological developments. As shown in Fig. 4, on the assumption that all inputs could be adjusted in a radial direction, Charnes et al. (1978) and Banker et al. (1984) proposed classical DEA models, called CCR model and BCC model. These two models stipulated that all inputs could be scaled down proportionately without reducing the outputs. CCR model stipulated that the scale returns were constant, but BCC model permitted variable scale returns. Because the weak efficiency problem exists when CCR model and BCC model are used for calculation, the efficiency might be overestimated, in which case the recognition ability of these models might be weakened. Therefore, the DEA model (Färe and Lovell 1978; Tone 2001) that allows all inputs to be contracted in different proportions was proposed because this model could also directly deal with the redundancy of over-specified inputs and output shortages. Due to transaction costs and regulations, not all inputs could be adjusted to the optimal level. Therefore, Ouellette and Vierstraete (2004) modified the DEA model according to the quasi-fixed input hypothesis. This means that the energy input could be adjusted and the other inputs would remain quasi-fixed in the short term. With the aggravation of environmental pollution, the DEA model, which could deal with reductions in energy inputs and pollutants simultaneously, was proposed. It calculated energy performance based on Shephard distance function (Färe et al. 1996). However, the model still had some problems dealing with bad outputs, and it did not conform to actual production rules. Hence, DDF was proposed to deal with the pollutant problem (Chung et al. 1997; Färe and Grosskopf 2004). For piecewise linear production frontiers, the measured efficiency would be overestimated when nonzero slack variables existed in the constraint conditions. SBM directional distance function was proposed to solve this problem (Fukuyama and Weber 2009). Because of temporal and regional differences, the DEA model was developed into the multi-period data envelopment analysis model and the meta-frontier data envelopment analysis model. The Malmquist-Luenberger index, which consisted of an efficiency change index and a technological progress index, was used to measure total-factor productivity changes for two different periods. The efficiency change index could further be divided into pure technical efficiency change and scale efficiency change (Chung et al. 1997).

Fig. 4
figure 4

The development and changes of DEA method

Full size image

Unlike DEA, the SFA method could set parameters and separate out random factors. The initial SFA model was proposed by Aigner et al. (1977) . With extensions in research practice, the parameter estimation of non-efficiency items, the panel data model, and the time-varying performance model were also expanding. Battese and Coelli (1992) proposed the SFA model for panel data, which set the non-efficiency term to obey a truncated normal distribution. In 1995, the improved SFA model was proposed (Carrer et al. 2015), with performance measurement and factor analysis incorporated into the same model. SFA needs to assume an equation form when constructing the production frontier and uses maximum likelihood estimation method to estimate the parameters. A diversity of equation forms increases the risks of using SFA, and therefore it is important to determine the equation form reasonably and accurately. To solve the problems of multiple inputs and outputs effectively, Yin et al. (2017b) applied the ray stochastic frontier model. The development and changes of SFA method are shown in Fig. 5.

Fig. 5
figure 5

The development and changes of SFA method

Full size image

Evaluation system

Against the background of the “smart city” and “Big Data information technology,” all kinds of computer tools have been applied to the city in both macro and micro aspects. Urban energy performance covers a huge amount of data and information and is closely related to the economic, social, ecological, environmental, and well-being aspects of a city. Therefore, it is necessary to evaluate and monitor these aspects through an urban energy performance evaluation system and to strengthen further the comprehensive management and sustainable use of urban energy. Energy performance evaluation systems have been applied in various fields, including the international energy performance architecture index, the Energy Star Program in the USA, the Energy-Related Products directive in the European Union, the Energy Performance Certification in the European construction industry (Pagliaro et al. 2015), and the Energy Saving Trust in the UK. These energy performance evaluation tools used in countries, enterprises, and products have played outstanding roles and provide important references for constructing urban energy performance evaluation systems.

Results and discussions

Extension of the urban energy performance concept

By reviewing existing studies related to urban energy performance, combining the evolution of the energy performance concept and the characteristics of urban energy consumption activities, it has become clear that the concept of urban energy performance needs to be redefined from the following three aspects. First, because the city is the main energy user, the measurement scope of urban energy performance should be based on the processes of production activities and production results. Second, the measurement of urban energy performance is not reflected by the performance of energy tasks (such as economic output tasks), but should include “task performance” and “contextual performance” (including energy environmental performance and energy well-being performance). Third, urban energy performance is measured based on inputs and outputs. Within the total-factor evaluation framework, urban energy performance is influenced by many input indicators, output indicators, and other factors (Knight and Rosa 2011; Wang et al. 2015a; Wu et al. 2017). Therefore, urban energy performance is defined as the level of producing optimal outputs based on energy and other inputs through urban development activities under the influences of certain social regulations and factors. Based on the above analysis and definition, future studies of urban energy performance should take the economy, the environment, and social well-being as perspectives and pay more attention to integrated performance on the economy, the environment, and social well-being.

Construction of new framework for urban energy performance evaluation indicators and influencing factors

Through the analysis described in Section 3.2, the evaluation indicators and influencing factors used in studies related to urban energy performance were set up based on the research objects and purposes, ignoring multi-dimensional standards related to the economy, the environment, and social well-being. For example, national, regional, industrial, and enterprise energy performance as well as urban energy efficiency paid more attention to energy economic performance (Boyd et al. 2008; Cengiz et al. 2018; Lin and Du 2015). Carbon emissions performance, urban environmental efficiency, and low-carbon ecological city evaluation paid more attention to energy environmental performance (Lin et al. 2014; Üstün 2015; Wang et al. 2015a). Energy well-being performance and urban sustainability performance measured energy performance from the social well-being perspective (Jorgenson et al. 2014; Munier 2011). Therefore, based on conceptual contents and the existing urban energy performance evaluation indicators and influencing factors, a new framework for urban energy performance evaluation indicators and influencing factors is constructed and is shown in Fig. 6.

Fig. 6
figure 6

New framework for urban energy performance evaluation indicators and influencing factors

Full size image

The idea of constructing the framework is as follows: based on the conceptual contents of urban energy performance and current urban development goals, the economy, the environment, and social well-being are proposed as perspectives for future studies on urban energy performance evaluation and factor analysis. Therefore, in Fig. 6, urban energy performance includes urban energy economic performance, urban energy environmental performance, and urban energy well-being performance, which breaks through the limitation of purely studying urban energy economic performance. Production and living activities are taken as the carriers of inputs and outputs, which mainly reflects the dynamic process of transferring energy and other inputs to outputs through a series of activities.

According to analyses of evaluation indicators and influencing factors used in studies related to urban energy performance evaluation, the framework of urban energy performance evaluation indicators and influencing factors can be divided into two modules: the evaluation indicator module and the influencing factor module. In other words, it is not only necessary to evaluate urban energy performance using evaluation indicators, but it is also important to explore performance differences based on influencing factors. Capital, labor, and energy are set as input elements, and gross regional product, carbon dioxide emissions, other pollutants, and average life expectancy (or life satisfaction) are taken as output elements. The first three output elements are used to reflect energy economic performance and energy environmental performance and life expectancy (or life satisfaction) is used to reflect energy well-being performance. Among these output elements, gross regional product and life expectancy (or life satisfaction) are desirable outputs, and the other outputs are undesirable outputs (Dietz et al. 2012; Knight and Rosa 2011; Wang et al. 2012; Wu et al. 2017; Yang et al. 2018). Policy and region are situational variables (Lin and Du 2015). The influencing factors include industrial structure, energy price, population density, home car ownership, climate factors, Gini coefficient, health expenditure level, and unemployment rate. The first five factors help to explore differences in energy economic performance and energy environmental performance (Cengiz et al. 2018; Keirstead 2013), whereas the last four factors contribute to exploring differences in energy well-being performance (Išljamović et al. 2015; Knight and Rosa 2011).

Directions for improvement in urban energy performance evaluation methods

A review of relevant evaluation methods reveals that DEA and SFA are the most widely used. Different evaluation methods have certain limitations due to their own variables and function forms, but they are also being improved as the field is studied further. Based on the new framework of urban energy performance evaluation indicators and influencing factors, the directions for future improvement of urban energy performance evaluation methods can be foreseen as described below.

First, to make the framework of evaluation indicators and influencing factors more consistent with actual urban development, undesirable outputs are included in the measurement. As the urban energy performance range expands, the average life expectancy or life satisfaction, which embodies urban energy well-being performance, is also included in the framework. Therefore, future urban energy performance evaluation methods should focus on dealing with multiple desirable and undesirable outputs on the premise of ensuring the evaluation model’s rationality and robustness.

Second, previous studies related to urban energy performance evaluation and influencing factors have been carried out in two stages, which undoubtedly increases errors in evaluating performance and analyzing performance differences. The improved SFA method creatively integrated energy performance evaluation and factor analysis into the same evaluation model, but this method was limited to a single output. The DEA model considered multiple outputs, but it did not include the functions described above (Carrer et al. 2015). Furthermore, future urban energy performance methods should gradually integrate performance evaluation and factor analysis into the same method model.

Third, as important situational variables, policy and region play irreplaceable roles. Therefore, future urban energy performance evaluations should make policy orientation clearer. The first point is to integrate undesirable outputs into the evaluation model, which makes it possible to evaluate the economic and environmental effects of policies and to define the degree and scope of policy implementation (Wang and Zhao 2017). The second point is to set the weights for input and output variables and to compare the differences in output expansion and pollutant discharge before and after implementing energy consumption constraints, so as to estimate indirectly the effectiveness of changing energy policies (Wang et al. 2017a; Zhou et al. 2016). The third point is to estimate indirectly the opportunity cost caused by environmental controls by comparing the output differences under pollutant constraints. The opportunity cost is the potential economic loss caused by environmental policy implementation. Urban energy performance measured under these various adjustment methods would objectively reflect the regulatory roles of policies, which would provide an important reference for rational formulation of energy conservation and emissions reduction targets and paths. In addition, regional differences would lead different cities towards different production frontiers, making the performance analysis more reasonable.

Construction of an urban energy performance evaluation system

At present, there are relatively few evaluation systems for energy performance, and there are even fewer studies on urban energy performance evaluation systems. Therefore, an urban energy performance evaluation system would have substantial application prospects. Based on extensive analysis of existing energy performance evaluation systems, this paper proposes a construction thought for an urban energy performance evaluation system, as shown in Fig. 7.

Fig. 7
figure 7

Construction thought for an urban energy performance evaluation system

Full size image

In the system planning stage, the needs of the users are collected. Combining with the basic requirements of the information management system, the design objectives for the urban energy performance evaluation system are put forward. Regional management institutions, urban government management institutions, urban planners, urban energy management institutions, urban energy management researchers, and urban residents are the main users of the urban energy performance evaluation system. Therefore, while maintaining scientific rigor and rationality, the system should be both convenient and easy to operate. The primary requirement of the urban energy performance evaluation system is to assist users in intuitively obtaining evaluation results for urban energy performance and the effects of different influencing factors.

In the analysis stage, it is necessary to analyze the structures, functions, and construction tools provided by existing energy performance evaluation systems, to compare the advantages and disadvantages of existing systems, and to determine a logical framework for constructing the urban energy performance evaluation system (Chang and Yu 2001).

In the system design stage, the functions of the urban energy performance evaluation system are initially set as login, information storage, information and data addition, information and data modification, information and data editing, urban energy performance calculation, urban energy performance comparison, urban energy performance query, urban energy performance analysis in different regions, chart export, and other functions. At the same time, based on the design pattern of a stand-alone system, the modules of the urban energy performance evaluation system are envisaged as a database, a distributed component server, and a human-computer interaction interface. After comparing the functions of several software development tools, the development tools for the urban energy performance system can be finally determined (Soutter et al. 2009; Yin et al. 2017a).

In the system development phase, the system interface is designed with reference to existing evaluation systems and focuses on attractiveness and practicality. In addition, according to the selected evaluation indicators, influencing factors, and method model, the parameters used to edit the program are calculated.

According to the arrangement and design of the first four stages, the developed urban energy performance evaluation system needs to be tested. The system would be improved according to the suggestions of different users.

Conclusions and prospects

Based on the query results from Web of Science, this paper has analyzed the authors, numbers, regions, and themes of the studies related to urban energy performance evaluation and has reviewed conceptual connotations, evaluation indicators, influencing factors, evaluation methods, and evaluation systems used. Therefore, the following conclusions can be summarized.

First, with the intensification of urban energy use and environmental problems, the number of studies on urban energy performance evaluation has increased year by year. In recent years, the growth trend has become more rapid, indicating that the attention paid to urban energy performance evaluation is increasing. The authors with the largest numbers of urban energy performance evaluation studies are from China, Belgium, and the USA. These countries provide more support to researchers in this field, and mutual collaboration occurs among these authors, which provides conditions for further studies. Italy, China, and the USA are the countries with the largest numbers of publications. It was apparent that developed countries and countries with higher energy consumption pay more attention to the issue of urban energy performance. High-frequency words such as “green,” “low carbon,” and “sustainability” reflect the goals and themes of studies on urban energy performance evaluation. Terms such as “energy performance,” “energy system,” “energy consumption and conservation,” “energy efficiency,” “environmental performance,” “economic performance,” and “social effect” expand the research perspectives of urban energy performance evaluation. The research content and the means of urban energy performance evaluation are illustrated by high-frequency words such as “city,” “performance evaluation,” “evaluation model and system,” “concept model,” “evaluation method,” “energy policy and strategy,” “energy management,” and “indicator system.”

Second, from the review results, the concept of urban energy performance lies in “performance” and should highlight the roles of urban production and living activities, the measurement datum of input-output, the measurement range of energy performance, and the regulating effects of other control variables. The evaluation indicators related to urban energy performance are identified in this paper and mainly include energy, capital, and labor inputs, economic outputs, average life expectancy (or life satisfaction), carbon dioxide emissions, and other pollutants. The influencing factors include industrial structure, energy price, population density, home car ownership, climate factors, Gini coefficient, health expenditure level, and unemployment rate. The evaluation indicators and influencing factors are different under different objects, contents, and objectives, but existing urban energy performance measurements fail to incorporate these evaluation indicators and influencing factors into the same framework.

Third, SFA, fuzzy comprehensive evaluation, DEA, regression analysis, and the Malmquist-Luenberger index are often used in studies related to urban energy performance evaluation. SFA and DEA are most commonly used to measure performance based on the gap between target and actual energy inputs or the gap between target and actual outputs. The difference between these two methods is that the former is a parametric method, but the latter is nonparametric. The fuzzy comprehensive evaluation method measures performance by assigning weights to the variables, and the regression analysis method uses various measurement models based on independent, dependent, and control variables. The Malmquist-Luenberger index is mainly used to reflect dynamic change trends. Each method has its own advantages and disadvantages. An evaluation system is the most convenient tool for urban energy performance information storage and calculation, but there is no professional evaluation system for urban energy performance.

Based on the above conclusions, future studies should focus on the following aspects. First, increases in urban energy consumption and in environmental, ecological, and social problems have led to a situation where traditional energy economic performance is no longer applicable to current demands for urban energy performance evaluation. Future studies should be guided by the actual development goals and development conditions of cities and expand their research perspectives and themes on urban energy performance evaluation.

Second, in future studies, the definition of the urban energy performance concept and the construction of a framework for urban energy performance evaluation indicators and influencing factors should regard the research perspectives of urban energy performance evaluation as a benchmark, enrich the contents, structure, and categories of concept, and include a comprehensive set of evaluation indicators and influencing factors. In addition, such studies should integrate the different variables into the same framework to evaluate urban energy performance with proper consideration of variable endogeneity problems.

Third, with the complexity of new frameworks for urban energy performance evaluation indicators and influencing factors, future urban energy performance evaluation methods should measure multiple inputs and multiple outputs, incorporate performance evaluation and factor analysis into the same method model, and reflect the regulatory roles of policies. Scientific evaluation results and performance differences would provide important references for policy formulation.

Fourth, in the era of Big Data and information, computer information management tools are already supporting all aspects of urban production and life. Therefore, future research and application of urban energy performance evaluation systems should be applied to manage basic information, energy consumption data, evaluation indicator data, influencing factor data, and evaluation results. Performance evaluation, performance analysis, and other functions also would be embedded in the evaluation system to meet the needs of urban development decision-makers and system users.