1 Introduction

Product manufacturing is a branch that has to constantly improve its competitiveness by increasing quality and reducing cost. Interactive simulation techniques contribute to this improvement by identifying potential optimisations without incurring in heavy physical testing processes [1–3]. There are many different approaches in the literature that are based in physical simulations and deterministic models that optimise industrial processes in terms of quality, time, raw material, etc. From a design perspective, Cherifi et al. [4] propose the use of methodologies such as TRIZ [5] to introduce new factors in the design process.

On the other hand, current machine tool industry is creating devices fully equipped with sensors that provide massive and heterogeneous data that, in many cases, it is difficult to incorporate in existing deterministic/physical models. However, this data can contain highly valuable information that properly characterised and modelled could dramatically improve optimisation processes. Furthermore, existing models can be extended to include new aspects of design into the same designing process, for example, energy consumption aspects could be introduced in quality/time/raw material design models.

As a specific use case of this approach, we propose the use of probabilistic methods to model the energy consumption behaviour and perform predictions, increasing in this way the competitiveness of aluminium injection processes. As one of the main limitations of probabilistic methods is the uncertainty, this paper is focused on measuring, delimiting and reducing this uncertainty in order to provide a satisfactory confidence level in optimisation and design processes.

The term measurement and verification (M&V) refers to the measurements and calculations necessary for reliably quantifying savings delivered by an energy conservation measure (ECM) [6]. Savings are determined by comparing measured use before and after implementation of an ECM, making appropriate adjustments for changes in conditions [6, 7]. The most common approach is to develop a model of energy consumption before the ECM is implemented and, then, compare the metered use after the ECM is implemented to the estimate resulting from applying the model to the post-ECM data. Care must be taken to measure the uncertainty of the estimate. The usual method for energy consumption modelling is to aggregate the data by period of comparison (e.g. month) and build a linear regression model with the aggregated data. Then, the uncertainty is measured using traditional statistics and linear regression theory. The range for the estimate provided by these tools can be quite wide. Although consensus is widespread on the societal and industrial potential of the results of policies focusing on the implementation of ECMs, there has not been much work directed at providing a narrower range of possible values. This is important because the narrower the range of possible values the more precisely the savings can be quantified. And conversely, when the range of possible values for the estimate is wide it might not be possible to determine whether energy consumption has decreased or increased.

This paper presents a method for modelling the energy consumption of a metal mold injection machine and estimating the range of possible values at a given level of confidence for any period of time containing a large number of measurement intervals. In the case study presented in the results section, the boundary of the savings determination is the motor of the injection machine, which in the reporting period has been retrofitted with an ECM. This paper is only concerned with the performance of the systems affected by the ECM.

In M&V terminology, the baseline period is the period of time selected as representative of facility operations before retrofit and the reporting (post-retrofit or post-ECM) period is the time following a retrofit during which savings are to be determined [8]. Additionally, the baseline model is the set of arithmetic factors, equations, or data used to describe the relationship between energy use or demand and other baseline data and the adjusted baseline is the application of the post-retrofit independent variable data to the baseline model to determine the baseline energy use or demand adjusted to post-retrofit conditions [8]. The proposed method uses the baseline data for fitting the model and then applies the model to the reporting period data to estimate the adjusted baseline against which to measure the energy savings. The contribution of the model is to combine traditional statistics and machine learning methods to take advantage of the fact that the period in which the energy savings are measured (the reporting period) is usually quite larger than the time intervals in which the energy performance measurements are taken to provide narrower limits for the possible values of the estimate of energy consumed during the reporting period. The proposed M&V method is amenable to automation in energy management systems.

2 Related work

The international performance measurement & verification protocol (IPMVP) is a standard first issued in 1996 that “provides an overview of current best practice techniques available for verifying results of energy efficiency, water efficiency, and renewable energy projects” [9]. The protocol is complemented by the American Society of Heating, Refrigeration and Air-Conditioning Engineers (ASHRAE) Guideline 14 ‘Measurement of Energy and Demand Savings’ [8]. IPMVP serves as a framework to determine energy savings resulting from the implementation of an energy efficiency program whereas ASHRAE’s guideline focuses on the technical details and helps formulate a M&V plan [9]. ASHRAE’s guideline 14 is designed for M&V applied to buildings, not industrial processes [10]. The IPMVP protocol defines four different M&V options: partially measured retrofit isolation (A), retrofit isolation (B), whole facility (C), and calibrated simulation (D). This paper focuses on option B, retrofit isolation, defined in the protocol [9] as follows:

Savings are determined by field measurement of the energy use of the systems to which the ECM was applied, separate from the energy use of the rest of the facility. Short-term or continuous measurements are taken throughout the post-retrofit period.

At the end of 2014, the International Organization for Standardization published the first edition of ISO 50015:2014 Energy management systems—measurement and verification of energy performance of organizations—general principles and guidelines [11]. ISO 50015:2014 can be applied alongside ISO 50001:2011 Energy management system. The latter specifies the requirements for establishing, implementing, maintaining and improving an energy management system [12], whereas the former establishes general principles and guidelines for the process of M&V of energy performance of an organization or its components [11].

There has been much work published regarding energy modelling and M&V of energy savings in buildings (see, for example, [13, 14]). In that context, the possibility of reducing uncertainty in energy-use predictions using measured data with finer time resolutions has been explored in combination with Gaussian Process modelling of energy use [13, 15]. There are not so many publications in the context of industry and manufacturing processes, even though one of the pillars of the Industry 4.0 movement is the analysis of data (including sensor and meter data). In the context of energy efficiency, the case for metering in industrial facilities has already been made elsewhere [16]. The US Department of Energy’s Superior Energy Performance Measurement and Verification Protocol for Industry (SEPMVPI) [17] establishes a protocol for M&V in industrial settings. This protocol requires the facility to conform to the ISO 50001 standard. Previously, there was no established method of measuring the energy consumption of machine tools [18]. This paper proposes another method for reducing uncertainty based on fine resolution measurement data in an industrial environment.

3 Method

This method requires the system of interest (i.e. the injection machine or one or more of its components) to be fitted with an energy meter that takes measurements at regular intervals. It also requires that the system logs all changes from active to inactive state, and viceversa. Active time is used as a substitute for production. Shut-down periods are excluded from the data.

3.1 Mean active power in a measurement interval

The mean active power (kW) measurement in an interval is modelled as the regression line [19] of the time (in h) the injection machine spends in an active state within the measurement interval. If the active time is denoted as X and the mean active power is denoted as Y, then the model is shown in Eq. 1 where \(\alpha \) and \(\beta \) are the parameters of the model, \(\alpha + \beta X\) is the linear regression estimate, and \(\xi \) is the error term. If more than one component of the injection machine have been sensorised, then there needs to be a linear regression model per component.

$$\begin{aligned} Y = \alpha + \beta X + \xi \end{aligned}$$
(1)

3.2 Prediction error estimation

Traditionally in linear regression, the range of possible values for the estimate for a given confidence level depends on the standard error of the estimate (\(SE_{\hat{Y}}\)) and is calculated with Eq. 2, where n is the size of the sample used to fit the model, q the number of independent variables in the regression model, and t is the t-distribution quantile for a given confidence level and the degrees of freedom (DF) of the model (i.e. \( DF = n - q - 1\)) [20]. The formula for the standard error of the estimate is shown in Eq. 3. Note that \(SE_{\hat{Y}}\) is calculated using the sample data used for fitting the model.

$$\begin{aligned}&\alpha + \beta X \pm ( t \times SE_{\hat{Y}} ) \end{aligned}$$
(2)
$$\begin{aligned}&SE_{\hat{Y}} = \sqrt{\frac{\displaystyle \sum \nolimits _{i=1}^{n}{(Y_i - \alpha - \beta X_i)^2}}{n-q-1}} \end{aligned}$$
(3)

However, as the model trained with the data from one period will be applied to the other period (the test set), it is interesting to find out the mean error of the estimate on the test set. Following a machine learning [21, 22] approach, the prediction error is estimated through tenfold cross-validation [23] using the data for the training period. For each fold i, the mean prediction error or residual, \(\mu _i\), and its variance, \(\sigma _i^2\), are calculated. Then, the expected prediction error of the model on the test set, \(\mu \), is the arithmetic mean of the mean errors for the folds (\(\mu _i\)) and its variance, \(\sigma ^2\), the arithmetic mean of the variances (\(\sigma _i^2\)) [24].

3.3 Model fitting

Once the model’s prediction error has been estimated, the linear regression model is fitted using all of the period data. The SEPMVPI protocol allows three primary methods for applying an adjustment model.

The forecast method compares the observed reporting period energy consumption to the adjusted baseline period energy consumption by training the model with the baseline period data and applying it to the reporting period conditions. The adjusted baseline period energy consumption is the estimated energy consumption that would have been expected at reporting period conditions (e.g. production levels and external factors), if the baseline operating equipment and practices were still in place [17].

The backcast method compares the observed baseline period energy consumption to the adjusted reporting period energy consumption by training the model with the reporting period data and applying it to the baseline period conditions to estimate the energy consumption that would have been expected at baseline production levels and external factors, if the reporting-period operating equipment and practices were in place.

Finally, the standard conditions method compares the adjusted reporting-period consumption to the adjusted baseline period consumption at a standard set of production levels and external factors (collectively known as standard conditions). This method is equivalent to fitting a forecast model and applying it at standard conditions to calculate the baseline period energy consumption. Then, fitting a backcast model and applying it at standard conditions to calculate the reporting period energy consumption. And, lastly, comparing the two energy consumptions calculated.

3.4 Energy consumption in an interval

Given the mean active power (kW) in an interval, the mean energy consumed (kWh) in the interval is the product of the mean active power by the duration (in h) of the measurement interval (i.e. the period). If the energy consumed is denoted by E and the period of the measurements by \(t_m\) (which is constant as the sensor takes measurements at regular intervals), then the energy consumed is calculated as in Eq. 4.

$$\begin{aligned} E = t_m Y = t_m ( \alpha + \beta X + \xi ) \end{aligned}$$
(4)

3.5 Energy consumption for a period of time longer than a measurement interval

The mean energy consumed in any period of time that contains more than one measurement interval [e.g. an 8-h work shift such as (2015-09-01 06:00, 2015-09-01 13:59)] is the sum of the energy consumed in the measurement intervals contained in the period. If the energy consumed in the period P is denoted by \(E_p\), the interval duration by \(t_m\) (which is constant, as mentioned earlier), the timestamp at which each measurement was taken by \(t_i\), and the number of intervals in the period by \(N_p\), then the energy consumed in the period is given in Eq. 5, where the first term in the sum is the energy consumption estimate and the second is the error term.

$$\begin{aligned} E_p = t_m \displaystyle \sum _{t_i: t_i \in P} ( \alpha + \beta X_{t_i} ) + t_m \displaystyle \sum _{t_i: t_i \in P} \xi _{t_i} \end{aligned}$$
(5)

By the central limit theorem (CLT), if there is a sufficiently large number of measurement intervals in the period of interest then \(\sum \nolimits _{t_i: t_i \in P} \xi _{t_i}\) follows a normal distribution with expectancy \(N_p \mu \) and variance \(N_p \sigma ^2\). 99.73 % of the values of a normal distribution are within three standard deviations of the mean. Therefore, 99.73 % of the values for the energy consumption in the period will be in the interval shown in Eq. 6. The endpoints of the interval are the 99.73 % lower and upper limits of the energy consumption range, respectively.

$$\begin{aligned} \begin{aligned}&E_p \in \left[ t_m \displaystyle \sum _{t_i: t_i \in P} ( \alpha + \beta X_{t_i} ) + t_m ( N_p \mu - 3 \sqrt{N_p} \sigma ),\right. \\&\left. \quad t_m \displaystyle \sum _{t_i: t_i \in P} ( \alpha + \beta X_{t_i} ) + t_m (N_p \mu + 3 \sqrt{N_p} \sigma ) \right] \end{aligned} \end{aligned}$$
(6)

whereas the range of possible values for the consumed energy given by traditional linear regression methods would be the one shown in Eq. 7.

$$\begin{aligned} t_m \displaystyle \sum _{t_i: t_i \in P} ( \alpha + \beta X_{t_i} ) \pm N_p t_m ( t \times SE_{\hat{Y}} ) \end{aligned}$$
(7)

When the number of measurement intervals in the period of interest is large enough so that the CLT applies, the range obtained with this method (Eq. 6) is narrower and provides a better estimate than the range obtained with traditional linear regression (Eq. 7).

3.6 Measurement of energy savings

The IPMVP protocol states that, in general, ‘energy savings are calculated by comparing the observed energy consumption during the reporting period to the adjusted baseline’ [9]. This can be applied directly to the forecast method, as shown in Eq. 8 where \(SAVE_{reporting}\) denotes the energy savings during the reporting period. The adjusted baseline, \(E_{reporting}\), is the energy consumption estimate given by the forecast model applied to the active time data of the reporting period and \(O_{reporting}\) is the observed energy consumption during the reporting period.

$$\begin{aligned} SAVE_{reporting} = E_{reporting} - O_{reporting} \end{aligned}$$
(8)

When using the backcast method, the savings are calculated by comparing the observed energy consumption during the baseline period to the energy consumption estimate given by the backcast model applied to the active time data of the baseline period [17]. Using the standard conditions method, the savings are calculated by comparing the energy consumption estimates given by the backcast and forecast models applied to the standard conditions [17].

4 Results

The proposed method has been tested with the data for the motors of an injection machine following the forecast method of the SEPMVPI. The forecast method is, according to ASHRAE [8], the most common method of adjustment. Mean active power demanded by the motors has been recorded at a period of observation of two minutes. The injection machine also automatically logs when there is a change in state, from active to inactive and viceversa. Thus, it is possible to calculate how long the machine has been in an active state within each of the mean active power measurement intervals.

The dates are 2015-09-14 to 2015-09-18 with 3299 records for the baseline and 2015-09-21 to 2015-10-02 with 6065 records for the reporting period.

Fig. 1
figure 1

Observed mean active power by measurement interval active time and regression line fitted with baseline data (Eq. 9) with lower and upper limits for the possible values of the estimate calculated with Eq. 2

Full size image
Fig. 2
figure 2

Estimated and observed energy consumption by 8-h shifts for the baseline and reporting periods using the proposed method (eq. 6)

Full size image

4.1 Model fit

The regression line for the mean active power fitted to the baseline data is shown in Eq. 9 and Fig. 1. This figure also shows the upper and lower limits of the range for the possible values of the estimate for the 99.73 % confidence level calculated with Eq. 2. The root-mean squared error is 5.516 (15.7 % of the mean value, 35.14). The prediction error estimated using cross-validation has a mean of \(-\)0.00078 and variance 31.

$$\begin{aligned} Y = 19.82 + 522.8 X + \xi \end{aligned}$$
(9)
Table 1 Energy savings estimate for the full reporting period under consideration
Full size table
Fig. 3
figure 3

Comparison of traditional linear regression and machine learning possible value ranges for the consumed energy estimate for a training and test set partition of the baseline data by 8-h shifts. a Consumed energy estimate using linear regression on data aggregated by 8-h shift. b Consumed energy estimate using the proposed method

Full size image

Figure 2 shows the observed and estimated energy consumption for 8-h shifts for both periods, baseline and reporting, calculated with the proposed method. Table 1 shows the numerical results. The total reporting period energy consumption is 5212.29 kWh and the adjusted baseline consumption is 6938.66 kWh. Therefore, the savings are 1726.37 kWh, i.e. 24.88 % of the adjusted baseline consumption. The percentage saving is 24.41 % with respect to the lower bound and 25.34 % with respect to the upper bound.

4.2 Comparison

This section compares three approaches for calculating energy savings using a training and test set partition of the baseline data. The first approach is the most used in industry and uses aggregate data to fit the linear regression model. In this case, the data (consumed energy and active time) has been aggregated by 8-h shift, the linear regression model has been fitted to this aggregate data, and the range of possible values for the estimate has been calculated using Eqs. 2 and 3. The result is shown in Fig. 3a.

Figure 3b shows a comparison of the consumed energy range obtained using the proposed method (Eq. 6) and the aggregate of linear regression estimate ranges for unaggregated data (Eq. 7). It can be seen in this figure that the estimates themselves are quite similar but the proposed method provides a much narrower range of values. The range of values provided by traditional linear regression methods is too wide to be of use in practice.

Fig. 4
figure 4

Temperature measurements for the Bilbao/Airport station retrieved from NOAA for baseline and reporting periods, where TMIN, TAVG, and TMAX are the daily minimum, average, and maximum temperatures, respectively

Full size image

4.3 Model validation

This section will show that the model is a valid model for calculating adjusted consumption according to the validation criteria specified by the SEPMVPI [17].

This protocol states that the variables considered for inclusion in the model must include production quantities, weather, input quantities and input characteristics, such as moisture content. The proposed method uses active time as a substitute for production as they are highly correlated in this case. Specifically, Pearson’s linear correlation coefficient between active time and part production in an 8-h shift is 0.9671 overall, 0.9853 for the baseline period and 0.9636 for the reporting period. The advantage of using active time is that data collection is easier, as the timestamp of active/inactive changes of state is recorded automatically, and, it is possible to calculate active time for any length of measurement interval. Production data, however, is only recorded once at the end of each 8-h shift. Both the intercept and the active time predictor variable in the linear regression model are significant with p value 0 and 0 (up to the computer’s 64-bit floating point precision), respectively.

Regarding weather conditions, the two periods are close in time (three consecutive weeks) so it can be assumed that weather and environmental conditions are similar and, thus, it is not necessary to include this variable in the model. Temperature data from the US Department of Commerce National Oceanic and Atmospheric Administration (NOAA) database for the station located at Bilbao (Spain) airport (41.1 km from Abadiño, where the plant is located) has been used to check this assumption. Figure 4 shows that both periods have very similar temperatures, so the extrapolation of the baseline model to the reporting period is reasonable. In order for the model to be applicable year-round, weather-related predictor variables would need to be included in the model and the model would need to be fitted with full-year data. Lastly, in the periods considered the injection machine is producing the same part, thus input quantities and input characteristics are the same in both periods.

Fig. 5
figure 5

Observed mean active power distributions by 8-h shift

Full size image

The mean active power measurements contain outliers, mostly very low readings that often but not always occur when active time is low (as can be appreciated in Fig. 1). These outliers have not been removed from the data because it is deemed important that, as a next step, they are analysed further. However, the effect of these outliers on the reliability of the model estimates has been studied. In Fig. 3b it can be appreciated that there are two shifts for which the observed energy consumption is outside the range provided by the proposed method. The two troughs below the CLT lower limit (shifts 2015-09-15 06:00 and 2015-09-16 14:00) correspond to the two 8-h shifts for which the observed mean active power distributions have considerably lower values, as can be seen in Fig. 5. However, there are other shifts which have a considerable number of outliers and for which the estimate provided by this method is a good approximation to the observed energy consumption, which is well within the range of values provided by the method. Therefore, it is not the presence of outliers that is compromising the reliability of the model, but the presence of shifts with a notably different distribution of mean active power readings within the same period. These shifts have a lower aggregate active time. However, shift 2015-09-14 06:00 has an aggregate active time similar but lower than that of shift 2015-09-16 14:00 and its mean active power readings distribution is not as positively skewed and its energy consumption estimate is a good approximation of the observed consumption. These events will be studied further in future work.

Regarding model testing (section 3.4.5. of the protocol), the p value of the F test of the overall model fit is 0 (up to the computer’s 64-bit floating point precision) and the threshold to be valid is 0.1. The active time predictor variable has a p value of 0 (up to the computer’s 64-bit floating point precision) and, thus, below the threshold of 0.2. Lastly, the coefficient of determination (\(R^2\)) of the model is 0.5019 (above the 0.5 threshold).

For the model to be valid for calculating adjusted energy consumption, the average of the predictor variables used to calculate the adjusted consumption from the model must fall within either the range of observed data that went into the model or three standard deviations from the mean of the data that went into the model [17]. In this case, the range of the only predictor variable, active time, is [0, 0.03333] for both the baseline and reporting periods (as can be seen in Fig. 1). As the model satisfies all the requirements of the validation tests set by the SEPMVPI protocol, it can be concluded that it is a valid model.

5 Conclusion

Traditional manufacturing industry is going through a process of significant evolution brought about by technological developments. Companies need to adapt to remain competitive. One such change is the quantitative monitorisation of all processes with the objective of improving efficiency. This monitorisation provides data, which then has to be transformed into actionable information. In the M&V field, measured data has to be processed with the objective of quantitatively estimating energy savings. In order to do this, the baseline energy consumption needs to be modelled.

This paper presents a method for developing a baseline model for an injection machine based on high frequency energy metering. The proposed method achieves an excellent fit for the baseline data and, thus, provides the means to estimate the adjusted baseline against which the energy consumption for the reporting period can be compared. The main contribution of the proposed method is that it also allows narrowing the range of possible values for the estimate and, therefore, the energy savings can be estimated better.

It is interesting to note that the baseline energy consumption can be modelled based only on the active time for baseline and reporting periods with similar environmental conditions (e.g. climate...) and production characteristics (e.g. production of the same part). To develop a baseline model applicable year-round, it will be necessary to include other factors, such as climate and production details, in the model.