Introduction

Dependence of humans on electricity is ever-increasing especially in this technology-driven world, where even the smallest of appliances needs electricity for its functioning. In such a situation, the resources are strained, there is an increase in carbon footprint and the goal of sustainability seems further far-fetched. Especially in the residential sector, a lot of energy wastage takes place and often goes unnoticed. If the end user is informed about the consumption, it can help and reduce wastage. Thus, the concept of informed consumption is vital. Following on this idea, a residence-specific, cost-effective energy management solution was developed. It includes the concept of Non-Intrusive Load Monitoring (NILM). NILM is based on the principle of energy disaggregation. A combined reading is disaggregated into device specific values. This is useful as it requires just a single hardware setup, thereby making it cost effective and less intrusive. The readings are sent to the cloud database and a user interface has been provided through which consumers will be able to view the energy expenditures for each registered appliance, monthly, yearly, values and the estimated energy bill. This feature will allow users to check for discrepancy in the received electricity bills by the electricity board at the end of every month; a problem encountered by many in India especially during the early pandemic days. Many citizens felt that the bills they have received are wrong and the bill calculations are not correct [1, 2]. The proposed system is, hence, labeled as ‘Smart’ Energy Management System as it combines the functionality of disaggregation and visualization utilizing the included algorithms and setup. Thus, four goals of Measurement, Management, Monitoring and Visualization have been combined in a single application.

Related Work

Several studies have shown the design of a home energy management system before. Various technologies have been used by researchers to achieve this goal. In [3], ZigBee technology has been used, where ZigBee-based modules are deployed to monitor appliance consumption in addition to a Power Line Communication (PLC) based gateway. [4] used a complete IoT based approach to monitor energy consumption using Wi-Fi and MQTT protocol. [5] also uses IoT approach in addition to Blockchain for controlling and synchronization of various IoT devices and deploys a microcontroller based circuit in the hardware module. In [6], an Arduino Uno and ESP8266 module is installed near the MCB and an interconnection of Relays was used to switch ON/OFF the appliances. Most of the above works use multiple hardware modules and interconnections to achieve appliance-specific energy monitoring. The problem with multiple device approach is that in place of a single hardware module used in the NILM approach many such modules will be required. This will naturally increase the cost of such system and will be tedious on the installation front. Moreover, in place of maintaining a single module in the NILM approach, maintenance of several modules are required in the intrusive approach. In [7], NILM-based approach is used. The algorithms used in this approach are Decision Tree and K-Nearest Neighbor(KNN). These are essentially and widely used as supervised classification algorithms. The classification algorithms only classify a state for a given device (say ON/OFF) when a combined power rating is given. Thus, this approach is used for load identification. However, it is also required to give the actual consumption value for a given device and not just its ON/OFF status which can be procured by algorithms like FHMM and CO and are, thus, used in this proposed system. With the help of these algorithms, this system gives the values of individual power consumption of each device as well. Through this proposed system, it has been attempted to offer a solution to monitor device specific energy consumption through a single hardware module in a non-intrusive method and also indicate the power consumption of each of the monitored devices as well. To achieve this, usage of NILM techniques and algorithms is done along with a single hardware module for data acquisition and web-based Dashboard as a visual interface.

Non-intrusive Load Monitoring Overview

Non-intrusive Load Monitoring (NILM) is a form of Appliance Load Monitoring (ALM) techniques. It is also called single point monitoring as it measures consumption from just one point (like an energy meter). This has many practical advantages like low hardware cost, low installation costs, easy installation, etc. This NILM concept is not new. It was proposed by George W. Hart, Ed Kern and Fred Schweppe in the 1980s which can be further reviewed in [8]. However, especially in India, it is still not much in practical usage and attempts are being made to further enhance it. The first step to NILM is Data Acquisition. Using suitable hardware set up data is acquired. These data consist of combined readings for an entire household or a specific measurement area. The concept of NILM can be mathematically defined as,

$$\begin{aligned} P(t)=P_1(t)+ P_2(t)+ P_3(t)+...P_n(t), \end{aligned}$$
(1)

where P(t) is the combined power rating at the point of measurement whereas \(P_1(t), P_2(t)\), etc. are ratings of individual appliances contributing to the combined power and n is the number of appliances. NILM disaggregates P(t) into individual appliance power. The first step to this disaggregation is data acquisition. The aggregated energy rating is captured using a suitable hardware at a specified sampling rate. Then, the acquired data are processed using various algorithms. Then, the appropriate load and its consumption are identified. The algorithms used for NILM are supervised or unsupervised. As per the supervised algorithms, the system has to be trained using an adequate amount of labeled data. Various datasets have been developed for NILM purposes; however, most of them, like Reference Energy Disaggregation Dataset (REDD) [9], Building-Level fUlly labeled dataset for Electricity Disaggregation (BLUED) [10], etc. have been developed for foreign households and their typical appliances. Indian data for Ambient Water and Electricity Sensing (iAWE) consists of both aggregate and sub-metered readings. More information of iAWE is available in [11]. However, to be able to appropriately test the dataset we have developed all our own datasets which have been elaborated in the further sections. A supervised algorithm Factorial Hidden Markov Model (FHMM) and Combinatorial Optimization (CO) have been used in the implementation of the proposed energy management module. FHMM is an improvisation of the Hidden Markov Model (HMM) which in turn is based on the Markov chains. Formally, we can represent a FHMM as a set of k HMMs that evolve simultaneously and independently over time. In the implementation of FHMM in the energy disaggregation, it initially assigns an HMM to every device whose energy has to be disaggregated. Then it combines all HMMs into a single one. This is the training phase. In the testing phase, it runs the Viterbi algorithm to find the most likely sequence of states given the observations. Combinatorial Optimization (CO) on the other hand, is a relatively simpler algorithm as compared to FHMM. During training, the power consumption of each device is clustered into values based on its different states. For example, if an appliance has consumption values as {20, 23, 18, 0, 0, 0, 2, 19}, then they will be clustered as {20, 0} corresponding to two power states. Then, a two dimensional array will be made in which each column will represent a device and the rows represent all the possible device combinations. During disaggregation the algorithm just finds the row whose sum is nearest to the aggregated signal. [12] gives in depth information about these algorithms in addition to additional algorithms of NILM.

System Design

This section discusses the design of the proposed system for energy disaggregation. The system can be divided into two parts—the Hardware module for data acquisition and disaggregation, whereas the Software module consists of the ML model, cloud database and the dashboard based web application to view the disaggregated data. Fig. 1 gives an overview of the system as a whole.

Fig. 1
figure 1

Block diagram of the proposed energy management system using NILM

Full size image

Hardware Design

The hardware module consists of a sensor for measuring the energy ratings and a microcontroller for measuring and processing of the data. Various sensors exist which measure AC values like ACS712, SCT-013, PZEM-004T. The former two sensors had certain limitations: only current or voltage values were obtained, signal conditioning circuits were required, etc. However PZEM-004T requires no additional circuitry to support its functioning, it has libraries aiding its operation in Arduino and Raspberry-Pi interfacing. Moreover, it provides six readings: voltage, current, frequency, power factor, power, energy. Hence, PZEM-004T energy measurement module was selected as a sensor. It uses a non-invasive Current Transformer (CT) sensor. The CT sensor has a primary winding a magnetic core and a secondary winding. The live or neutral wire acts as the primary winding. The data are read through the TTL interface. The measuring range of the module is up to 10 A as it is and up to 100 A with the additional current transformer module. The 100 A setup has been used in this system. The microcontroller used is Raspberry Pi 3B+ due to its compatibility with Python programming language (since most of the coding and algorithms have been developed in Python), internet connectivity and an ability to support a machine learning module in itself. Fig. 2 indicates the overall hardware module circuitry. As is previously mentioned, due to the presence of a single module for hardware the cost for multiple sensors and modules with their installations is reduced to a great extent. The module primarily needs a microcontroller and a sensor for its implementation which brings down its cost and makes the system cost-effective as opposed to its intrusive variant which needs multiple such modules to be installed for each appliances required to be monitored.

Fig. 2
figure 2

Raspberry Pi with PZEM-004T interfacing

Full size image

Software Design

From the software perspective, the entire operation can be divided into three steps. The first step is disaggregating the acquired power readings. There are certain algorithms like Decision Tree or KNN which tell us the ON/OFF status of the appliance. However, FHMM and CO tell us this status as well as give the value for power consumption of each appliance. Moreover, these algorithms have been included in the NILM Toolkit [11] which is used to standardize the various algorithms and datasets to aid researchers. Thus, the FHMM and CO were selected in this approach. The disaggregated values from both these algorithms are compared in further sections. The Simple Load Disaggregation (SLD), a python based library has been used for the implementation. A detailed overview of SLD can be found in [13]. The trained machine learning model is loaded into the Raspberry Pi 3B+. The readings are collected through the sensor and sent to the microcontroller via TTL interface. The microcontroller reads these values and processes them through the algorithms. The disaggregated values are acquired and are sent to the cloud database. For the purpose of this implementation MongoDb Atlas Cloud Database service is used. It is a NoSQL form of cloud database and thus offers flexibility. It is a document database, which means it can store data in JSON or BSON like documents. It also supports array and nested object values and allows for flexible and dynamic schemas. The values are stored with specific timestamps to extract and display data at weekly, monthly, yearly time frequencies. The values from the hardware module are sent to the cloud database. The last block of the energy suite is the creation of a visualization interface for the user. This is in the form of a dashboard like structure where the user will be able to monitor the readings. The visualization gives users control over their energy expenditure. If any discrepancy is observed, it can be acted upon easily. Various plots are being plotted containing different parameters such as total energy consumption, total power consumption, appliance-wise consumption in percentage, monthly consumption, power factor, frequency, units of electricity consumed, etc. The Figs. 3, 4 shows the created visual interface

Fig. 3
figure 3

Dashboard line charts

Full size image
Fig. 4
figure 4

Dashboard gauge charts

Full size image

Hence using the hardware module, the electricity readings can be measured and the algorithms will disaggregate them into individual device-specific readings. Through the dashboard users can keep an eye on their personal consumption. Thus, with this setup using a single hardware module user will be able to easily monitor and measure the personal energy expenditure. The visualization interface presents the users a view of their consumption habits and it has been seen that such view can inculcate an energy saving behavior in users. Thus, this view can be called as a feedback to the consumers which in turn helps in inducing a behavior towards efficient energy consumption. Hence, this system can be said to provide this feedback mechanism.

Datasets

For this implementation specific datasets were created. These datasets pertaining to the implementation facilitated appropriate testing of the entire setup. Although there are a few pre-existing datasets for NILM and energy disaggregation purposes, the electrical appliances used in those datasets are varied and large in number. Even though it was possible to train our models using them, it was not easily possible to test those models accurately due to unavailability of so many appliances and the setup. Four datasets were created and trained using two algorithms—Factorial Hidden Markov Model and Combinatorial Optimization. Both the algorithms follow distinct methods for disaggregation as discussed in the earlier section. For the preparation of the datasets, a sub-metering approach was used where a separate sensor was attached to each device to get its individual values whereas another sensor was connected to the main meter providing a supply to both the devices. The description of each dataset is as follows:

  • Dataset 1: In the first dataset, a 9 W LED Bulb along with the hairdryer was used. Since the consumption of hair dryer was very high as compared to the LED bulb, the algorithm gave poor results. The algorithm rejects the LED bulb values and hence the accuracy is impaired. It was, thus, inferred that, the appliance in consideration for disaggregation should have comparable values. (If there are appliances with comparatively lower consumption values, readings of such can be combined into one during practical implementation. For e.g., if the reading of a single light bulb is low for the purpose of disaggregation the entire lighting system of a household or of two to three rooms can be combined.)

  • Dataset 2: This dataset was made between a 100 W incandescent light bulb and hair dryer operated in a single mode. Due to the single mode of appliances, the power consumption of each appliance was almost constant and the values of it were comparable unlike in that of dataset 1. This resulted in great accuracy for disaggregation.

  • Dataset 3: This dataset was made for a 100 W incandescent bulb but this time the hair dryer was operated in both its modes (HIGH and LOW) to test for hidden mode identification by the algorithm. Although the accuracy degraded than dataset 2 giving out some false negatives as well but all in all it was still better than dataset 1.

  • Dataset 4: This dataset was made between a 9 W LED bulb and a 5 W LED bulb. There were no false positives or false negatives but the accuracy was not as good as the dataset 2. Overall the algorithm becomes less accurate if power consumption values are very less.

The combined readings as well as the individual appliance readings taken for each dataset are graphically represented in the Figs. 5, 6, 7, 8, respectively.

Fig. 5
figure 5

The graph for Dataset 1. The blue line indicates the total power whereas the yellow line gives the hair-dryer values. The red line at the bottom indicated LED bulb values which evidently is very low compared to total rating

Full size image
Fig. 6
figure 6

Dataset 2—Hair dryer in single mode. The blue graph gives the total power, yellow the hair-dryer power and the red as the incandescent lamp power

Full size image
Fig. 7
figure 7

Dataset 3—Hair dryer in multi-mode. The blue graph gives the total power, yellow the hair-dryer power and the red as the incandescent lamp power

Full size image
Fig. 8
figure 8

Graph of Dataset 4. Blue line gives total power, the yellow gives the 9 W bulb power and the red line gives the power for the lower rating bulb

Full size image

The advantage of these datasets are that they are practically evaluated. These datasets incorporate not just different types of appliances but also their combinations in various modes. These can be further leveraged by researchers for research and development regarding the same topic of energy disaggregation. Moreover, the performance of these datasets was evaluated based on certain parameters like Normalized Error in Assigned Power (NEP) and Fraction of total Energy Assigned correctly (FTE).

  • Normalized Error in assigned Power (NEP): The sum of the differences between the assigned power and actual power of appliance n in each time slice t, normalized by the appliance’s total energy consumption. It can take any non negative value but lower values are desirable.

    $$\begin{aligned} \frac{\sum _{t}|y_t^{(n)} - \hat{y}_t^{(n)}|}{\sum _{t} y_t^{(n)}}, \end{aligned}$$
    (2)

    where \(y_t^{(n)}\) is the predicted power for \(n^{th}\) appliance at time t

  • Fraction of total energy assigned correctly (FTE): The overlap between the fraction of energy assigned to each appliance and the actual fraction of energy consumed by each appliance over the data set. Value varies between 0 and 1. Higher value is desirable.

    $$\begin{aligned} \sum _{t} \min {\left( \frac{\sum _{n} y_t^{(n)}}{\sum _{n,t} y_t^{(n)}}, \frac{\sum _{n}\hat{y}_t^{(n)}}{\sum _{n,t}\hat{y}_t^{(n)}}, \right) }, \end{aligned}$$
    (3)

    where \(y_t^{(n)}\) is the actual power and \(\hat{y}_t^{(n)}\) is the predicted power for \(n^{\text {th}}\) appliance at time t

Implementation and Results

The dataset was divided into training and testing and the predicted output was again verified with the actual values of power consumption. It was made to run for both the types of algorithms, FHMM and CO; the results of which have been compared further. The first dataset gave very poor disaggregation results and as per discussed earlier it was majorly because the power consumption of one appliance was negligible in comparison to the other. The second dataset in which the appliance 1, i.e., the hair dryer was operated in just a single mode, whereas the incandescent lamp also had a constant 100 W consumption. This gave good results for both FHMM and CO. The following figures analyze these outputs where the graphs compare actual and predicted output of each device and the device-wise share of the total consumed power.

Fig. 9
figure 9

Dataset 2: Appliance 1 prediction and actual values

Full size image
Fig. 10
figure 10

Dataset 2: Appliance 2 prediction and actual values

Full size image
Fig. 11
figure 11

Dataset 2: Actual power consumption share

Full size image
Fig. 12
figure 12

Dataset 2: Predicted power consumption share

Full size image

In Figs. 9 and 10, the blue line gives the actual power consumed, whereas the red line indicates the predicted power output. A good overlap suggests good prediction accuracy. The Figs. 11 and 12 show the consumption share. The actual values show 57.7% and 42.3% consumption share for hair dryer and lamp, respectively. Similar analysis has been obtained for each dataset depicted in the Figs. 13, 14, 15, 16. Due to two modes, the accuracy has reduced in dataset 3 than earlier dataset but it is still good.

Fig. 13
figure 13

Dataset 3: Appliance 1 prediction and actual values

Full size image
Fig. 14
figure 14

Dataset 3: Appliance 2 prediction and actual values

Full size image
Fig. 15
figure 15

Dataset 3: Actual power consumption share

Full size image
Fig. 16
figure 16

Dataset 3: Predicted power consumption share

Full size image

Here, the first graph (Fig. 13) gives predictions for the incandescent lamp whereas the second one for hair dryer (Fig. 14). Due to the multi-mode operation of the hair dryer in this case the accuracy has been reduced to some extent. The blue lines give actual appliance consumption, whereas the red lines give the predicted values over the time of measurement. Similar to Dataset 2, a consumption share pie-diagram has been plotted as well. For this the actual percentage of share was 58.1% and 41.9% of total power for appliance 1 and appliance 2, respectively, whereas the algorithm predicted it as 60.4% and 39.6% of total power for appliances 1 and 2, respectively (Figs. 15, 16). The results of Dataset 4 (Figs. 17, 18, 19, 20) were not as promising to the previous two datasets owing to its very small power consumption values. The blue lines and red lines depict actual and predicted power consumption, respectively, and the Figs. 17 and 18 gives the values for appliance 1 and appliance 2 which in this case were LED lamps of different wattage (9 W and 5 W). The difference in appliance 1 power consumption and its prediction is just 0.5 W but it appears amplified due to its small value.

Fig. 17
figure 17

Dataset 4: Appliance 1 prediction and actual values

Full size image
Fig. 18
figure 18

Dataset 4: Appliance 2 prediction and actual values

Full size image
Fig. 19
figure 19

Dataset 4: Actual power consumption share

Full size image
Fig. 20
figure 20

Dataset 4: Predicted power consumption share

Full size image

For the power consumption share, the actual values were noted as 61.8% and 38.2%, respectively, for each appliance, whereas the predictions mention it as 65.9% and 34.1%, respectively.

The above results have been plotted for the FHMM algorithm. Similar analysis was done for the CO algorithm as well. The actual and predicted values were observed and the accuracy and performance of each algorithm for each dataset was determined using the evaluative metrics like NEP and FTE defined in the earlier section. The Table 1 gives values for the FHMM algorithm for all datasets (except dataset 1 as the accuracy obtained was low), whereas the Table 2 gives the same values for CO algorithm.

Table 1 Dataset results for FHMM algorithm
Full size table
Table 2 Dataset results for CO algorithm
Full size table

The value of NEP should be as low as possible, whereas for FTE it should be as high as possible. Thus, accordingly, it can be observed that for dataset 2 exactly identical performance has been observed for both the algorithms, owing to the fact that this dataset consisted of both appliances working at high and comparable powers and in the same mode, thereby making it easier for both algorithms to disaggregate them . The second algorithm with air dryer operated in two modes sees FHMM having superior performance for appliance 1, i.e., the incandescent lamp, whereas for hair dryer the CO performs in a better manner. CO offers better FTE in dataset two albeit with a small difference. In the fourth dataset, the appliance power consumption values were small and as CO is very sensitive to small changes in values, the performance of FHMM was overall better.

Conclusion

It has been observed that model gives best performance when used to disaggregate appliances having comparable values of power consumption, i.e., not having a device with very negligible power consumption as compared to the total consumption. Since while practical implementation, a user will anyway have more concerns over his/her consumption of high-powered appliances, this will not act as a blocker during implementation. Moreover as the complexity increases, i.e., when the number of hidden modes or number of appliances increase algorithms shows slightly decreased accuracy. Through the help of such a system a general idea can be obtained for each appliance and its power consumption and such system can be implemented in smart home deployments. This feedback can prove beneficial in using energy efficiently and thereby taking a step towards sustainability.

Future Scope

The proposed system has a wide scope to it. Such systems can be installed in residential, commercial and educational spaces to effectively monitor and manage the energy consumption. To further enhance the performance more comprehensive datasets pertinent to specific use-cases can be created. Another important use-case can be the Time-of-Day tariff system which is not yet implemented in India but is operational in many parts of the world and can be implemented in India very soon. More information on this tariff system can be read in [14]. The proposed energy management system can be enhanced to include provisions for this system. The algorithm can suggest suitable time for usage of certain appliances to save energy and monetary expenditure simultaneously. Through the installation of such systems, discrepancies in electricity bills can be verified, electricity theft can be curbed and forecasting features can be implemented to forecast energy patterns for future time spans. Thus, the proposed system can be used in quite a many ways to achieve efficient energy consumption leading to financial, environmental and many such benefits.