Keywords

1 Introduction

Manufacturers are modifying their business and operations model to move towards environmental sustainability . The reasons for this trend include stricter regulations (due to climate change issues), a need to increase competitive advantage, a need to protect and strengthen brand/reputation and financial incentives from grants, subsidies and cost savings [1]. For instance, increasing operational efficiency and the need to protect and strengthen their brand and reputation are some of General Motors’ motivations to adopt an aggressive sustainability approach [2].

Monitoring and controlling energy consumption of electrical appliances are important processes in energy management systems (EMS) . Such systems are widely utilized in buildings, homes and factories to achieve optimal energy efficiency [3,4,5]. On top of this, these systems provide users a breakdown of their energy usage so that they are able to better respond by controlling the energy consumption of relevant appliances with high-energy usage. Likewise, monitoring energy consumption of equipment in a manufacturing workcell is part of an important process of energy consumption control .

Sophisticated and advanced energy sensors have made it possible to effectively monitor energy consumption of equipment in the workcell via a smart device in real time. For instance, Moreno et al. [6] developed an IoT-based, smart building energy management system (BEMS) , using a combination of RFID and IR sensors to monitor energy consumption. Their case studies illustrated that energy savings of about 23% can be achieved when this system is utilized.

However, it is worthy to note that in a manufacturing setting, stakeholders like new workers, eco-consultants and/or authorities are unfamiliar with the equipment terms, thus making it challenging for them to visualize the energy consumption of each equipment in the workcell from the smart device. These challenges increase further when it comes to complex manufacturing operations like the production of hybrid medical devices (e.g. CNT-PDMS (carbon nanotubes-polydimethylsiloxane ) based on artificial trachea prosthesis [7,8,9]), which make up of a combination of both synthetic and biological components. Such manufacturing processes involve many uncommon equipment which terms are usually unknown to these stakeholders, thus making it difficult for them to consult or regulate manufacturers on their energy control efficiencies.

AR technology’s potential use for monitoring purposes has been investigated by various researchers. Zollmann et al. [10] introduced an approach to utilize AR technology for on-site construction site monitoring . Their field tests results confirmed great potential for the proposed methodology in other industries. In another work by Fard et al. [11], the authors proposed a four-dimensional AR model for automated construction progress monitoring. Their preliminary results proved potential benefits of utilizing this technology for progress monitoring in the construction industry. On a side note, AR technology has already started being utilized for assembly training and inspection purposes in some manufacturing companies. For instance, in a Harvard Business Review case study by Abraham and Annunziata [12], it has been reported that Boeing is utilizing AR technology for wiring harness assembly, and this led to an increase in productivity by 25%. As such, an AR-based energy monitoring concept will add value with minimum cost because of its easy integration in the inbuilt system together with these features. Moreover, either most of these proposed AR-based methods have a prefixed data programmed or they require manual updating of data in the system. Hence, there is also a need to consider an AR-based approach that allows automatic, real-time updating of data in the system.

In order to aid this group of stakeholders in the monitoring of energy consumption process, we propose a novel, augmented reality (AR) and Internet of Things (IoT) -based energy monitoring conceptual design. It aims to not only give users an option to have an overall view of the energy consumption in the workcell but also to allow users to visualize energy consumption of each equipment in the actual field. With the help of AR technology, one can immediately pinpoint equipment with constant high consumption rates while walking in real environments and give better advice to control energy consumption by exploring ways to reduce their energy usage. With IoT technology, automatic, real-time updating of data in the system is possible. This paper highlights the potential of the proposed energy monitoring conceptual design in aiding users to better visualize the energy consumption patterns of individual equipment in the workcell.

2 AR Energy Monitoring Architecture

Figure 6.1 shows the AR energy monitoring architecture which consists of three main components: (a) the equipment set, (b) the processor and (c) the AR display device. Figure 6.2 is the system flowchart.

Fig. 6.1
figure 1

AR energy monitoring architecture. Energy data transfer is indicated by green line; visual detection of marker(s) is represented by blue line

Full size image
Fig. 6.2
figure 2

AR energy monitoring system flowchart

Full size image

The equipment set comprises of all the equipment that require energy usage. Each equipment will be tagged with a unique marker for identification purposes [13]. On top of this, each equipment is attached with a wireless energy sensor to enable real-time measurements of energy consumption and convenient energy data transfer [14]; this energy data will be wirelessly transferred to the processor. Next, the processor will collate the energy data from all equipment, and process and convert them to display mode for AR (e.g. graphical and chart forms that will be augmented on the AR display device). The reason for having this conversion process is to provide better visualization for users at a later stage, thus allowing them to easily identify equipment with high-energy usage and to compare energy usage among all equipment [15]. Subsequently, the processed energy data (in display mode) is wirelessly transferred to the AR display device.

The AR display device, which is usually a lightweight, and a portable device (e.g. smartphones, tablets, head-mounted display (HMD), AR glasses) [16,17,18] are utilized to present the processed, well-packaged energy data to the user while he/she carries the device around in the real workcell environment. When the device’s camera detects the marker tagged on the equipment, the AR display of the energy data will appear to the user on the device [19].

In summary, the proposed AR energy monitoring architecture aims to allow users to better visualize energy consumption levels of each equipment in the actual field. Using markers to identify each equipment and using a processor to produce well-packaged energy data for each equipment, it is technically possible to adopt this approach with the aid of an AR display device.

3 Conceptual Illustrations

In order to illustrate the concept of the proposed AR energy monitoring architecture, we utilized a small-scale workcell with various equipment.

3.1 Description of the Small-Scale Workcell

The small-scale workcell is located in an equipment room in our laboratory. We selected three various equipment that consume different amounts of energy. Equipment 1 is a Panasonic refrigerator-freezer, model NR-BN221SNSG, that is used to store chemical agents. Equipment 2 is an Akarui Digi 38L dry cabinet that is used to store items that are sensitive to humidity and dust. Equipment 3 is a regular Dell personal computer. All these equipment are switched on 24/7 and tagged with different marker designs. The small-scale workcell and the respective marker designs are illustrated in Fig. 6.3.

Fig. 6.3
figure 3

Small-scale workcell with various equipment and their respective marker designs

Full size image

3.2 AR Energy Monitoring

We used an Android tablet and an MS Windows laptop for the AR display device and the processor, respectively. An application using the software Unity3D and Vuforia 3D is built to enable marker detection and to augment the virtual processed energy data on the tablet.

The virtual processed energy data is illustrated in Fig. 6.4, and the AR energy monitoring process is illustrated in Figs. 6.5 and 6.6. In Fig. 6.4, the energy data, which are collected from the sensors, are converted by the processor to display mode for AR. Figure 6.4a–c represent the energy consumption pattern graphs for equipment 1, 2 and 3, respectively. Figure 6.4d represents the overview comparison of energy consumptions among the three equipment. These processed energy data will be presented to the user appropriately via the AR display device when he/she aligns the device’s camera vision with the respective tagged marker as clearly illustrated in Fig. 6.5.

Fig. 6.4
figure 4

Virtual processed energy data : energy consumption patterns for (a) equipment 1, (b) equipment 2, (c) equipment 3 and (d) overview comparison chart

Full size image
Fig. 6.5
figure 5

Conducted AR energy monitoring process for (a) equipment 1, (b) equipment 2, (c) equipment 3 and (d) activation of overview comparison chart

Full size image
Fig. 6.6
figure 6

Overview of user’s monitoring experienc e during experiment

Full size image

The AR energy monitoring process requires two steps: (1) finding the tagged marker on the equipment of interest and (2) aligning the device’s camera vision with the marker. Figure 6.5a–c illustrates the monitoring process for equipment 1, 2 and 3, respectively, step 1 on the left and step 2 on the right side of the figure. Figure 6.5d, on the other hand, illustrates how the user can activate the overview comparison chart by ‘pressing’ on the virtual ‘overview’ button. Figure 6.6 illustrates an overview of how the user can utilize the system to effectively monitor energy consumption levels of individual equipment in the workcell.

4 Discussions

From above, we demonstrated that the proposed AR energy monitoring architecture is feasible and effective. Firstly, from Figs. 6.4 and 6.5, we can observe that the system provides better visualization to the user via the virtual processed energy data; the graphical and chart data formats make it easier for the user to analyse data. Secondly, the system provides real-time and accurate energy data to the user. As such, the user does not need to estimate energy consumptions for each equipment based on electricity bills and/or equipment power ratings. Thirdly, the system allows the user to conduct on-site monitoring. As such, the user is able to immediately pinpoint equipment with constant high consumption rates while walking in real environments. The system also provides convenience to the user as it involves wireless technology and lightweight and portable devices. The involvement of wires and bulky and heavy devices will hinder the monitoring process in the workcell.

Nevertheless, the proposed system has several limitations . Firstly, the additional costs of wireless energy sensors and AR display devices may deter manufacturers to adopt this system [20,21,22]. Nevertheless, we believe that with rapid technological advancements and increasing production efficiency, these costs will decrease significantly over time. Secondly, the system heavily relies on markers. The disadvantages of using markers for identification purposes are having unstable tracking and limited tracking range; its tracking system is based on a few features; and the range of camera vision from which the markers are visible is limited [13]. Thirdly, the illustration example utilizes a tablet as the AR display device. Although the tablet is lightweight and portable, it still poses a hassle for the user as it has to be carried around on hand in the workcell [21, 22]. Lastly, the amount of virtual processed energy data (as shown in the illustration example) that is provided to the user may not be sufficient enough to reach an accurate conclusion on energy usage. This is because energy consumption measurements can be done in varying units (e.g. kWh, joules, kgoe) and in varying time periods (e.g. annually, monthly, daily) [23,24,25]. Including these variations as part of the provided virtual processed energy data may enable better judgement of the energy consumptions by the user. Table 6.1 summarizes the advantages and limitations of the proposed AR energy monitoring architecture.

Table 6.1 Advantages and limitations of proposed AR energy monitoring architecture
Full size table

5 Conclusion

The increasing need for manufacturers to move towards environmental sustainability calls for an urgent need to improve energy management systems that include the monitoring and control of energy consumptions. Although smart systems have the potential to effectively manage energy levels, it can be foreseen that with increasing competition and costs, manufacturers will face with a dilemma between maintaining energy consumption levels and increasing output. As such, stakeholders like eco-consultants and authorities play an important role to constantly keep these manufacturers in check.

As this group of stakeholders can be unfamiliar with the equipment terms in a complex manufacturing setting, it will be beneficial to consider an energy monitoring system that will effectively aid users in visualizing the energy consumption patterns of individual equipment in the workcell. In this paper, we illustrated the potential of the proposed AR energy monitoring concept that enables users to immediately pinpoint those equipment with constant high consumption rates while walking in real environments. Hence, this also allows them to give better advice to control energy consumption by exploring ways to selectively reduce energy usage.

Further studies and development are required for the proposed system. Hardware aids such as head-mounted display (HMD) and AR glasses could be used to enhance effectiveness and user experience. Wearing headgears, as compared to carrying portable smart devices around, may be a more convenient and visual solution. We also intend to improve the presentation of the virtual processed energy data by including more data in varying units and time periods. Marker-less tracking system will also be considered to reduce the reliance on markers. Testing the fully developed AR energy monitoring system on human subjects and comparing the user experience between utilizing the proposed AR energy monitoring system and the conventional method of analysing data from a device in an enclosed space (e.g. office, laboratories) are important.

The proposed system will enable users to better visualize the energy consumption patterns of equipments within a workcell. The collected data could be analysed over time. It is a platform for us to research and develop visual cues for effective monitoring and control of equipment energy consumption. Computational intelligence software is used to generate the appropriate visual cues.