Keywords

1 Introduction

The introduction of the iPhone about 10 years ago radically changed the market for mobile phones. Featuring a large screen with a touch display, it combined several functions and features that all required separate devices before. Suddenly, a camera, an MP3 player, a telephone, an internet-ready small computer, and many more could be held in one’s hand. It was not the first smartphone to enter the market, but the iPhone was the first successful one, thus kick-starting a market turnover.

Today, more actively sending mobile devices than people can be found in most industrialized countries. The maturity as well as the saturation for these devices can be described as quite high in the majority of these markets. Still, smart devices are mostly used in a private environment—as personal organizer and device for surfing the web, etc. On the other hand, current studies are predicting a high growth potential for such devices in the industrial environment, mostly in automation and factory control: by the year 2025, according to a study from PricewaterhouseCoopers, 75% of all smart devices will be found in the area of industrial automation [1]. This is a major turnover in the market for these devices.

Smart devices are very often perceived as key enablers for company digitalization, as they are rather cheap and provide simple ways to introduce smart capabilities into an industrial environment. In a wider sense, they play a key role as technical assistance systems for the integration of workers in a digitalized factory (see Fig. 1).

Fig. 1
figure 1

The stepwise approach towards Industry 4.0

Full size image

Typically, the first step of a producing company towards Industry 4.0 —a term, that was introduced in 2011 to describe the endeavors of the federal government and the industry to enable German industry to be prepared for the future of production—or Smart Manufacturing lies in the collection and processing of data, thus turning them into information. Smart devices as technical assistance systems depict the next logical step in the usage of this information on the shop floor and in real time. Integrating and connecting all machinery is typically addressed as a third step, as this means higher efforts regarding machine control, interfaces, networks, and many more. Only then, technically more sophisticated topics such as autonomy or decentralized control can be addressed [2].

When looking at the Industrial Internet of Production , smart devices represent the communication and information exchange layer, thus clustering, evaluating, and aggregating all data coming from the different software layers, machines, and sensors, as depicted in Fig. 2 [3]. Today, most of them are commonly used as representation layer, while storage intensive calculations, etc., are run on external servers and computer. With increasing calculation power and storage, more and more data integration, analytics, and modelling can be achieved locally on these devices.

Fig. 2
figure 2

Internet of production

Full size image

Four major fields have been identified for the initial industrial application of smart devices: logistics, assembly instructions, quality control, and maintenance. In logistics, glasses can be used as a hands-free option to display real-time information regarding, e.g., which parts to pick up, where to ship them, etc. First successful applications have already been introduced to the market. In manual assembly, a shift towards more customized or individual products leads to a higher complexity and variety for the workers, which again depicts an interesting field of action for smartglasses: Assembly instructions can be provided locally, with a direct view on the final product. First studies show savings potentials of up to 30% in assembly time when compared to classic, paper-based descriptions [4]. In quality control, smart devices can display evaluation instructions directly to the quality personnel. Through guided processes and direct feedback of pictures or videos, this process again can be digitalized and thus upgraded efficiently.

For maintenance, smart devices and especially glasses depict a promising technical solution to provide instructions and historical information as well as close the feedback loop directly. When it comes to data sources, the maintenance process depends heavily on everything that happened to the specific equipment over time. Starting from first engineering drawings to production information and the service history, the digital twin of the equipment to be maintained plays a major role for the personnel involved, see Fig. 3. The three major distinctions towards the machine or equipment to be maintained are: equipment as planned (design phase), equipment as built (after manufacturing and assembly), and equipment as serviced (history of earlier repairs and services). All of these together constitute the current status of the system to be maintained, while at the same time might be documented in totally different systems [5]. The role of the digital twin as storage tank for all information from the history of a system has thus been stressed extensively in the scientific literature, as, e.g., in [6].

Fig. 3
figure 3

Different databases and lifecycle steps for a system under maintenance

Full size image

The following chapter provides an introduction of the possibilities and challenges for smart devices in maintenance processes. The chapter is structured as follows: after the introduction, an overview over the state of the art is given in the second section, including a definition of terms and descriptions of the individual smart devices as well as market shares of the devices and potentials each smart device offers. In the third section, application examples in maintenance are given, including local data analytics and communication for condition monitoring, remote expert solutions, and process data visualization for process monitoring. The fourth section focuses on limitations and challenges smart devices face, including hardware limitations, user acceptance, information compression on smart devices, and legal aspects. The fifth section briefly summarizes the content of the entire chapter.

In the context of the book “Predictive Maintenance in Dynamic Systems,” the chapter at hand introduces smart devices as mobile user interfaces, which provide possibilities to integrate humans into modern IT infrastructures in manufacturing companies and thus help humans to take on new roles in maintenance processes. The chapter shows that local data analysis and condition monitoring, process monitoring, and remote expert solutions for maintenance are among the benefits that smart devices provide in the field of predictive maintenance.

2 State of the Art

This section provides a general introduction to smart devices. First, important terms and concepts are presented. Different devices are then categorized and characterized. A view on the market introduces the different vendors and operating systems as well as their importance based on market share. Based on the hardware properties, device selection criteria for different applications and boundary conditions are derived.

2.1 Definition of Terms

In general, smart devices are electronic, mobile devices, which provide functionalities via sensor-based information processing and communication. Smart devices can run applications, programmed for various use cases. With cameras, microphones, and other sensors, they connect humans to the environment and the digital world[7].

For presenting information to humans, Milgram has defined a reality–virtuality continuum , which characterizes different levels of integrating virtual content into the real world. Different devices can be classified on this continuum as shown in Fig. 4 [8].

Fig. 4
figure 4

Different degrees of merging virtual and real world

Full size image

In virtual reality , the content is separated from the real world by using head-mounted displays [9]. The headset’s position is tracked and movements are translated into the virtual reality. The user can interact and manipulate the virtual world with position-tracked controllers, which often represent the users’ hands or tools [10]. Assisted reality overlays information, e.g., user manuals or process information with the real world. Mixed reality merges virtual and real world even more than assisted reality. The visual elements are augmented in such a way that they appear to be part of the real world. This requires tracking of the headset’s position, which can be either visually or sensor based (e.g., gyroscopes or accelerometers). Visual tracking is often marker based. Without markers, methods of computer vision are used to identify objects and their position. For augmented and mixed reality applications, a mix of these tracking methods is usually used. Augmented reality has the highest degree of merging real and virtual content. Devices like Microsoft HoloLens project holograms in the viewer’s field of view. Their position can be fixed in real environments, and the user can move freely around the objects.

2.2 Physical Devices/Hardware

There are different types of smart devices, which comply with the definition above. They differ in hardware design but also in functionality. Popular devices used today are smartphones, tablets, smartglasses, and smartwatches. Smartphones and tablets are quite similar and are therefore described together.

2.2.1 Smartphones and Tablets

Smartphones are handheld computer devices, which feature wireless network connectivity (via WLAN and cellular networks) and other wireless technology-like location services (GPS, GLONASS, and Galileo), Bluetooth, and NFC [11, 12]. The telephone function is becoming more and more of a minor matter, in view of the large range of functions provided by smartphones. Smartphone operating systems (i.e., Android and iOS) can run applications programmed for a wide range of industrial use cases. Smartphone CPU and GPU performance has multiplied in recent years. Therefore, they are increasingly capable of running compute-intensive applications. For user interaction, current smartphones generally have large (high-resolution) touchscreens on the front. Most smartphones integrate cameras (front and/or back) and other sensors, like gyroscopes, accelerometers, barometers, proximity sensors, and ambient light sensors [13]. Depending on the CPU and GPU usage, modern smartphones provide between 1 and 2 days of batterylife.

The technological innovations stagnated in recent years and improvements are mainly limited to ever-faster processors and better cameras. It can be concluded that smartphones as device category are commonly used in private sectors and characterized by high technological maturity levels.

From a technological perspective, tablets mainly have the same functionalities as smartphones. Usually, they do not provide telephone features. The touchscreen size typically varies between 7 and 13 in. Regarding sensors and wireless connectivity, they are on a par with smartphones. Tablets usually deliver greater performance and battery life due to their larger size, enabling manufacturers to pack larger batteries.

2.2.2 Smartglasses

Apart from their shape, smartglasses are relatively similar to smartphones. They provide similar functionalities in a different construction. Smartglasses (optical head-mounted displays) project information into the user’s field of vision through mainly three technologies: optical see-through displays, video see-through displays [14], and retinal projection [15]. It can be distinguished between monocular and binocular smartglasses, whereas monocular smartglasses use a single display unit. In addition, smartglasses are characterized by different levels of combining real and virtual world as described above [16].

Most smartglasses also feature a wide variety of sensors like gyroscopes, accelerometers, microphones, and cameras. They can use the sensors to track their position and orientation in space, which is necessary for augmented and mixed reality applications.

Smartglasses in general suffer from short battery life between 1 and 6 h. Extended battery life can be achieved using wired external batteries [15]. The field of view of smartglasses is still rather small, compared to the human eye (20–60° horizontally vs. 180°), which causes a limited area where the virtual information can be placed without turning the head [17]. Compared to smartphones and tablets, smartglasses are characterized by lower technological maturity. This is mainly due to low battery life, less hardware robustness, and poor ergonomics (low resolution, low field of view, and mostly high weight; see sect. 4).

2.2.3 Smartwatches

Smartwatches are another hardware category used in manufacturing environments. In general, smartwatches are wrist-worn devices, featuring computational power, integrated sensors, and can connect to other devices through the internet [18].

Concerning the hardware, smartwatches mostly integrate touchscreens for information display and interaction. They include sensors, which are, e.g., gyroscopes, accelerometers, barometers, light sensors, and heart rate sensors. Bluetooth and WLAN are used for wireless connectivity and GPS and/or GLONASS for localization [19]. In addition, many watches are waterproof or water resistant. Battery life ranges from one to multiple days, depending on usage and processing requirements of the running applications [18].

Smartwatches are mostly designed to function in interaction with a smartphone. They can relay notifications and alarms from the smartphone to the user’s wrist. The smartphone acts as an interface between the watch and external systems such as MES or CAQ-systems [20].

Compared to smartphones and tablets, smartwatches have some key advantages for providing information. Because they are wrist worn, information can be accessed quicker, with less obstruction and “hands-free” [19]. In addition, haptic feedback can be more reliable than acoustic or vibration feedback coming from a smartphone in a person’s pocket (e.g., in loud manufacturing environments) [20].

Disadvantages are the low processing power and the small screen. However, smartwatches are still a rather young device category, and huge improvements have been made in the last few years.

2.3 Market View

For assessing vendors and operating systems, it is useful to differentiate between the different device categories. Especially, the smartglasses vendors have little to no overlap to the well-established leaders on the smartphone and tablet market.

Smartphones and Tablets

The single biggest vendors for smartphones and tablets are Samsung and Apple with 34.2% in the third quarter 2017 on the smartphone, respectively, 40.8% on the tablet market, see Fig. 5 [21, 22]. Regarding operating systems (OS), there are only two with significant market share: Android and iOS. Android is based on Linux and is developed by the Open Handset Alliance, which is led by Google. In the first quarter 2017, it had a market share of 85.0%. During the same period, iOS, which is Apple’s smartphone and tablet OS, had a market share of 14.7% [23]. Most other smartphone and tablet vendors are using Android, which they customize to provide features not available by default.

Fig. 5
figure 5

Smartphone and tablet market share for the third quarter 2017 [21, 22]

Full size image

Smartglasses

The market for smartglasses is much smaller, compared to the smartphone and tablet market. In 2016, only 16 million head-mounted displays were sold [24]. In the same time, 1.5 billion smartphones and 175 million tablets were sold [22, 25]. Vendors in the smartglasses market are new startups mixed with traditional electronic manufacturers. Notable manufacturers are, e.g., Atheer, Epson, Google, Meta, Microsoft, ODG, and Vuzix. There are no reliable sales numbers indicating the market leader. What can be said though is that Android is the leading operating system for smartglasses [15]. One important exception is the Microsoft HoloLens, which runs a version of Windows 10. Microsoft has also presented a platform called “Windows Mixed Reality” which provides a framework for AR apps and hardware.

Smartwatches

Their sales numbers are projected to double from 2017 to 2021, see Table 1. Apple’s WatchOS, Samsung’s Tizen, and Google’s Wear OS are the relevant operating systems. Whereas WatchOS and Tizen are exclusive to Apple and Samsung, respectively, Wear OS can be used by every interested hardware vendor.

Table 1 Sales of wearables as forecasted 2017 (in million units) [24]
Full size table

2.4 Device Selection and Potentials

Different categories and smart devices have been established in the previous section. For their application, it is important to match the expected operating conditions with the specific suitability of the device. Choosing a smart device that does not meet the requirements of a specific use case is one of the biggest threats in application projects with wearable devices—even if the use case per se might have big potentials regarding productivity gains.

In order to prevent such mistakes, a guideline will be introduced to specifically guide stakeholders in the process of device selection as well as potential and effort estimation. Figure 6 shows a general approach for an implementation procedure.

Fig. 6
figure 6

Implementation approach for smart devices

Full size image

The approach follows the phases “Demo,” “Proof of Concept,” “Pilot project,” and “Rollout.” Within the first phase, decision makers should find out, whether the targeted combination of software and hardware works in general. This can easily be found out during a short demo that might only take some minutes, therefore has almost no effort but will result in no measurable benefit except for the fact that afterwards the solution has been falsified or verified under laboratory conditions. When this demo phase has been passed, a proof of concept should follow to analyze if the targeted solution also works in the use case’s boundary conditions. While the demo might have proved to work under laboratory conditions, realistic conditions might result in the opposite. In case, the solutions also pass this phase of the implementation procedure, the assumed business case should be verified within a proof of concept project. That means that the solution should temporarily be implemented to evaluate if its application over weeks or months results in the desired productivity gains. After the business case has been proven, a rollout is the final step to constantly gather positive productivity effects.

The following criteria are important aspects for device selection but do not claim to be exhaustive. In maintenance, many processes are manual and require the worker to use both hands. In general, devices, which allow hands-free operation, are better suited for these processes. A higher degree of automation usually requires less human interaction, which makes them less suited for smart device support. In addition, the duration of processes is an important factor. Because humans need time (~200 ms) to react to visual stimuli [26], longer processes, like maintenance (which normally is not bound to strict cycle times), are better suited for smart device applications. The longest process duration is limited to the device’s battery life. Processes which require many and/or very complex steps to perform are better suited for smart device application. The device can provide a detailed explanation and visualization for every step. This is especially useful for processes, which are not performed regularly, like special repair tasks. It is also possible to support workers on how to use the required tools to perform these tasks.

The environment conditions in which the smart devices will be used are an important factor regarding device selection. First, there is temperature, which the manufacturers only guarantee a specific window of operation for the devices to work properly in. The same applies to humidity. This makes some devices unsuitable for rough environment conditions, e.g., maintenance applications in very humid parts of the world. Excessive noise can impede the use of, e.g., voice commands or acoustic feedback, like alarms. When using smart devices, dirt can also have a negative impact, especially for touchscreen usage. The same applies to vibrations. From an organizational perspective, processes with a lot of necessary documentation are better suited for smart device usage. The documentation can be done right on the device, accessing the company’s databases, providing seamless integration. Also, if additional data is required for performing the required tasks, smart devices can easily provide with the information. Therefore, processes with a lot of additionally necessary information are better suited for smart device usage.

In the early phases of the introduced implementation approach, device selection can be supported by tools using the described criteria. The tools can prevent decision makers from being already stopped in a demo phase of a solution (see Fig. 7).

Fig. 7
figure 7

Evaluation tool for smartglasses selection [27]

Full size image

Besides an overview of different smart devices that are available at the market and their detailed technical specifications, the tools can offer a questionnaire that allows users to systematically describe their use cases. After sending the questionnaire, a knowledge- and experience-based matching algorithm is applied that gives recommendations about preferred hardware for the entered use case. In addition, a rough estimation of implementation efforts is made which depends on several factors, but especially depends on the integration level into existing IT infrastructure, which usually requires customization and integration programming efforts.

Finally, knowledge exchange between users of the platform is offered to complement the systematic guideline approach with human interactions like commenting the guideline results in order to continuously improve the guideline’s underlying heuristics [27].

To give an example of the evaluation, two different application scenarios will be presented. The first example is a maintenance application in an indoor automotive assembly environment (climate controlled). It is an unplanned manual repair and therefore requires guidance on the system’s components and their interaction. The worker is supported by providing manuals for the equipment. The general process duration is high, compared to, e.g., assembly lines with a fixed cycle time. For the second scenario, a repair task of a construction machine will be considered. The maintenance takes place in hot, humid conditions in the field. There is less routine of the mechanic, because he is not specially trained for the task. Figure 8 shows a comparison of the two applications and their suitability for smart device usage derived from the criteria described above.

Fig. 8
figure 8

Machine maintenance in climate-controlled assembly shop (left) compared to maintenance in a construction environment with hot and humid conditions (right)

Full size image

These radar charts can provide a decision-making basis for assessing smart device potentials in maintenance and other industry-related use cases. They can be used for a preselection of suitable processes. However, the specific suitability of a process must be examined in detail, as there are always new devices on the market and this classification can only provide an orientation.

3 Application Examples in Maintenance

This section provides examples for different applications for the use of smart devices in production system maintenance. First, an example for local data analysis and communication for condition monitoring is given. This includes the presentation of a real-time worker information system as the core of solutions for worker assistance in condition monitoring tasks. Second, the application of smart devices for remote expert solutions is presented. Remote expert solutions enable maintenance engineers to communicate with machine experts via video live stream, to collaborate on fixing problems. Finally, an example is presented which shows how smart devices have changed the way information is displayed to workers in case of process data visualization for process monitoring.

Critical to the successful use of smart devices in the industrial environment is the integration and linking into the relevant system landscape. Instead of a stand-alone solution, planning systems such as enterprise resource planning (ERP), manufacturing execution systems (MES), computer-aided quality (CAQ), and the machine itself exchange data. This is the prerequisite to use smart devices as an integral part in the different applications. Common use cases for smart devices in maintenance applications are found in the area of condition monitoring, remote expert solutions as well as process monitoring [28, 29].

Condition monitoring describes the process of recording machine data for checking the current machine status. This allows the identification of irregularities or errors in the system. Moreover, a condition monitoring system can make predictions about the future system behavior by means of a combination of the analysis of the current system state and historical data. Thus, the monitoring system can detect faulty states early or plan maintenance activities and intervals. Remote expert solutions help to accelerate and improve the maintenance process. Those systems enable engineers to communicate with experts via live streams. Being able to look into the machine while simultaneously displaying all the relevant information on the spot also makes it possible to predict the future state of the machine. Finally, systems connected to other systems such as ERP provide an overview of the state of the entire system on the shop floor. The combination of these three use cases enables companies to use smart devices on a large scale for predictive maintenance.

3.1 Local Data Analysis and Communication for Condition Monitoring

Condition monitoring enables the maintenance engineer to identify the current system status and allows the derivation of future recommendations for action. Prerequisite is the use of real-time machine data in order to interpret and analyze it and to derive actions subsequently. This is useful, e.g., for troubleshooting, maintenance, or predicting future system states. For applications, providing real-time information of the machine to the worker, a direct information exchange between the machine and smart devices is necessary. Information here is often time critical and requires short-term action and intervention options (e.g., in the event of sudden malfunctions or tool changes). Therefore, in the following a real-time worker information system is presented, see Fig. 9. It enables direct communication between smart devices (e.g., tablets and smartglasses) and the machine control. The system supports the machine operator in planned and unplanned maintenance activities.

Fig. 9
figure 9

Communication between machine and device

Full size image

As stated, data exchange between mobile terminal and machine control is a prerequisite for real-time systems. Direct socket communication enables the data exchange and eliminates a separate arithmetic unit. Smartphones and tablets are used as hardware. Communication with the machine control system takes place, for example, via OPC-UA . For this purpose, the control technologic PLC interface (programmable logic controller) is implemented device-specific. By this, transmission is enabled for many different devices. Other (manufacturer-specific) protocols can also be implemented in the system. A communication protocol ultimately defines the rules and syntax of how data of specific inputs and outputs can be got or set (read and write functions). By constantly retrieving actual data from the controller, real-time information such as machine status or tool condition is transferred and further processed in the information system. The data transport is done wirelessly, e.g., over Wi-Fi.

The information system forms the core of solutions for worker assistance in condition monitoring tasks. In addition to the real-time machine data, the information system also provides a library with specific video manuals and documentation, e.g., manuals in PDF file format. Particularly in the field of “training,” users can use video manuals, for example how to replace a tool during maintenance. This is displayed directly on the tablet or the glasses conveniently and location-independent. It gives the worker the information he or she needs without being dependent on paper-based instructions or PC terminals. When using smartglasses, the operator can open these manuals parallel to her or his work, since the glasses provide the information via an integrated, semitransparent display.

The direct communication between the device and machine enables real-time data analysis within the information system. For monitoring reasons, the operator can see the latest machine and order information, such as progress, remaining time, machine status, or overall system effectiveness (OEE ). This promises real-time transparency for the employee, because the controller transfers raw data continuously to the smart device. The smart devices then further process and visualize the data locally, condense it into key performance indicators (KPI ), and perform automatic updates.

In case of unexpected disturbances, interruptions, or errors, the user automatically receives information, e.g., as a pop-up. Predefined error libraries and codes give the machine operator direct messages via the smart device. Examples of such error and fault information are opened safety devices such as doors or necessary tool changes in the mechanical machining of components. The interpretation of the raw data for a possible incident, the derivation of instructions for the user, and their communication also take place directly in the information system on the smart device. A manual error analysis by error retrieval at the machine terminal is obsolete. This reduces the reaction times in case of unexpected disturbances and can increase the OEE. So, all the required information, such as manuals or repair information for the specific error, are available directly during maintenance.

The described system for machine-related operator support represents a tangible extension of the classical interface between machine control and worker. In the age of Industry 4.0, the system enables direct retrieval, local processing of data as well as the visualization of the correspondingly condensed information via smart devices. In this way, the system supplements the classical human–machine interface (HMI ) on the machine terminal by using a flexible and real-time-capable information system directly at the shop floor.

Current challenges and research requirements lie in particular in the integration of different control systems and corresponding communication protocols. In addition to actual data exchange strategies, this also includes the interpretation of the control-specific raw data and the subsequent information compression. In addition, current systems are often limited to communication between a device and a single machine control. In industrial use cases, direct communication from one device to multiple controllers is desirable. For this, it is necessary to define meaningful access routines. For example, a pairing of device and machine could be done via a scan of a machine ID (e.g., QR code ) or the automatic recognition of surrounding production machines via Bluetooth or NFC . In the field of predictive maintenance in particular, the predictive models must be further improved and generalized. These improved and generalized models can provide more accurate predictions for wider use cases.

3.2 Remote Expert Solutions

In manufacturing companies, machine downtimes can considerably influence the productivity and result in high costs. New studies show that 82% of surveyed companies have been confronted with unplanned machine failures within the last few years. Most typical reasons were hardware- and software-based malfunctions followed by human errors. On average, machine failures lasted for 4 h and involved costs of two million dollars. Almost 50% of companies believe that downtimes can be decreased when machines are able to request for help by themselves and when they use cloud-based functions to support failure diagnostics [30].

To accelerate machine failure handlings supported by machine experts, mobile devices with remote expert system s can be applied. Those systems enable maintenance engineers to communicate with machine experts via internet connection and provide a live video stream showing the failed machine. Thus, the machine expert can immediately support the troubleshooting and provide professional instructions for a proper failure handling. For maintenance engineers, smartglasses serve as a practical platform for remote expert systems. Their mobility allows them to stay on the shop floor and to use the integrated camera module for sharing their own perspective of the failed machine. The integrated headset enables speech-based communication while being hands-free. Since the machine expert can be consulted immediately without regard to the current location, travel costs can be saved. The machine expert can use a computer or a tablet PC. Similar to classical applications with video conference functionalities, remote expert systems have to be installed on the devices of both conversation partners. As shown in Fig. 10, those systems provide different functions and options depending on the specific device and role of the user.

Fig. 10
figure 10

Connection of maintenance engineer and machine expert via remote expert system

Full size image

Since most available smartglasses on the market use popular operating systems such as Android, the access to the camera module is standardized. Moreover, remote expert systems connect to a local wireless router with the device-integrated wireless module. The external machine expert, who receives the camera image, can easily guide the maintenance engineer through the troubleshooting and failure handling processes. To support these processes, remote expert systems provide different useful functionalities. For example, the machine expert can add and remove different elements such as symbols, textboxes, images, or checklists to the camera image via drag and drop. Due to automatic synchronization with the smartglasses of the maintenance engineer, those elements will also show up on their screens. This way, the machine expert is able to guide the maintenance engineer to the right spot of the machine and to write comments that can include information about the next steps.

Remote expert systems allow immediate failure handling guided by machine experts and therefore, downtimes and resulting costs can be decreased. For machine suppliers, those systems provide new opportunities to create profitable business models. Up to now, 81% of companies state that aftersales services do not significantly contribute to the profitability due to limited capacities, which result into long reaction times [31]. The ad hoc connection of service employees and customers via a remote expert system can be considered as one feasible approach for an efficient use of personnel capacities.

3.3 Process Data Visualization for Process Monitoring

One essential target of Industry 4.0 is to provide employees with the information they need at the right time to carry out their processes efficiently and to fulfill current quality requirements. This can be realized with modern process data visualization systems that are interconnected with the production’s software landscape . Available systems providing CAD, CAM, ERP, MES, MDA, and CAQ functionalities are used as different data sources [4]. Instead of classical computer terminals, smart devices are used, providing a high grade of mobility. With their application, information does not need to be actively requested at a fixed location on the shop floor. Smart devices can display information at any time when it is needed without considering the employee’s current location. Audio or vibration signals are typical instruments to gain the employee’s attention. Figure 11 illustrates how process visualization systems combine different data sources of the production’s software environment. Employees who are equipped with such a system can be provided with various information such as technical product specifications, process and quality data as well as machine condition data. The final choice of information that is visualized on the screen is adapted to the individual needs and functions of the specific system user. For example, machine operators receive information about single processes, while maintenance engineers mainly receive condition data of machines that are under their responsibility.

Fig. 11
figure 11

Process monitoring through mobile devices

Full size image

In case of an occurring machine failure, maintenance engineers can use the integrated menu structure to get access to digital machine handbooks (machine as planned ), to specific information about machine components (machine as built ), and to machine-related maintenance histories (machine as serviced ). Such a high availability of information promotes time-efficient troubleshooting and failure handling. With the help of data processing algorithms, various key performance indicators such as the overall equipment effectiveness (OEE) of machines can be calculated and displayed. Predictive algorithms inform machine operators and maintenance engineers about future quality outcomes and machine states. In case of predicted qualities that are outside the tolerances, process chains and parameters can be adapted. Maintenance engineers can be informed about possible machine failures before their occurrence. To provide an impression of modern process monitoring applications, an example is given in the following. Figure 12 shows a system that visualizes a precision glass molding process.

Fig. 12
figure 12

Process monitoring during precision glass molding [32]

Full size image

In contrast to grinding and polishing processes for production of optical lenses, precision glass molding describes a replicative molding process. With short cycle times and its ability for production of complex lens geometries with stable qualities, this technology depicts an optimal approach for mass production. In addition, due to the fact that every lens geometry requires a specific and expensive mold, precision glass molding can mostly be found in productions with high output rates of the same products. During the molding process, force and temperature sensors deliver data from different positions within the mold. Since the molding of the glass blank cannot be observed visually, the data acquired during the process is combined with a simulation that visualizes the molding based on a three-dimensional model. Thus, besides the monitoring of the current process, this system supports gaining new process knowledge because it visualizes the correlations between the geometrical specifications of the mold and the resulting forming, forces, and temperatures. This knowledge enables the optimization of process parameters and mold designs. From a maintenance perspective, the acquired data can help to derive the current and future wear state of the tool. This can support maintenance engineers to initiate tool repairs or changes before the output quality is considerably influenced [32, 33].

Process data visualization is an important component of Industry 4.0, which increases process and machine state transparency and therefore, promotes process knowledge building and an increase of experience regarding machine behaviors. From a short-term perspective, visible process data supports operators and maintenance engineers to fulfill predefined quality requirements and to decrease machine downtimes. The long-term application allows continuous optimizations of process parameters and maintenance strategies, e.g., event-based maintenance, that enables preventions of discard, rework, and machine downtimes.

4 Limitations and Challenges

This chapter provides and discusses current limitations and challenges related to the use of smart devices for industrial maintenance applications . First, hardware limitations are discussed. These hardware limitations include human-related limitations like wearing comfort, application-related limitations, which are set by limited accuracies of sensors and cameras, and environment-related limitations like high temperature or dust. The second subsection focusses on user acceptance and emphasizes that an appropriate and practical system design is required to achieve general user acceptance. After that, information compression, which is necessary due to the compact design and reduced possibilities of user interaction on smart devices, is discussed. Finally, legal aspects are considered by describing legislative requirements originating from EU directions referring to safety and health requirements for the workplace, work equipment, and data protection .

4.1 Hardware Limitations

In addition to the variety of possibilities, using smartglasses in industrial environments also leads to certain limitations and challenges. These challenges can be divided into three categories:

  • Human-related limitations

  • Application-related limitations

  • Environment-related limitations

Human-related limitations, for instance, include wearing comfort. Smartglasses should not affect the user’s comfort, even if the user wears them over a longer period. Weight of the smartglasses as well as the glasses’ fit to the user’s head are crucial factors in terms of wearing comfort and will also contribute to the user’s acceptance for the smartglasses.

Application-related limitations are limits that result from the hardware sets in terms of accuracy of sensors and cameras. Since sensors have a defined range of measurement inaccuracy, not all glasses are suited for all applications. Smartglasses can only be utilized, if their sensors and cameras fulfill the requirements, which the application defines. For example, if smartglasses are used for technical service through remote expert software, the smartglasses’ camera has to be capable of recording high-resolution videos, even when the caller is in motion, so that the receiver of the call can also identify small details (e.g., a tool identification plate) in the streamed video. Another application-related limitation is the battery. The battery life has to be suitable for the applications. As bigger and heavier smartglasses tend to have a bigger battery, the trade-off between comfort and battery life has to be evaluated for every application case.

Environment-related limitations are limits that result from environmental influences on smartglasses. Environmental influences include water, dust, temperature (hot and cold), and atmospheric corrosion. The IP Code provides information on the degree of the device’s solid particle protection and liquid ingress protection. Dust and water can damage smartglasses, if they are not selected according to their IP Code. High or low temperatures can also damage smartglasses. As smartphones tend to have a less efficient battery during very low temperatures, the same applies to smartglasses, since they use the same type of lithium-ion battery. Many smartglasses are approved for temperatures near to room temperature. The temperatures of many production facilities exceed these temperature values, though. Another environmental aspect is explosion protection. Electrical devices can possibly become a source of ignition and only the use of certified, intrinsically safe devices is allowed in explosion-hazardous areas. As many smartglasses lack an approval for explosion-hazardous areas, their usage within these areas is strictly limited.

4.2 User Acceptance

Nowadays, working population represents a cross section of different generations and corresponding backgrounds regarding the use of and the familiarity with digital solutions in their daily work. Future digitalized production systems—Smart Factories —require workers to operate with and within the world of data. Here, smart devices represent important interfaces between worker and interconnected production machines and software systems. The worker’s specific technical affinity is strongly related to his or her generation and level of training. However, as the smartphone shows, the professional or generation background does not prevent a widespread use of smart devices across almost all sections of population in the private sector.

The industrial sector faces similar characteristics for its future. However, this requires a broad user acceptance by appropriate and practical system design. General requirements for the system design and its interfaces can be derived from ISO 9241—Ergonomics of Human-System Interaction. This series of standards defines boundary conditions and design rules/guidelines for physical aspects such as workplace design and posture. Furthermore, major topics of those standards are related to software ergonomics. Here, aspects such as dialogue management, user interfaces, or interactive system features are considered [34].

Besides generalized guidelines such as ISO 9241, specific end-user requirements need to be taken into account when designing smart device applications for maintenance purposes. A participative approach represents a key success factor. Therefore, maintenance personnel and experts should be actively included during the system development by structured gathering and incorporating their requirements and feedbacks. Figure 13 provides a recommendation of methods to systematically include end-user requirements and feedbacks during different phases of the development, implementation, and rollout of smart maintenance systems-based mobile devices.

Fig. 13
figure 13

Recommended methods and approaches towards participative system design

Full size image

Besides the general system design, the user acceptance regarding the implementation of new systems correlates with its level of adaption during the rollout period. The introduction of digital applications in operational processes such as maintenance activities is always related to a change to the employee’s way of working. Therefore, change management is a crucial and central aspect for the rollout of smart device applications in maintenance. Experienced maintenance worker might feel left behind or less valued in case their established and proven procedures are replaced—respectively adapted—by new technologies such as interactive failure documentation using augmented reality or guided repair procedures via remote expert solutions. This aspect can be described according to the worker-specific perception of its own competence. A generalized development of this perception during the implementation of changes is visualized in Fig. 14.

Fig. 14
figure 14

Perception of competences during the rollout of changes [31]

Full size image

It can be seen that the worker’s reaction to those changes develops from an initial shock, refusal, stepwise acceptance of the new technologies and a related perception of decreased competence towards a learning curve characterized by improved knowledge and integration. According to this model, the process results in a perception of increased competence [35]. However, it needs to be pointed out that it is in the nature of things that employees might be skeptical of changes and corresponding implications on their daily work and perception of own competences. Thus, it is all the more important to include the end-user in the development of smart device applications for maintenance following a participative approach as outlined before.

4.3 Information Compression on Smart Devices

Smart devices provide several advantages such as capability for mobile applications. However, due to their compact design and reduced possibilities for user inputs and interactions, the provision of information is not comparable to classic methods such as printed documentation or PC terminals. Documents such as drawings, quality plans or working instructions (e.g., pdfs) are generally provided as extensive information containing all details. Consequently, workers might be overstrained, as documentation needs to be reviewed in terms of relevant information for the very specific task. This aspect represents a major potential for improvement when using smart devices, as the provided content is limited to the relevant information. The direct connection of smart device maintenance applications to superior software and planning systems (e.g., for providing relevant maintenance instructions and checklists) allows to query the very specific and didactically prepared information, rather than entire manuals or process documentation. However, this requires information compression on the chosen device.

Depending on the device technology, display sizes, resolution as well as general features such as audio recording, playback, or vibration capability vary. A current research focus is therefore related to the device- and user-dependent compression of information for industrial applications such as maintenance support via digital repair plans or ad hoc documentation via mobile devices. The general scheme of this information compression is outlined in Fig. 15.

Fig. 15
figure 15

Scheme for information compression on different devices

Full size image

It is necessary to identify the trade-off between loss of information and mental overload through unnecessary or redundant information. While detailed process descriptions or system plans can be provided and interpreted via tablets through intuitive operations such as scrolling or zooming, the use of smartglasses or smartwatches for the same information could lead to confusion rather than support. Consequently, maintenance procedure might even be delayed and more complicated. Here, short and concise requests for subtasks as text instructions, videos, or schematic sketches could be used instead of extensive documentation.

4.4 Legal Aspects

Further challenges for the implementation of smart devices for maintenance activities are also related to legal conditions. Some of those legislative requirements originating from EU directions shall be presented at this stage. Those directions particularly refer to safety and health requirements for the workplace and work equipment as well as data protection.

For countries of the European Union, 89/654/EEC defines minimum safety and health requirements for the workplace [36]. This document also defines specific requirements for mobile virtual display units. Therefore, this regulation is also applicable for maintenance activities supported by smartglasses or tablets. As an example, this document requires a temporally limited use of those mobile display units, except there are no other technical solutions available for the specific tasks. However, in case they are required and need to be used to execute specific maintenance operations as work equipment, they are covered by 2009/104/EC, which defines minimum safety and health requirements for the use of work equipment [37]. According to this regulation, the employer is required to assess the functional safety of those mobile devices on a regular basis to avoid any hazards for the workers. Defect devices, e.g., indicated by hotter battery systems, shall not be used anymore as they represent potential hazards. In general, employers are required to measure and to achieve improvements in the safety and health of workers at work. For the European Union, this aspect is regulated by 89/391/EEC [38]. The employer has to assess the working conditions for maintenance activities and to introduce countermeasures in case of any hazards. Therefore, personal protective equipment (PPE) is mandatory—especially in the field of maintenance. However, in case smart devices are used for maintenance purposes, the compatibility of smart devices and PPE needs to be guaranteed. Due to this reason, there are different smartglasses on the market that can be easily combined with PPE such as helmets.

In the age of Industry 4.0 and fully connected digitalized manufacturing systems, data protection plays a crucial role. Especially, from employee and work council perspective, the protection of individuals with regard to the processing of personal data is of major interest. For the European Union, this aspect is regulated in 95/46/EC, respectively, 2016/679 regulation. Personal data can clearly be related to a specific person, respectively, worker [39, 40]. Personally identifiable data acquisition, data processing, and data use can only be considered as legal if the affected person authorizes it or if it is legally required or allowed. The acquisition of this kind of data—e.g., for assessing the employee’s performance or working speed by use of mobile devices—is critical and should be punished according to this regulation. Even the recording of other (uninvolved) persons when using smartglasses during maintenance documentation requires the explicit permission of the specific person. Therefore, measures need to be executed to guarantee anonymity of data and a limitation of recorded data that is coherent with legislative boundary requirements when introducing smart devices for maintenance purposes.

It can be concluded that several legislative requirements are formulated for work equipment, working conditions, and data acquisition and processing to guarantee safety standards as well as sufficient data protection. Some of those regulations are applicable to the use of smart devices for maintenance applications. To be coherent with those regulations, measures and data acquisition and processing strategies need to be implemented to ensure a lawful implementation of smart devices.

5 Summary

This chapter described the role of smart devices in production system maintenance. With their specific features such as mobility, interconnectivity, and processing performance, they serve as technical assistance systems and promote the integration of workers into the digital factory.

In a definition of terms and a subsequent market view, different types and technologies of smartphones, tablets, smartglasses, and smartwatches were introduced. Besides technical differences, leading providers of different operating systems and hardware were named.

To underline the key role of smart devices in terms of modern maintenance, three application examples were described. The first one described the interface between smart devices and machines through direct socket communication. Specific benefits of local data analysis and condition monitoring were pointed out. Occurring machine downtimes can be decreased with the application of remote expert system that connects maintenance engineers with machine experts through the internet by using smart devices. Their advantages and functionalities were described in the second example. The last example introduced data visualization systems for process monitoring on smart devices as mobile solutions to provide maintenance engineers with machine information without regarding their current location on the shop floor.

In the last part of this chapter, limitations and challenges that are connected to the integration of smart devices into the shop floor were discussed. For each specific application, a different technology can be seen as most suitable. Technical differences and limitations of smart devices are defined by their specific operating systems, technical interfaces, calculation of power, storage, and battery capacities, and by their environmental working conditions. Moreover, the application of smart devices and software can be restricted regarding user acceptance, information compression, and legal aspects.

Considering all existing limitations and requirements, smart devices are forward-looking technologies that promote location-independent and need-based information exchanges on the shop floor. For maintenance, those advantages are crucial to enable quick reactions to machine failures and to unacceptable changes of machine conditions.