Keywords

1 Introduction

Industry 4.0 is the fourth stage of the industrial revolution, which plays a key role in the development of the third industrial revolution. It is a digital transformation process from automated factories to intelligent factories. Industry 4.0 was referred to as an integration of information and communication technology in industrial manufacturing when it was first started. Industry 4.0 is the ongoing automation of traditional manufacturing processes, enabling industries to share data and collaborate with various elements involved in the supply chain. The various technology elements associated are smart machines, IIoT, augmented reality, machine vision, smart sensors, big data, robotics, digital twin, additive manufacturing, and predictive maintenance along with cyber-physical systems which play a key role in digitally connected factories. Smart manufacturing is concerned with reducing all the time elements such as introduction, growth, maturity, and decline stages involved in a product life cycle. Industrial robots are one of the applications of Industry 4.0 which play a key role in this digital transformation of factories whose positioning and operations must be planned appropriately [1].

The digital technology used in this project is augmented reality (AR) which is used to train users of industries adopting smart manufacturing. AR has a large market value if implemented in many industries and organizations. Augmented reality converges the digital and physical world giving a better edge to a range of applications. The physical world is three-dimensional and the data of this would be embedded on 2D screens and pages, and this difference between the physical and digital world prevents human capabilities for better usage of data. This disadvantage can be bridged by AR. AR changes the user’s perception of the real world or uses certain widgets like images, labels, and tags to overlay virtual data on the physical world. AR creates business value by becoming part of a product through devices like AR display heads-up, smart glasses, wearables, and also its experience can be viewed on mobile phones. AR reduces the cognitive load on humans by digital information overlay. The key capabilities of AR are visualize, instruct and guidance, and interaction with users. Visualize refers to the visualization of internal features of a real complex system, instruct and guide describe step-by-step guidance on tasks like product design and assembly operations, and interaction feature enables the activation of a virtual user interface by replacing physical controls like buttons by using smart glasses and headsets.

The Internet of Things (IoT) also plays a key role in providing real-time data for fault diagnosis and other applications. This has embarked new IoT-powered augmented reality applications in many ways like system replacement, training, etc. Fault diagnosis is one of the major applications of the combined technology of AR and IoT [2]. Robotic welding is one of the use cases of fault diagnosis. The welding mechanism plays a key role in the development of hardware systems. With the continuous development process in the industries, manual welding has been eliminated since it is more difficult and also impacts the health of the system. This may lead to a delay in product production [3]. Apart from this, other researchers have identified various applications of AR in maintenance. Diagnosis, training [4], performance measurement [5] are some of the studies which are currently ongoing in the industries.

This paper gives a detailed explanation of how critical systems of robotic welding cells are analyzed, after which the faults are monitored from tower light through real-time data, and users are instructed to take corrective actions to diagnose the fault identified in safety systems by AR experience created through Vuforia Studio. Continuous monitoring of the system is a must to identify faults and inform the user to correct them.

2 Proposed System

Industrial robotic welding cell, which is a flexible manufacturing cell like any other machine, requires frequent maintenance. Fault diagnosis is the primary step to effective maintenance which identifies faults, and correction is done as a remedy. Different software is involved in the overall process. Thingworx IoT platform has its key role in collecting the monitored data of the safety systems fault and Vuforia Studio helps in creating and publishing an AR experience. AR experience can be created through the following tracking methods in Vuforia Studio. They are as follows:

  • Marker-based AR requires a marker to activate an augmentation.

  • Markerless AR offers the control to the user to choose where they would like to place the content such as based on location.

  • Projection-based AR is a video projection technique that expands and strengthens video data by launching images on 3D spaces.

  • Superimposition AR uses object recognition where the augmented image partially or fully replaces the original image [6].

In this work, a thingmark-based target is used. Figure 1 displays the AR development workflow which describes the hardware and software components involved in the development process of augmented reality applications for fault diagnosis.

Fig. 1
figure 1

Block diagram of process flow [7]

Full size image

2.1 Hardware and Software Components

Robotic Welding Cell (RWC)

The welding cell used in our application performs MIG welding and additive manufacturing. It is a flexible manufacturing cell—meaning it has its inventory, material handling system, and various parts in random order. The order can be placed using an app or manually using an HMI. There are two robots for performing the welding operations. The important components involved in the robotic welding cell are listed as follows.

Two robots

There are two robots involved in the process. The first one is the welding robot, which is a 6 degree of freedom articulated robot with the model—KUKA KR8 R1420 arc HW used to weld the workpiece, and a welding torch is mounted on this robot. The second is a material handling robot that moves the pallet fixture inside the welding cell with the model—KR30 R200 and is used to hold the workpiece.

Fronius Mig Welding Power Source

It provides the power for welding through a welding torch present on a welding robot and supplies current to it. The model is Fronius TPSI 400 CMT [8].

Torch Cleaning Mechanism

After the welding process, the welding torch attached to the welding robot has to be cleaned. The torch is immersed into the torch cleaning solution to remove the burr after the welding process. The model is Reamer V Easy.

Storage Rack

It is used to store the workpiece and welded part.

Human–Machine Interface

It is the interface between the controller and the user where most of the controls are run through it. The model is EXOR EX715 [9].

Safety Systems and Sensors

To detect errors or faults inside the RWC, various safety systems are installed, which must work efficiently to protect the workspace and the people working in the area. Safety components are as explained below.

Light curtain

Light curtains are present in this station for safety. If there is a breach in the light curtain, the machine will stop and it will move to a safe position after the light curtain is reset. This is an advanced method of securing the area around hazardous machines. The transmitter unit consists of light-emitting diodes (LEDs) that emit invisible infrared light pulses when energized by the light curtain’s timing and logic circuitry [10].

Emergency stops

If the emergency stop is pressed, the machine will be reset and stopped. If any kind of mal-performance in the machine is observed, then this switch must be pushed which disables the power supply of the welding cell causing the system to stop immediately. This button is connected to the control circuit for its use in an emergency crisis.

Safety interlock

The door is integrated with safety solenoid interlocks. When the door is open for maintenance or an operator is working inside the cell, the programming will be in an idle state. Once the door is locked, the system resumes.

Tower Light

The tower light indicates the safe working of safety systems which have three lights in them. It is used to alert the user in case of any malfunction in the system. It consists of red, amber, and green colors. It is an L-bracket aluminum tube mounting.

Sensors (Data on IO-Link Master)

IO-Link is an industrial communications networking standard involved in the connection of sensors and actuators to a type of field bus or ethernet [11]. IO-Link master collects the data from these sensors and sends them to plc to control the output. It also sends data regarding the health state of the sensor and monitored data.

Figure 2 shows the control architecture of RWC working which represents the controllers and related electronics involved along with cloud systems in the process.

Fig. 2
figure 2

RWC control architecture V2

Full size image

Creo Parametric and Creo Illustrate

Parametric is a software product of Creo used for 3D modeling. It is design software for creating parts and assemblies. Through this CAD models can be integrated into Vuforia Studio for AR implementation. Illustrate is an animation-creating software tool for 3D models that helps to apply animation sequences to guide a user by connecting to the AR experience [12].

Thingworx and Vuforia Studio

Thingworx [13] is a developer platform to create IoT-powered augmented reality to collect real-time data and integrate it into Vuforia Studio which is a web-based tool having CSS stylers, 2D and 3D widgets creation environment where AR experience is created and published. This can be viewed through Vuforia view app [14].

2.2 Methodology

Various steps are involved in implementing AR on RWC for performing fault diagnosis. Figure 2 describes the flowchart of the methodology overflow. The steps are as discussed below (Fig. 3).

Fig. 3
figure 3

Implementation plan

Full size image

CAD Model design

The basic parts involved in robotic welding cells are designed in Creo Parametric software considering the workstations which involve load/unload area, inventory storage area, inspection, and operations area. Using the same software, the components designed are then assembled to create a final assembly of the RWC model. Later, the safety systems are designed separately as per the standard model numbers considering the safety design parameters like threshold distance, vertical height placement, user-centric, etc. These are then assembled onto the final RWC model at the specific locations. Tower light is placed on top of the cell for easy monitoring, HMI and emergency pushbuttons are placed at the front side of RWC for user-friendly handling, and a safety interlock is located on the door for a more secure entry into the station along with speedy progress.

Creating Animation Sequences

The 3D CAD model created in Creo Parametric is imported to Creo Illustrate to create animation sequences for the fault diagnostic operations. The animation sequences have been created for all the safety systems separately for the user to receive guidance to rectify the errors. The exploded view and motion effects along with color characteristics have been implemented in the model to provide detailed user guidance on fault diagnosis. This platform has features like camera and capture which play key roles in sequence and animation creation. RWC model sequences are then imported as a ‘pvz’ file to Vuforia Studio for creating an AR experience. The figures of the 3D model and its illustration created are displayed below (Fig. 4).

Fig. 4
figure 4

RWC CAD model illustration

Full size image

Importing real-time IoT data

The Thingworx platform is used to import real-time data, for fault diagnosis operations. In this work, tower light indicates whether a safety system is working normally or has a particular error that leads to safety system breakdown. This tower light data will be monitored to indicate faults of the safety system and diagnose them by using the reset button. Safety systems are used when there is a particular error or fault occurring inside the robotic welding cell which has to be monitored first. For instance, if any part of the system, for example, the robotic arm has stopped working, this fault must be monitored to indicate the user to use safety systems to diagnose the error. Thingworx data is integrated with Vuforia Studio for further implementation of AR. In the Thingworx platform various entities, subscriptions, and events are created.

Creating and Publishing AR experience

Vuforia Studio is used to create AR experiences that can be viewed through the Vuforia view application on mobile. There are two modes—first is the 3D canvas where a target is set using thingmark with a marker width of 0.02 cm and the 3D sequenced model is imported on it and adjusted to a scale of 0.3. After this operation, the 2D canvas is worked where various 2D widgets like buttons, labels, card data, popup, etc. are imported to create an effective AR experience for fault diagnosis. The 2D widgets are bound to various services and events are triggered. Through external data, the IoT data tag can be imported and integrated into Thingworx. CSS styling code is implemented for a better user experience. For a better AR experience, UI design was first worked in proto software to create mockup screens and then applied in Vuforia Studio. Multiple screens are created to take the user through different options. It is then published and experienced through a QR code and Thingmark scanner (Fig. 5).

Fig. 5
figure 5

Mockups and 2D widgets design for AR implementation

Full size image

3 Results

The results of an effective AR experience for fault diagnosis are viewed in the form of a 3D visual display in the physical world. In the fault diagnosis performance, tower light display indication plays a key role in this proposed project. When the tower light indicates red, it infers that there is a particular error or human intervention in the safety system, upon which reset button is pressed in Vuforia Studio AR experience created to bring the safety system back to the error-free device and smooth working. In case the error continues, further user guide instructions and sequences are described. For example, in the case of a light curtain, when the transmitter or receiver is rotated by 10 degrees, the system will start working smoothly and then the tower light indicates green color. In case the safety system is currently in use, then the tower light indicates yellow color. All the processes of tower light in the fault diagnosis as described are integrated into the actual AR experience. The functional working results of AR experience are represented in physical view as shown in Fig. 6. Vuforia view app is used to view the AR experience in physical view by scanning the thingmark generated. Table 1 describes the comparison between the conventional approach and the AR experience used in fault diagnosis application [15].

Fig. 6
figure 6

AR experience in physical (Vuforia) view application

Full size image
Table 1 Comparison of conventional approach and AR approach
Full size table

4 Conclusion

This paper proposes an integrated approach to implement AR on industrial robots for fault diagnosis using real-time IoT data which can be experienced via mobile phones. AR and IoT use case implementation in any application impacts huge benefits to the user as well as industries. Fault diagnosis, machine monitoring, and maintenance training are some of the use cases where these key technologies along with data spread play a vital role. Remote maintenance is another key advantage of implementing AR in industries. The fault identification at an early stage reduces severe breakdown and production losses, which is most needed in the current generation of industries. In this work, fault diagnosis on one application use case of industrial robots is performed. It can also be performed on various applications of industrial robots such as finding looseness in a robotic mechanism, predicting system response and tuning the control system, and controlling the material inflow and welding torch mechanism, etc. Predicting the life of various systems using AR and IoT data collected from sensors is one of the key applications implemented in Industry 4.0. Results suggest that safety systems can be analyzed easily using IoT data as indicated on the tower light. Future developments would be focused on implementing AR experience for various other applications on wearables and heads-up displays.