The Core of the Fourth Revolutions: Industrial Internet of Things, Digital Twin, and Cyber-Physical Loops

Abstract

Like with the previous revolutions the goal of the fourth revolution is to make manufacturing, design, logistics, maintenance, and other related fields faster, more efficient, and more customer centric. This holds for classical industries, for civil engineering, and for NDE and goes along with new business opportunities and models. Core components to enable those fourth revolutions are semantic interoperability, converting data into information, the Industrial Internet of Things (IIoT) offering the possibility for every device, asset, or thing to communicate with each other using standard open interfaces, and the digital twin converting all the available information into knowledge and closing the cyber-physical loop. For NDE this concept can be used #1 to design, improve, and tailor the inspection system hard- and software and #2 to choose and adapt to best inspection solution for the customer or to enhance the inspection performance. Enabling better quality, speed, and cost at the same time. On a broader view, the integration of NDE into IIoT and Digital Twin is the chance for the NDE industry for the overdue change from a cost center to a value center. In most cases, data gathered by NDE is used for a quality assurance assessment resulting in a binary decision. But the information content of NDE goes way deeper and is of major interest for additional groups: engineering and management. Some of those groups might currently not be aware of the benefits of NDE data and the NDE industry makes the access unnecessarily difficult by proprietary interfaces and data formats. Both those challenges need to be taken on now by the NDE industry. The big IT players are not waiting and, if not available on the market, they will develop and offer additional data sources including ultrasonics, X-ray or eddy current.

1 Introduction

The cyber-physical ecosystem introduced by Industry 4.0 and NDE 4.0 [1,2,3,4,5,6,7] is based on digitization, digitalization, and digital transformation [8, 9]. Its core component is the cyber-physical loop processing digitized data representing one or multiple physical properties, like financial data, design data, data from production, data from operation, data from (NDE) sensors, or data from classical NDE inspections. The accumulated data is used for some data processing, like feedback, trending, or predictive maintenance. The results are visualized to gain knowledge which can eventually be used to invoke the necessary actions for process improvements [3, 10].

Cyber-physical loops or feedback loops have already been utilized for some decades by implementing the various interfaces from the various data sources and some data processing and visualization (mostly manual, computer-assisted data processing and visualization using e.g. Matlab or R). Most of those loops (see Fig. 1) use proprietary isolated solutions leading to multiple challenges; in particular, if reused or repurposed later for different applications.

Fig. 1

Challenges of cyber-physical loops (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Cyber-physical loops discussed in conjunction with the fourth revolutions are different: completely digitally transformed cyber-physical loops seamlessly integrating data sources and emerging technologies [2]. This publication focusses on how to build such cyber-physical loops, discusses the core elements, and shows how to connect the emerging technologies. All of it with focus on NDE. This “big picture” ecosystem and the number of points discussed in the following might sound a bit overwhelming. However, if taken on step-by-step and accompanied and tailored to the needs it can be reached.

1.1 Digitization, Digitalization and Digital Transformation

The ecosystem presented in the following is based on all three: digitization, digitalization, and digital transformation. Unfortunately, the ambiguous use of those three terms in public can be quite confusing. Moreover, most languages, like German, Spanish, and Japanese, do not even differentiate between digitization and digitalization, even though the digitization and digitalization activities have little in common. The only commonality between the two terms (besides the similarity in notation) is that digitalization requires digitization.

In simple terms, digitization is the transition from analog to digital and digitalization is the process of using digitized information to simplify specific operations [8, 11]. The digital transformation uses digital infrastructure and applications to exploit new business models and value-added chains (automated communication between different apps of different companies) and therefore requires a change of thought process. Digital transformation requires collaboration to improve customer’s digital experience. There is one more term here—Informatization, which is the process by which information technologies, such as the World Wide Web and other communication technologies, have transformed economic and social relations to such an extent that cultural and economic barriers are minimized [12]. Informatization is the path from analog, digital, and digitalize to digital transformation [9].

2 Industrial Internet of Things (IIoT)

The first step within the cyber-physical loop is the conversation of one or many physical properties into digital data and the combination or fusion of the data. For most Industry 3.0 applications, this process was mostly implemented proprietary. A typical example is the so-called automation stack. This section will describe the way from the automation stack to the Industrial Internet of Things, from proprietary interfaces to the digital transformation of communication. The next section will focus on data processing.

2.1 Automation Stack

In a digitized industrial production environment (“Industry 3.0”) the techniques and systems in process control are classified using the automation stack, which is also called automation pyramid or 5-layer model (see Fig. 2). The automation stack represents the different levels in industrial production. Each level has its own task in production, whereby there are fluid boundaries depending on the operational situation. This model helps to identify the potential systems/levels for Industry 4.0 and NDE 4.0 interaction. However, validity of this model needs to be discussed regarding Industry 4.0 and NDE 4.0.

Fig. 2

The Automation Stack. (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Level 0 (process level) is the sensor and actuator level for simple and fast data collection. The field level is the interface to the production process using input and output signals. The control level uses systems like programmable logic controllers (PLC) for controlling the equipment. Supervisory control and data acquisition (SCADA) of all the equipment in a shop happens at shop floor level. SCADA systems usually also provide some dashboard functionality to monitor production on the shop floor level. Manufacturing execution systems (MES) are usually used for collecting all production data and production planning on the plant level. Finally, Enterprise Resource Planning (ERP) systems control operations planning and procurement for a company. Systems for Product Lifecycle Management (PLM) or Design are usually not included in the automation pyramid (as the automation pyramid visualizes the automation during production and not during the lifecycle of a product) but should be connected to both the MES and the ERP systems. NDE equipment, if connected at all, is usually on the control or field level.

The information flow for the planning of production comes from the ERP system and is broken down to the field/process level (meaning the communication starts at the top level of the pyramid and is communicated to the bottom layer). Once production is running the data is collected by the field/process level, is condensed in several steps (in the different levels), and finally the key-performance indicators (KPI) are stored in the ERP system (meaning the communication starts at the bottom levels of the pyramid and is communicated to the top level). For this information flow in both directions’ interfaces need to be implemented between the levels. Depending on the number of systems or devices in a level the number of interfaces to be implemented can be exhausting.

This is in particular true between the control and the shop floor level. A typical industrial shop floor usually contains several thousand of PLCs. Each PLC is manufactured by an OEM and most of them have their own proprietary interfaces or APIs (application programming interface) which all have to be implemented individually to allow the integration into SCADA.

Moreover, in many production environments, no digital interfaces are implemented between MES and SCADA. This is why in a lot of production environments still analog (paper-based) or not-machine-readable digital (Email or PDF) solutions are used—like paper-based routing sheets. However, such solutions require human action, are highly error-prone (like entering the 10-digit serial number of a certain component), and hinder the information flow of information from production to the ERP system.

Thinking about one of the main goals of Industry 4.0—the improvement of industrial production by analyzing data—this works best if data from operations planning, financial planning, procurement, and sales is combined with the data from production. But this is not working as #1 a lot of devices from the control and field level are not integrated and #2 due to paper-based routing sheets. Meaning this system requires a major revision.

Another mayor idea of Industry 4.0 is that every device and system (including all NDE equipment) is able to communicate with each other device and system. All this independent of the level of the automation pyramid. Therefore, not only interfaces between two adjacent levels would be necessary but interfaces between all devices and systems in all levels. This implementation effort for all the interfaces would prevent Industry 4.0. This is one of the reasons why standardized, open, and machine-readable interfaces become key for Industry 4.0 and this is why companies will have to shift from proprietary interfaces to standard interfaces if they want to survive the ongoing fourth industrial revolution. Looking onto the member lists of the ongoing standardization efforts shows that most of the big players (for example SAP, Microsoft, Siemens) are beginning to understand this. Unfortunately, a lot of small and medium companies are still ignoring this development.

In other areas of industry (e.g. civil engineering, owner-operator, recycling) similar automation stacks can be identified with similar issues.

2.2 The Idea of the IIoT

The idea of the Industrial Internet of Things (IIoT), as shown in Fig. 3, and similar concepts in other areas is to overcome the challenges of the automation stack by eliminating all interfaces between the levels of the automation stack and to integrate all (relevant) data seamlessly.

Fig. 3

Industrial Internet of Things and how currently NDE is integrated (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

The automation stack is focused on production. In contrast the IIoT is a holistic approach. It starts with the initial idea, includes design and lifing considerations (for example by integration of CAD, fracture mechanics, or other design and lifing systems or software), production, operation, and maintenance. It even includes EOL (end of life) and enables a circular economy (repurpose, reuse, recycle). The idea is to integrate all the data before, during, and after the lifetime of the product. To include data captured during production in the supply chain, during product lifecycle management, from service, from structural health monitoring, and from a variety of sensors. Including the integration of cloud services and cloud storage, or database systems. Also, data from public information sources or data bought (like market research data) should be integrated. All this integration requires standardized, open, and machine-readable interfaces.

NDE inspections are performed during manufacturing in the supply chain, in production, in service, and in maintenance. Also structural health monitoring and sensors in general provide nondestructive evaluation results. NDE inspections are performed manually, automated, or automatically, are evaluated and a report, containing the Key-Performance Indicators (KPI) of the inspection, is generated. Currently most of those reports are printed, signed, and archived, either paper or scanned PDF based. All to eventually provide a decision for quality assurance. A GO / NO GO decision. This is why NDE is currently seen, by some customers, as a cost center [2, 3].

With the current way of archiving NDE reports (in paper format or as a scanned PDF in some database) the data stored on the report, the KPIs written down on the report, can only be read by computers under huge effort. So, in most cases this information is lost for further processing (besides manually retyping all information into data-base systems). Similarly, archiving the raw or processed data and metadata of inspections in proprietary formats of the manufacturer of the system means that the data can only be accessed by other systems #1 if the manufacturer allows and enables the access and #2 after implementation of a converter. In case a manufacturer decides to terminate certain products or data formats, or the manufacturer goes out of business, or the software is not supported by modern computers anymore the data will be lost.

This current situation is tragic. NDE data should not be limited to a use for quality assurance. NDE data provides way more value. NDE data can be used for example to make lifing calculations more accurate. NDE data can be used to calculate more exactly when the next maintenance must be performed. NDE data can shift e.g. from schedule based and condition-based monitoring to predictive and prescriptive maintenance. But this would require that engineering, in particularly engineering using statistical methods, gains access to the results, to the data. This will convert NDE from a cost to a value center and requires integrating NDE into the IIoT (see Fig. 4).

Fig. 4

Industrial Internet of Things and how NDE should be Integrated (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

NDE 4.0 is the chance for NDE to free itself from being considered a pure cost center and to free itself from the niche it is currently restrained in. Comparing with other fields NDE has the benefit that a lot of inspection equipment is already installed. Meaning the hardware invest for the customers to integrate NDE into the IIoT is negligible. Mainly the software needs to be enabled.

Such an integration requires a change of the thought processes of the inspectors. Inspectors are trained to provide the information critical for quality assurance, but they do not necessarily understand the needs of engineering. The authors have heard comments such as “I don't inspect chips”. Meaning the inspectors were not willing to inspect areas which will, in later manufacturing steps, be removed. In the context of Industry 4.0, all information is important. Test results from areas that will later be machined also contain valuable information that can be used, for example, to improve lifing models.

2.3 How to Integrate NDE into the IIOT—Basic Ideas

NDE as an integral part of the product development process, industrial production, and industrial operation, provides the quality assurance means needed by industry. As Fig. 5 shows, NDE is typically performed during initial production (at the asset OEM and/or in the supply chain), at certain intervals in operation, and after the EOL of an asset. In addition nondestructive sensor technology is used for monitoring and evaluation during production and assembly as well as structural health monitoring (SHM) or condition monitoring (CM) during operation.

Fig. 5

NDE during the lifetime of a product (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

In the following it will first be discussed how NDE is integrated into the product development process, second, in more detail, how NDE is integrated into serial production and maintenance, and third which interfaces need to be implemented for a holistic implementation of NDE into the IIoT.

During the product development process (see Fig. 6), the specifications for production and inspection are created through the cooperation of experts from design, material sciences, production, and NDE. Those specifications are field tested to optimize design and inspections. The value of NDE can already be seen here, as NDE offers a look into the prototypes and can therefore make a significant contribution to improving design and production. This requires interfaces for the statistical evaluation of the data (together with the process data from the inspections).

Fig. 6

Typical product development process (Fig. 7 provides a more detailed description of the situation during inspection in serial production and operation). (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

The data that can be obtained during the subsequent serial production and service give an even better picture of the components produced and their joints. This allows further improvements in design and production. In addition, they allow the next generation of products to be optimized (feedforward).

Figure 7 shows a closer look at the serial production and the inspections in the supply chain and during operation. Starting with material suppliers, who already carry out inspections on the raw material, through inspections at the component suppliers to the inspections at the OEMs, who assemble the final product. After all, the user is responsible for commissioning and service checks after certain operating times until the asset reaches its end of life and NDE supports the reuse, repurposing, or recycling. All these inspections provide results that could be integrated into an Industry 4.0 world through appropriate interfaces and thus, as described above, could contribute to improving production, and design and maintenance.

Fig. 7

(Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Typical supply chain with inspection steps in serial production and operation.

Figure 8 shows the interfaces of each individual inspection step. The input interfaces marked in green supply the order data, provide the inspector with information on the component, serve to correctly set the devices, the inspection, the mechanics, and the evaluation and to document the results in accordance with the specifications.

Fig. 8

Typical sequence of an (automated) inspection in serial production or during operation (can in principle be used for manual testing). (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Digital transformation of these input interfaces will help to support the inspector in his work, to avoid errors in the inspection, to optimize the inspection and to ensure a clear, revision-safe assignment of the results by digital machine identification of a component.

On the output side, the inspection system status information and the inspection results are generated. The inspection system status information could be used for maintenance and to improve the inspection system itself. The inspection results consist of the actual test data, the raw and processed data and the metadata (meaning the framework parameters of the inspection and evaluation), and finally the reported values. As mentioned above, the reported values represent the key performance indicators (KPIs) of the inspection. For industry, interpreted data are the easiest to evaluate. Therefore, the reported values are currently the most relevant data of the inspection. Consideration should be given to whether the currently reported values are sufficient for NDE 4.0 purposes or whether the results to be reported should be extended for statistical purposes and thus for greater benefit to the customer.

2.4 Semantic Interoperability

If a human sees some data, some information, like a certain number, humans are in some cases able to directly understand its meaning. Like the number 42. Most of the readers will immediately know that 42 is the answer to the ultimate question of life, the universe, and everything according to the novel “The Hitchhiker's Guide to the Galaxy” by Douglas Adams [13].

If the number 42 is entered in a computer, it will be converted into a binary number. However, the computer will not know how to do the interpretation of such a number. A computer could conduct a search. The current Wikipedia article on the number 42 provides 18 mathematical, 6 scientific, 5 technological, 5 astronomical, 10 religious, and multiple other meanings in popular culture. The number 42 could also be (see Fig. 9) the length of a truck, the gain of a UT instrument, the 42^nd day of the year, or the authors weight. Even knowing that the number 42 represents the gain of a UT instrument the questions arise: was the gain established before or after calibration, at which day, which probe, which component, which unit of measurement, which instrument, according to which specification, …

Fig. 9

What could be the meaning of the number “42”? (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

For a computer to identify a certain information without doubt, to exchange data between computer systems with a unique, common meaning, to enable machine readability semantic interoperability is needed. This is achieved by adding metadata to the data which is to be stored and linking each data element to a controlled, shared vocabulary. The meaning of the data is transmitted with the data itself in a self-describing "information package" that is independent of any information system. It is this shared vocabulary and the associated links to an ontology, which form the basis and capability for machine interpretation, inference, and logic.

Semantic Interoperability converts data into information, it allows combining information from different sources within a so-called namespace, it allows data-fusion, it allows creating data formats and data base structures, and it allows the direct use of the information in digital twins and for further data processing with, for example, using AI.

2.5 Interfaces for IIoT

The need for standardized, vendor-independent interfaces was discussed before. But what are the interfaces in this context? Is it the question regarding the physical interface? The question regarding USB, WIFI or 5G? The question regarding TCP/IP, http, XML, or OPC UA? Before further discussion, the term interface must be specified in more detail.

2.5.1 OSI Level

The OSI model, see Fig. 10, gives an overview of the different abstraction layers of digital interfaces and helps to select the interfaces that are decisive for NDE 4.0. The lowest level represents the physical connection, i.e. the cable or the radio connection, this is where WIFI, the various 5G variants, or low-power wide-area-networks (LPWAN) can play important roles. The first OSI layer, the transmission of the individual bits, runs via this connection. The information to be transmitted is combined with transmitter and receiver addresses and other information in the data link layer to form frames. Information packets are "tied" in the network layer and combined into segments in the transport layer.

Fig. 10

The OSI layers—a model for visualizing the degree of abstraction of interfaces (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

The layers above are the so-called host layers. The session layer is responsible for process communication. The presentation layer is responsible for converting the data from a system-independent to a system-dependent format and thus enables syntactically correct data exchange between different systems. Tasks such as encryption and compression also fall into this layer. Finally, the application layer provides functions for applications, for example with application programming interfaces (API).

The application layer is the communication layer that is decisive for Industry and NDE 4.0. However, semantic interoperability (not to be confused with syntactic) needs be added on top for an appropriate Industry 4.0 communication. The physical connection (USB, WLAN, 5G, ...) is more-or-less irrelevant.

An example of an application layer protocol is HL7 (Health Level 7). HL7 is the protocol used in healthcare to ensure interoperability between different information systems. HL7 (besides DICOM - see below) should therefore be one of the interfaces for Medicine 4.0 and the communication can run over various physical connections. Other protocols such as OPC UA, Data Distribution Service (DDS) or oneM2M are gaining ground in the industrial world (For those who might ask: the authors do not see an application of HL7 for NDE or Industry).

2.5.2 IIC Core Connectivity Standards

The Industrial Internet Consortium (IIC) approaches to define the Industrial Internet of Things in its specifications. In Volume G5 [14] Industry 4.0 interfaces are discussed. Those discussions are based on the Industrial Internet Connectivity Stack Model, which is similar to the OSI model, however compared to the OSI model it combines the three host layers to one so-called framework layer. Based on this model it compares the interface protocols OPC UA, DDS and oneM2M with Web Services (see Table 1). Every interface protocol is considered a Connectivity Core Standard and the need for Core Gateways between the Connectivity Core Standards is emphasized. This brings the benefit that every connectivity standard can be used, and the information combined using the gateways between the standards.

Table 1 The IIC core connectivity standards

DDS is managed by Object Management Group (OMG) and focusses on low-latency, low-jitter peer-to-peer communication with a high Quality of Service (QoS). It is data-centric and does not implement semantic interoperability.

OneM2M is a connectivity standard mainly for mobile applications with intermittent connections and low demands regarding latency and jitter. Semantic interoperability implementation is ongoing.

WebServices use the Hypertext Transfer protocol (HTTP) known from the internet. It is primarily for human user interaction interfaces but implementations like REST (Representational State Transfer) or SOAP (Simple Object Access Protocol) allow the use of webservices for computer-computer communication. Semantic interoperability can be reached using the Web Ontology Language (OWL).

OPC UA, discussed in detail below, is mainly used in the manufacturing industry. In contrast to DDS it is object oriented and provides semantic interoperability.

For NDE applications OneM2M could be of benefit for mobile devices. WebServices are ideal for human-computer interaction and could be used for operator interfaces to store and read information regarding the component to be inspected. Low-latency and low-jitter communication is not necessary for typical NDE equipment; therefore, DDS will not be considered further. OPC UA, being the standard protocol for manufacturing and due to the included semantic interoperability, seems like the ideal interface for NDE 4.0.

2.5.3 OPC UA

The high-level communication protocol / framework that is currently established in the manufacturing Industry 4.0 world is OPC UA [15, 16]. OPC UA has its origin in the Component Object Model (COM) and the Object Linking and Embedding (OLE) protocol. OLE was developed by Microsoft to enable users to link or embed objects created with other programs into programs and is used extensively within Microsoft Office. COM is a technique developed by Microsoft for interprocess communication under Windows (introduced in 1992 with Windows 3.1). This standardized COM interface allows any program to communicate with each other without having to define an interface separately. With the Distributed Component Object Model (DCOM) the possibility was created that COM can also communicate via computer networks.

Based on these interfaces, a standardized software interface, OLE for Process Control (OPC), was created in 1996, which enabled operating system independent data exchange (i.e. also with systems WITHOUT Windows) in automation technology between applications from different manufacturers.

Shortly after the publication of the first OPC specification, the OPC Foundation was founded, which is responsible for the further development of this standard. The first version of the OPC Unified Architecture (OPC UA) was finally released in 2006. OPC UA differs from OPC in its ability not only to transport machine data, but also to describe it semantically in a machine-readable way. At the same time, the abbreviation OPC was redefined as Open Platform Communications.

OPC UA uses either TCP/IP for the binary protocol (OSI layer 4) or SOAP for web services (OSI layer 7). Both Client-Server and Pub-Sub architectures are supported by the OPC UA communication framework (see Fig. 11). Based on this, OPC UA implements a security layer with authentication and authorization, encryption, and data integrity through signing. APIs (Application Programming Interfaces) are offered to easily implement OPC UA in programs. In the .net framework OPC UA is even an integrated component. This means that the users do not have to worry about how the information is transmitted. This is done completely in the OPC UA framework. The only thing that matters is what information is transmitted.

Fig. 11

(© Vrana GmbH, based on [17]). Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

OPC UA architecture

As Fig. 11 shows, the OPC information model already defines some basic core information models in which models are defined that are required in many applications. In addition, companion specifications exist for product classes such as field devices (FDI), robots or scales. These companion specifications provide semantic interoperability and are therefore the basis for Industry 4.0, the basis for smooth I4.0 interfaces and communication and result in any OPC UA-enabled device being able to interpret data from others. In addition, there may also be manufacturer-specific specifications for the exchange of data between the devices of one manufacturer.

OPC UA Pub/Sub enable One-to-Many and Many-to-Many communications. Moreover, OPC UA TSN (Time Sensitive Network) will make it possible to transfer data in real time and to extend OPC UA to the field level. The OPC UA specifications are also currently being converted into national Chinese and Korean standards.

Moreover, it is planned to start the development of an NDE companion specification for OPC UA in a joint project between DGZfP, VDMA and OPC Foundation.

OPC UA is, like HL7 in healthcare, the standard for an interface to the manufacturing Industry 4.0 world. In the same way as in medical diagnostics, large amounts of data are in some cases generated with NDE (in OPC UA larger files are split into smaller packages—e.g. the OPC UA C++ Toolkit has a maximum size of 16 MB). Computed tomography (CT), automated ultrasonic testing and eddy current testing can easily result in several GB per day that need to be archived long term. In the healthcare sector those large data files resulted in the development of DICOM (Digital Imaging and Communications in Medicine) alongside HL7.

2.5.4 DICOM

DICOM is an open standard with semantic interoperability for the storage and communication of documents, image, video and signal data and the associated metadata as well as for order and status communication with the corresponding devices. This will enable interoperability between systems from different vendors, as Industry 4.0 is striving for.

In health management, this leads to the necessity of interfaces between HL7 and DICOM (see Fig. 12). This interface is usually found in the PACS (Picture Archiving and Communication System) server. In the process, patient and job data are translated from HL7 to DICOM for communication to the imaging devices. Information about the order status, about provided services (e.g. "X-ray image of the lung ") as well as written findings and storage locations of the associated images are communicated back. The returned data, texts and references would usually be referred to in industry as KPIs (Key Performance Indicators).

Fig. 12

Interaction between HL7 and DICOM (©DIMATE GmbH, Germany)

The central system for the "process logic" in hospitals is the HIS (Hospital Information System; comparable to an ERP system in industry), which communicates with all other systems via HL7. All image, video and signal data are stored in DICOM format in PACS, which is designed to handle large amounts of data and is the central system for archiving and communicating the data.

2.5.5 Digital Workflow in NDE with OPC UA and DICONDE

For the NDE world, this system can be transferred from HL7 and DICOM as follows (see Fig. 13): The Industry 4.0 world consists of ERP (Enterprise Resource Planning) or MES (Manufacturing Execution System) servers for production planning or as a production control system and assets supply data via OPC UA. A transmission of order data for inspections as well as a return transmission of notifications and inspection results (KPIs for storage in the MES) can be mapped via OPC UA. An integration of maintenance and calibration data of NDE equipment via OPC UA is also conceivable.

Fig. 13

Proposed interaction between OPC UA and DICONDE (©DIMATE GmbH, Germany)

With a few exceptions, however, the raw data generated during tests are too large to be communicated reasonably via OPC UA. Like HIS in a hospital, ERP and MES are not designed for the administration, communication and archiving of large amounts of image, video or signal data, such as is generated in radiography, computed tomography, automated ultrasound and eddy current testing or SAFT/TFM. Therefore, it makes sense to store the raw data outside the OPC UA world in a revision-proof way. The DICONDE standard offers itself as protocol and data format offering semantic interoperability. DICONDE is based on DICOM and has been adapted by ASTM to the requirements of the various NDE inspection methods [18,19,20,21,22,23]. In radiography the DICONDE standard fits very well to the requirements of the users. There are already many manufacturers who store their data in the DICONDE format and have implemented the DICONDE communication interfaces, for example for the digital query of inspection orders, whose IDs are then automatically stored in the metadata of the DICONDE files and thus ensure structural integrity between NDE raw data and ERP/MES. DICONDE is also currently established as the standard in the field of computer tomography. Similar to healthcare, an entity that "translates" order data and reported values between OPC UA and DICONDE makes sense.

In ultrasonic and eddy current testing, however, the medical requirements are further apart from the requirements of NDE. Although the DICONDE standard strives to define suitable data formats [18,19,20,21,22,23], these are currently not supported by device manufacturers. It is necessary to clarify at which points the manufacturers still see a need for action.

Contrawise, DICONDE can be easily implemented for the connection of visual inspections, e.g. photos in the field of dye penetrant and magnetic particle inspection and videos in the field of endo- and boroscopic tests.

2.5.6 Data Security, Data Sovereignty, Connected World, and Data Markets

The basis for all interfaces is confidence. This is why data security and sovereignty are key. In particular considering a connected world, where every company is connected with every other company and where data will be a commodity.

Data security [24] is a means for protecting data (for example in files, emails, clouds, databases, or on servers) from unwanted actions of unauthorized users or from destructive forces. Data security is usually implemented by creating decentralized backups (to protect from destructive forces) and by using data encryption (to protect from unwanted actions).

Data encryption is based on mathematical algorithms which encrypt and decrypt data using encryption keys. If the correct key is known encryption and decryption can be accomplished in a short time, but if the key is not known the decryption becomes very challenging for current-day computers (several months or years of calculation time) and the data is therefore secured from unwanted access. However, with computers becoming increasingly more powerful over time, encryption keys and algorithms need to become more challenging over time. And data encrypted with old algorithms or too short keys need to be re-encrypted after some time to keep it safe. The only measure ensuring data encryption over time is to use keys which have the same length as the data to be encrypted and which are purely random. One of the few methods to create such keys is quantum cryptography, which is still quite expensive in installation.

Where data security is the necessary basis, data sovereignty goes one step further protecting data [25]. Data sovereignty guarantees the sovereignty of data for its creator or its owner. Data itself, if not artistic, is legally not protected by any copyright. Therefore, if a dataset is submitted to somebody else currently only individual contracts hinder the receiver from forwarding or selling the data (even if submitted using data encryption). Therefore, two measures have to be implemented to guarantee data sovereignty. #1 legal documents need to be prepared and #2 software and interfaces need to be implemented to restrict the use on receiving side to the rules of the submitting side.

The International Data Space Association (IDSA) is working on both. IDSA develops standards and de-jure-standards based on the requirements of IDSA members, works on the standardization of semantics for data exchange protocols and provides sample code to ensure easy implementation.

One of the key elements IDSA is implementing are the so-called IDS connectors (International Data Spaces Association 2019) which guarantee data sovereignty (see Fig. 14). Both the data source and the data sink have certified connectors. The data provider defines data use restrictions. The data consumer connector guarantees that the restrictions are followed. For example, if the data provider defines that the data consumer is allowed to view the data once the data will be deleted by the consumer connector after the data was viewed. This enables also the producer of the data to decide which customer can use his data in which form as an economic good, for statistical evaluation or similar.

Fig. 14

IDSA: Connected Industrie 4.0 World [26]

Data sovereignty will enable secure digital communication between companies; within the connected world. This information represent a value in itself. Data becomes an asset, a commodity. There is a market for information and it is important to use it. The way to this market are the interfaces discussed in this publication.

2.6 How to Integrate NDE into the IIOT—How to Proceed

With IIoT, OPC UA, WebServices, DICONDE, and IDSA, protocols and interfaces have already been created in industry to implement “NDE for Industry 4.0”. In order to make NDE an integral part of the Industry 4.0 world, cooperation is required—for example by establishing semantic interoperability by using OPC UA Companion Specifications, and WebServices Web Ontology Language.

With DICOM/DICONDE there is an advanced interface and a well-developed open data format available. DICOM/DICONDE already offers semantic interoperability and its standardized and open ontology can be used as a base for the NDE ontologies for the standard Industry 4.0 interfaces mentioned in the paragraph above.

For NDE technologies with large data volumes, DICONDE is an ideal addition to the industrial interfaces (similar to the combination HL7 and DICOM). This means that interfaces/mappings from DICONDE to the Industry 4.0 world (OPC UA) are needed. For NDE technologies with small data volumes, it is necessary to decide, depending on the application, whether a direct interface is created using OPC UA or whether these are first stored in the DICONDE world and then transferred to the OPC UA world, in order to summarize all test results in one place. In addition, it is necessary to check which steps are required to be able to use DICONDE for ultrasound and eddy current.

Figure 15 (based on Fig. 8) shows an idea for the integration of an NDE system into an Industry 4.0 landscape. Using OPC UA for most of the input and output files and to use DICONDE for archiving the actual RAW and processed data files including the connected metadata.

Fig. 15

(Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Idea for the integration of an NDE system into an Industry 4.0 landscape.

In general, a revision-safe and secure storage must always be ensured. The retrievability, integrity and sovereignty of the data is key. Most of those requirements are already implemented in DICONDE and OPC UA.

Other open data-formats for NDE data, like HDF5, can be seen as alternatives to DICONDE. However, for most inspection situations the standardized open information models of DICONDE, which enable machine readable data using semantic interoperability, surpass the information models of the other data formats. Also, revision-safe and secure data-storage needs to be implemented in addition.

In order to ensure the interests of NDE in the Industry 4.0 world and for the development of the necessary ontologies, cooperation with Industry 4.0 must be strengthened.

With the integration of NDE into the IIoT and with the incorporation of semantic interoperability models (ontologies) NDE data will be transformed to information (see Fig. 16). This allows data fusion between different NDE methods, between classical NDE and NDE sensors, between NDE and other material tests, and between NDE and ALL other data. This is the path from proprietary interfaces to the digital transformation of communication which can now be used for the digital transformation of data processing: digitals twins.

Fig. 16

The digital transformation of data communication to establish the cyber-physical loop (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

3 Digital Twin

According to MedicineNet a twin is one of two children produced at the same pregnancy. Twins who develop from a single ovum are called monozygotic or identical twins. They have identical genomes. This leads to the situation that twin studies are a key tool in behavioral studies.

An (ideal) digital twin is a virtual representation of an asset and like a monozygotic twin it shows the same behavior and development as the asset. An asset can be anything from a manufacturing device, sensor, component, product, system, process, service, operation, plant, business, enterprise, software, to a person, operator, or engineer. A digital twin connects the physical world with the cyber world. Digital twins can be used for behavioral or development studies of the asset represented. The first idea to this concept was introduced 2002 by Michael Grieves [27].

3.1 Digital Twin of a Person

To create a digital twin of a person (see Fig. 17) information about the person must be collected. This includes information about the type (in the case of the author: mammal, human, male), about ancestors and relatives, physiology of the body, psychology, and information documenting status, development, and behavior (like financial, occupation, social, family, friendships, partnerships, sexual, leisure, or professionally related information). The physiological information will contain information inherited from the type (like the general constitution of a human body, including the skeleton, the organs, …), the peculiarities of the instance, the as-is condition of the body, and some real-time information. One way to determine the peculiarities and the as-is condition of the body is chemical, physical, and radiologic testing (NDE!). Real-time information sensors will provide data regarding heart rate, sleeping patterns, … (SHM!) and will support the information from the radiologic testing.

Fig. 17

The digital twin of a person with the core components of every digital twin (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

All this data, all this information can then be used to predict the behavior of the person or respectively developments regarding the person. For example, it could be used to predict the financial success of a person, buying patterns, risk propensity, movement profiles, work attitude or to gain information about the personality. This can be used for example to automatically tailor adds, to calculate insurance rates, credit ratings, health ratings, … The data could also be used to supports doctors in providing reasonable diagnostics and treatments, to predict future health issues, or even to calculate the lifetime.

This shows the value of the data once the data is processed by an appropriate digital twin. This is why all of the big players in the IT industry are collecting data about everybody. The more data they have—the better the predictions. Similarly, for insurance companies or government agencies.

3.2 Basic Concepts

An ideal or complete digital twin of a person would incorporate ALL data and would be able to answer ALL questions regarding the person. With the data collected of a person, social media platforms, purchasing platforms, insurance companies and governments have already implemented partial digital twins.

Some of those digital twins are/will be unbelievably valuable to us and some of them extremely scary or even harmful. This is one of the reasons why protecting personal data is so important.

How the create value out of information. The most straight forward way is to visualize data and its cross correlations. This can be supported by statistical evaluation tools, by algorithms and by simulation tools for data processing.

Those algorithms can either be implemented deterministically, based on knowledge about physical or other correlations, heuristically, or statistically. Deterministic solutions are ideal because they will provide accurate results even if the data base is small and even outside of the data spectrum available. Filters and reconstruction algorithms (like computed tomography, SAFT or TFM) are considered deterministic algorithms. However, in some cases the correlations are unknown or so complex that deterministic implementations are not possible or too expensive. Once a sufficient data base exists heuristic or statistical algorithms, like artificial intelligence, machine, or deep learning, can become handy.

This means:

• 1. If the correlations are known and not too complex deterministic algorithms should be used

• 2. If the correlations are NOT known or too complex and if a sufficient data base exists heuristic or statistical algorithms should be used

• 3. If the correlations are NOT known or too complex and if the data base is NOT sufficient more data should be collected, simulations performed and/or the correlations worked on

Simulation are computer programs for the recreation of specific realities for various purposes, such as training, entertainment, or the analysis of systems too complex for theoretical or formulaic treatment. Simulations are virtual experiments and like real-life experiments they can be used to gain knowledge about certain processes or system behavior. Simulations are based on input data and with more data they will provide more accurate results. The data obtained by simulations can be used to enhance the data base obtained by experiments. However, simulations need to be validated to assure appropriate results.

Simulations are usually used to solve “normal problems” by assuming a certain set of parameters and a simulation model and seeing the output. For digital twins usually the opposite is of interest, solving “inverse problems”—meaning taking the result, applying the model backwards and obtaining the parameters. Simulations can also be used to solve inverse problems—however this usually takes tremendous computing power. For example simulations can be performed for thousands of parameter sets which afterwards are used to develop algorithms, for example by training a convolutional neural network with the results of the simulation.

Figure 17 visualizes the digital twin of a person. The digital twin is represented by the blue box and contains the key elements of every digital twin:

• 1. Information (Data with Semantic Interoperability)

• 2. Data Processing (Algorithms, Statistical Evaluation and Simulation Tools)

• 3. Visualization & Action (Action is either generated manually by visually analyzing the data and the results of the data processing or it is generated automatically)

• 4. The cyber-physical loop between the person and the persons digital twin

Like the digital twins of a person digital twins can be created for every asset. As mentioned above an asset can be anything from a component, product, process, or system. And digital twins scale from components, processes, operations, plants, businesses, enterprises, to governments and countries.

3.3 Nesting

Looking in industrial manufacturing a digital twin for the complete enterprise, digital twins on plant level, digital twins on shop floor level, and digital twins for every single device can be imagined. The digital twin of the enterprise will contain all the digital twins on plant level. A digital twin on plant level will contain all the digital twins on shop floor level and a digital twin on shop floor level contains all the digital twins of the devices. This is called nesting (vertical axis in Fig. 18: Landscape of digital twins. On the vertical axis nesting of digital twins is shown and on the horizontal axis the digital thread.) and follows the automation stack showed in Fig. 2. Every digital twin on lower levels will inherit the properties of the digital twin at higher levels.

Fig. 18

(Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Landscape of digital twins. On the vertical axis nesting of digital twins is shown and on the horizontal axis the digital thread.

A similar nested structure can be found for every product in operation. For example, the digital twin of a civil airline company will contain the digital twins of all the airplanes. The digital twin of an airplane contains the digital twins of wings, cockpit, fuselage, and engines. And the digital twin of an engine will contain all the components which build the engine.

For NDE this concept of nesting can be taken further. The digital twin of an NDE system will be part of the digital twin of the shop floor. The digital twin of the NDE system will contain the digital twins for the mechanical automation, for the detectors and sensors, for the evaluation software and for the operator.

3.4 Digital Twins of Personnel

Personnel, i.e. operators or inspectors, are also represented by a digital twin. For example, there may be a digital twin for a level 3 ultrasonic inspector specializing in the inspection of castings. This inspector receives his task via a tablet, or an augmented reality platform and the results are stored digitally by the inspector. This shows that Digitalization, Digital Transformation, Industry 4.0, and Digital Twins are NOT striving for the deserted factory. For Industry 4.0, networking is crucial, and the results must be available digitally. It does not require automation. For some work steps, especially repetitive tasks, it makes more sense to use automated solutions. In other work steps the human being is more effective.

3.5 Digital Thread

Another viewing angle into industrial manufacturing is from the viewpoint of the product (see also Fig. 5). First the idea for a new product is born, the product is designed, raw material produced, individual components manufactured, assembled to a product, operated until the product reaches its end-of life (EOL). After it’s end of life the product is disassembled and the material of the components gets recycled. Each of those steps can be represented by a digital twin. And all the digital twins over the lifetime of a product are connected by the digital thread as shown by Fig. 18 (horizontal axis). The digital twins during lifetime relate to various companies. Raw material and components will usually be produced by suppliers, assembly will be performed by an OEM, operation by an owner-operator and the activities after EOL by specialized companies. This means the digital thread needs to be handed from one company to the next during the lifetime of an asset.

Such digital threads can be created for every asset. Every manufacturing device should have its digital thread, every process, every software and even every company. The digital thread for an enterprise will start with the initial idea for the company. The company will grow, procure other companies, and will eventually go out of business.

3.6 Digital Twin Type, Instance and Aggregate

A Digital Twin Type or Prototype (DTP) is a digital twin for an asset before starting the production/creation. Such digital twins usually incorporate the initial idea, the design requirements, drawings, CAD models, results of destructive tests, material properties, bill of materials, …

A Digital Twin Instance (DTI) is a digital twin of a certain instance of an asset. DTI will contain the information from the DTP. Such digital twins usually incorporate the as-is geometry (Metrological information from the components and the assembled product), the peculiarities and internal structure of the components and product (NDE), sensor data from manufacturing and operation, bill of processes, service records, operational data, …

A Digital Twin Aggregate (DTA) is the aggregation of all the DTIs. Considering an airline company, it could be an aggregate of all airplanes, of all airplanes of a certain type, of all engines, or of all seats in all airplanes.

3.7 Digital Twin Interrelation

Figure 18 shows on the horizontal axis the digital thread of an asset (in this example of a product in production). A machine used for a certain production or inspection operation is applied at a certain point in time regarding the digital thread. Therefore, the nested structure of such a machine is shown vertically and crosses the digital thread at a certain point of time. At that point, the data from the nested production digital twin becomes part of the digital thread.

However, it is not just one horizontal and one vertical axis. It is multiple. And all of them are interrelated. To summarize:

• 1. Every asset has a digital thread

• 2. Every asset is part of a nested structure

This leads to a 2D net of digital twins.

Moreover, the different instances of a type constitute a third dimension orthogonal to the paper. As not necessarily always the same machine was used for production the digital twins of the various instances will interact with different branches of the nested production related digital twins. This is creating a 3D net of digital twins.

3.8 Reference Architectural Model Industry 4.0 (RAMI 4.0)

As discussed above a digital twin can be created for every asset. Either a complete digital twin or multiple partial digital twins. Figure 18 shows horizontally the life cycle & value stream of an asset which is described by its digital thread, containing multiple digital twins, starting with DTPs for the type and continuing with the DTIs for the instance.

Moreover, Fig. 18 showed that every asset is contained in hierarchical structure. This leads to a digital nesting structure for the digital twins.

There is a third dimension which was not discussed up to the moment: the abstraction layers, the architecture: Starting with the asset, continuing with the integration, which represents the connection between the physical and the cyber world, the communication of the data, the conversion of data into information, the actions gained from the information, and finally how all of this influences the business.

This structure, with its three dimensions, was already identified and visualized by the Plattform Industrie 4.0 in 2015 (see Fig. 19) and was given the name Reference Architectural Model Industry 4.0 (RAMI 4.0). RAMI 4.0 is described in detail in DIN SPEC 91345 [28].

Fig. 19

Slightly modified RAMI 4.0 (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

The Life Cycle & Value Stream (IEC 62890 [29]) axis of RAMI 4.0 represents the value chain and the life cycle of an asset, starting with the development and usage of a new type, through the production of the instance to the usage of the instance. Compared to RAMI 4.0 as detailed in DIN SPEC 91345, Fig. 19 contains in addition the initial idea, EOL, and potential production within the supply chain, just like Fig. 5. The digital implementation of the Life Cycle & Value Stream is the digital thread as shown in Fig. 18. The term "type" is used to identify a new asset type, such as a new X-ray inspection system. Instance refers to the test facilities that have been built.

The hierarchy levels (IEC 62264 and IEC 61512) correspond to the layers of the automation stack (refer to Fig. 2), besides the top level “Connected World”, and to the digital nesting structure of digital twins (Fig. 18).

On the architecture axis (Layers) the lowest layer (Asset) represents the physical object. The "Integration Layer" is the transition layer between the physical and the information world. "Communication", "Information" and "Functional Layer" are abstraction layers for the communication and the "Business Layer" describes the business perspective.

RAMI 4.0 provides a nearly complete picture of the cyber-physical landscape and provides an excellent possibility to locate interfaces, digital twins, cyber-physical loops … within the Industry 4.0 landscape. However, RAMI 4.0 is asset centric or to be more exact—it considers single instances of assets. Therefore, it does not consider the interaction with other assets and it does not consider aggregation. This is where the digital twin interrelation, as proposed by the author, takes RAMI 4.0 to the next level.

3.9 General Concept of a Digital Twin

As already indicated above every digital twin consists of three main elements shown in Fig. 20.

Fig. 20

Illustration of the concept of a digital twin (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

Information is generated out of data by establishing semantic interoperability. Moreover, the data needs to have a reliability information. This does not only count for data from NDE. Even financial data is only accurate to a certain account. This reliability information is needed so that the data can be used appropriately in data processing. For NDE this means that any data which is supposed to be used in a digital twin should contain reliability information. This is why the importance of PoD (Probability of Detection) will drastically increase with NDE 4.0. The information used in the digital twin will most likely not be stored within the digital twin. More likely the digital twin will have access to database systems, to hard drives, to clouds, and to the IIoT which is combined using their semantic interoperability. This is the big data input for the digital twin.

Data Processing uses, as mentioned above, algorithms, statistical evaluation, and simulation tools for the big data processing. For such calculation’s conventional computers, AI, ML, DL or even quantum computers can be used. Typical data processing approaches are: feedback, trending, predictive and prescriptive maintenance, probabilistic lifing, behavioral analytics, risk modelling, reliability engineering.

Visualization using extended reality, dashboards, or other aids to visualize the information, the cross correlations of the data, and the results of data processing will lead to a gain of knowledge.

This knowledge can finally be implemented to improve production, maintenance, and design. This conversion from knowledge into Action can either happen manually, after interpretation of the visualization, or automated. The ideal scenario for a digital twin is to perform all those actions in real time. Digital twins are living, learning models.

The difference of a digital twin to the data processing implementations of the last decades is the step from digitalization to digital transformation. From proprietary implementations to a data processing eco-system. An eco-system allowing to implement all data and information sources, an eco-system allowing to use various applications and visualization tools available or to implement new ones, an eco-system allowing to automatically create action. And it needs to be scalable so that it can be used both for the “low-hanging” use cases using partial digital twins or the more challenging, more complete digital twins.

4 The Cyber-Physical Loop

The digital twin is the core element of the cyber-physical loop as shown in Fig. 21 and closes the cyber-physical loop: The data is collected and digitized, converted into information by semantic interoperability, combined with other information in the IIoT, processed by the digital twin to create knowledge, which finally leads back to actions in the physical world. As mentioned before those actions can be triggered manually after gaining knowledge by performing an interpretation visually or the actions can be automatic.

Fig. 21

The digitally transformed cyber-physical loop (Author: Johannes Vrana, Vrana GmbH, Licenses: CC BY-ND 4.0)

A digital twin is the living, learning key-component of the cyber-physical loops. Digital twins are connected in a 3D net (perhaps further dimensions will be identified in the future). The current digital twins have to be identified as partial digital twins as not all data is incorporated and as the data processing capabilities are not designed for all possible purposes. Comments like “CAD is the digital twin” are kind-of correct. CAD can be seen as a very simple digital twin, which encapsulates some dimensional data, provides some data processing and some visualization capabilities.

As a complete digital twin can use ALL data—including all NDE data—shows that statements like “NDE is the digital triplet” must be identified as what they are: marketing. Just like Industry 5.0 or NDE 5.0.

5 Discussion and Outlook

The cyber-physical loop, including its core components IIoT, semantic interoperability, and digital twin is the core of the digital transformation. The core of the technology behind the various fourth revolutions—in industry, in NDE, etc.

IIoT allows to seamlessly connect every device with every other device, within the company, between companies, with the cloud, and within the connected world. Semantic interoperability is the means to achieve machine readability and data fusion.

Digital twins close the cyber-physical loop by taking data from the IIoT, processing and visualizing the data and creating knowledge, which can be used to create action in the physical world. For sure not only the input to the digital twin is established by the IIoT but also the output. This can eventually leads to multiple digital twins interacting with each other.

The cyber-physical loops created using IIoT and digital twin lead to digital transformation, replacing proprietary interfaces and applications by a new scalable open eco system. This eco-system will also enable the data market.

For NDE this is great news. NDE can pick up the existing interfaces to the IIoT, which allows access to the IIoT, to digital twins, to the cyber-physical loops. NDE is one of the most valuable data sources and the NDE community needs to get onto this subject by defining the ontologies.

IF DATA IS THE NEW OIL, THEN NDE 4.0 IS THE NEW OIL RIG. – RIPI SINGH [30]

In an upcoming paper the concept of the cyber-physical loops will be taken one step further by discussing the various loops and by identifying the associated business cases.