EIMDC: A New Model for Designing Digital Twin Applications

Abstract

With the development of communications and big data, digital twin as a novel paradigm has been received insentive attentions. However, there are some huge challenges in designing digital twins due to the complexity of digital twin applications. Firstly, most existing approaches merely focus on customized development, they are not general enough to tailor multiple applicaiton domains. Secondly, it lacks down-to-earth methodology for leading the designing process. Thirdly, it is tricky for developers to develop high valuable applications in real scenarios. To conquer these challenges, in this paper, we propose an EIMDC model for designing digital twin applications. It is comprised of entity, infrastructure, model, data and context. The entity is used to depict the physical entities mentioned in applications. The infrastructure exhibits the supporting infrastructure for enabling the digitalization of the physical entities. The model specifies the behavior of digital twin including geometric physical modeling, data-driven model and mechanism model. The data illustrates the data in cyberspace sensing from physical entites. The context represents the application context for digital twins. Finally we use a SMT production line case to show the effectiveness of the proposed model.

1 Introduction

Digital twin as a noval computing paradigm has been widely applied in multiple domains with the rapid development of digital transformation and big data [17]. The original meaning of digital twin depicts a mapping model between cyberspace and physical space, which is used to represent the fusion of cyber world and physical world. In 1991, the first idea of digital twin was coined by David Gelernter [15], mirror worlds were given in his descriptions. However, Dr. Michael Greaves (University of Michigan) firstly applied the digital twin concept to manufacturing and formally announced the concept of digital twin software in 2002 [16]. Finally, in 2010, NASA [22] came up with a new term -“digital twins”, it specified that a digital twin was an integrated multiphysics, multiscale, probabilistic simulation of a built vehicle or system that uses the best available physical models, sensor updates, fleet history, etc, to mirror the life of its corresponding flying twin.

Many emerging digital twin applications have been developed in various domains, such as manufacturing industry, electric power industry, automotive industry, etc. Designing a digital twin application actually is a tricky task due that it refers to multidisciplinary integrations and various technologies, typically including IoT, Big Data, AI, 3D Visualization, etc. Most of existing design approaches for digital twins are oriented to specific domains, In [21], many cases are investigated in wind turbines, product management, healthcare centers. Some researchers examines the digital twin for greenhouse horticulture [12], which applies it to study cultivation or climate control via IoT systems. Also some researchers use digital twin to post-harvest handling in agriculture [14]. Though these investigations in digital twin have been widely applied in corresponding domains, it lacks general design methodology guidance for how to start a digital twin design.

To address the design concerns, we propose a EIMDC model for digital twin applications. The core idea is to abstract the design process as five critical elements, which are entity, infrastructure, data, model and context, respectively. The entity is to depict the physical objects for digitalizaiton in physical space. The infrastructure provides a technical supporting for enabling the digitalization of these entities. The data illustrates all the data related to digitalization of entities. The model is used to represent the behavior of entities in cyberspace. Finally all the data and model for the entities are presented in the context for use.

The reminder of this paper is organized as follows: In Sect. 2, we present the related works in digital twins and their applications. The EIMDC model is described in detail in Sect. 3. For Sect. 4, we present a case about industrial production line to examine the proposed model. Finally the conclusions are given in Sect. 5.

2 Related Works

Digital twin has attract many attentions in industry and academia due to its mix of virtual and actual reality. The authors in [18] employ deep reinforcement learning to realize industrial robot grasping based on digital twin. In [25], it proposes an energy digital twin framework for industrial energy management. It aims to enable a reduction in carbon and environment protection. In material domain, Artem et al. [19] leverage digital twin to carry out a laser flash experiment for helping improving the thermal performance of metal. Wu et al. [24] propose they use digital twin models for tunnel geological environment and the data is used to represent the multi-feature geological environment.

There are also some design modeling for digital twins. In [13], the authors propose an integrated framework for the management of digital twin via Petri net. It is inclined to use IoT modeling for digital twins. Robert et al. [23] present a model-based data integration for product management based on digital twins. Rosario et al. [11] propose a cognitive digital twin for maintenance management, which uses an ontology approach to develop digital twin.

Despite these works in digital twins have considerable effect on diverse application scenarios. However, they partly solve the design or modeling issues of digital twin, it lacks a generic model for leading the design of digital twin applications.

3 EIMDC Model for Digital Twin Applications

In this section, we propose an EIMDC model for design digital twins to conquer the design challenges and provide a design methodology for digital twins. In our proposed model, we specify the design of digital twins as five critical parts. They are comprised of entity, infrastructure, model, data and context. As the Fig. 1 shown, first, to design the digitial twin applications, the entities in the physical space should be carefully investgated for digitalization. Then, we need powerfully digital infrastructure for transforming the entities into cyberspace. Furthermore, the data and model are used to describe the components in digital twin applications. Finally, the context should be clearly designed for these applications based on data and model given in cyberspace.

Fig. 1.

The EIMDC model for the design of digital twin applications. The entity denotes the physical entity in the real world for digitalization. The infrastructure represents the computing infrastructure for sensing the physical entity and transforming them into data and model. The data demonstrates the data in cyberspace sensing from physical space. The model illustrates the physical model, mechanism model and data-driven model. The data and model are effectively fused into the context for presenting the value of digital twins.

3.1 Entity

The entities refer to all the physical objects needed to be trivialization in the physical space. The physical objects are usually divided into three types:

• Nature Objects. All these objects are natural existence or created by nature in physical space. For example, water, rock, mountain, etc, which can be seen as the most common physical objects in real world. These objects can be as the sensing object for digitalization or as the background objects for real context.

• Artifacts. They depict the human-created objects to be used for transforming nature. They can be products for daily life using, such as chairs, desks, ladders. Also they can be machines made by humans, such as electronic terminals, computer number control, mounters, etc.

• Physical Phenomenon. They typically are the nature phenomenon, like weather, heating power, etc. In digital world, we can specify the raining, snowing or temperature objects for fully reflecting the physical world.

To design the digital twin applications, all the entities of physical space should be firstly listed in detail. Then we should confirm their locations in the physical space. A three-dimensional reference system is usually used to depict the locations of these physical entites, which are also exactly represented in cyberspace via 3D modeling technologies.

3.2 Infrastructure

The infrastructure is the digital infrastructure to transform the physical entities into the digital objects of cyberspace. Typical infrastructures are sensor, actuators, communications, IoT platform, Bigdata platform, AI platform,3D and visualization platform. Multiple digital technologies are utilized to be integrated to enable the digitalization of physical entities.

Sensors and Actuators. To sensing and controlling the physical entities in physical space, sensors and actuators are employed for collecting and catting on the data from physical entities. For example, in industrial domain, we need to use sensors to sense and to manage the humans, machines, materials, technical and environments in digital twin applications. For management of persons, video sensors can be used to gain the status data of workers, and to guarantee to execute the standard for manufactures. The status data of machine running in industrial production line is precisely measured by vibration sensors, electronic sensors, pressure sensors. The data of materials is managed via PLM and MES, it can also be recognized using the video sensors. Also the data of technics is gain from MES for data processing. Regard to the environment, for example, temperature sensors and humidity sensors can be used for getting data of environment monitoring. Otherwise, usually in factory, PLC technologies are employed to realize the operation on industrial equipments, further controlling the machines running in factory based on the sensing data.

Communications. To transmit the data sensing from physical entities by sensors, a suitable communication technology should be chosen according to the scenarios of digital twin applications. For meeting the their demands, a ultra-low latency communication is required due that digital twins need real-time interaction between cyberspace and physical space. Usually we choose the communication protocol according to the context of digital twins. In relatively easy wiring environment, a wired communication solution can be used, a proper transmission method is optical fiber with high bandwidth. Another emerging solution can use high-speed power line communication (HPLC), which transmits the data through power line carrier. The main advantages of HPLC are that it makes good use of existing power infrastructure and tailors the requirements of anti-interference for complex electromagnetic environment. For complex and rugged environment, wireless communication technologies can be selected for guaranteeing the transmission of sensors or terminals data. Many available wireless communications include NB-IOT, Lora, WiFI, Bluetooth, 4G/5G. For little data collecting scenarios, we can choose Lora for transmission of digital twin data at restricted range. In more complicated environment, 5G can be considered as an appropriate solution for sending twin data under the context of a large scale twin data transmission due to its characteristics with ultra-high speed bandwidth and ultra high reliability and low delay communication.

IoT Platform. IoT platform aims to solve the management issues of collecting terminals for twin data. This digital infrastructure supports shelding various transmission protocol at bottom and provides a shipment of twin data for upper big data platform. Also it can manage the terminal devices to meet the requirements of heterogeneous device access. Many existing IoT platforms can be used as an IoT digital infrastructure for supporting digital twin applications, such as Thingswrox [9], Microsoft Azure IoT Suite [5], IBM Watson IoT [3], AWS IoT [1], Saleforce IoT Cloud [7], GE Predix [2], etc. The IoT platform paves a way for twin data gathering.

Big Data Platform. Big data platform as an infrastructure is used for twin data store and processing. To meet the real-time computing demands of digital twin applications, a real-time data warehouse is needed to be built based on big data platform. The data warehouse are divided into ODS (Operation Data Store) layer, DWD (Data Warehouse Detail) layer, DWS (Data Warehouse Service) layer and ADS (Application Data Store) layer. Data can be real-time synchronization in ODS layer, and subsequently data is cleaned for removing null and empty values in DWD layer, then forming a theme wide table in DWS layer, finally all the resulted data are moved to ADS layer for invoking. Most of current data warehouse are built via Hadoop-enabled clusters, we can use Kudu as a real-time storage for twin data, and for data computing, Flink can be considered as a computing engine to twin data analysis. To seek a lower delay, we can use Flink CDC to synchronize the data to offer for Kafka, which can be used as a real-time data warehouse from ODS layer to DWS layer. The resulted data is self-synchronizing to ADS layer for 3D and visualization platform.

AI Platform. AI platform offers a deep insight based on the data from big data platform. There are multiple AI platforms for selection, we need to leverage MLOps [20] to achieve the life cycle management of machine learning, which should support diverse AI frameworks (Tensorflow, Pytorch, Mxnet, etc.) and integrate model training, model verification, model serving. AI platform is the basis when we design a data-driven model in the digital twins. Typical AI platform built tools, such as MLflow [6] and Kubeflow [4], can be employed to enable the construction.

3D and Visualization Platform. 3D and Visualization Platform provides the presentations of digital twin applications. It should offer the following functions:

• 3D Scenarios Management. It provides 3D virtual scene library editing function, and supports geometric transformation and editing of various scenes, and provides application templates of domain scenes;

• Scenario Script Management. It supports custom scene action, and meanwhile supports to achieve scene animation control through custom programming;

• 3D Model Management. It supports 3D modeling file management and can enable common CAD file import and edit;

• Mechanism Model Management. It supports the import of mechanism library files of related industries and docking with scenarios;

• Authority Management. It supports scene digital twin user system control and privacy protection;

• Data Source Management. It supports the management of a variety of data sources, including commonly used data relational database access, API access, CSV/Excel import, support data sources and scene model binding.

For realizing the requirements, existing solutions can be partitioned into two-folds: one is evolution from game visualization engine. For example, Unity for 3D, Unreal4. Another is WebGL based technologies, which achieve binding between OpenGL and Javascript. The major shortcomings of game visualization engine are that they must install a heavily client for running the applications. The advantage for WebGL based framework is that it can be run in any browser enabled computers without extra clients. However, it requires the display card to accelerate the 3D rendering within browser. We need to choose the framwork according to the context requirement. The design considerations include deployment cost, access delay, quality requirement, etc.

3.3 Data

All the physical entities in physical space are mapped into the twin data of cyberspace. When we design a data solution for digital twin applications, we should focus on managing two types of twin data: geometric data and physical sensing data from real world.

For gemetric data, the physical entites can be designed and modeling data is generated by 3D modeling software, like Maya, 3DMax, Blender, etc. The output of gemetric model data file can be .obj, .fbx, .3dx.

Regarding to physical sensing data, they may come from IoT platform and legacy systems. The IoT platform collects the sensing data from physical entities saved into relational database. The legacy systems can be the existing management systems in an organization. We need to extract them into big data platform and meanwhile examine their meta data to form a data directory and dictionary. This is ready for built the data warehouse.

3.4 Model

Model is a critical part in digital twin applications, which provides an approximate representation for physical space. The model can includes three-fold when designing a digital twin model.

Geometric and Physical Model. In cyberspace, geometric model given in Sect. 3.3 plays a role in simulating the multi-level geometrical shape of physical entities. The locations of geometric model are assigned according to actual physical positions. The physical model aims to simulate the physical low, such as motion, ray of light, collision. In many 3D platforms, like Unity, WebGL, etc, they can provide the encapsulation for these physical model. When we need to design a physical model for a digital twin application, we can expediently invoke the available interfaces of in-built physical engines of these platforms.

Mechanism Model. Mechanism model uses logics and rules to depict the relationships among digital entities in cyberspace. Physical model is essentially a machinist model closely related to domain and industry. For industrial domain, these mechanism model can be developed via domain-specific industrial software. For instance, in electronic industry, matlab is usually used to simulate the mechanism of the power grid. These machinists are enabled via math equations (state equation, differential equation, etc). Each domain in fact may have its mechansim simulation software. Dynamic mechanism model for a centrifugal machine is built via Fluent. To design the mechanism model, the interface to integrate these simulation software should be offered with interactions 3D and visualization platform.

Data-Driven Model. The data-driven model is constructed based on collecting data from physical space due to unknown domain mechanisms. For actual applications, we leverage the AI technologies to mine the knowledges from twin data. The data-driven model is quite fit for the scenarios of prediction and diagnosis for industrial domain. When we devise a fault diagnosis model for equipment in digital twins, we can use decision tree based learning approach to achieve the diagnosis based on the sensing data for the equipment and fault data. Also a predictive maintenance model for equipments can be gain by deep neural network training, which can be developed and served through AI platform.

3.5 Context

For digital twin applications, the data and model will be fused into the context to provide the application value. Actually, the physical space is a continuous space, however, the cyberspace is a discrete space which is comprised of diverse contexts. The context in essence is an abstraction for physical space. In the digital twins, we can get a global perspective for data compared with really physical space. The context refers to three key elements which are humans, machines and things.

Humans. Regarding to humans, they to some extent denote the persons participating into the activities under the specific context in physical space. The digitalization of human activities are clearly depicted in digital twin applications. We should carefully consider the activities when designing a context of digital twins.

Business Flow. Business flows are the running behaviors which are simulated for real world in the context according to the built model and data.

Presentation. The business flow and activities are presented by using 3D and visualization based on data-driven pattern, and their data generally originate from diverse sensing systems including sensors and IoT platform for different contexts.

4 Case Study

In this section, we use a Surface Mounted Technology (SMT) production line of power terminal as a case to show the design of digital twins. For the electronic manufacture, SMT is the most popular technology and process in electronic assembly industry in which a pin or short lead surface assembly component is installed on the Printed Circuit Board (PCB) surface or other substrate surface and is welded and assembled by reflow welding or immersion welding. To monitor the SMT production line, we need to realize the operation of SMT via digital twin technologies.

We specify the design process according to our proposed design model as entity, infrastructure, data, model and context.

4.1 Entity

For the first step of design, we should clearly give all the physical entities in SMT production lines, where the production equipments and products are the critical entities for SMT digital twin. They are summarized as follows:

PCB. PCB is a series of entities for SMT digital twin. It is composed of an insulating base plate, connecting wires and a pad for assembling welded electronic components. It has the dual functions of conducting circuit and insulating base plate. It can replace the complex wiring, realize the electrical connection between the components in the circuit, not only simplifies the assembly of electronic products, welding work, reduce the wiring workload under the traditional way, greatly reduce the labor intensity of workers. And it can reduce the volume of the whole machine, reduce the cost of products, improve the quality and reliability of electronic equipment.

Automatic Unloader. The automatic unloader is used to unload the PCB into the production line, it is a device for improving the efficiency of PCB shipment. Laser Engraving Machine. Laser Engraving Machine aims to engrave text or images in the PCB. Adium carving refers to laser engraving, is through the laser beam of light which can lead to the chemical and physical changes of the surface material and engraved traces, or through the light can burn part of the material, show the required etched graphics, text. According to the different carving methods, they can be divided into dot matrix carving and vector cutting.

Solder Paste Printer. Solder Paste Printer is responsive for finishing the solder printing of the PCB. The PCB is fixed on the printing positioning table first, and then the solder paste or red glue is printed on the corresponding pad through the steel net by the left and right squeegee of the printing press.

Solder Paste Inspection. Solder Paste Inspection tester is a kind of SMT Inspection equipment that calculates the Solder Paste height printed on PCB by triangulation based on the principle of optics.

Chip Mounter. Chip Mounter is a device that accurately places surface mount components on PCB pads by moving mount head.

Smooth Switching Machine. Smooth switching machine is a kind of accessories in industrial automation production line which is mainly used for the transfer function of tooling board.

Reflow Soldering Oven. The reflow soldering process is the soft soldering of mechanical and electrical connections between the solder ends or pins of the surface assembly component and the PCB pad by remelting the paste soft soldering material preassigned to the PCB pad.

Coveryor. The Coveryor is to realize the shipment between chip mounter and reflow soldering oven.

AOI Inspection Machine. AOI inspection machine through high-definition CCD camera automatically scans PCB products, collects images, test points and qualified parameters in the database are compared. After image processing, it checks out the defects on the target products, and displays/marks the defects through the display or automatic signs for maintenance personnel to repair and SMT engineering personnel to improve the process

Automatic Loader. Automatic loader is allocated into the end positions of SMT process and to be used for collecting the completed boards.

4.2 Infrastructure

In our case, there are 4 SMT production lines, all the data about production lines are saved into MES. Also we use 8 electronic terminals(sensors and processors) to collect the energy data, where 4 terminals to be used for reflow soldering oven and 4 terminals for the total energy consumption of each production line. To enable the digitalization of physical entities of SMT. For the communications, a HPLC is employed for energy data transmission of production lines, and a 4G transmission for reflow soldering oven.

IoT platform is enabled via an opensource platform, ThingsBoard [8]. The collecting data is transmitted to this platform, and it provides the data sources for big data platform. The big data platform is built based on Hadoop clusters, it includes 8 cluster nodes and extracts the data from MES and energy database. An off-line data warehouse is built based on dophinschuduler and Hive, and a real-time data warehouse is built by Flink and Kafka, which are stored into the ADS table for 3D and visualization platform.

We use a WebGL enabled framework [10] for 3D and visulization platform. The Javascript script is utilized to simulate the behavior of production lines.

4.3 Data

The data of presentation in SMT digital twin includes energy data and MES data from equipment of production lines. The meta data items of energy data contain active energy, reactive power energy, active demand, total apparent power calculated by minute. The data items of production line are good product quantity, yield, energy consumption per unit yield, total energy consumption for production line, today maximum power load. Other data is fault data for each prodution line, which records the device state in MES.

Fig. 2.

The SMT production line.

Fig. 3.

The energy monitoring of reflow soldering.

4.4 Model

For the gemetric model of each prodution line, we construct the 3D model with Maya to generate the equipment model with .obj and .mtl file format. These equipment models are imported into the 3D and visualization platform for edit. As Fig. 2 shown, the locations of 3D SMT production line are assigned according to the real production lines.

The mechansim model about production line is to calculate the energy consumption per item. Furthermore we enable the production of energy consumption per item by day, which is illustrate in Fig. 2.

Fig. 4.

The production information of SMT.

Fig. 5.

The fault warning of the equipments.

4.5 Context

The context of digital twin in SMT production line, refers to the operation persons of the equipment of the production line. For the business flow, they can achieve the virtual inspection for the production line. Moreover, we can monitor the SMT production information in digital twin context as given in Fig. 4, they present the energy information and yield information for each production line. Otherwise, in Fig. 5, when a production line equipment occurs a fault, they can be monitored and real-time warning in the digital twin environment.

5 Conclusions

In this paper, we present a novel model for designing digital twin applications which refers to five core parts: entity, infrastructure, data, model and context. Each element can represent an abstraction for enabling the digital twins. The design firstly requires to define the physical entities for digital twin applications, then the key infrastructures should be built to support the digitalization of these physical entities. The digitalization is exhibited via data and model in cyberspace, and finally they realize the fusion to the context for show their value of applications.

Regarding to the future, we should focus on more researches on the topics about the digital evaluation mechansims with the change of objects for physical space.