Keywords

1 Introduction

In the age of digitalization and connectivity, Industry 4.0 has dramatically revolutionized manufacturing and production through the employment of cyber-physical systems (CPS), the Internet of Things (IoT), and digital twins. The confluence of these technologies has paved the way for the burgeoning field of the Internet of Robotic Things (IoRT). However, despite substantial advancements, there are still hurdles to seamlessly integrating these constituents into a holistic system. The present study aims to bridge this gap by delving into the application of a CPS and a digital twin for the governance of a three-degree-of-freedom (3DoF) robotic manipulator.

Our investigative approach adopts a comprehensive methodology, beginning with the mathematical modeling of the kinematics of the robotic manipulator, followed by simulation to anticipate system behaviour across various scenarios in conjunction with designing its digital twin. Ultimately, we instituted and assessed the system remotely in a regulated environment. This method enabled us to gauge the efficacy of the system whilst identifying and resolving potential issues.

The insights garnered from this research underline that integrating the CPS and digital twin in steering the robotic manipulator yields significant enhancements for tool users, whether they are local or remote. This conclusion reinforces the viability of our proposition and accentuates the potential of these technologies in the progression of Industry 4.0.

We surmise that the successful deployment of our proposal represents a notable advance in support for Industry 4.0. The findings furnish a robust foundation for subsequent investigations, facilitating the exploration of more intricate and scalable applications of these concepts, thus bringing an innovative perspective to robot control in an IoRT setting.

The paper is structured with Sect. 2 dedicated to a literature review, exploring the various global implications of IoRT. This section concludes with a brief bibliometric study which underscores the value of fostering these research themes in Latin America. Section 3 highlights the methodology employed, encompassing the kinematic analysis performed on the robot, its simulation, and ultimately the implementation of the system. Section 4 examines the results garnered from the research conducted, and Sect. 5 articulates the conclusions drawn.

2 Literature Review

The concept of the Internet of Robotic Things (IoRT) has been coined in recent years as an extension of Internet of Things (IoT) systems. Recent work [17] has established that this new concept seeks to extend the use of intelligent things in different fields. One major challenge of this emerging line of research is the development of algorithms which allow the execution of “intelligent” actions through the information the devices obtain from the network in which they operate. It is also essential to define that terminology, such as cyber-physical systems [11] and [9] It gains more preponderance by providing these agents with the particularity that their interaction would not only be at the physical level of inputs and outputs but also has a facet in the world, which would enhance the decision-making capacity that could be provided in networks of agents that exchange information among themselves.

According to [20], Internet of Things (IoT) networks enable technology elements for the Internet of Robotic Things (IoRT). These networks make it possible to leverage more complex control structures, which consider the interaction of various actors in the system. This implies that these heterogeneous and distributed networks’ interaction can lead to significant performance improvement. In robotics, the unique characteristics and inherent capabilities of each component of these networks can help physically simplify on-board robots. [4], while simultaneously enriching their characteristics through the information provided by each agent. This includes sensory data, control commands, status updates, and real-time feedback. Implementing IoT networks in IoRT enables robots to operate in more intelligent and adaptive environments [8].

Considering the anteroom proposed in the literature and according to the definition by [14] and [1], the structure of an IoRT-type system can be stratified according to layers, as shown in Fig. 1. Under this consideration, a scheme will be proposed that allows the interaction of the system at the aforementioned levels along with its subsequent physical implementation for behavioral testing, providing the possibility of operation for non-remote users.

Fig. 1.
figure 1

Layers in a system that is described as IoRT, [14]

Full size image

This collection of elements thus defines the existence of three interconnected layers in the application of IoRT. The first is the physical layer, which refers to the robotic agent and its unique characteristics. The second layer is a combined network and control layer in which both elements require each other in a continuous symbiosis, with the network acting as a hub for interaction and control actions due to the interaction of devices exchanging information. Here, devices are understood not only as robots but also as sensors that provide additional information to the system. Finally, the third layer is a services and applications layer, where robots’ interactions with the environment can be observed.

These elements define the existence of three interconnected layers on the Internet of Robotics. This conceptual framework creates macro-networks for information exchange, a concept coined as the Web of Robotics Things (WRT) [13], which seeks the integration of architectures that allow real-world elements to become part of the World Wide Web.

In this study, multiple application areas are suggested, taking advantage of distributed networks that provide robots with a greater degree of autonomy and allow them to interact with their environment and with each other in more sophisticated ways. This can improve efficiency and open up new possibilities in diverse applications, from manufacturing and logistics to healthcare and agriculture. For example, [12] applied this concept to solutions in agriculture, while the work of [18] applies it to autonomous vehicles for the deployment of wireless sensor networks. Additionally, in the medical field, [5] eds a system for dispensing drugs through a web-based system. In all cases, the robots can adapt and respond to unexpected environmental changes, providing unprecedented resilience and flexibility in automation operations.

Considering the above, there is an inherent integration between the agents that form an IoRT-type network and the system in which they operate. This integration is an essential condition for these agents’ existence since, as shown in the layer model, a formal relationship is established between the network layer and the application layer. The latter hosts the services that these systems can provide. This is examined in depth in the work of [19], where a model of interaction between the elements that would comprise a system with these characteristics is proposed, emphasizing the importance of this interrelation for the effective functioning of the IoRT system.

Up to this point, the IoRT strategies are vast and multiple applications can be observed; however, one underexplored facet is their linking with elements of recreation of robotic actions. According to the literature, the concept of Digital is an essential point regarding Industry 4.0. Based on the points from [15] and [16] these and other technologies, such as artificial intelligence in robots [2], are elements which are transformed into means for the growth of Industry 4.0. Finally, other technologies that allow the development of digital elements are virtual and augmented reality. [7] indicated that this demonstrated advances for the manufacturing industry from the perspective of multi-level environment visualization, highlighting the linkage between the virtual and cyber worlds.

In the literature analysis, [3] provides some applications that cross the use of digital with robots [21]. It is desired to improve efficiency and energy awareness in smart manufacturing through crucial solutions such as real-time monitoring or simulation techniques. For this reason, it is interesting to conduct a study on the bibliographic references that involve the topics addressed in this work. We can note this in Fig. 2, which presents the relationships between the fields of IoT and systems; however, the robot concept is isolated. This indicates that according to the WoS database with the search criteria “TC = Internet of Robotics things* AND Digital Twins*) AND WC = (Industry 4.0)” of the keywords used, the IoRT concept is a space that allows research on the mutual interaction of its constituent elements.

Fig. 2.
figure 2

Graph of studied topics according to WoS database.

Full size image

Most importantly, these studies benefit from a dense global collaboration network, as illustrated in Fig. 3. This graph highlights the limited interaction between Latin America and the significant powers working in these critical areas to develop innovative solutions. There is an opportunity of significant interest here to promote and strengthen the interconnection of Latin American research with these global innovation centers. This interconnection could foster the development of solutions adapted to the specific needs and characteristics of the region, while contributing to the diversity and richness of the global body of knowledge in these essential areas.

Fig. 3.
figure 3

World map of collaboration according to WoS database.

Full size image

In response to the challenges presented, and in line with our research objectives, we present a significant and novel contribution to the Internet of Robotic Things (IoRT) field. Our proposal focuses on integrating three degrees of freedom (3DoF) robots that can be remotely operated and visualized through a digital twin which replicates its movements, thus coordinating with its physical performance.

While there is abundant literature defining concepts and theorizing about IoRT-type systems, there is little research that presents practical solutions to the challenges and effectively integrates the different layers of the analysis framework. This is what makes our work stand out, since we are not only developing theories and models but also carrying out a concrete and actual application that will demonstrate their value in real situations. This practical and application-focused approach can catalyze future developments in this field, thus contributing to the progress of Industry 4.0.

3 Methods

In the development of this research, a strategy based on three fundamental stages (modeling, simulation, and implementation) is adopted to integrate the constituent elements of a robotic agent in an IoRT environment.

It starts with modeling, where a mathematical and physical model of the robotic agent is built, covering all fundamental characteristics and behaviors. An equivalent digital twin is also generated to represent the robotic agent in virtual space. The second stage, simulation, uses the model to predict system behavior in various situations. This process is essential to identify and resolve potential problems before real-world implementation.

Finally, in the implementation stage, the system is applied to the robotic manipulator in a controlled environment. During this phase, the behavior of the system is monitored, and data is collected for further analysis.

The experiment was designed to evaluate the effectiveness of integrating the cyber-physical system and the digital twin in managing the robotic manipulator. Through the tests performed, data were collected to evaluate system performance and identify improvement opportunities.

3.1 Modeling and Simulation

Using the Denavit-Hartenberg (DH) algorithm [10] a kinematic model was built for the robotic manipulator shown in the image provided. This algorithm is a standard method in robotics to represent manipulator kinematics. The canonical form of the matrix resulting from the application of the DH algorithm is presented in the Eq. (1), where: \({\boldsymbol{R}}_{z,\theta } \) represents the rotation matrix, \(\boldsymbol{q} \) coordinate vector of the form\(\begin{bmatrix}x_i&y_i&z_i\end{bmatrix}^{T} \) and n the link to be analyzed.

$$\begin{aligned} \begin{array}{@{}l}T_0^{n}=\begin{bmatrix}{\boldsymbol{R}}_{z,\theta }&{}\boldsymbol{q}\\ 0^{T}&{}1\end{bmatrix}\end{array} \end{aligned}$$
(1)

Based on the above, the manipulator under study is described through the following set of matrices evaluated for each link. The kinematics obtained can be seen in Eqs. (2), (3) and (4), where \(c\theta _i \) represents the function \(\cos \theta _i \) and \(s\theta _i \) shall be the function \(\sin \theta _i \). The indicated models allow us to determine the positions of the links, and use simulations to determine the working volume of the robot, as shown in the following figure Fig. 4.

$$\begin{aligned} \begin{array}{@{}l}{\boldsymbol{T}}_1=\begin{pmatrix}c\theta _1&{}0&{}s\theta _2&{}0\\ s\theta _1&{}0&{}-c\theta _2&{}0\\ 0&{}1&{}0&{}l_1\\ 0&{}0&{}0&{}1\end{pmatrix}\end{array} \end{aligned}$$
(2)
$$\begin{aligned} \begin{array}{@{}l}{\boldsymbol{T}}_2=\begin{pmatrix}c\theta _1c\theta _2&{}-c\theta _1s\theta _2&{}s\theta _1&{}l_2c\theta _1c\theta _2\\ c\theta _2s\theta _1&{}-s\theta _1s\theta _2&{}-c\theta _2&{}l_2c\theta _2s\theta _1\\ s\theta _2&{}c\theta _1&{}0&{}l_1+l_2s\theta _1\\ 0&{}0&{}0&{}1\end{pmatrix}\end{array} \end{aligned}$$
(3)
$$\begin{aligned} \begin{array}{@{}l}{\boldsymbol{T}}_3=\begin{pmatrix}c(\theta _2+\theta _3)c\theta _1&{}-s(\theta _2+\theta _3)c\theta _1&{}s\theta _1&{}c\theta _1(l_3c(\theta _2+\theta _3)+l_2c\theta _2)\\ c(\theta _2+\theta _3)s\theta _1&{}-s(\theta _2+\theta _3)s\theta _1&{}-c\theta _1&{}s\theta _1(l_3c(\theta _2+\theta _3)+l_2c\theta _2)\\ s\left( \theta _2+\theta _3\right) &{}c\left( \theta _2+\theta _3\right) &{}0&{}l_1+l_2s(\theta _2+\theta _3)+l_2s\theta _2\\ 0&{}0&{}0&{}1\end{pmatrix}\end{array} \end{aligned}$$
(4)
Fig. 4.
figure 4

Behavior of the robotic manipulator in different displacement positions.

Full size image

The importance of this graph lies in the determination and understanding of the singularities of the system, thus avoiding the generation of mechanical stresses when the system reaches maximum angular points. For this case, we use the Jacobian matrix, described by \(\begin{array}{l}\begin{bmatrix}\dot{x}\\ \dot{y}\\ \dot{z}\\ \omega _x\\ \omega _y\\ \omega _z\end{bmatrix}={\boldsymbol{J}}_{6xn}\begin{bmatrix}\dot{{\boldsymbol{q}}_1}\\ \dot{{\boldsymbol{q}}_2}\\ \vdots \\ \dot{{\boldsymbol{q}}_n}\end{bmatrix}\\ \end{array} \), where \({\dot{\boldsymbol{q}}}_n \) represents the linear and angular velocities of each link. On the other hand, n is the number of links which each equipment unit has. In order to clarify the model explanation, a rotational type link can be described as shown in Eq. (5).

$$\begin{aligned} \begin{array}{@{}l}{\boldsymbol{J}}=\begin{bmatrix}{\boldsymbol{J}}_v\\ {\boldsymbol{J}}_\omega \end{bmatrix}=\begin{bmatrix}{\boldsymbol{R}}_{i-1}^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\times \left( {\boldsymbol{q}}_n^{0}-q_{i-1}^{0}\right) \\ {\boldsymbol{R}}_{i-1}^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\end{bmatrix}\end{array} \end{aligned}$$
(5)

From the above, we can conclude that the angular values taken by \(\theta _2 \) y \(\theta _3 \) are critical to the system according to the determinant of \(det(J_{6x3}) \). This behavior can be better appreciated when the aforementioned angles take the same value for the evaluated case \(180^\circ \). This situation appears in Fig. 5, and this behavior is essential during system implementation to avoid equipment damage. Generalization of the associated elements in the particular case under analysis appears in Eq. (6).

$$\begin{aligned} \begin{array}{@{}l}{\boldsymbol{J}}=\begin{bmatrix}{\boldsymbol{J}}_v\\ {\boldsymbol{J}}_\omega \end{bmatrix}=\begin{bmatrix}{\boldsymbol{R}}_0^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\times \left( {\boldsymbol{q}}_3^{0}-{\boldsymbol{q}}_0^{0}\right) &{}{\boldsymbol{R}}_1^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\times \left( {\boldsymbol{q}}_3^{0}-{\boldsymbol{q}}_1^{0}\right) &{}{\boldsymbol{R}}_2^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\times \left( {\boldsymbol{q}}_3^{0}-{\boldsymbol{q}}_2^{0}\right) \\ {\boldsymbol{R}}_0^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}&{}{\boldsymbol{R}}_1^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}&{}{\boldsymbol{R}}_2^{0}\begin{bmatrix}0\\ 0\\ 1\end{bmatrix}\end{bmatrix}\end{array} \end{aligned}$$
(6)
Fig. 5.
figure 5

Singularities when the angular values of links 2 and 3 are equal.

Full size image

3.2 Implementation

The digital twin is created using Unreal Engine [6] because it establishes communication with hardware through a serial communication plugin. For its precise design, the range of movement is established based on the analysis obtained from the simulation to emulate the behavior of the physical object. The environmental visualization appears in Fig. 6.

Fig. 6.
figure 6

3 DoF digital robot model.

Full size image

The system integration follows a layered architecture proposed by [1] and [19]. The application of the different elements comprising the model strata has been developed, as shown in Fig. 7. This application of the model includes two supervision elements based on a virtual twin. One operates parallel to the robot at one location, while the other can be observed and controlled remotely. This action allows remote communication and control, and observation of the device. Data is stored in a database via direct communication between the cyber-physical device (NodeMCU) and the database.

Fig. 7.
figure 7

Proposal for applying an IoRT system according to what is established in the literature.

Full size image

4 Results and Discussion

Our main findings, their novelty and our contribution to existing knowledge is summarized below:

  • Design of the digital twin in Unreal Engine: Unreal Engine was used as the design environment for the digital twin due to its characteristics and features to establish communication with hardware through a serial communication plugin. This choice provided a robust and versatile platform for developing the digital twin.

  • The motion ranges and behavior of the digital twin: The motion ranges of the digital twin were established based on the analysis and simulations performed. It was possible to emulate the behavior of the physical element accurately, which allowed us to have a digital twin that accurately represented the natural behavior.

  • Layer architecture and supervision elements: System integration followed a layered architecture proposed in the literature. Considering the models proposed by the literature, the necessary elements for implementing the system were added. Two supervision elements based on the virtual twin structure were developed.

The application possibilities of the system in more demanding environments will depend on scalability. However, integrating multiple tools is an advance that could be enhanced in industrial scenarios.

5 Conclusion

We demonstrated the potential of integrating cyber-physical systems and digital twins to control robotic manipulators. The developed digital twins and the implemented layered architecture provide a solid basis for managing and optimizing automated systems in the context of Industry 4.0. The results contribute to advancing and understanding the interconnection between cyber-physical systems, digital twins, and robotics, laying a foundation for future developments and applications in the field. As a future work, we propose the integration of industrial robots and creation of control algorithms that perform better in the physical element and with its digital twin.