1 Introduction

Telerobotics is a new field of robotics, which has attracted much researchers attention in recent years [1,2,3]. The term telerobotic Internet refers to a remotely controlled robot system surveyed through the Internet. The growing interest in this area is stimulated by promoting the Internet, which provides access to various computing resources virtually anywhere in the world. The increasing demand of the Internet as a means of communication for telerobotic system is that it also uses a standard communication protocol; and the physical support is available for the application of telerobotics, eliminating the need to develop an exclusive and expensive communication system. The internet does not provide a guaranteed quality of service (QoS); it gives a number of limitations and difficulties, such as bandwidth constraints, transmission delay, packet loss, connection loss, etc. the above situation influences the performance of Telerobotic systems based on internet [4,5,6], which is a new field in the last decade [7].

There are two generations of internet controlled robots. The first generation was mainly based on a simple robotic arms or mobile robots controlled by human operators [8, 9]. These systems operate in a structured environment with little uncertainty and a lack of local intelligence. Mercury [8] and the Australian Telerobot [10] were the forerunners of real implementations of such systems. Since then, dozens of these systems have been put online by research teams worldwide [5, 6, 9, 11, 12]. The second generation of internet controlled robots focuses on autonomous mobile robots operating in dynamic and uncertain environments [13]. The main characteristics of this generation are that the robots are autonomous and reactive allowing them to navigate and cope with uncertainty in the real world. The objective in building this generation of internet controlled robots is achieving remote control and monitoring [14]. Internet based teleoperation of complex systems generates more problems such as system unavailability, inaccuracy and instability. This is due to the quality of service on the unsecure internet and especially the unstable delay from “end to end” [13, 15].

The generic architecture of telerobotics is the same in most applications. The main problem is to reduce the influence of the RTT fluctuations. The impact of QoS is obvious when controlling a mobile robot via the internet network [5]: when we send commands to the mobile robot to start the movement followed by a stop request, it will result in further displacement \({\varvec{\Delta }} \mathbf{d}= ({\varvec{\Delta }} \mathbf{x}, {\varvec{\Delta }} \mathbf{y}\)) which is proportional to the RTT fluctuation as shown in Fig. 1. When performing the desired path (blue path AG), we’ll have an accumulation of all these errors and the robot reaches another goal G\(^{\prime }\) instead of the initial goal G (red path AG\(^{\prime }\)).

2 Related works

Remote operation of complex systems using the Internet as a communication channel generates more problems such as system downtime, vagueness and instability. This is due to the quality of service on the unsecured internet and especially the volatile end to end delay [13, 15, 16]. To solve this problem, and Bambang Riyanto T. developed a client-server architecture based on VIRTUAL REALITY to remote control a robot manipulator with five degrees of freedom, called Mentor [11]. The majority of these researches use a surveillance control system which enables the operator to issue high-level commands. As the internet delay time, the design of direct control system that allows the user to control the continuous movements of robots may not be easy. Such control systems have been proposed, but are not sufficient to mitigate the impact of delay on the Internet. To overcome the delay problem, Three approaches [6] were often used: the aiding predictive approach, the Simulating, display and planning approach [13, 17,18,19,20], and the event-based approach [21, 22]. We find also other approaches; “Move and Wait” strategy [4], which is typical for space robots and was first applied to the long-distance communication. But this solution takes a lot of time to move the robot to the target because the operator must stop the robot every time before moving on to the next task, this is why various solutions have been proposed, and we are interested in this article strategy (speed Limit and Delay Module approximator) [4] and GEMMA-Q [23] and [24], these solutions are used to control the speed of a mobile robot according to the quality of service so that the robot reaches the desired position. However, it is not easy to find a uniform SLM for all users.

Our objective is on the field of mobile robot teleoperation. In this area, some telerobotic systems on the Internet have used the direct control [25,26,27]. Obviously, direct control is not suitable for mobile robots controlled by internet because of the limitation of bandwidth and delays during the long distance transmission. To overcome these problems, we proposed two strategies [28, 29] to remote control a mobile robot (Pioneer P3AT) [30] in a network without guaranteed quality of service.

In the first strategy [29], we have significantly improved our (client-server) architecture system and human-robot interface. The client application contains a Java applet, visual feedback and a 3D application to simulate the behavior of our system (robot Pioneer mobile P3at) and the server application contains an architecture that allows us to predict in real time the (RTT) using the HOLT WINTERS method [29, 31, 32] to obtain a good assessment of the quality of service (QoS) which helps control the robot safely and accurately. In the second strategy (THE TOMPORESATION OF THE COMMANDS) [28], we used the same client interface, but the application server contains an architecture, which permits us to delay the commands sent to the robot with a known and constant delay to avoid random variations in RTT. In this article, we proposed two strategies to remotely control the mobile robot Pioneer, and we compared them with another strategy in order to examine and address their advantages and disadvantages.

This paper is organized as follows: in Sect. 3, we will mention the Architecture of a teleoperation system; then we will present three control strategies in Sects. 5, 6 and 7. Man-robot communication interface presented in Sect. 8 and the experimental results are discussed in Sect. 9; the summary of comparison is described in Sect. 10. Finally, conclusions and perspectives are presented in Sect. 11.

Fig. 1
figure 1

Path result with direct control

Full size image

3 Architecture of the teleoperation system

The generic tele-robotics architecture is the same in most applications. The main problem is how to reduce the influence of the variation of the delay on the Internet (RTT), the impact of QoS appears clearly on a mobile robot, because the command reaches the robot through the internet network and after a delay \({\varvec{\Delta }} \mathbf{t}\); the robot that moves at the speed V can travel a further distance \(\Delta \)d before receiving the command of turning or stopping as shown in (Fig. 1). We note that the errors concerning the waypoint stacks, the robot then does not reach the desired position.

4 Virtual reality and remote control

The main problems of the telerobotics are the time delay, bandwidth and the quality of the video signal. The technologies of Augmented and Virtual Reality (AR) (VR) can be used to prevent these problems. In [20, 33, 34], the authors used the RA to limit the effect of delay in an industrial application of telerobotics. Similarly, the RV was used in [35,36,37,38] showing that 3D maps can be more effective in a localization task, even if it takes longer to achieve. The robots used in [9, 17,18,19] are mobile robots. The benefits of virtual reality interface for remote operation of the authors are the ability to use visual aids, personalized display, attractive presentation of data and the redundancy of the screen data.

In telerobotics, the video is generally poor due to which the VR can help users get a better view and understanding of the system they want to control remotely. It can also be used to compensate for the delay induced by the networks.

Fig. 2
figure 2

Global view of the proposed architecture [29]

Full size image

In the next part of this article, we will examine three control strategies (Speed Limit Module, Improved speed limit module and the tomporesation of the commands) and meet their advantages and disadvantages.

5 Speed limit module (SLM)

The second strategy used to solve the problem of latency is the Speed Limit Module (SLM), this have been used in order to control a simple mobile robot using the idea of quality level GEMMA- Q SLM to design an SLM strategy, that is to say, researchers [4] have varied the speed of the robot according to the delay time measured at time tn (the current delay time), this solution is only effective when they have slight variations in delay time, so it’s difficult to find a uniform SLM for all users. For this problem, we have proposed two strategies to remotely control a mobile robot (Pioneer P3AT) in a network with no quality of service guaranteed.

6 Improved speed limit module (ISLM) [29]

Our proposal is a client-server architecture (Fig. 2) [29] through which the operator (user) can send commands and receives information from the remote system. Users from graphic simulator send commands (forward, backward, left and right) to the medial server (step 1). Then, they must take pre-treatments before sending commands to the robot (step 2); this step is the most important where we evaluate the Internet state (good, weak or Bad) (Table 1), this is done by measuring the (RTT) between the client and the medial server then storing these latters in a database so that we can extract them in order to predict the next RTT using the Holt-Winters method. The operations of measuring, storing and retrieving data occur every 2 seconds. After the prediction operation; we vary the speed of the robot according to the predicted RTT (step 3). Many experiments were conducted in the workplace in order to select the appropriate speed for each level. This step sets the operation of the robot according to four levels of QoS:

  • Level W1 Normal functioning with the possibility of safe-controlling the robot when predicted RTT is between 1 and 500 ms

  • Level W2 Slow motion functioning when predicted RTT is between 501 and 1300 ms

  • Level W3 Extra slow-motion functioning; in this case, it is hard to precisely control the robot trajectory. We will stop the robot movement if this phase lasts for a long time when predicted RTT is between 1301 and 2000 ms

  • Level W4 Immediate turning off of the robot when predicted RTT is superior to 2000 ms. As in the case of a communication loss between the client and the medial server or between the medial server and the robot; the robot will automatically stop.

6.1 Holt-Winters method

The Holt-Winters method is mostly favoured among the exponential smoothing techniques in the case of observation series which have secular trend and seasonality. They operate simultaneously the smoothing of three terms corresponding to the local level estimates of the seasonally adjusted series (\({L}_{t})\) the slope of the pattern (\({b}_{t})\), and the seasonality (\({S}_{t})\). We can mention at least two methods; one of them is suitable for series admitting a multiplicative decomposition and the other corresponding to the additive decomposition [31].

6.2 The multiplicative version of Holt-Winters method

Let us note‘s’ being the natural frequency of the series. The system of equations is given by [31]:

  • Smoothing of the level:

    $$\begin{aligned} L_t =\alpha \frac{x_t }{S_{t-s} }+(1-\alpha )(L_{t-1} +b_{t-1}) \end{aligned}$$
    (1)

  • Smoothing of the trend :

    $$\begin{aligned} b_t =\beta (L_t -L_{t-1})+(1-\beta )b_{t-1} \end{aligned}$$
    (2)

  • Smoothing of the seasonality:

    $$\begin{aligned} S_t =\gamma \frac{x_t }{L_t }+(1-\gamma )S_{t-s} \end{aligned}$$
    (3)

    where \({L}_{t}\) the smoothed level of the series in t; \({x}_{t}\) the observer value of the series in t; \({S}_{t}\) the seasonal coefficient in t; \({b}_{t}\) The estimated tendency in t.

The Holt-Winters model has the advantage of incorporating a seasonal component and thus achieves the calculation of the prediction in a single treatment. Three separate smoothing are performed:

  • Smoothing of the level with a smoothing parameter \(\alpha \in [0,1]\);

  • Smoothing of the trend with a smoothing parameter \(\beta \in [0,1]\);

  • Smoothing of the seasonality with a smoothing parameter \(\gamma \in [0,1]\);

To obtain a good result of prediction, it is necessary to carefully choose these coefficients.

6.3 Choice of smoothing parameters

To start the smoothing process, we must select a value for the constant \(\upalpha \). This choice is very important because it determines the future prediction through the degree of weight that is assigned to the recent past and the farthest one.

Various estimation procedures have been established. The most classic one is to retain the value of \(\upalpha \) that minimizes the difference between the prediction and the realization of the known part of the chronologic series. Another approach is to develop procedures for regulation and control of the constant change of smoothing. Thus, in case of systematic prediction divergence, \(\upalpha \) is automatically adjusted to fit this change in structure.

The most commonly used principle of the calculation technique is as follows:

For a given range of values of \(\upalpha \), ß and \(\upgamma \) with a “pace” close enough to (0.05), the predictions are simulated and the sum of squared error prediction is calculated. We retain the value of \(\upalpha \) which gives the minimum sum of squared deviations (SSD). This technique can be generalized to determine the three parameters \(\upalpha \), ß and \(\upgamma \). The retained value of the coefficient is that which corresponds to the minimum of the term:

$$\begin{aligned} SSD=\mathop {\sum }\limits _{t=1}^s {(x_t -S_t)^{2}} \end{aligned}$$
(4)

6.4 The prediction

The Holt-Winters method helps in realizing predictions from historical data. This technique allows giving great importance to the closest historical data and less importance to furthest ones. The prediction of the horizon h is given by the equation (5):

$$\begin{aligned} \hat{{x}}(t,f)=(L_t +h.b_t).S_{t-s+h} \end{aligned}$$
(5)

The initialization of the algorithm requires knowledge of the initial values \(L_1,\ldots ,L_s,b_1,\ldots ,b_s,\) and \(S_1,\ldots ,S_s\).

In this article, we took into consideration the first six values of RTT known in advance (\(\hbox {s} = 6\)), and the initial estimates of the values are given by the following equations:

$$\begin{aligned} L_t= & {} \frac{x_1 +\cdots +x_s }{s} \end{aligned}$$
(6)
$$\begin{aligned} b_t= & {} \frac{1}{s}\cdot \left[ {\frac{x_{1-s} +\cdots +x_1 }{s}+\cdots +\frac{x_{2s} +\cdots +x_s }{s}} \right] \end{aligned}$$
(7)
$$\begin{aligned} S_t= & {} \frac{x_t }{L_t } \end{aligned}$$
(8)

The Holt-Winters method to explore real-time predicting of RTT. This technique is largely used for the analysis and treatment of time series data. Real measurements of the RTT were taken between 4 geographically distant nodes. The nodes are as follows:

The prediction results which have shown in the Figs. 3, 4 and 5, confirm the efficiency and precision of the proposed method. For more information, see our previous work [32].

Fig. 3
figure 3

The prediction of RTT by Holt-Winter between USTO (Algeria) and UoB (France)

Full size image
Fig. 4
figure 4

The prediction of RTT by Holt-Winters between (USTO) and (UMKB-Algeria)

Full size image
Fig. 5
figure 5

Prediction of RTT by Holt-Winters between (USTO-Algeria) and (ANISP-Iran)

Full size image
Fig. 6
figure 6

System and procedures of Delayed Commands [28]

Full size image

7 The tomporesation of the commands (Toc) [28]

Both previous strategies (SLM and ISLM) are able to control the robot via internet, but it is not easy to accurately reach the desired position when a sudden change occurs in the RTT, especially in the SLM strategy [4]. At this problem, we proposed a new architecture that allows us to minimize the error in the path of the robot controlled via the Internet; the idea is to reach to the desired position “G” (Fig. 1), we must have a constant value for the robot speed “V” and the time to move the client command to the robot “T” (Eq. 9). It is easy to give the speed of the robot a fixed value, but the problem is in the “T” value which varies according to time and in a random way. To fix this, we proposed an algorithm that allows us to delay sending commands to the robot with a constant delay time (TMC = 500ms) as shown in (Fig. 6).

$$\begin{aligned} \Delta \hbox {d}=\hbox {V}.\Delta \hbox {t} \end{aligned}$$
(9)

First, the remote client sends the commands (forward, back, left and right) at time T1 and the robot receives them at time T2. Then, we calculate the time to go of the command \(\hbox {TMP} = \hbox {T}1- \hbox {T}2\) at the server level (Robot) then we test whether TMP <500 (ms) next, we delay the command with a time (\(\hbox {RLC} = 500-\hbox {TMP}\)), otherwise we will stop the robot. Finally, we give an explanation for each module of our man-machine communication interface.

Fig. 7
figure 7

Virtual environment interface

Full size image

8 Man–robot communication interface

Our platform consists of a remote host PC and the a medial PC communicating with each other by client-server type of internet using the Internet communication protocol TCP/IP (Fig. 2). The operator via a graphical interface of a Man-Machine communication remotely controls the movements of the mobile robot Pioneer 3AT (front, back, left and right) through a medial PC (medial server) that communicates with the robot through WIFI in a local area network (LAN), while having the opportunity to see the scene through the camera which is transmitted via the transmission channel (internet).

The Man-machine interface grants the operator three essential tasks, it supports user commands, plays video images and manages the virtual simulator of the robot for the purpose of providing the user with vision on the tasks the robot must run. The figure below (Fig. 9) illustrates the main window of the developed interface; the latter consists of two parts:

8.1 The client

The client application was developed by the Java programming language (Java NetBeans) which is installed in the host PC. It provides in turn the following functions:

  • Managing a graphical man-robot communication interface

  • Managing the communication to and from the robot, by sending instructions to the user.

  • The reception, display video images from the camera.

  • The management and interpretation of commands given by the user.

8.2 The simulation of the robot

We have developed a graphic simulator (Fig. 7) of our robotic system, which is called (Pioneer mobile robot simulator P3T). This simulator was developed with game engine Unity 3D software. We designed this simulator in order to test different strategies in real conditions of use. The virtual robot reproduces the actual behavior of the real robot, with constant speed corresponding to the maximum speed of the real robot (400 mm/s). This simulator was developed to compensate for the delay caused by the network which is responsible for the latency in the transferring of videos to the client (user); for these reasons we rely too much on this simulator to control the robot remotely.

8.3 Design of the virtual world

Virtual simulations represent a specific category of simulation that utilizes simulation equipment to create a simulated world for the user. Virtual simulations allow users to interact with a virtual world.

The use of VR in our system allows to give several advantages to the user such as:

  • Allows to see the configuration of the robot during the execution of a task without having to wait for the return of the actual information.

  • Can help users get a better view and understanding of the remote system they want to control.

  • To compensate for the delays caused by the networks, by providing returns of the information free of the delay to the operator.

  • The takeover of machines by means of web technologies and mobility: Smartphones, tablets, tablets-pc. This takeover can be done by the machine in the workshop, as from any place in the factory, or even from the company.

8.4 The medial server

We have developed two architectures to control the robot; these architectures have been installed in the medial PC between the client and the robot. Our architectures consist of four major parts:

A. time synchronization

It is an application developed in C at our laboratory (Laresi) to synchronize time between the client and server to assure that the time is the same in both PCs.

B. Multimedia application

The camera continuously delivers video images of the scene to the client using the video streaming technique.

C. Evaluation of the quality of service

In this section we evaluate the QoS using a statistical method (Holt-Wintres) which permits to predict in real time the RTT in order to vary the robot speed according to the predicted value. This section is related to the ISLM strategy.

D. The temporization of commands

The objective of the second strategy (TOC) is to reach to the desired position “A”. To do that, the speed of the robot must have a constant value “S “as well as a constant time to go value “t”, this is why we have proposed an algorithm to fix the value of “t”.

8.5 The server

We have developed a server application using the programming language C\(++\) application which is installed PC integrated in the robot (Fig. 8), it performs the following functions:

  • Managing the communication to and from the Medial server by sending information about the state of the robot and receiving instructions from the user.

  • Control the movement of the robot.

  • Managing the sensors of the robot.

Fig. 8
figure 8

Pioneer-3AT server interface

Full size image

8.6 Internet communication

The main problem of tele-robotics applications is the limitation of bandwidth and delays during the long distance transmission. To overcome these obstacles, we must substantially improve our architecture systems and man-machine interface. We used client-server architecture with a

TCP/IP protocol. All links between the operator and the workplace are implemented using TCP/IP sockets to allow long-distance remote operation via the Internet.

9 Experiments

In this work, two types of experiments were performed. The first is the synchronization between the movements of the virtual object (simulator) and the actual movement. To move the robot from the point A to the desired position G, the human operator either uses the interface buttons or the keyboard commands (forward, backward, left, right and STOP). The purpose of this experiment is to demonstrate how to control the robot so that it can navigate through a complex space using virtual reality as mentioned in Sect. 2.The results show that the 3D simulator corresponds with the actual robot motion.

The second experiment examines the effectiveness of the proposed architectures on the influence of time delay on the trajectory of mobile robots, and to do this we conducted teleoperation experiments on different distances away from the operator and the robot, respectively (100 m, 5 km, 130 km and 700 km), in which the human operator remotely controls the real robot which is located in a large complex room via internet (see Fig. 10). The remote operator observes the work environment through streaming video feedback. When the distance between the operator and the robot is great, the video transmission will have some latency; in this case, we do not know the exact position of the robot which is why we developed the 3D simulator (Fig. 9).

In the ISLM strategy [29], the real robot and the virtual robot can move continuously without adjustments, with a constant speed for the virtual robot and a variable speed for the real one. This solution allows the robot to reach the target destination accurately. For example, the remote operator sends the command ’Walk forward’ to the real robot, meanwhile the virtual robot will move forward before the real one due to the actual delay time caused by the network. To compensate for this delay, we vary the robot speed according to four levels, each level is given based on the predicted RTT as shown in Table 1, whenever the predicted RTT value increases, the speed will automatically decrease and the vice versa; so when we send the command ‘STOP’, the real robot will stop in the desired position in which the virtual robot stopped.

Fig. 9
figure 9

Control based on a virtual model

Full size image
Fig. 10
figure 10

Teleoperation results

Full size image
Table 1 Classification of the quality of service according to the predicted RTT
Full size table
Table 2 Comparison of performance under various teleoperation strategy
Full size table

The second proposed solution [28] to mitigate the impact of the delay induced by the network is the architecture that allows us to delay the commands sent from client to server (robot) with a known unit of time (500 ms), this solution allows us to know with a very small error the delay time for the operator to adjust and locate the position of the robot. Next, we compared three strategies of control (SLM, ISLM and TOC) in order to meet their advantages and disadvantages.

10 Summary of comparison withother approaches

In this paper, we proposed two solutions (ISLM [29] and TOC [28]) to control the mobile robot Pioneer 3AT in a network without a guaranteed quality of service, then, we compared them with another strategy (SLM [4]). We focused on two parameters for this comparison: the variation of the RTT and the distance between the user (human operator) and the robot, as shown in Table 2.

The SLM strategy is able to control the robot with safety in two cases; in case of slight variations of RTT and the second has a relationship with the distance between the user and the robot, because it is difficult to find a uniform SLM for all users [4]. To eliminate this problem, we have suggested the ISLM strategy in which the main novelty is the architecture that grants us the security when controlling the robot thanks to the good assessment of the levels of quality of service; this assessment is carried out with the prediction of RTT using the Holt-Winters method. This method can predict the RTT with very acceptable precision as shown in our previous work [32] subsequently the robot will work with the appropriate speed. Table 2 shows how effective of the ISLM strategy is in case of sudden variations of RTT even if the users are located away from robot. However, we encountered difficulties to control the robot with accuracy and safety in case of frequent internet interruptions occur. But this problem was solved by the TOC strategy, this latter allows us to set the delay time that is to say, the robot has a constant speed unlike the other two strategies, which reduces the risk of colliding the robot with space objects. But this strategy is still limited because when the delay time goes beyond the 500 (ms) threshold, in other words, if the distance between the user and the robot is great, we cannot control the robot.

11 Conclusion

In this article, two architecture have been proposed to improve the quality of control of a remote system. First, we have established a network communication between the client and the Pioneer robot P3AT to compensate for the latency induced by the network. We have developed two architectures; The first strategy allows us to vary the robot speed according to the predicted RTT using the Holt-Winters method, this method was chosen for the purpose of getting a good assessment of the quality of service; and the second allows us to delay the commands sent from the client to the robot with a known unit of time in order to avoid the change in time delay. Moreover, we made a comparison between the proposed architectures and another strategy to examine and address their advantages and disadvantages.

We have also developed a simulator, which is a virtual test bench; it arouses much interest because it helps us see the robot configuration before, after and during the execution of a task. The operator will have a full imagination of the action to be done by the robot.

The experiments that we experienced have lead us to figure out the advantages and disadvantages of the three strategies in this article. In future work, it would be interesting to improve or combine two architectures to control the robot, regardless of the quality of service.