A Telepresence-Guaranteed Control Scheme for Teleoperation Applications of Transferring Weight-Unknown Objects

Jinfei Hu, Zheng Chen,, Xin Ma, Han Lai, and Bin Yao,

Abstract—Currently, most teleoperation work is focusing on scenarios where slave robots interact with unknown environments. However, in some fields such as medical robots or rescue robots, the other typical teleoperation application is precise object transportation. Generally, the object’s weight is unknown yet essential for both accurate control of the slave robot and intuitive perception of the human operator. However, due to high cost and limited installation space, it is unreliable to employ a force sensor to directly measure the weight. Therefore, in this paper, a control scheme free of force sensor is proposed for teleoperation robots to transfer a weight-unknown object accurately. In this scheme, the workspace mapping between master and slave robot is firstly established, based on which, the operator can generate command trajectory on-line by operating the master robot. Then, a slave controller is designed to follow the master command closely and estimate the object’s weight rapidly, accurately and robust to unmodeled uncertainties. Finally, for the sake of telepresence, a master controller is designed to generate force feedback to reproduce the estimated weight of the object. In the end,comparative experiments show that the proposed scheme can achieve better control accuracy and telepresence, with accurate force feedback generated in only 500 ms.

I. INTRODUCTION

WITH the development of mechatronic systems [1]-[7],teleoperation robots [8] provide an effective solution for exploration in unstructured or hazardous environments that are difficult for humans to access directly. In the past few decades, teleoperation systems have been successfully applied to submarine robotic missions, space exploration [9], nuclear waste cleanup [10], medical telesurgery [11], and rehabilitation [12]. Basically, teleoperation applications can be divided into two categories: unknown environment interaction (UEI)tasks when the slave robot contacts the environment and weight-unknown object transferring (WUOT) tasks when the slave robot transfers objects in free space.

Currently, most teleoperation work deals with UEI situations. In this category, stability and control accuracy should be guaranteed firstly in the presence of potential time-delay and environmental uncertainties. In addition, the feedback force generated by the master robot should give the operator a sense of direct operation of the environment, which is referred to as telepresence [13]. This can be achieved by force feedback such that the master impedance matches the slave impedance[13]. For UEI tasks, the communication channel in the teleoperation robot system is widely modeled as a two-port network, and then different bilateral control algorithms are employed, such as the four-channel method [14], [15], the wave variable method [16], the time-domain passivity control approach [17], the disturbance observer [18], the radical basis function-based neural network method [19], [20] and the model predictive control approach [21].

The other typical teleoperation applications are the WUOT applications [22], [23]. Related tasks include nuclear waste cleanup, drone object transfer tasks [24], robot grasp tasks[25], or hydraulic robot manipulation tasks [26]. As shown in Fig. 1, in these applications, the slave robot should follow the master trajectory closely; besides, the master device should provide force feedback, so that the operator has an intuitive sense of the weight of the object, i.e., good telepresence.However, the weight is usually unknown in advance. This not only introduces dynamic uncertainties to the system, making it difficult for the slave robot to follow the master command accurately, but also leads to poor telepresence. On the other hand, there are lumped unmodeled uncertainties in the slave robot, such as modeling errors and disturbances, which need to be dealt with effectively for robustness.

Fig. 1. WUOT teleoperation tasks.

One straightforward solution to the WUOT problem is to use force sensors to measure the payload. The sensors can be installed on the wrist [27], in the joints, or on the base of the slave robot. Among these three ways, commercial force/torque sensors on the wrist are usually expensive, fragile [28]and impractical to be installed in confined space tasks. The latter two ways are to use commercial collaborative robots[29], such as Baxter, KUKA iiwa, or UR5. However, these robots are expensive and cannot directly measure the weight of objects compared to wrist sensors. Therefore, in WUOT applications, it is an effective way to make full use of system dynamics to estimate the object’s weight and compensate for its effects.

Some work is published about WUOT teleoperation applications. In [23], adaptive neural network control is proposed for the teleoperation system to compensate for the dynamic uncertainties by unknown payloads. But such neural network control only compensates for the lumped effects of the payload, rather than estimating the weight of the object.Therefore, the force feedback cannot be generated by the master device because it depends on the object’s weight. In[30], [31], an asymmetric teleoperation control scheme is proposed for object transferring tasks where master and slave devices are different. In their schemes, the force feedback is based on the tracking errors of the slave robot, so the operator cannot intuitively perceive the payload weight. In [22], a direct adaptive robust control approach is proposed for the WUOT teleoperation problem. The weight of the object transferred by a 7 degrees-of-freedom (DoF) robot is estimated by a direct parameter adaptation algorithm. However, the traditional direct type adaptive method which is driven by tracking errors [32] cannot guarantee rapid and accurate weight estimation. So the force feedback related to the weight estimation will become lagging and fluctuating. This will deteriorate both the telepresence for humans and the control accuracy of the slave robot.

Above all, in WUOT teleoperation applications, the problem of achieving accurate control and good telepresence simultaneously has not been addressed well yet. Therefore, we propose a novel control scheme for WUOT teleoperation applications. In this scheme, the workspace mapping between master and slave robot is firstly established, by which human operation is converted to the slave robot motion command.Then, a direct/indirect adaptive robust controller (DIARC) is designed for the slave robot to follow the motion command closely and estimate the object’s weight rapidly and accurately. With the object’s weight estimated accurately, a master controller is designed to generate force feedback to reconstruct the weight, aiming to provide human operators with a good sense of presence. In addition, to deal with other uncertainties by frictional modeling errors and disturbances,an additional dynamic compensation term is designed to observe and compensate for the slow time-varying components of the lumped uncertainties.

The research in this paper mainly contributes to the following aspects. 1) A telepresence-guaranteed control scheme is proposed for teleoperation applications where accurate transferring of a weight-unknown object is required. The essence is that the master controller uses force feedback to reproduce the slave-side object’s weight which is estimated on-line. 2) A slave controller is designed to estimate the object’s weight on-line and follow the master command under dynamic uncertainties. The comparative experiments validate that both better control accuracy and better telepresence can be achieved.

This paper is organized as follows. The problem formulation is presented in Section II. Section III introduces the workspace mapping and reference trajectory generation for the slave robot. In Section IV, the proposed master controller and the slave controller are introduced. Comparative experiments are carried out in Section V to show the merits of the proposed scheme. Finally, conclusions are drawn in Section VI.

II. DYNAMICS AND PROPOSED CONTROL SCHEME

In this section, the slave and master robot dynamics and their properties in WUOT teleoperation applications are introduced. Then, the whole problem is split into 3 subproblems and formulated mathematically. Finally, the proposed control scheme is introduced.

A. Master and Slave Dynamics

Fig. 2. Proposed control scheme.

B. Proposed Control Scheme

In many teleoperation tasks such as medical or rescue applications, it is a common task to transfer a weightunknown object with different master and slave devices. For these applications, the proposed control scheme divides the whole task into three sub-problems, i.e., the workspace mapping, the slave controller design, and the master controller design, as shown in Fig. 2. Specifically,

1) With the master trajectoryqm(t) generated on-line by human operator, design a mapping method to generate a

III. SLAVE REFERENCE TRAJECTORY GENERATION

Since the master and the slave robot are with different configurations and sizes, referred to as asymmetric teleoperation in [30], workspace mapping is firstly designed in this section, followed by the slave robot trajectory generation.

A. Workspace Mapping

The master device used in this paper is the 6-DoF Geomagic Omni, with an encoder in each joint. The slave robot is the 6-DoF Comau Racer3 industrial robot. However, the two devices have different workspaces with their Denavit-Hartenberg (DH) parameters shown in Tables I and II.

TABLE I DH PARAMETERS OF COMAU-RACER3

TABLE II DH PARAMETERS OF GEOMAGIC OMNI

B. Trajectory Generation

A. Slave Robot Control Law

B. Slave-Side Object’s Weight Estimation Algorithm

C. Master Robot Control Law and Passivity Analysis

The first three joints of the master device, i.e., Geomagic Omni are equipped with motors for force feedback. And the master controller for these three joints will be designed in Cartesian space as the manufacturer provides access to command the Cartesian space force directly. Therefore,instead of joint space dynamics described in (1), the following Cartesian space dynamics is used in the master controller design:

Therefore, the operator can perceive the payload weight more intuitively when it is estimated more accurately.

Generally, teleoperation applications suffer from the stability issues under time-delay, which as pointed in [38], is due to the power signals (force and velocity) transferred in the communication channel. However, in the proposed scheme,the variable transferred in the communication channel is the estimated system parameter, i.e., non-power signals, which makes the communication channel passive under arbitrary time-delay [38]. Note that the master device, the slave robot and the operator are also passive, the passivity of the entire system can be guaranteed automatically. Therefore, even the existence of time-delay will not lead to stability issues.

D. Theoretical Results

Remark 4:Practically, to reduce the effect of the friction modeling errors at low speed, the adaptation for the payload estimation can be turned off when the joint velocity is less than a threshold. The theoretical results still hold because of the separation of control law and adaptation law. Such modification will be further explained in the experimental results later.

V. COMPARATIVE EXPERIMENTS

In this section, comparative experiments are carried out to show the superiority of the proposed control scheme, i.e.,better control accuracy under dynamic uncertainties and better telepresence by reproducing the payload weight more accurately.

A. Experiment Setup

The hardware used in the experiments is shown in Fig. 3.The master device is Geomagic Omni [40], with encoders in all the 6 joints to generate motion command and motors in the first 3 joints for force feedback. According to the manufacturer, the maximum force output is 3.3 N. The slave robot is the 6-DoF Comau Racer3 industrial robot with a 19-bits encoder in each joint and 3 kg maximum payload capacity.The workspace mapping and the slave control algorithm are executed on the control desk of the slave robot with 1250 Hz sampling frequency, whereas, the master control algorithm runs in MATLAB at 1000 Hz. The communication is by the Quanser data acquisition board and the B&R Automation X20AI4632 module. The communication delay is designed to beT= 200 ms. The Geomagic Omni and the Quanser data acquisition board are both connected to a PC, in which Matlab/Simulink can be used to transfer the command trajectoryqm(t) to Quanser analog output (AO) in real-time.The B&R Automation X20AI4632 module is used as analog input (AI) to transferqm(t) to the control desk of the slave robot. In turn, the estimated object’s weightm?L(t) is transferred from the slave robot to master robot in the similar way.

Fig. 3. Tele-robot system setup.

The experiment is to mimic a periodical back and forth movement for the tele-robot where the slave robot holds an object with unknown weight. The operator manipulates the master robot to generate a trajectory on-line to control the slave robot. The master command trajectoryqm(t) is not shown for simplicity and the slave desired trajectoryqd(t) is recorded in Fig. 4. Note thatqd(t-T) rather thanqd(t) is shown in Fig. 4 becauseqdis delayed byTin communication with respect toqm. Also note thatqd(t) is obtained by applying the workspace mapping (7) and the low-pass filter (8) toqm(t).

Fig. 4. Reference trajectory qd(t-T) for the slave robot.

To verify that the proposed method can deal with objects with various weight, two different payloads are attached to the end-effector. We refer the experiments with 1.445 kg payload as Set I and experiments with 2.275 kg payload as Set II.

B. Performance Indices

C. Experiment Results

The proposed control method is compared with the baseline method in [22]. The workspace mapping approach and the master controller are kept the same, while the slave controller is modified to the one in [22]. Note that for the fair comparison, the command trajectoryqm(t) for the proposed and baseline method should be the same. However, human operator cannot move along the same trajectory twice.Therefore, the command trajectory used for the baseline method is the recorded trajectory when conducting the experiment for proposed method.

The master controller is designed in (31) with parameters specified as

The slave robot trajectory tracking errors are shown in Figs. 5 and 6, respectively. In terms of control accuracy under uncertainties, the proposed method achieves smaller tracking errors with the equivalent control feedback gains, especially in joints affected by the payload significantly, i.e., joints 2, 3, 5.The quantitative control accuracy performance is shown in Fig. 7, where only results of Set II are presented in Fig. 6 due to the page limit.

Fig. 5. Slave robot tracking errors in Set I. The proposed method achieves higher control accuracy, especially in those joints affected by the transferred object significantly, i.e., joints 2, 3, 5.

Fig. 6. Slave robot tracking errors in Set II.

The payload weight estimation histories are shown in Fig. 8.Note that in experiments, the slave robot joint frictions are discontinuous when the joint velocities cross zeros, which makes the friction model inaccurate in low speed movement[41] and results in the payload estimation drift. Therefore, as explained inRemark 4, the adaptations are reasonably turned off when the velocities are smaller than a threshold, which leads to certain unchanged periods of the payload estimations in Fig. 8. Besides, in the proposed method, only the data of joints 2 and 3 which are greatly affected by the payload is used in the payload estimation to reduce the on-line computation. With such modification, all the theoretical results still hold thanks to the separation of the payload estimation law and the slave control law. The lumped uncertainty estimation of the proposed method is shown in Fig. 9 for Set II. Note that Figs. 8 and 9 represent the estimation of two kinds of uncertainties as explained inRemark 2.

Fig. 7. The maximum and RMS values of the slave tracking errors in Set II.

Fig. 8. The payload estimation histories. By separating control law and adaptation law, the proposed controller can estimate the unknown object’s weight more rapidly and accurately according to system input/output (joint control torque/position and velocity). In contrast, the adaptation of the baseline method is driven by tracking errors, so the estimation histories have the similar shape as the tracking errors as shown in Figs. 5 and 6. When the tracking errors fluctuate due to the back-and-forth trajectory, the estimation does as well.

The master Cartesian space trajectory and the feedback force are shown in Fig. 10. For clarity, only the trajectory from 6 s to 10 s is shown. In the force feedback, the reproduced payload weight is shown in Fig. 11. In terms of telepresence performance, it is obvious in Fig. 8 that the payload can be estimated more rapidly and accurately in the proposed method. As a result, in Fig. 11, the force feedback on the master side can be reproduced more rapidly and accurately by the proposed scheme. A quantitative comparison is made in Fig. 12 based on the results in Fig. 11. Therefore,the operator can have an intuitive sense of payload, i.e., the proposed scheme can achieve better telepresence.

Fig. 9. Lumped uncertainties estimation of the proposed method in Set II.

Fig. 10. The master device Cartesian space trajectory and force feedback from 6 s to 10 s. As the command trajectories for the compared controllers are the same, the added damping forces are also the same. Whereas, the reproduced object’s weight for are not the same as their payload estimation histories are not the same.

Fig. 11. Reproduced object’s weight by the master device. The payload weight can be reproduced more rapidly and accurately in the master device,i.e., achieving better telepresence performance.

VI. CONCLUSION

Fig. 12. The force feedback speed and accuracy in Set II. The force feedback time is very fast with only 500 ms to reach 9 0% of the ground truth in transient state and the force feedback RMS error is less than 0.05 N in steady-state.

This paper proposed a telepresence-guaranteed control scheme for teleoperation system with an unknown slave-side payload. The basic idea is that the master device reproduces the payload weight so that the operator can manipulate it intuitively. Specifically, the proposed scheme consists of 3 components, the mapping method, the master controller and the slave controller. The mapping method maps the master motion to the slave motion, which can generate a reference trajectory online for the slave robot. Then the slave controller drives the slave robot to follow the trajectory closely and estimate the unknown payload accurately. The master controller reproduces the payload weight estimated on-line for the sake of telepresence. The experiment results show that both better control accuracy and better telepresence can be achieved by the proposed methods. The accurate force feedback only needs 500 ms to be generated.