Motion Planning and Object Grasping of Baxter Robot with Bionic Hand

Abstract

Grasping and moving objects is a natural behavior in human daily life, whereas it turns into an enormous challenge with robots. To analyze the difficulty of grasping and moving target objects, a arm-hand system is performed with 7-DOF dual arms robot and bionic hand in this paper. A numerical method is proposed to solve the problem of arm motion planning. And a novel grasping strategy is proposed for enabling bionic hand to grasp efficiently. Finally, the effectiveness of the proposed methodology is demonstrated using both computer simulation and physical experiment.

This work is supported by Science and Technology Commission of Shanghai (15411953500) and Science and Technology Commission of Shanghai Municipality under “Shanghai Sailing Program” (16YF1403700), Shanghai University Youth Teacher Training Assistance Scheme (ZZSD15088).

1 Introduction

Nowadays, robots are widely used in numerous industrial and scientific circumstances to replace humans as executors of some dangerous and complicated works. But in daily life, task like grasping objects are still an enormous challenge that robots face. Thus, some researches have been developed to find the solution of robot uses in daily life [1,2,3,4]. A dual-arm robotic system is presented to combine the motion planning with task assignment to have objects manipulated in cluttered environment [5]. Besides, objects grasping of different sizes and shapes with robot arms and hand coordination is shown in [6]. It is important to appropriately control the motion of robot arms while robots accomplish the target of grasping objects.

Object manipulation in cluttered environment is a task that humans perform daily. Different types of problems are found when humans are replaced with robots that have to perform this type of tasks. Problems like the motion planning, the computation of the object grasping, and the computation of a path to move the object from its initial configuration to a new goal configuration. This kind of problems are considered classics in the literature in robotics [7,8,9].

To enable the movement of robot arm from initial to the goal configuration, a methodology of motion control is proposed in this paper. A numerical algorithm based on Newton-Raphson method is illustrated to solve the inverse kinematics of robot manipulator. Unlike analytical solution, the numerical algorithm could easily figure out the joint angles. In addition, the purpose to grasp objects stablely requires a strategy to adapt robot hand to the object pose. In order to accomplish the objective of having the objects grasped stablely, the strategy of grasping different sized and shaped objects is proposed in details.

This paper presents an implementation of motion planning and object grasping of Baxter robot with bionic hand using methodology proposed. The simulation and experiment are established in ROS (Robot Operation System) [10]. MoveIt plugin [11] is used to study the performance of the robot arm-hand system. At the end of this paper, the result of manipulator experiment of grasping object is shown.

2 System Specification

In order to have the objects grasped as well as motion planning, a system is set up include three parts: 1. Baxter robot, 2. SHU-II bionic hand and 3. Kinect sensor. The SHU-II hand and Kinect sensor are attached to the robot with program run on PC. The details of components is illustrated in the following content.

2.1 Baxter Robot

Baxter robot is a humanoid robot as shown in Fig. 1 that includes a head-pan with a screen, located on the top of torso, can rotate in the horizontal plane followed by a torso based on a movable pedestal and two 7-DOF (degree of freedom) arms installed on left/right arm mounts respectively. Each arm has 7 rotational joints and 8 links, as well as an interchangeable gripper which can be installed at the end of the arm [12]. But the gripper is not operating well and unsteady to grasp objects. Thus, an bionic hand named SHU-II hand [13] is adopted. And because of its anti-collision design, Baxter robot is s able to sense a collision at a very early time instant, before it hits badly onto a subject. Thus it is a good choice to fulfill safe human-robot interaction.

Fig. 1.

Baxter robot with 7-DOF and the structure

2.2 SHU-II Hand

The SHU-II hand is equipped with force sensor that is used to grasp objects. Figure 2 shows the prototype of the SHU-II robot hand consisting of 5 under-actuated fingers and each finger is controlled by a DC motor with a gear box. A servo operates the thumb to be positioned parallel to the other four forefingers. A palm is designed to carry five fingers including their motors, servo and encoders. The thumb has four joints with four-degrees-of-freedom (DOF) and each finger has three joints with 3-DOF, which means it has sixteen degrees of freedom like real human hand. The size of the artificial hand is 1.1 times the human hand size.

Fig. 2.

SHU-II hand

The under-actuated hand with anthropomorphic motion can adaptively grasp objects with different sizes and shapes like the human hand. Because of its higher integration, it can be very easy to install and test on the other platform.

2.3 Vision

Total coordinated system is consists of robot, SHU-II hand, and the vision system. This vision system provides the object information that is obtained by Kinect sensor showed in Fig. 3. The object image processing goes as follows: 1. Information of depth and RGB colors of image is collected by Kinect. 2. Information is segmented by the image depth. 3. Target identification based on objects’ shapes and colors. 4. Kinect based frame is converted to the robot based frame. Hereafter the robot gets the 3D coordinates of objects of the robot frame.

Fig. 3.

Baxter with arm-hand

2.4 Operation Flow

The main controller is operated on ROS (Robot Operation System), and consists of three threads. One is ROS operation thread called Main thread. It gets the command from user then starts the main process. The forward kinematics of robot arm is computed, then the initial value of trajectory is set which called the original position. The main thread gets the object position from second thread Vision thread, and convert the Kinect based frame to Robot based one. Hereafter, serial data is sent to the SHU-II hand then it will grasp the object we chosen before. A feedback is returned once the grasp is complimented. Thus the robot arm begins to move. Place the object at the placed position (Fig. 4).

Fig. 4.

Main flow of Robot arm-hand system

Fig. 5.

The height and width of object

3 Motion Planning and Grasp Strategy

3.1 Motion Control of Robot Arm

The whole system is completed under the ROS system, and we obtain the DH parameters of Baxter from its URDF (Unified Robot Description Format) file [14]. The D-H parameters of robot left arm measured by the mechanical structure (Table 1) can be utilized to establish the D-H equation in formula (1):

Table 1. DH notation table of the left arm

$$ ^{i - 1} T_{i} \, = \,\left[ {\begin{array}{*{20}c} {c\theta_{i} } & { - s\theta_{i} c\alpha_{i} } & {s\theta_{i} s\alpha_{i} } & {\alpha_{i} c\theta_{i} } \\ {s\theta_{ii} } & {c\theta_{i} c\alpha_{i} } & { - c\theta_{i} s\alpha_{i} } & {\alpha_{i} s\theta_{i} } \\ 0 & {s\alpha_{i} } & {c\alpha_{i} } & {d_{i} } \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] $$

(1)

Where $c$ and $s$ represent the trigonometric of cos and sin respectively. Thus the position and orientation of the end effector described by coordinate in a Cartesian coordinate system can be calculated by each transformation matrix in formula (2):

$$ {}^{0}T_{n} \, = \,{}^{0}T_{1} {}^{1}T_{2} \cdots {}^{n - 1}T_{n} \, = \,\left[ {\begin{array}{*{20}c} {n_{x} } & {o_{x} } & {a_{x} } & {p_{x} } \\ {n_{y} } & {o_{y} } & {a_{y} } & {p_{y} } \\ {n_{z} } & {o_{z} } & {a_{z} } & {p_{z} } \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] $$

(2)

And ${}^{i - 1}T_{i}$ could represents as $A_{i}$, so the formula (2) could be rewritten as formula (3):

$$ {}^{0}T_{n} = A_{1} A_{2} \cdots A_{n} $$

(3)

Forward kinematics analysis of Baxter robot determines workspace of the robot arms, from which we could obtain the jacobian matrix of robot. Jacobian matrix of robot will establish the relationship between joint motion and hand motion. And the relationship can be represented shortly as formula (4):

$$ D\, = \,JD_{\theta } $$

(4)

Where $D\, = \,[\begin{array}{*{20}c} \nu & \varpi \\ \end{array} ]^{T}$, $D_{\theta } \, = \,[\dot{q}_{1} \cdots \dot{q}_{n} ]^{T}$,and the formula showed above can be rewritten as formula (5):

$$ \left[ {\begin{array}{*{20}c} \nu \\ \varpi \\ \end{array} } \right]\, = \,\left[ {\begin{array}{*{20}c} {J_{11} } & \cdots & {J_{1n} } \\ \vdots & \ddots & \vdots \\ {J_{61} } & \cdots & {J_{6n} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\dot{q}_{1} } \\ \vdots \\ {\dot{q}_{n} } \\ \end{array} } \right] $$

(5)

Where $[\begin{array}{*{20}c} \nu & \varpi \\ \end{array} ]^{T}$ is the linear velocity and angular velocity of robot hand which is equally to $\left[ {\begin{array}{*{20}c} {\nu_{x} } & {\nu_{y} } & {\nu_{z} } & {\varpi_{x} } & {\varpi_{y} } & {\varpi_{z} } \\ \end{array} } \right]^{T}$ while $[\dot{q}_{1} \cdots \dot{q}_{n} ]^{T}$ shows the angular velocity from different joints, $n$ is the DOF of robot. For us $n$ is equally to 7 as Baxter robot has 7-DOF. Hereafter $J\, = \,[\begin{array}{*{20}c} {J_{1} } & \ldots & {J_{n} } \\ \end{array} ]$ is the differential kinematic relation of them. The jacobian matrix could be computed from its kinematic equation. With regard to a matrix:

$$ J_{i} \, = \,\left[ {\begin{array}{*{20}c} {J_{1i} } \\ {J_{2i} } \\ {J_{3i} } \\ {J_{4i} } \\ {J_{5i} } \\ {J_{6i} } \\ \end{array} } \right]\, = \,\left[ {\begin{array}{*{20}c} {(p_{6}^{i} \times n_{6}^{i} )_{z} } \\ {(p_{6}^{i} \times o_{6}^{i} )_{z} } \\ {(p_{6}^{i} \times a_{6}^{i} )_{z} } \\ {(n_{6}^{i} )_{z} } \\ {(o_{6}^{i} )_{z} } \\ {(a_{6}^{i} )_{z} } \\ \end{array} } \right] $$

(6)

The elements of it can be calculated one by one with specific equation and $(p \times n)_{z} \, = \,(p_{x} \times n_{y} - p_{y} \times n_{x} )$ where $i$ varies from 1 to $n$.

$J_{i}$ is the $i_{th}$ column of jacobian $J$ and it is based on the kinematic:

$$ {}^{i}T_{n} \, = \,A_{i} A_{i + 1} \cdots A_{n} \, = \,\left[ {\begin{array}{*{20}c} {n_{x} } & {o_{x} } & {a_{x} } & {p_{x} } \\ {n_{y} } & {o_{y} } & {a_{y} } & {p_{y} } \\ {n_{z} } & {o_{z} } & {a_{z} } & {p_{z} } \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] $$

(7)

And $p_{6}^{i}$ is present by formula (8):

$$ p_{6}^{i} \, = \,\,[\begin{array}{*{20}c} {p_{x} } & {p_{y} } & {p_{z} } \\ \end{array} ]^{T} $$

(8)

An analytical methodology of inverse kinematic computation for 7-DOF redundant manipulator may be difficult. But with the numerical methodology it can be solved easily. The problem of finding the inverse solution of the robot is considered as a problem of solving a set of nonlinear equations. Newton-Raphson method is an optional method for this problem. The problem rewritten as equations with joint angles unknown like:

$$ f = \left[ {\begin{array}{*{20}c} {f_{1} (\begin{array}{*{20}c} {x_{1} } & \cdots & x \\ \end{array}_{n} )} \\ \vdots \\ {f_{n} (\begin{array}{*{20}c} {x_{1} } & \cdots & {x_{n} } \\ \end{array} )} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} 0 \\ \vdots \\ 0 \\ \end{array} } \right] $$

(9)

Where $x_{1}$ to $x_{n}$ is the angles of different joints. From the equation mentioned above, we have:

$$ J = f^{{\prime }} = \left[ {\begin{array}{*{20}c} {\frac{{\partial f_{1} }}{{\partial x_{1} }}} & \cdots & {\frac{{\partial f_{1} }}{{\partial x_{n} }}} \\ \vdots & \ddots & \vdots \\ {\frac{{\partial f_{6} }}{{\partial x_{1} }}} & \cdots & {\frac{{\partial f_{6} }}{{\partial x_{n} }}} \\ \end{array} } \right] $$

(10)

It means jacobian matrix consists of elements with respect to the partial differential of function $f_{1}$ to $f_{6}$.

And finally the joint angles will be calculated:

$$ \left[ {\begin{array}{*{20}c} {q_{1(k + 1)} } \\ \vdots \\ {q_{n(k + 1)} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {q_{1(k)} } \\ \vdots \\ {q_{n(k)} } \\ \end{array} } \right] + J^{ - 1} Err $$

(11)

$q_{i(k + 1)}$ is the $i_{th}$ joint angle obtained after $k + 1$ times. $J^{ - 1}$ is the inverse matrix of jacobian $J$.

$$ Err = f(x_{k} ) $$

(12)

Represents the difference between the current value and the target value after the $k$ times iteration. And the final solution will be obtained when the difference is close to 0.

3.2 Grasping Strategy

The robot grasping of different sized and shaped objects is a complex problem to be solved [15]. A strategy of objects manipulation is proposed in this part. Because the shape of any object can be viewed as a combination of several special ones like sphere and cuboid, the methodology to grasp these specific objects could be generally extended.

Just like human beings seize things, the grasping pose of robot hand is changed on the basis of postures of different objects. And the approach to grasp is sometimes distinct with same thing of changed pose. The whole task could be divided into two aspects: 1. the position robot hand moved and 2. the pose of robot hand to grasp the objects stablely. Strategy to solve the problems is performed after the analysis of objects structure.

Sphere and cuboid is taken for the analysis of objects. By the experience human hand seizes, the grasping point is always on the longer side of object. As shown in Fig. 6, the $Z$ axis represents the hand approach direction to the object, and $X$ axis is from the paper outward which matters the pose to grasp.$Y$ axis is vertical to the plane shaped by $X$ and $Z$ axis. The height and width of object are represented as $H$ and $W$ in Fig. 5. $d_{\hbox{max} }$ and $d_{\hbox{min} }$ is defined as the biggest and least of $H$ and $W$.

Fig. 6.

Coordinate of hand

$$ \left\{ {\begin{array}{*{20}c} {d_{\hbox{max} } = \hbox{max} \left\{ {H,\left. W \right\}} \right.} \\ {d_{\hbox{min} } = \hbox{min} \left\{ {H,W} \right\}} \\ \end{array} } \right. $$

(13)

As mentioned before, the $Z$ axis known as approach direction is parallel with the normal vector of plane shaped by the direction vector of high and wide side which is obtained from the vision process. Similar to $Z$ axis determined, the direction vector of $Z$ axis and $d_{\hbox{max} }$ confirm the one of $X$ axis.

Some objects have the similar $H$ and $W$ like sphere, it’s not necessary to choose which pose to grasp. Thus, a criterion named $d_{cr}$ is defined to divide the objects into different groups. If $\Delta d = d_{\hbox{max} } - d_{\hbox{min} }$ is smaller than $d_{cr}$, the pose is not defined. The parameter of $d_{cr}$ is decided by $d_{\hbox{max} }$ and the grasp range $d_{rg}$ of bionic hand as shown in formula (14).

$$ d_{cr} = \left\{ {\begin{array}{*{20}c} {d_{\hbox{max} } ,\begin{array}{*{20}c} {} & {d_{\hbox{max} } \le d_{rg} } \\ \end{array} } \\ {d_{rg,} \begin{array}{*{20}c} {} & {d_{\hbox{max} } > d_{rg} } \\ \end{array} } \\ \end{array} } \right. $$

(14)

Because of the shape and size of SHU-II hand, there is an offset attached to the grasp point. The strategy to grasp cuboid and sphere is simulated in Fig. 7.

Fig. 7.

Grasp simulation

4 Experiment

4.1 Motion Planning

When the Cartesian position and orientation of robot are known, we can calculate the 7 joint angles based on the algorithm illustrated in Sect. 3. The simulation result in Fig. 8 shows the accuracy of the solution and well plan of the trajectory.

Fig. 8.

Motion plan of robot arm

4.2 Object Grasping

The arm-hand system comprises Baxter robot, Kinect sensor of vision and SHU-II hand. The whole system is operated on ROS and the movement of manipulator is decided by algorithm mentioned in Sect. 3. Strategy to grasp is used to grasp the object stablely. The experiment selects the green apple and cola jar as targets. The Figs. 9 and 10 shows the validity of the Motion Planing and Object Grasping of Baxter Robot with SHU-II hand.

Fig. 9.

Baxter robot recognizes and grasps green apple

Fig. 10.

Baxter robot recognizes and grasps cola jar.

5 Conclusion

In this paper, a robot arm-hand system which is comprised of a 7-DOF humanoid robot, a bionic hand and vision sensor is shown. We have proposed an effective numerical method to accomplish the task of robot arm motion planning. And the strategy for bionic hand to grasp object is illustrated. The target object is grasped successfully. The main difficulty for the implementation is that sometimes the position of grasp object obtained by vision sensor is not precise enough. So the offset of bionic hand is necessary to grasp the object successfully. The simulation and experiment show the validity of the robot arm-hand system.

Now the main objective of grasping target object is implemented. The motion planning based on the numerical method is verified to be correct. Using them we could build a robot arm-hand system easily, but further experiment and analysis of grasping strategy need to be carried out in our future work.