Accuracy of Robotic Elastic Object Manipulation as a Function of Material Properties

Abstract

We deal with the problem of thin string (1D) or plate (2D) elastic material folding and its modeling. The examples could be metallic wire, metal, kevlar or rubber sheet, fabric, or as in our case, garment. The simplest scenario attempts to fold rectangular sheet in the middle. The quality of the fold is measured by relative displacement of the sheet edges. We use this scenario to analyse the effect of the inaccurate estimation of the material properties on the fold quality. The same method can be used for accurate placing of the elastic sheet in applications, e.g. the industrial production assembly.

In our previous work, we designed a model simulating the behavior of homogeneous rectangular garment during a relatively slow folding by a dual-arm robot. The physics based model consists of a set of differential equations derived from the static forces equilibrium. Each folding phase is specified by a set of boundary conditions. The simulation of the garment behavior is computed by solving the boundary value problem. We have shown that the model depends on a single material parameter, which is a weight to stiffness ratio. For a known weight to stiffness ratio, the model is solved numerically to obtain the folding trajectory executed by the robotic arms later.

The weight to stiffness ratio can be estimated in the course of folding or manually in advance. The goal of this contribution is to analyse the effect of the ratio inaccurate estimation on the resulting fold. The analysis is performed by simulation and in a real robotic garment folding using the CloPeMa dual-arm robotic testbed. In addition, we consider a situation, in which the weight to stiffness ratio cannot be measured exactly but the range of the ratio values is known. We demonstrate that the fixed value of the ratio produces acceptable fold quality for a reasonable range of the ratio values. We show that only four weight to stiffness ratio values can be used to fold all typical fabrics varying from a soft (e.g. sateen) to a stiff (e.g. denim) material with the reasonable accuracy. Experiments show that for a given range of the weight to stiffness ratio one has to choose the value on the pliable end of the range to achieve acceptable results.

1 Introduction

The robotic manipulation with the soft material remains a challenging task due to the high degree of freedom of the soft material. One of the studied manipulation skills is the ability to fold the material accurately. The folding requires a folding trajectory which depends on several material properties. Estimation of these properties is inaccurate or omitted in real environment and then the folding itself is inaccurate as well. The folding inaccuracy as a function of the material properties is studied in this contribution.

Fig. 1.

The accurate folding visualization for the simplest scenario, where folding line (green, dashed) divides the sheet into two equally sized parts. The one side of the sheet is grasped and folding trajectory is followed in order to move the moving part on the top of the fixed part. One or two grippers are required for grasping depending on the width of the sheet. (Color figure online)

The fold is specified by the folding line as shown in Fig. 1. The folding line divides the sheet into two parts. The goal of the folding is to move the left part on the top of the right part. The folding is achieved by following the folding trajectory. In the simplest scenario, the folding is aligned with the sheet edges and divides the sheet into two equally sized parts (Fig. 1). In this paper, all performed experiments assume the simplest scenario. The correct folding in the simplest scenario results in aligned edges of the sheet. The quality of the fold can be quantified by the distance between edges, which is zero if folding was accurate.

The goal of this paper is to analyse the fold quality for different robotic folding methods. In robotic folding, the folding trajectory is generated for the given sheet properties. The folding trajectory starts with grasping of the one side of garment while the opposite side lies freely on the folding surface. The trajectory depends on several sheet properties, for instance the size, density and stiffness. Generated trajectory is used to perform the folding by a robot. If the unsuitable trajectory is generated, the resulting fold is inaccurate. In practice, the sheet properties are measured or estimated in advance. The estimation might be inaccurate and the designed trajectory is thus unsuitable for the real material. In the following two sections, the methods used for generation of the folding trajectory are described. The rest of the paper examines several experiments performed both in simulation and in real robotic testbed. These experiments analyze the effect of the properties estimation inaccuracy on the resulting fold.

The key contribution of the paper is a methodology, which select a folding trajectory for a given range of the material properties. This trajectory provides the satisfactory folding results for all materials from a given range. It also gives a possibility to pre-compute the trajectories and thus provides a fast planning algorithm for the folding trajectory design. This is demonstrated for all typical fabrics used for garments, where we will show that only four trajectories are required to fold all fabrics.

2 Related Work

To the author’s knowledge, there are four different approaches for the folding trajectory design. Two of them are independent of material properties and the trajectory is described by a set of simple curves. The other two approaches depend on the material properties and are planned based on simulation.

The first approach for trajectory design is gravity based folding method [2], denoted as linear trajectory hereinafter. The trajectory shape was derived assuming the infinitely flexible material and infinite friction between the garment and the folding surface. Such assumptions are unrealistic and lead to imprecise fold as shown in work [6], where the linear trajectory was compared to the circular trajectory. The circular trajectory proved to be more accurate when the folding was performed on jeans. The derivation of the circular trajectory was build on the assumption of the rigid material with frictionless joint located in the folding line. The linear as well as circular trajectory do not depend on the material properties. The design of these trajectories is thus fast and it is the preferable solution if the folding accuracy requirement is not high.

The other two approaches relay on the simulation and requires the material properties to be known in advance. The first method was designed in [5], where the simulation software Maya was used. Authors described the material by a single parameter called shear resistance. In the simulated environment, the garment with a known material is folded and the resulting shape is analyzed. The folding trajectory is parametrized by a Bézier curve and the curve parameters were optimized until a sufficiently accurate fold was obtained. The shear resistance was measured by an operator before the actual folding.

In our previous work [7], we have designed another method for folding trajectory design. Our model describes the garment by a set of ordinary differential equations derived based on the static equilibrium of forces. The linear relation between the bending moment and the garment curvature is assumed, ignoring the effect of hysteresis [4]. Single parameter is required to characterize the material properties and this material parameter is a weight to stiffness ratio. The several methods for an estimation of this parameter were described in [8] and we use technique called a free fold test, originally described in [9]. The folding trajectory is found by solving a Boundary Value Problem [1, 3], where boundaries characterize the folding requirements. Our method design a single trajectory for a given material and this trajectory results in the zero displacement from the expected position. In this work, an effect of the weight to stiffness ratio estimation inaccuracy is analyzed with respect to this approach for a folding trajectory design.

The methods presented here are not restricted for analysing of the garment folding only. Our model describes a wide range of materials and the accuracy analysis can be used in other fields as well. For example, the similar model was used for the task of undersea pipes laying using the J-lay method [10] or for a pipeline installation and recovery in deepwater [11].

3 Model

The model described in [7] will be briefly introduced in this section. The model is valid for homogeneous rectangular sheets and for homogeneous 1D strings. We have shown [7], that the material can be described by a single parameter called weight to stiffness ratio and denoted by a symbol $\eta$. The weight to stiffness ratio $\eta$ is a combination of the material density $\rho$ and the material bending stiffness K in a form:

$$\begin{aligned} \eta = \frac{\rho _a\, b \,g}{K}, \end{aligned}$$

(1)

where $\rho _A$ is area density, b is a width of a rectangle and g stands for a gravitational acceleration. In a case of a 1D string, the term $\rho _A \, b$ is replaced by a length density. The density as well as the bending stiffness are constant due to the assumed homogeneity of the material. The larger weight to stiffness ratio represents softer and/or heavier material.

According to [7], the string element (see Fig. 2) is modeled by a set of following differential equations:

$$\begin{aligned} \frac{\mathrm {d}}{\mathrm {d}s} \left( \tilde{T}(s) \cos \theta (s) + \frac{1}{\eta }\frac{\mathrm d^2 \theta (s)}{\mathrm d s^2} \sin \theta (s) \right)&= 0,\end{aligned}$$

(2)

$$\begin{aligned} \frac{\mathrm {d}}{\mathrm {d}s} \left( \tilde{T}(s) \sin \theta (s) - \frac{1}{\eta }\frac{\mathrm d^2 \theta (s)}{\mathrm d s^2} \cos \theta (s) \right)&= 1 - f(y(s)),\end{aligned}$$

(3)

$$\begin{aligned} \frac{\mathrm {d} {x}(s)}{\mathrm {d} s}&= \cos \theta (s), \end{aligned}$$

(4)

$$\begin{aligned} \frac{\mathrm {d} {y}(s)}{\mathrm {d} s}&= \sin \theta (s), \end{aligned}$$

(5)

where ds is an element of a string, $\theta (s)$ is an angle between the element and the horizontal axis, $\tilde{T}(s)$ is a scaled tension force T(s) and f(y(s)) is a force used to incorporate the contact of the element s with the folding surface. The variables x(s) and y(s) stand for the position of the element. The scaled tension force acts in a direction, which is tangent to the element and is related to the tension force as:

$$\begin{aligned} \tilde{T}(s) = \frac{{T}(s)}{\rho _A \, b \, g}. \end{aligned}$$

(6)

The contact force f(y(s)), which supports the part of the string laying on the table is modeled as:

$$\begin{aligned} f(y(s)) = \frac{1}{y^2}10^{-6}, \end{aligned}$$

(7)

which implies that the folding surface is horizontal with the zero height. The force is already normalized by the term $\rho _A \, b \, g$.

Fig. 2.

The model of the string.

3.1 State of the Model

The model differential Eqs. (2), (3), (4) and (5) could be transformed to the six first order ordinary differential equations. Six boundary conditions (BCs) are thus required to specify the state of the model. For the string of the length l, the boundary conditions restrict the start ($s=0$) or the end ($s=l$) of the string to be e.g. on the specific position or to have zero tension force. The BC at the position $s=0$ are denoted by superscript ‘$^-$’ and the BC at $s=l$ are denoted by superscript ‘$^+$’. When six BCs are specified, the BVP solver [3] is used to find the state of the model.

During the manipulation, there are many states of the model, which have to be solved. One such state is a completely folded string. It can be found by specifying the following boundary conditions:

$$\begin{aligned} \theta ^- = 180^\circ , \quad \frac{\mathrm {d}\theta ^-}{\mathrm {d}s} = 0, \quad x^- = l,\ \ \frac{\mathrm {d}\theta ^+}{\mathrm {d}s} = 0, \quad x^+ = l, \quad y^+ = 0. \end{aligned}$$

(8)

The folded states will differ depending on the weight to stiffness ratio $\eta$. The several folded states for different weight to stiffness ratio are shown in Fig. 3. It can be seen, that the string curvature located in the folding line is changing according to $\eta$. In our experiments, we have measured the maximal height of the string in the folded state, and denoted it as $h_m$. This height was used to estimate $\eta$ for the given material [8]. The relation between $\eta$ and $h_m$, which was used for the estimation, is shown in Fig. 4.

Fig. 3.

Folded states for different weight to stiffness ratio $\eta$.

Fig. 4.

Relation between weight to stiffness ratio $\eta$ and maximal height of the folded string.

Fig. 5.

Folding phases. The linear and circular trajectories are shown in blue and green, respectively. The forward folding phase is shown on the left side and backward phase is shown on the right side. (Color figure online)

3.2 Folding Trajectory

The folding trajectory describes the pose of the robot gripper in time. In our model, the garment states satisfying the folding requirements are found and the trajectory is computed from these states. Two different types of boundary conditions were used and are denoted as the forward and the backward folding phase. Both folding phases are shown in Fig. 5. In the forward folding phase, the string is lifting until the touchdown point reaches its final position. The backward folding has a fixed touchdown point position and can be divided into two parts: (a) before the string touch itself and (b) after the touch occurs, which continues until the string is completely folded. Different weight to stiffness ratio results in different folding trajectory as shown in Fig. 6.

4 Experiments

In the ideal folding scenario, the weight to stiffness ratio $\eta$, is estimated accurately in advance. This estimate is then used to generate the folding trajectory which is executed by the robot. In reality, the estimation of the weight to stiffness ratio might be inaccurate and then the designed folding trajectory is not suitable for the given material. The purpose of these experiments is to analyse the quality of the fold when $\eta _t$, used for trajectory generation, differs from the material weight to stiffness ratio $\eta _m$. We fold several materials both in simulation and in real robotic testbed and the resulting folds were analysed based on the displacement d (Fig. 7). Displacement measures the oriented distance between the expected and real position of the grasped side of the string.

Fig. 6.

Folding trajectory for the different weight to stiffness ratio $\eta$.

Fig. 7.

The displacement measurement. The displacement d measures the quality of the fold and the maximal height $h_m$ is used to estimate the material weight to stiffness ratio $\eta _m$.

Two undesirable events influence the displacement: the slipping of the strip and the bending behaviour. In this paper, the effect of bending behaviour is studied and the slipping is avoided by considering several assumptions. It is assumed, that the friction between the garment and the folding surface is high enough so the slippage of the non grasped side is zero. Furthermore, it is assumed that the friction between the garment layers is high enough as well, so the upper layer cannot slip on the lower layer. The slipping of the upper layer is unfavorable because it is hard to predict the amount of the slippage due to the elasticity of the real material and due to the high variance of the soft material friction behavior. Furthermore, it is not decidable in our model whether the upper layer will slip on the lower layer or the both layers will slip on the folding surface.

Fig. 8.

Simulated displacements - two different visualizations of the same experiment. The displacements for the selected trajectories (generated with $\eta _t$) are shown for all simulated materials $\eta _m$ on the left side. On the right side, the displacements of the selected materials are shown for all trajectories. The cross and circle marker stand for the linear and circular trajectory, respectively.

4.1 Simulated Displacement

To obtain the simulated displacement, the boundary conditions have to be modified such that the given trajectory $\varvec{u}(t) = \begin{bmatrix} u_x(t),&u_y(t),&u_\theta (t) \end{bmatrix} ^\top$ can be followed. The slippage of the upper layer is forbidden in the simulation and the after touch correction of the displacement is not possible. The simulation is described by the following BCs:

$$\begin{aligned} \theta ^- = 180^\circ , \quad \frac{\mathrm {d}\theta ^-}{\mathrm {d}s} = 0, \quad x^- = l,\ \ \theta ^+ = u_\theta , \quad x^+ = u_x, \quad y^+ = u_y. \end{aligned}$$

(9)

The simulation is stopped when the upper layer touches the lower layer and the displacement is computed.

For the purpose of the experiments, 24 different trajectories were generated. Two of these trajectories were linear [2] and circular [6]. Other trajectories were generated for $\eta _t$ interpolated logarithmically between the values: $10^3$ and $10^6$. These trajectories were then used to fold several materials, which differ in weight to stiffness ratio $\eta _m$ in a range from $10^3$ to $10^6$. The measured displacements are shown in Fig. 8. It can be seen, that the displacement is zero in the case where $\eta _t = \eta _m$. It represents the situation in which the weight to stiffness ratio was estimated accurately before trajectory generation - i.e. the ideal scenario. The displacement is more or less enlarging as the estimation is moving away from the ideal scenario. Furthermore, the displacement is not symmetric. For the situation where $\eta _t > \eta _m$, the displacement is lower then for the situations where $\eta _t < \eta _m$. It suggests that one should select the trajectory on the pliable end of the known range of the material weight to stiffness ratio $\eta$. It means, that for the given $\eta _m$, we should use the larger $\eta _t$ for the trajectory generation. It will results in the satisfactory folding for all materials in the range. Moreover, the smaller range will results in smaller displacement.

Fig. 9.

Measured real displacement.

4.2 Real Displacement

We performed a real experiment with the CloPeMa testbed, where an industrial arm was used to fold the fabric strips. Strips with different weight to stiffness ratios were folded by each generated trajectory. The weight to stiffness ratios were estimated using the maximal height of the folded strips. To avoid the slipping of the strips on the table, the non grasped side was fixed to the table with the tape. The measured displacement together with simulated displacements for the estimated material are shown in Fig. 9. The measurements show, that simulation overestimated the oriented displacement for all situations except one, where $\eta _t = \eta _m$. Nevertheless, the hypothesis that trajectory on the pliable end should be selected is confirmed for the real displacements too.

4.3 Model Inconsistency

The simulated displacements overestimation suggests that there is an inconsistency in our model. To analyse this inconsistency, we performed two experiments: the first to check the repeatability of the real displacement and the second to compare simulated and real shapes of the strips during folding.

The repeatability experiment was performed for one trajectory ($\eta _t = 3.7\cdot 10^5$) and one strip ($\eta _m = 5.5 \cdot 10^4$). The strip was folded 5 times and the measured variance of the displacement was 0.5 mm. It shows satisfactory repeatability of the real robotics folding.

The second experiment compares the simulated and real strip states. For the comparison purposes, the camera was added to the testbed and its field of view was geometrically calibrated. The calibration was used to project the simulated model into the image. The simulated state and the real state were compared visually. We observed, that the model is inconsistent in the states shown in Fig. 10. The inconsistency is probably caused by the missing hysteresis model between the bending moment and the curvature. The hysteresis modeling in the bending of fabrics was described in work [4]. In our future work, we will add the hysteresis component into our simulation in order to resolve this inconsistency. The bending model will then be nonlinear and the more complex estimation technique for material properties will need to be developed. Nevertheless, this inconsistency in the model does not influence the selection of the folding trajectory for the fabrics.

Fig. 10.

The inconsistency in the model bending behaviour. The real strip is red, the simulated is shown in green color and the followed trajectory is blue. The difference between the simulation and the real strip was negligeble until the situation shown in 10a was reached. During the situation 10a, the simulatated model skips in a single step considerably. The real strip catches up the simulation with a small delay but the touch point was shifted w.r.t. shown in simulation 10b. (Color figure online)

4.4 Fabric Folding

Our last experiment measured the displacement for whole range of typical fabrics when folded by four trajectories only. We divided the fabrics strips into four categories based on the estimated weight to stiffness ratios $\eta _m$. For these categories, the trajectory on the pliable end of the range (generated by $\eta _t$) is selected and the folding is performed on the CloPeMa testbed. The measured displacements are shown in Table 1. The table shows, that displacement is maximally 10 mm for all used fabric strips of length 1000 mm. For the purpose of robotic garment folding, this is an acceptable displacement. The displacement can be further reduced if more categories are used.

Table 1. Measured displacements when only four trajectories are used for fabric strips folding. The estimated weight to stiffness ratio is denoted $\eta _m$ and the weight to stiffness ratio $\eta _t$ was used to generate the folding trajectory. The height of folded strip $h_m$ was used for the estimation.

5 Conclusions

This contribution examined the robotic folding accuracy as a function of the material weight to stiffness ratio $\eta$. Several experiments were conducted in simulation as well as in real robotic testbed. The experiments measured the folding accuracy based on the displacement of the excepted and the real position of the grasped side of the completely folded strip. Different materials, varying in weight to stiffness ratio $\eta$ were folded and we observed that the trajectory on the pliable end of the known range provides the satisfactory folding accuracy. Based on the observation, we proposed a methodology for folding of the materials with roughly estimated weight to stiffness ratio. The methodology designs a trajectory for the range of the weight to stiffness ratio and accurate estimation is thus not necessary. The demonstration of the methodology was provided for the fabric strips. We shown, that only four trajectories are required to fold all typical fabrics used for the garments.

In the experiments, we observed the inconsistency between the simulation and real strips. This inconsistency causes an overestimation of the simulated displacement. We think, the inconsistency can be resolved by adding the hysteresis into the model and we will address this task in the future work.