Introduction

The field of agriculture is facing new challenges worldwide in terms of supply and demand. Deployment of large-scale smart farms and precision agriculture is expanding and the development of agricultural robots is on the rise. For example, the traditional harvesting of fruits and vegetables is a labor-intensive operation that demands tedious manual operations. However, the introduction of agricultural robots increases efficiency and ensures competitiveness by reducing harvest-dependent labor (Shamshiri et al., 2018). A recent study has categorized fruit harvesting using robots into perception, manipulation and end-effector (Jun et al., 2021).

Various types of fruit perception methods have been studied for the purposes of autonomous harvesting (Kamilaris & Prenafeta-Boldú, 2018). It is difficult to identify fruits and vegetables due to issues such as the target fruit (the fruit to be harvested) overlapping with other fruits or leaves, unstable lighting conditions and occlusion. One of the ways to solve this is by using deep learning technology. Hou et al. (2016) employed convolutional neural networks (CNNs) to input fruit images directly into the network without feature extraction, improving the training time efficiency. Since then, various applications have emerged alongside region-based convolutional neural networks (R-CNNs), which combine CNNs with region-of-interest (ROI) approaches. Bac et al. (2017) detected orchard fruits, including mangos, almonds and apples using Faster R-CNN, which merges region proposals and object classification and localization into one unified deep object detection network. The Mask R-CNN technology accurately differentiates between ripe and unripe strawberries, marks object areas with bounding boxes and extracts these areas from the background at the pixel level. This algorithm overcomes the difficulties of poor universality and robustness using an existing machine vision algorithm in an unstructured environment (Yu et al., 2019). The ability to automatically detect the outline of an object by retrieving information about that part can also be introduced. Lin et al. (2019a) presented a framework for detecting a wide array of fruit types in the natural environment. Sub-fragments of interest were detected by bidirectional partial shape matching (PSM) and were grouped as fruit candidates using a novel probabilistic Hough transform (PHT). They were eventually excluded as false candidates using a support vector machine (SVM). Lin et al. (2019b) proposed a global point cloud descriptor (GPCD) algorithm based on angle/color/shape information.

The various fruit data showed high detection precision and recall with averages of 0.879 and 0.821, respectively. Furthermore, fruit perception has been applied in a new form that integrates stereo vision and simultaneous localization and mapping (SLAM) systems (Chen et al., 2021).

Robots and intelligent machines applied in agriculture bear witness to the development of modern agricultural technology. One of the classic topics, the kinematic control of manipulators, has many applications in agriculture. Arad et al. (2020) described common motion control as manipulation to move closer to the target fruit and then transfer the fruit to a container after harvesting. Kurtser and Edan, (2020) presented the traveling salesman paradigm (TSP) as a methodology for planning the work sequence of a harvesting robot by considering both cost and traveling time. This method reduced the harvesting time by up to 12%. Wang et al. (2022) proposed a novel trajectory planning method for a fruit harvesting robotic manipulator to generate smooth motion based on cubic polynomial functions, reducing the average harvesting time by 36.77%. The method can be used in real-time and detects continuous collisions. With the growing interest in harvest automation, multi-manipulators that reduce harvesting time and improve productivity have been attracting increasing attention. Using field experiments, Barnett et al. (2020) confirmed that efficient work distribution of kiwifruit harvesting using multi-manipulators reduces harvesting time. SepúLveda et al. (2020) used multi-manipulators to visualize their workspace in 3D for harvesting aubergine in a collaborative manner. The decision-making process was based on the 3D position of aubergine centroid, which is its center of mass.

Mechanisms for end-effectors have been explored by considering fruit characteristics. Xiong et al. (2019, 2021) developed a strawberry-only end-effector equipped with a cable-driven gripper for strawberry harvesting robots, whose design considers the characteristics of strawberries that grow. It has been designed to swallow hung strawberries from below and ensure a stable harvest by efficiently handling fragile strawberries. In this way, control strategies of the agricultural robot end-effector and gripper are being explored by placing importance on fruit characteristics (Zhang et al., 2020). Several types of end-effectors for fruit and vegetable harvesting have been developed, as shown in Fig. 1. Bac et al. (2017) developed the fin-ray and lip-type end-effectors by identifying the characteristics of sweet pepper, which are difficult to detect due to their wide leaves. The fin-ray end-effector adjusts its four fingers to the fruit’s curvature for gripping and then cuts the pedicel with the scissors attached to its fingers. The lip-type end-effector uses a suction cup and vacuum sensor to grab the fruit and then close both the lips to cut off the pedicel. The two lips move independently of each other; thus, information on the posture of the pedicel is not required. The harvest success rates of lip-type and fin-ray end-effectors were significantly improved to 2% and 6%, respectively, when the harvesting environment was complex. The improvements were 26% and 33%, respectively, when the harvesting environment was simplified. Chiu et al. (2013) designed four fingers with foam sponge pads inside an end-effector to minimize damage to the fruit when gripping. Although it showed high accuracy, the sample size (25 sample tomatoes) was remarkably small and the experimental data collected in only one environment was unreliable. Silwal et al. (2017) used a harvesting method by holding apples and rotating them. The proposed picking pattern included a combination of horizontal pull, inclined pull and twisting. Wang et al. (2019) designed an end-effector with a bite mode based on a snake’s head mechanism. Citrus fruits growing in their natural environment are very random; thus, for a single citrus fruit, it can be assumed that the fruit is spherical in shape and that the interaction point between the citrus and its stalk is on the harvesting plane.

Fig. 1
figure 1

Growth patterns of various fruits and the recently developed robot end-effectors for harvesting

Full size image

Due to the end-effector structure, fruits such as tomatoes, apples and kiwis are damaged due to their fragile nature and are prone to bruising (Ji et al., 2021; Zhang et al., 2018). Therefore, end-effectors should be designed by integrating modules that can assist in cutting and grasping, such as suction modules and soft air pillows (Saleh et al., 2020). In addition, a transport module has been developed to save transportation time (Mu et al., 2020). As such, agricultural end-effectors should be designed from the perspectives of cutting, identifying and transporting fruits upon harvesting (Shintake et al., 2018). In the cutting module, the issues of disease transmission and cutting fruits and vegetables with fiber need to be explored. To solve these problems, several methods have been employed, such as repeatedly immersing the cut surface in skimmed milk (Van Henten et al., 2002) and cutting stems with high voltage (Bachche & Oka, 2013). However, these methods eventually lead to blade replacement; thus, simpler mechanisms are required because a complicated mechanism is difficult for users to replace.

Grasping has been applied in two ways to separate the fruit from the plants as a suction module: cutting and separating. In the former case, Fujinaga et al. (2021) showed a 15% damage rate due to a suction cutting device; in the latter case, soft and delicate fruits that are sensitive to impact, such as tomatoes and strawberries, have been harvested by a finger-type gripper with a success rate of only 72.86% (Liu et al., 2020). To solve this problem, Jun et al. (2021) used a cutting method with a suction module, a kirigami-based suction pad (Choi et al., 2019), to minimize the damage to tomatoes. Through this design, items can be grasped using a soft vacuum suction cup that adheres well to the curves of fruits and vegetables. To the best of the authors’ knowledge, not many studies on transporting modules have been conducted, where a significant harvesting time is consumed. Therefore, it is necessary to develop this module to solve this problem.

The objectives of this study were as follows: (a) to design methods that enable easy cutting and prevent disease transmission, considering fibers in fruits and vegetables; (b) to design a method that can grasp target fruits with minimal damage compared to the existing finger-type end-effector; (c) to design a transporting module for reducing the total harvesting time; and (d) to conduct field experiments to evaluate the performance of the end-effector that integrates each novel module designed in (a), (b) and (c). The research hypothesis is that regardless of the pedicel’s angle, when it enters the cutting area of the end-effector, it will have a high harvest success rate and reduce the harvesting time. To prove this hypothesis, this study developed an integrated end-effector that can enter from various angles and conducted a performance evaluation experiment (Fig. 2).

Fig. 2
figure 2

The end-effector structure integrating the cutting, suction, and transporting modules (Jun et al., 2020)

Full size image

Integrated end-effector design: cutting, suction and transporting modules

Jun et al. (2020) developed the required functionality for the harvesting end-effector by subdividing the operations into three modules. In this study, the holistic requirements for harvesting robots were considered. These requirements were designed by classifying modules. The modules designed were tested and checked for possible applications to fruits. The classified modules and their goals are as follows:

  • The cutting module’s goal is to detach fruits from the pedicel, which is an essential harvesting process.

  • The goal of the suction module is to prevent damage to the fruit and assist in the processes of approaching and cutting.

  • The goal of the transporting module is to respond to the various postures of the end-effector for reducing transporting time.

  • The proposed integrated end-effector has a simple mechanism and includes cutting, suction and transporting modules.

Cutting module

Cutting the pedicel is a fundamental requirement during harvesting. Most farms use scissors as a cutting tool to guarantee ease of use and performance. A previous study used a scissor structure for cutting (Jun et al., 2020). However, most end-effectors with scissor mechanisms cannot be easily replaced because the scissor blades are designed to have complex shapes or are mounted internally.

Complex shapes cannot prevent disease transmission because they are difficult to disinfect and clean. When harvesting fruits or vegetables, methods to prevent disease transmission can usually be divided into disinfection of equipment after harvesting (e.g., spraying a disinfectant and thermal cutting) and replacement of harvesting equipment (e.g., cutters). However, the limitation is that disease transmission cannot be prevented completely by spraying a disinfectant. Therefore, the proposed circular saw-type end-effector has a structurally simple mechanism for replacement and maintenance. If the blade is changed periodically, disease transmission can be prevented during harvesting.

Circular saw with traction cutting unit

Generally, a circular saw cuts owing to the rotating motion of its blade in a direction perpendicular to the object. However, a circular saw experiences the unpredictable movement of hanging fruits (e.g., bouncing off and slippage). Consequently, cutting is impossible without grasping the pedicel. To solve this problem, a previously developed traction cutting unit (TCU) was adopted (Jun et al., 2021). Owing to the grasping of the TCU, the proposed cutting unit prevents the undesired movement that hinders cutting. The cutting process using the TCU is shown in Fig. 3.

Fig. 3
figure 3

Cutting process using the traction cutting unit (TCU): a the pedicel pushes the traction supporter; b after the pedicel is entered, the traction supporter returns to its initial position; c the pedicel is pulled by the linear servo actuator and cut by the circular saw; and d the linear servo actuator returns to its initial position

Full size image

Cutting and traction force

As shown in Eq. (1), the required traction force, \({F}_{t}\) to traction through the TCU can be modeled using the back drive force of the linear servo actuator, \({F}_{bf}\). \({F}_{repulse}\) is the repulsive force generated by cutting with the circular saw blade and \({F}_{react}\) is the pedicel reaction force against the traction. The back drive force of the linear servo actuator used in this study was 46 N.

$$ F_{\min .t} > F_{bf} + F_{repulse} + F_{react} . $$
(1)

However, it is difficult to theoretically determine the generated \({F}_{repulse}\) during the cutting process and \({F}_{react}\) by traction. As evident from Wang et al. (2020), cutting energy and speed are correlated. The results depend on state variables, such as the moisture content and cutting angle, even for the same fruit. Furthermore, variables such as the plant growth shape, pedicel state and pedicel location are often uncertain. Further research on \({F}_{repulse}\) and \({F}_{react}\) needs to be carried out by considering the influence of multiple parameters in the future. In this study, the required \({F}_{t}\) was experimentally identified and the back driving force of the linear actuator used was sufficient to drive the traction supporter of the end-effector. For the other required cutting forces \({F}_{c}\), see Eq. (2) given by Kopecký et al. (2014).

$$ F_{c} = k_{c} \cdot b \cdot e \cdot \frac{{v_{f} }}{{v_{c} }}, $$
(2)

where \({k}_{c}\) is the specific cutting resistance, \(b\) is the width of a saw kerf, \(e\) is the workpiece height (depth of cut), \({v}_{c}\) is the cutting speed and \({v}_{f}\) is the workpiece feed speed. To define the mathematical model for power \({\overline{P}}_{cw}\) during the cutting process using the saw blades, it is necessary to apply Eq. (3) based on the sawing kinematics for circular saw cutting (Fig. 4), presented in Orlowski et al., (2013), to include the subsequent steps of overcoming friction during cutting. Notations for the parameters used in Eq. (3) are shown in Table 1.

$$ \overline{P}_{cw} = F_{c} \cdot v_{c} + P_{ac} = \left[ {z_{a} \cdot \frac{{\tau_{\gamma } \cdot b \cdot \gamma }}{{Q_{shear} }} \cdot h_{m} \cdot v_{c} + z_{a} \cdot \frac{R \cdot b}{{Q_{shear} }} \cdot v_{c} } \right] + \dot{m} \cdot v_{c}^{2} \left( W \right). $$
(3)
Fig. 4
figure 4

Sawing kinematics of circular saw cutting

Full size image
Table 1 Symbols for cutting and traction force
Full size table

Preliminary experiment

A separate motor was used to generate sufficient rotational speed in the cutting blade and enable traction at an appropriate velocity. As shown in Fig. 5, a high-speed DC motor (MB3540-12V, Motorbank, Republic of Korea) for sawing and a linear servo actuator (L16-100-63-6-R, Actuonix Motion Devices Inc., Canada) for traction were used to construct the cutting module. The cutting module verified the performance by cutting the pedicel of the fruits and vegetables. A total of 50 experiments were conducted in a testbed environment in a lab for each fruit and vegetable, and the following success rates were obtained: tomato 85% (3.5–6 mm), sweet pepper 70% (8–16 mm), eggplant 95% (6–12 mm) and cucumber 100% (5–7 mm).

Fig. 5
figure 5

The proposed cutting module: a top view and b bottom view

Full size image

Suction module

Detachment from the pedicel was achieved by cutting or separating during harvesting. Detachment by grasping is not suitable for fruits with soft surfaces because a large force is applied to the contact surface. Therefore, the suction method is mainly used for supplementary purposes rather than directly grasping each fruit during harvesting. A specific target within a cluster can be easily grasped according to the shape of the suction cup. This also helps prevent collision damage during the approach process caused by the material and structure used. A vacuum motor is connected through a flexible hose to install a vacuum motor in a separate space, thereby miniaturizing the end-effector and increasing space utilization.

Shape of the suction cup

The shape and material of the suction cup are significant components affecting its performance. The authors designed a suction cup to ensure the flexible grasping of fruits with related, unstructured features. As shown in Fig. 6, the outer suction cup in the form of a skirt adequately wrapped itself around objects having large diameters. The internal suction cup separately sealed the space to maintain suction. Thus, owing to the use of significantly soft materials, damage due to collision during the approach process was prevented. Any impact was buffered by maintaining an adequate gap between the contact surfaces in the perpendicular direction.

Fig. 6
figure 6

The suction cup with the kirigami structure (Jun et al., 2021): a multi-layered structure and b shape deformation by suction

Full size image

Suction force

A suction vacuum can be generated by the pressure difference between the inner and outer surfaces. In such cases, the holding force due to the vacuum pressure, \(P\), is directly proportional to the opening surface area and is calculated by using the holding force (\({F}_{h}\)) and surface area (\(A\)) of the suction pad opening. However, this simple formula does not consider the numerous variables that determine lift. To account for these and other external factors, a safety factor of at least 1.5 or 2.0 should be used for smooth or porous/oily objects, respectively; moreover, a safety factor of approximately 4.0 should be considered when the vacuum suction cup is in a vertical position. In this study, the following formula was used (where \(S\) denotes the safety factor):

$$ F_{h} = \left( {P \times A} \right) \times S. $$
(4)

Additionally, the force required for moving and lifting by suction on a vertical surface from the ground can be expressed by the following equation (where \(m\), \(\mu \), \(g\) and \(a\) denote the mass, friction coefficient, gravity and acceleration, respectively):

$$ F_{h} = \left( {m/\mu } \right) \times \left( {g + a} \right) \times S. $$
(5)

However, during harvesting, fruits are already connected to the pedicel and only a minimum suction force is required to prevent the fruit from being pushed out during the cutting process. The required suction force is affected by the reaction force mentioned in the Cutting Module section, as shown in Fig. 7, which expresses the relationship between the forces.

Fig. 7
figure 7

Relationship between the forces on the fruit by suction

Full size image

Preliminary experiment

The suction cup of the desired shape was created in a mold via 3D printing. The inner and outer suction cups were prepared in separate molds and integrated by bonding. Both suction cups were made of silicone, a soft material. Suction was generated from the outer motor into a flexible hose. The cups were tested for suction with the same type of fruits and vegetables as the holding experiment.

The suction force acted vertically downward on the fruit.

The results are summarized in Table 2. When applying a suction force of 80 kPa, the suction cups endured a suction diameter of 52.4–98.3 mm and a maximum weight of 289.1 g. However, after removing the suction force, the fruit was naturally separated from the suction cup. Although complete adsorption was not achieved in some cases, surface damage by adsorption was not observed.

Table 2 Experimental results of the suction module
Full size table

Transporting module

The transport success rate depends on changes in robot posture required to cut the pedicel. Specifically, grasping and transportation are difficult when the target fruit is in a cluster. Transportation directly affects the success rate and efficiency of harvesting. To address this, a module was developed to maintain a suitable posture continuously. It was connected to the loading area to form a path through which the harvest could be moved, thus including the transportation process (Fig. 8). Consequently, the manipulator can be caught and transferred through the module without any additional operation. The pedicel must fall toward the entrance after being cut and caught. The larger the entrance size, the more stable the transport; however, the range of movement of the harvesting robot is limited. To address this drawback, a design incorporating the properties of several fruits was used (Eizicovits et al., 2016; Kondo et al., 2010). The fruit cut from the pedicel is then dropped vertically to the ground. Thus, for most fruits, the center of gravity from the pedicel is within the diameter of the module, thereby enabling stable transport.

Fig. 8
figure 8

Stable posture to transport falling fruits: a maintaining a vertical posture and b moving to the horizontal plane

Full size image

Transport mechanism

The authors developed a mechanism such that the end-effector could approach and move perpendicular to the ground, regardless of the robot’s posture, to carry the harvested fruit. The structure can rotate separately from the cutting module in the direction of the axis at the end of the manipulator. Consequently, as shown in Fig. 8a, it can always maintain a horizontal state relative to the ground through its weight within a specific range without any additional control. Figure 8b shows the degree of freedom enabling stable transport even when tilted in a horizontal plane; this was achieved through weight and no additional control was required. The module moved by weight returns to its original position after a certain angle with respect to the horizontal plane via spring elasticity. Consequently, it enables natural movement in the forward–backward direction depending on the tilt and can increase the success rate of transporting.

Fruits can, however, collide with the entrance when they fall. They thus need protection during a collision. In addition, even if they collide, the falling direction of the detached fruits must guide them to the inside of the transport mechanism. Thus, it is necessary to prevent harvested fruits from falling outside. Accordingly, a cover, made of the same silicon material as the suction cup, was added to the entrance. Its shape is attached above the entrance and serves to guide the cut fruit into the transporting module.

Preliminary experiment

The transporting module, including the cover and basket, was used to conduct a preliminary experiment for verifying its performance. The transport success rate was confirmed for free-falling fruits from each position by varying the robot postures and cutting angle in the experiments. In addition, an experiment was conducted on fruit diameter regardless of the species and transporting failure was defined as a fruit not entering the entrance. The experimental results have been summarized in Table 3. In most cases, transportation was easily achieved in the range of constant postures (± 90°) for angular changes in the cutting module.

Table 3 Experimental results of the transporting module
Full size table

Experiment for integrated end-effector

Lab experiment

Experimental design

A harvesting robot system was designed to evaluate the performance of the end-effector with the three integrated modules (each module was validated by preliminary experiments). The robot system comprised a manipulator, mobile platform and integrated end-effector, as shown in Fig. 9. The integrated end-effector was attached to a mobile manipulator (i.e., unmanned ground vehicle and manipulator). The authors carried out the experiment in a lab testbed, which was similar to an actual tomato smart farm. The testbed was configured to cluster tomatoes similar to real fields with unstructured positions for harvesting. The experimental scenario was designed based on manual harvesting. According to the experimental scenario, the end-effector followed the control sequence shown in Fig. 10.

Fig. 9
figure 9

Experimental setup of the proposed end-effector

Full size image
Fig. 10
figure 10

Diagram of harvesting control sequence for the novel end-effector, including three modules

Full size image

Experimental results

Two lab experiments were carried out. The first experiment aimed at investigating the application of end-effectors in a range of environments. To confirm the environmental effect, the experiment was carried out under the same conditions, such as the arrangement of tomatoes and the trajectory of the manipulator were repeated 50 times on the testbed. The results were analyzed by the harvest success rate and failure cases. The detailed results are summarized in Table 4. The results showed 43 successes (86.0%) and 7 failures (14.0%) out of 50. The performance of the end-effector was compared to a recent harvest robot review article. Davidson et al. (2020) showed an average harvest success rate of 79.0%. The proposed end-effector demonstrated a higher rate of detachment: 86.0% versus 79.0%. However, the comparison was not sufficiently meaningful because each study has a different definition of success rate. Thus, the end-effector was analyzed from the perspective of failure cases.

Table 4 Analysis of lab experimental results using the proposed end-effector
Full size table

The failure cases were divided into three categories: the pedicel not entering the cutting area of the end-effector (i.e., approach), the pedicel entering but not cutting (i.e., detachment) and the pedicel not entering the transporting module after cutting (i.e., transport). The failure rate was the highest during approach because when the end-effector collided with the surrounding objects, such as other fruits or leaves, the collision caused the tomato to move undesirably. Therefore, the tomato did not enter the desired position (i.e., cutting area) and harvest failed.

The second experiment aimed at Investigating the fruit yield according to transporting time. To this end, 20 experiments were performed by observing the same conditions as the first experiment. The fruit yield depends on the harvesting time, which includes cutting, grasping and transporting. The average picking time (cutting and grasping time) reported by Davidson et al. (2020) was 11.5 s. However, the average picking time does not include the transporting time. Therefore, it is necessary to investigate the transporting time separately. In the second experiment, the transporting time averaged 19.1 s. Assuming that daylight was available for 12 h at the facility, through a simple calculation, the proposed end-effector could additionally harvest approximately 3757 (fruit yield/robot, day) fruits, as shown in Fig. 11. These calculations are based on the average picking time presented in Davidson et al. (2020) and the experimental results on transporting time.

Fig. 11
figure 11

Comparison by including the transportation process: a harvesting time and b fruit yield

Full size image

Field experiment

Experimental design

The performance of the proposed end-effector was verified through preliminary and lab experiments. To validate the effectiveness of the proposed end-effector, field experiments were conducted in actual tomato smart farms. The experiments were carried out under various conditions by including three different environments, fruit species and tilting of the pedicel. The details of the different conditions are shown in Fig. 12. It was impossible to adjust the experimental angle for each pedicel because fruit growth was unstructured. Therefore, the authors adjusted the angle of the y-axis of the end-effector. The experiment was conducted for a total of five cases (− 30°, − 15°, 0°, 15°, and 30°) for each entry angle. For accurate case setting, the posture of the manipulator was adjusted. After aligning the pedicel with the \(x\)- and \(z\)-axes, as shown in Fig. 13, the end-effector was moved straight along the \(y\)-axis direction to ensure that the pedicel entered the cutting area. The harvest sequence is shown in Fig. 14.

Fig. 12
figure 12

Field experiment environments and thickness of the pedicel: Morning-tomato farm (Place A 5–8 mm); smart farm belonging to the Korean Rural Development Administration (Place B 8–11 mm); Culti-farm (Place C 4–8 mm); and the structure of the cluster tomatoes on the right

Full size image
Fig. 13
figure 13

Experimental setup and harvesting method for the cluster tomatoes. The co-ordinates of the \(z\)-axis were arbitrarily set in the direction of the pedicel and adjusted to be parallel to the co-ordinates of the end-effector. The parallel state was referred to as 0° and the angle was adjusted by rotating the y-axis. The manipulator was moved forward along the \(y\)-axis to enter the cutting area

Full size image
Fig. 14
figure 14

Harvest in progress according to the harvest sequence of the end-effector. The upper- and lower-line figures show the side and top views, respectively. a The harvest start stage was adjusted so that the traction supporter of the end-effector approached the pedicel closely. b The pedicel passed through the traction supporter and entered the cutting area. c In the traction supporter, the pedicel was isolated. The linear servo actuator was operated to start traction. Simultaneously, the DC motor was operated and the circular saw rotated to cut the pedicel; thus, d the pedicels were cut by the circular saw

Full size image

Experimental results

The results of the harvest success rate experiments are shown in Fig. 15 and Table 5. A total of 160 cluster tomatoes were harvested, with 129 successes and 31 failures. The performance of the end-effector was evaluated by the harvest success rate and harvesting time for each case. The harvest success rate can be defined as a situation in which the cluster tomato pedicels were cut by the harvest sequence of the end-effector, as shown in Fig. 10. The failure cases were divided into two categories: the pedicel not entering the cutting area of the end-effector (i.e., approach) and the pedicel entering but not cutting (i.e., detachment).

Fig. 15
figure 15

Results of the field experiments according to the end-effector angle: a harvest success rate and b harvesting time

Full size image
Table 5 Analysis of field experiment results using the proposed end-effector
Full size table

The success rate for the total harvest was found to be 80.6%. It is worth mentioning that this result was achieved by varying the angle of entry. In addition, the success rate by environment was measured as 76.2%, 88.2% and 89.0% for Places A, B and C, respectively; the harvest success rate at Place C was the highest. The harvest success rates were 75.0%, 71.9%, 93.8%, 81.2% and 81.2%, in the order of − 30°, − 15°, 0°, 15° and 30°, respectively. The highest was at an end-effector angle of 0°, followed by 15°, 30°, − 30° and − 15°. The good performance was due to the high success rate at 0° (93.7%). Also, the performance at the other angles was about 80.6%, which could be entered from a variety of angles. These results were more reliable because they were obtained through experiments conducted in various environments. When harvesting cluster tomatoes, the average time was 15.5 s. Each harvesting time was 20.2, 16.0, 13.5, 13.7 and 14.1 s, in the order of − 30°, − 15°, 0°, 15° and 30°, respectively, at an end-effector angle of 0° (followed by 15°, 30°, − 15° and − 30°). In addition, the harvesting time in each environment was 16.0, 21.5 and 9.5 s for Places A, B and C, respectively; the best harvesting time was at Place C.

Next, the harvest success rate and harvesting time were calculated for the number of tomatoes in a cluster. The results are shown in Fig. 16a. When three tomatoes were in a cluster, the harvest success rate was slightly higher compared to cases of one or two tomatoes in a cluster. Therefore, the pedicel easily entered the cutting area owing to the strong vertical force (i.e., pushing force on the traction supporter). However, as shown in Fig. 16b, the results indicate that a large number of tomatoes caused the problem of increasing the harvesting time. The reason for this issue is that the heavier the cluster tomatoes were, the greater was the load (i.e., vertical force) on the linear servo actuator operating the traction supporter.

Fig. 16
figure 16

Results of field experiments considering the number of tomatoes: a harvest success rate and b harvesting time

Full size image

Discussion

Failure cases

The reasons for failure observed in the lab and field experiments will be discussed in this section. The reasons for harvest failure can be largely divided into three problems:

  • Incomplete fruit detachment from the pedicels.

  • Difficulty in cutting owing to the thickness of the pedicel.

  • Collision between the stem and end-effector interrupting pedicel entry into the cutting area.

In the preliminary experiments, the traction and cutting performances were evaluated for the pedicel that completely entered the cutting area. However, the field experiments involved the pedicel entering the cutting area.

The first main reason for harvest failure, incomplete detachment, was caused by the tearing of the stem owing to the plant fiber. Compared to Places B and C, the stem in Place A tore easily. These fibrous properties caused the fruit to detach while being attached to the pedicel (i.e., incomplete fruit detachment). The results of the harvest success rate in Places A, B and C (76.2%, 88.2% and 89.0%, respectively) partially show that the stem of Place A could easily tear apart.

The second main reason for harvest failure, difficulty in cutting owing to the thickness of the pedicel, caused an increase in harvesting time. To confirm this problem, the thickness of the tomato pedicels in Places A, B and C was measured. The tomato pedicel thicknesses in Places A, B and C varied between 5–8, 8–11 and 4–8 mm, respectively (Fig. 12). It was confirmed that the tomato pedicels in Place B were thicker than those in the other two environments. As evident from the graph in Fig. 15, a longer harvesting time was recorded for Place B compared to the other environments. However, the harvest success rate was almost similar to Place C (89.0%). Instead, the harvest success rate of Place A (76.2%), which had a medium pedicel thickness, was the lowest. This result shows that the thickness of the pedicel only affected the harvesting time.

The final main reason for harvest failure was the collision between the stem and end-effector, which interrupted pedicel entry into the cutting area. When the stem and pedicel angles were close to 90° (Fig. 12), the end-effector was interrupted by the stem and could not enter. To solve this problem, an adjustment of the end-effector angle was considered. As shown in Fig. 17, the interruption problem of pedicel entry was solved by adjusting the angle. In other words, by adjusting the posture of the end-effector, obstacles (e.g., the stem) could be avoided, leading to a successful harvest.

Fig. 17
figure 17

The problem of the tilted end-effector: a As the cross-sectional area of the pedicel increases, the harvesting time increases; b as the contact surface increases, the torque required to push the traction supporter becomes insufficient

Full size image

Additionally, there were some other reasons for harvest failure caused by the tilted end-effector. Due to the tilted end-effector, the pedicel entering the cutting area skewed diagonally, as shown in Fig. 17a. It did not cut perpendicularly to the circular saw of the cutting module but cut diagonally. The cross-sectional area of the pedicel thus increased compared to when the pedicel was cutting vertically. For this reason, when the angle of the end-effector was changed, as shown in Fig. 15b, the harvest time increased. Figure 17b shows an area in which the traction supporter and the pedicel’s contact surface overlap increased. Because of this, the pedicel was closer to the pin axis and interfered with the movement of the traction supporter. Thus, more force was required when the pedicel pushed the traction supporter to enter the cutting area. These issues should also be considered. Most of the harvest failure scenarios were observed during the first sequence: the pedicel entering the cutting area, the first of the proposed end-effector’s sequences. To confirm this, the harvest success rate was calculated separately in the event of successful traction and was found to be 94.4%. In other words, if the traction process was reached, most fruits can be harvested and most failure scenarios can be attributed to traction failure. Therefore, by improving the pin structure for rotating the traction supporter in Fig. 18b, the performance can be sufficiently improved if obstacles are minimized when entering the cutting area.

Fig. 18
figure 18

The tilted end-effector avoids the stem and cuts the pedicel: a end-effector angle of 0° and b end-effector angle of 30°

Full size image

Improved harvesting robot

In the future, the authors aim to introduce improvements to design robust harvesting robots with the goal of harvesting various fruits autonomously. Such improvements can be largely divided into three categories: (1) force/torque control of the circular saw for robust detachment; (2) decision of the cutting point for an autonomous harvesting system; and (3) extending the research to harvest a variety of fruits.

First, the authors are currently conducting a study to control the force/torque of the circular saw for robust detachment regardless of the type of fruit or vegetable. An experiment was conducted, details shown in Table 5, to calculate the harvest success rate separately in the event of successful traction; it was found to be 94.4%. This means that the success rate of detachment was sufficiently high and the proposed constant force/torque control was effective in detaching the tomatoes. However, applying a constant force may prove problematic when expanding the system for application to various fruits. To solve this problem, the cutting module, which can control the force/torque of the circular saw, is being further developed.

Second, to harvest using the manipulator, fruit perception must be achieved; in particular, detecting the pedicels for determining the cutting point is important (Arad et al., 2020; Barth et al., 2019). The authors are currently conducting a study to detect the pedicel and determine the cutting point. Following successful determination, owing to the wide cutting area of the cutting module, the proposed end-effector can respond to various fruits. Consequently, the authors believe that the detection method and proposed end-effector can be configured into an improved harvesting robot.

Finally, the authors are developing a harvesting robot for expanding the range of applications by identifying other harvestable fruits, such as cucumber, Korean melons and paprika. For cucumber harvesting, the authors are working on developing a cucumber harvesting robot based on a human-centered harvesting method. This harvesting method comprises genetic algorithm-based ordering, where the cutting point will be approached through robust visual servoing. The proposed end-effector and these three improvements will be incorporated in the future and the effectiveness of the systems will be verified through field experiments as soon as possible.

Conclusions

In this study, a novel harvesting end-effector has been proposed to improve harvesting efficiency. The proposed end-effector comprises cutting, suction and transporting modules. The cutting module should provide sufficient cutting force and prevent disease transmission for end-effectors. The authors thus developed a robust cutting module with a simple mechanism to address these problems. A suction module was also designed to prevent possible damage owing to the processes of grasping and approaching. Finally, the transporting module was utilized to reduce the harvesting time by developing a transporting module that is maintained in the direction of gravity for transportation.

The performance of the end-effector, which integrates the three modules, was evaluated through lab and field experiments. The field experiments were conducted in three tomato smart farms under varying conditions, e.g., different fruit species and pedicel thicknesses were considered, and 160 cluster tomatoes were harvested. The performance was measured in terms of the harvest success rate and harvest time. The success rate for the total harvest was found to be 80.63%. At the end-effector angle of 0°, the harvest success rate was high (93.8%) and the harvest time was short (13.5 s). These results indicate that the proposed mechanism can contribute to efficient harvesting. Further, the failure cases were analyzed in-depth for addressing the limitations of end-effectors. The authors have also discussed a future study to improve the performance of harvesting robots. These recommendations will motivate subsequent studies to expand the application to various fruits and vegetables and will prove beneficial for improving the performance of harvesting robots.