Robotics for Precision Viticulture

Abstract

Grapes are included in what are called specialty crops, fruits of high return value typically set in orchards, which account for 50% of the total value of crop production in the USA, and about 70 billion Euro per year, representing 22% of the total output value of the agricultural sector for Europe in 2014. This chapter exposes the advantages that robotics can bring to grape production, the key challenges faced by the introduction of robots in the vineyards, and some of the current technologies being applied to overcome such challenges.

5.1 Technological Needs, Barriers, and Current Solutions for Competitive Vineyards

Grapes are included in what are called specialty crops, fruits of high return value typically set in orchards, which account for 50% of the total value of crop production in the USA, accounting for $60 billion in 2005 (Burks et al. 2008) and reaching $76 billion in 2012 (USDA 2012). In Europe, specialty crops are valued at about 70 billion Euro per year, representing 22% of the total output value of the agricultural sector in 2014. The fruit and vegetable sector alone accounts for about 45 billion Euro, with a total production of 40 million tons of fruit. In the transformation of grapes into wine, Europe is the global market leader accounting for 45% of the world’s wine-growing area in 2014, 65% of production (167 Million hectolitres), 57% of global consumption and 70% of exports in global terms (Wine Institute 2016). The stable and privileged position of a wine in the global market depends on its long-term reputation, which takes considerable effort to attain but can be lost rapidly when a given standard is not assured. It is a known fact in viticulture that the best wine is made in the vineyard rather than in the winery, because grapes of high quality are the best guarantee for producing excellent wines. When the grapes are medium quality, efforts in the winery might correct certain defects, but will unlikely lead to premier wines. In Europe, Spain, France and Italy account for 32% of the total area devoted to vineyards in the world, followed by China with 11%, Turkey with 7% and the USA with 6% (Fig. 5.1a). The total production of wine in the world has increased 6.4% from 26,544 million litres in 2011 to 28,230 million litres in 2014. France, Italy, Germany and Spain alone account for 49% of global production, that is, almost half of the worldwide production as shown in Fig. 5.1b (Wine Institute 2016), resulting in 13,833 million litres of wine being produced in these four European countries only.

Fig. 5.1

World figures in the wine industry: a vineyard area and b wine production

It is possible to make great wine by chance or a good recipe, but not consistently; only by measuring and controlling key factors can the best wine be ensured year after year (Cox 1999). To make good wine consistently, it is necessary to test the grapes weekly to associate certain tastes with certain changes in measured properties. Weekly monitoring in the crucial weeks preceding harvest will allow the development of new management strategies for harvesting grapes in diverse zones at different times, which avoids mixing grapes of different degrees of maturity, a common source of poor wines. However, modern production of wine grapes, i.e. that based on objective and precise field data, is inefficient for the majority of growers for the following reasons:

• Monitoring cost: it is expensive to acquire field data. It can be done only once a year in general, which deters the updating of field information and of assessing the evolution of vines during the growing season. The nitrogen content, for example, varies continuously as fertilizer or water are applied.

• Low rate of sampling: it is not feasible to ask an operator to obtain sample data every metre, and as a result, measurements are usually sparse, say once every 400 m² (20 m × 20 m). With data from a sparse sample, conclusions can easily be biased. As a result, a weekly assessment of grape ripening in the six weeks prior to harvesting tends to be either too sparse or unaffordable.

• Weight of current hand-held devices: recording data for hours with a handheld device of several kilograms of mass (typically 2–6 kg) becomes exhausting for the operators, who also have to walk in the sun, usually in the summer.

• Costs of service providers: there are some service providers who can provide maps from airborne information, but they tend to be of low resolution. If several measurements are needed to determine how the plants evolve during the season, the cost of around 80 € ha⁻¹ (8camera 2016) means that monitoring 30 ha six times, for instance, will cost 14,400 € per season, which is prohibitive. The easiest way to have full control of information is by having full control of the scouting machine: you pay once and can use it as many times as needed. This is how farmers and field managers typically want to operate with machinery, and consequently is a promising approach to reach commercial success.

• There are very few suitable commercial robots to work in vineyards or other open agricultural fields. The majority of robots that exist today are at the laboratory stage and typically represent scientific proofs-of-concept. They are too complex and not reliable enough to cope with a 6- to 8-hour working timeframe. Some initial initiatives, however, are appearing but they only operate in small areas where technicians can assisst quickly and cost-efficiently.

5.1.1 Fertilization, Nutritional Status and the Estimation of Nitrogen Content

The greenness of a plant has traditionally been an accepted indicator of plant health. Some handheld devices such as SPAD^® (Spectrum Technologies, Inc., Aurora, IL, USA) are used to determine deficiencies in leaves, mainly nitrogen, by estimating chlorophyll activity. However, these small meters have to be clamped over leafy tissue to calculate the chlorophyll index. Even though they are non-invasive, the need to clamp the leaves prevents these devices from being implemented on vehicles, and are consequently not efficient for robotic applications. An indirect way to assess nitrogen content, and therefore its deficiency, has been done by plotting the normalized difference vegetation index (NDVI) of canopies, an optical method based on the enhanced reflectance from a healthy canopy in the infrared spectrum. Differences in reflectance can be considerable between weak and healthy plants. The NDVI can be monitored from the air. An aerial map covering 10 ha with approximately 200 images obtained at a height of 80 m and reaching a resolution of 30 mm pixel⁻¹ might cost around 800 € plus transport of equipment and operators to the test site (8camera 2016). On the other hand, NDVI can also be determined from a ground vehicle such as a conventional tractor, a utility vehicle or a robot. Some handheld devices can be fixed to conventional farm equipment to monitor nitrogen content (Fig. 5.2). Alternatively, machine vision in the near infrared band can be used to estimate the relative variation of a vine’s canopy coverage within a vineyard, when images are recorded from the top of a moving vehicle equipped with a GPS to generate a map of plant vigour (Saiz-Rubio and Rovira-Más 2013).

Fig. 5.2

Ground-based NDVI estimation: a CropCircle^® ACS-470 kit as a handheld device and b GreenSeeker^® RT200C mounted on a conventional farm vehicle

5.1.2 Pruning and Pre-pruning

Pruning is a crucial operation in viticulture because it influences the development of the vine in the forthcoming season. Although pre-pruning the vines is easy to mechanize, pruning them in winter requires skills typically gained through years of experience, and it is consequently done manually by dexterous operators. Because of the lack of skilled labour in the winter to perform this operation, some wine-producing areas in Europe have indicated the need to introduce automation for this delicate task. A pruning robot was developed by Botterill et al. (2017). It is a mobile platform that straddles the row of vines. The plants are completely covered, such that sunlight is blocked to benefit computer vision processing. Images are taken with three cameras as it moves. The computer vision system builds a three‐dimensional model of the vines and an artificial intelligence (AI) system decides which canes to prune. An articulated arm of six degrees of freedom executes the cuts.

5.1.3 Irrigation and the Control of Water Stress

Some vineyards, and even entire wine-producing areas, do not use irrigation in vineyard management. However, when available, precise control of water stress by suitable rates of irrigation might become an influential factor in the final quality of the grapes and of the future wine. Such control requires constant feedback on the state of the plant, which evolves continuously during the production season and especially in relation to the weather. The measurement of canopy temperature as an indicator of stress was first identified in practice in 1981 (Jackson et al. 1981) with the definition of the Crop Water Stress Index (CWSI). Temperature differences between stressed and unstressed plants have encouraged the use of thermal images to assess water status. In addition, the continuous decrease in cost of compact thermographic cameras that can be mounted on agricultural vehicles, and even small aerial vehicles, has extended its use from initial defense applications to commercial civilian use including agriculture. However, there are still many technical hitches that limit their generalized use for automated solutions from field vehicles (Stoll and Jones 2007):

• The monitoring of stomatal activity in leaves requires the robust exclusion of sky, soil and grapes from infrared images.

• Sunlit canopies give a far wider range of temperature variation than shaded areas.

• Reference surfaces are required to calibrate the thermal images and estimate the temperature of leaves under wet and dry conditions prior to applying the CWSI.

5.1.4 Grape Harvesting: Deciding the Most Critical Moment for Winemaking

According to Cox (1999), there are three factors, sugar, titratable acidity (TA) and pH, that can be tracked weekly after véraison (colour change in red grape berries identifying ripening) and that will reach optimum levels when the grapes are ready to harvest. The pH is related to TA, but differs from it in significant ways because the pH of grape juice might or might not be correlated with the amount of tartaric acid. Unfortunately, these three properties require some grapes to be obtained and the juice extracted to measure these chemistry-based properties. This makes it impossible to measuring them ‘on-the-fly’ and by non-invasive techniques (fast measurements without touching the grapes), which are fundamental for an automated solution such as airborne imagery (remote sensing) and ground-based platforms carrying monitoring sensors onboard (proximal sensing from farm equipment and field robots). In general, the statistical significance of these measurements is weak because of the lack of intensive sampling. In addition, laboratory analyses require several days to obtain the data and are typically too costly for average producers if they want to have a well-sampled vineyard. The monitoring of traditional key properties that determine the ripening status of grapes, namely sugar, acidity and pH, has to be done manually by sampling certain bunches in the field. According to experts, it should be done regularly in the weeks before harvesting to obtain meaningful results. As the grapes grow under the canopy, aerial images cannot reach them, therefore remote sensing and proximal sensing from aerial images (unmanned aerial vehicles or drones) cannot be used for this purpose. Only ground-based monitoring is feasible for determining the maturity of red grapes.

The measurement of anthocyanins in the (red) grape skin provides an alternative and indirect method to assess maturity. Anthocyanins have been chosen as markers of phenolic maturation because their evolution with ripening is equivalent to that of skin tannins (Agati et al. 2007). This has resulted in the development of new sensors. The spectrophotometer Spectron^® (Fig. 5.3a) announced by Pellenc (Matese and di Genaro 2015) and the Multiplex^® (Fig. 5.3b), released by Force-A (Orsay Cedex, France), are two examples of the growing interest in developing handheld sensors. However, there is currently no off-the-shelf sensor that can estimate the maturity status of grapes from a moving platform before harvesting. To obtain a map of anthocyanins before harvesting and at an adequate sampling rate would require the services of a company with a handheld device to walk along the rows and make multiple measurements. The data, the map and its scientific interpretation would also incur costs.

Fig. 5.3

Handheld maturity assessment: a Spectron^® (Pellenc) and b Multiplex^® (Force-A)

The generation of a manual map of anthocyanin levels in red grapes is possible with a handheld device such as those depicted in Fig. 5.3. Typical coverage might involve measuring 400 bunches per hectare, i.e. a point every 10 m × 10 m as the average of four representative grape bunches. At present, there is no commercial device to measure the anthocyanin content on-the-fly. Even though the European-funded VineRobot project (VineRobot 2014) worked for three years on a novel device to assess anthocyanins from a moving robot by combining computer vision and fluorescence, it was not feasible to measure anthocyanin levels at a distance of 40 cm from the grapes. Fluorescence-based sensors like the one shown in Fig. 5.3b usually analyse a circular spot of reduced diameter, typically between 4 and 8 cm. If the spot contains over 3% of green matter, the fluorescence reading is usually unreliable, and therefore must be discarded from maps of maturity. The reason behind this rationale is the large response of green matter to fluorescence when compared to anthocyanins, which typically masks the readings even with few leaves, stems, tendrils or peduncles. The resolution affordable with handheld devices, say 400 measurements per ha, would result in working cells of 100 m². However, an automatic system onboard a robot could obtain several measurements per metre in each scanned row if grapes are not overly occluded. This can be prevented by defoliation, a traditional operation in many vineyards to increase the sun’s radiation on maturing bunches.

5.2 Semi-autonomous Solutions: Decision-Making for Man-Driven Vehicles

A middle ground on the way to autonomous solutions for vineyard management might be affordable to many growers who already possess state-of-the-art farm machinery. The unprecedented availability of massive data sets from a new batch of novel sensors, as described in the previous section, can lead to a new way to use standard machinery. Robust machines that have proved reliable in the field, will increase their efficiency when intelligent decisions resulting from data recorded precisely in the field are added to the decision-making process. The following examples explain how to use nitrogen content in leaves and the amount of anthocyanin in grapes to enhance the performance of fertilizers and grape harvesters.

5.2.1 Variable-Rate Fertilization with Prescription Maps

The determination of vegetation indices from moving platforms, as shown in Fig. 5.2b, can provide a basis for variable-rate application of fertilizers in the vineyard so that vigorous vines do not get an excess of nutrients and weaker plants receive what they need to produce a satisfactory yield. The rate of fertilizer application can be determined from a digital prescription map that the machine understands. Such a map will include spatial information for locating the vehicle in the field and the recommended dose to apply. A highly varying dose as the vehicle moves is not usually practical, but if homogeneous zones with similar needs are identified, different rates can be applied efficiently to specific areas of the field, provided the data have been mapped accurately. These zones might be large areas, or alternatively, square cells with sides ranging from 1 to 10 metres according to the accuracy of the sensor, the resolution of the map and the grower’s management approach. Figure 5.4a shows a field map depicting the nitrogen balance index (NBI; Cerovic et al. 2015), an indirect way to assess the variation in foliar nitrogen in a vineyard recorded by a robot along the trajectory of Fig. 5.4b, which was registered with an onboard GPS receiver. The rows are spaced 2.4 m apart and have a length of approximately 105 m. The cells represented in Fig. 5.4 are squares of 4-m side, which have been obtained by averaging all the measurements within the 16 m² of each cell.

Fig. 5.4

Real-time NBI maps: a grid maps with 4 m × 4 m cells and b vehicle trajectory

The NBI is the ratio of chlorophyll content to epidermal flavonol leaf content, and can be used as an indicator of the nutritional status of the plant. It is less sensitive to phenology because it reflects the availability of nitrogen better than the two indicators used separately, which have inferior quality as estimators. The statistical correlation between NBI and nitrogen content in mg per gram of leaf, unfortunately depends on the side of the leaf being measured and the cultivar, but a practical relation for adaxial measurements on Pinot Noir vines, for example, was given by the equation NBI = –0.4 + 0.62 N (mg g⁻¹) (Vinerobot 2014). Even though space has been discretized in Fig. 5.4a to avoid intense changes in the actuation of the solenoids that adjust fertilizer doses, the prescription maps could be simplified further by reducing the doses to a smaller set of choices, such as high and low, for example. This downscaling of rates can be achieved by several approaches, from a simple resetting of rates to more sophisticated clustering techniques. Kriging (Oliver 2010) has been used in precision agriculture to smooth data that vary spatially and to interpolate from relatively sparse data that is spatially correlated. Kriging involves predicting values from neighbouring data at unsampled places using the model parameters fitted to an experimental variogram, therefore, it requires sufficient data from which to compute a variogram; at least 100 points.

Figure 5.5a shows a grid map with 4-m square cells filled with nitrogen measurements from a fluorescence sensor (NBI*100), and all the measurements within a given cell were averaged. The median value of the 25 cells considered for each moving window was applied to each averaged cell (Saiz-Rubio and Rovira-Más 2016) to obtain the simplified map of Fig. 5.5b, which would be more practical for field operations. This new map has resulted in zones with similar characteristics, facilitating the future implementation of automated tasks such as variable-rate applications (VRA) of fertilizer. Figure 5.5b can be used as the basis for a prescription map to fertilize according to nitrogen deficiencies detected in the field and recorded by a robotic platform. Agricultural geographic information systems (GIS) software is used to create prescription maps. A prescription map tells the controller of an intelligent vehicle how much product has to be applied at each location of the field. Most agricultural GIS packages can create prescription maps in multiple formats (Norwood et al. 2009). Further research, however, will be needed to determine if these procedures have any effect on crop growth and fruit bearing, which is the final objective of applying precision techniques in the vineyard.

Fig. 5.5

Smoothing operations in field maps as a basis for variable-rate applications: a raw measurements and b smoothed map with moving averages

5.2.2 Differential Harvesting with Intelligent Mechanical Harvesters

A long-term wish of wine makers and vineyard growers has been differential harvesting, in which a field is harvested at different periods to avoid mixing grapes of uneven maturity. Until now, this has not been practical either for most manual harvesting or with mechanical harvesters. However, the advent of new machines that can read GPS instructions and interpret digital maps provides the potential for differential harvesting, especially with vehicles that can carry two independent bins where grapes may be placed according to onboard computer commands. Although cutting-edge harvesters endowed with intelligent behaviour and new physical capabilities will be necessary for advanced harvesting techniques, there are still important steps that need to be solved before differential harvesting can be achieved, such as the provision of precise harvest-readiness maps. The anthocyanin level of red grapes will be an important component in such maps, but other complementary properties might help, such as the nitrogen content in leaves. Figure 5.6 shows a plot of the evaluation of four wines by scoring their main oenological properties on a 0–5 scale. The four wines come from the same vineyard, but the grapes used to make them come from separate sub-zones with distinct contents of nitrogen in the leaves (N) and anthocyanins in the grapes (A). Two levels for each property were established (high and low), resulting in four combinations A + N–, A–N–, A + N + and A–N + . Wine tasting experts finally concluded that the best wine was that made with a large anthocyanin content and a small nitrogen content (A + N–), as plotted in Fig. 5.6.

Fig. 5.6

Wine properties (Visual; Odour; Aroma; Mouth) as a combination of fundamental field properties nitrogen (N) and anthocyanins (A) (Courtesy of Les Vignerons de Buzet, France)

The mathematical combination of several maps, each one plotting a relevant field property, is feasible provided their axes, coordinates, origin and units are compatible. The grid maps of Figs. 5.4 and 5.5, with a local origin and plain coordinates East and North, provide a convenient way to fuse field data. The nitrogen distribution of Fig. 5.4a could be further simplified to only two levels (N + and N–) and used to group harvesting zones by following the philosophy of Fig. 5.6. Based on a cell-to-cell comparison, maps generated automatically from a moving vehicle could be fused with manually-generated maps such as those showing the spatial distribution of titratable acidity, must pH, sugar content or yield, to define a quality index for the future wine (Rovira-Más and Saiz-Rubio 2013). The various properties of the wines represented in Fig. 5.6 indicate that, as expected, the amount of anthocyanin in red grapes is an important property for classifying the oenological potential of a wine. However, the discussion raised in Sect. 5.1.4 demonstrates that, even though manual measurements with handheld devices are feasible, for a sampling rate to provide statistical significance it is better to make measurements on-the-fly from a moving vehicle. Figure 5.7a depicts real-time generated maps of anthocyanins measured in a vineyard of Merlot grapes with an experimental fluorescence device carried by a vineyard robot. Notice that this map is less populated than the nitrogen map of Fig. 5.4a because of the need to measure anthocyanins at a moving spot in which reliable estimates only occur with less than 3% of green matter at the spot, the rest being occupied by the grapes. The actual trajectory followed by the robot is shown in Fig. 5.7b.

Fig. 5.7

Real-time anthocyanins maps: a grid maps with cells 4 m × 4 m and b vehicle trajectory

Interpolation techniques, such as kriging, have been extensively used to interpolate data at places where there are no measurements. This procedure, however, does not guarantee a better representation of reality than a map with empty spaces like Fig. 5.7a. In fact, smooth interpolated maps might mask very variable data whose averaging might lead to the wrong decisions being made, so caution must be the rule when analysing data with large dispersion. If advanced harvesters can currently carry two bins (A–B) at most, and the anthocyanins map is used to determine automatically into which bin grapes must be loaded, it makes no sense to produce digital maps of more than two levels. Figure 5.8 shows a simplification of the anthocyanins map of Fig. 5.7a to only two levels (high and low) to obtain a reasonable number of zones for applications similar to that illustrated in Fig. 5.6.

Fig. 5.8

Simplification of Fig. 5.7a for a more practical zoning with two levels of anthocyanins (±)

5.3 Autonomous Solutions: The Advent of Agricultural Robots

Even though most vineyards have vine-supporting structures that can assist navigation, the problem of autonomous guidance is a complex problem because of the considerable uncertainty and the extraordinary risks involved with farm machinery. A vehicle in the open field is subjected to many disturbances caused by a dynamic environment such as changing illumination, fluctuating weather and unpredictable obstacles including tools, other machines, animals, or even people working in the field. Barren fields ready to be sowed require guidance commands from satellite-based positioning systems for automating farming tasks; whereas, vineyards typically have vines following a particular arrangement. Robotics and automation greatly benefit from vertically-oriented supporting systems, such as VSP (vertical shoot positioning), rather than the more traditional goblet training system. Although agricultural robotics is growing at present, the commercial offer of farm robots is very limited, yet many research groups at universities, government agencies and private corporations are making considerable efforts to develop robotic solutions to actual problems found in agricultural fields. The following points address several crucial challenges in the long journey from basic semi-autonomy to fully autonomous farm robots. Initial attempts, as the platform of the products mentioned in Agati et al. (2007), give a good idea of the growing interest in these technologies.

5.3.1 Reliability and Safeguarding as the Highest Priority

The systematic accumulation of sensors in automated applications, not always indispensable, have often resulted in weak solutions when challenged by the harsh environments of farm fields over an extended period of time. There is a big difference between a 10-minute demonstration and regular equipment operations during the entire season. The trade-off between complexity and reliability is key, and as a result we should verify carefully that adding a new component is strictly necessary to meet the end-user requirements because each new component will involve more complexity, and therefore a greater likelihood of failure (Vinerobot 2014). Fail-safe conditions may be enhanced by introducing redundancy in the system and by designing a reliable safeguarding network. To do so, the following features should be considered:

• Robots are usually designed to be proficient in defined environments, thus no operations outside pre-defined settings should be allowed. In the case of viticulture, for example, robots should not operate outside the vineyards. Global navigation satellite systems (GNSS) receivers should warn when a vehicle leaves the confidence zone set by the user.

• Canopy or terrain disturbances may induce unstable behaviour in the navigation engine of robots, putting them at risk after getting too close to surrounding vines or supporting structures. For such situations, it is necessary to stop the robot automatically and safely before it collides with other objects, and in case the non-contact system fails, halt the robot as soon as an obstacle is touched. For the latter case, a frontal bumper often becomes an efficient solution.

• There are many causes, some of them unpredictable, that can make a robot perform erratically or unstably; therefore, a network of emergency stop push buttons must be mounted and evenly distributed on the robot’s exterior so that anyone in the vicinity can stop it without potential harm.

• An intelligent vehicle that can operate autonomously receives instructions from one or several computing units. If a power shortage affects the normal performance of the main computer and ancillary components, the consequences may be lethal for the robot’s integrity. This is especially important when the robot is powered by electric batteries because the electronic network of the robot might behave randomly if battery power decreases below a threshold. Therefore, close monitoring of the power system is important for stability during the robot’s operational time.

Figure 5.9 shows the safety network implemented in the first prototype of the VineRobot (2014). Four emergency stop (E-stop) push buttons have been placed near each corner of the four-sided body of the robot (only two are visible in Fig. 5.9). When any of the buttons is activated, a relay cuts the power to the wheel motors and turns on the red warning light at the same time the buzzer sounds. The three sonar sensors mounted on the bumper and facing forward are programmed to stop the robot if an obstacle is detected at less than 50 cm from its front. Similarly, the rear sonar provides assistance for reverse manoeuvres at the headlands. If the frontal ultrasonic sensors do not halt the robot before an immediate collision, a gentle push on the bumper would fire the same relay that stops the wheel motors and issues an acoustic warning. When the robot of Fig. 5.9 was evaluated in actual vineyards, and after several hours of continuous operations in the field, a weak voltage in the battery system resulted in irregular behaviour of the stereo camera, which in turn froze auto-guidance images and eventually made the robot go astray. To avoid power-induced instabilities, the status of both the 12 VDC and 24 VDC power systems was tracked independently by a light bar display near the monitor, and also by an indicator included in the graphical user interface (GUI, yellow bars; Fig. 5.15a). When the voltage from either power system dropped below a predefined threshold, the robot sent a warning message and was stopped safely, disengaging automatic mode (orange light off) and only allowing manual operations (blue light on).

Fig. 5.9

Explanatory diagram of a safety network for a vineyard robot

5.3.2 Physical Requirements and Mechanical Design

Field testing with robots in real environments has shown that it is important, especially in agriculture, not to overlook the mechanical design to focus only on sensors, electronics and software development. A robotic platform that is supposed to compete, and optimally outperform, conventional farm machines will have to traverse all kinds of uneven and rough terrain, perform many hours of continuous operation and endure tough outdoor conditions including unexpected rain, high humidity, extreme heat in the summer or cold in the winter, and occasional strong winds. Consequently, the mechanical structure of a robot must withstand friction, vibration, wear, vertical accelerations (shocks) caused by bumpy terrain and even occasional branches hitting or scratching its external cover. In addition, the power delivered by the batteries or combustion engine must be conveyed efficiently to the tyres, which implies making the right choice when designing the transmission system and the steering mechanism. Trying to solve mechanical problems with software tends to be futile and often catastrophic. The following list reviews some key aspects under consideration when designing agricultural robots:

• The materials with which the supporting frame and the external cover of the robot are built must be resistant to corrosion, waterproof and strong. Aluminium and steel are good candidates for the structure, whereas external bodies made of malleable polymers leave room for creative designs. Special attention must be paid to the joints through which water and dust may penetrate and deteriorate the inner electronics, typically not well fitted for outdoor conditions. The design of chassis and body must give priority to practical needs rather than aesthetics so that replacing a battery or repairing a linkage does not require dismantling the entire robot.

• Effective transmission involves selecting the right set of mechanical components so that the final torque and rotational speed in the tyres optimizes the available power for versatile performance. In general, moderately-sized robots powered by electric batteries cannot handle complex drivetrains comprising clutches, multi-gear boxes or torque converters. Rather, they benefit from simplified approaches in which a reduced set of gears link electric motors and tyres. Yet, the selection of these gears is crucial to ensure the expected performance of the robot in all foreseen situations; the wrong speed will make the robot inefficiently slow or dangerously fast, whereas a lack of torque will compromise its roving capacity.

• Regardless of the precision achieved in the navigation control commands, if they are not properly executed by the steering system, the robot will not reach the desired position at the right time. Therefore, it is essential to define the steering strategy and design of the steering mechanism. Sharp and small corrections are needed for straight guidance, but large wheel angles will be necessary to complete headland turns successfully. For Ackerman geometry, the wheelbase and maximum turning angle of the front wheels are critical parameters to determine the turning radius of the robot, which is key in the automatic execution of headland turns. An efficient way of protecting the steering actuators of autonomous vehicles, particularly electric motors, is by limiting the sweeping movement of tie rods with end-of-stroke switches, avoiding extreme angles, friction wear and overheating of drive cards (Rovira-Más et al. 2015a).

• It is impossible to predict the properties of the terrain where the future robot will have to navigate during its lifespan. Even if the terrains were known, the effect of weather and farming tasks on the ground would alter their tractional capacity. Consequently, a compliant suspension system can considerably improve the mobility of the robot in the vineyard by increasing the likelihood of keeping the four wheels in contact with the ground all the time. Wheel slippage is unavoidable in off-road terrain, but limiting it will have a positive effect on the navigational capabilities of robots, mainly when negotiating the sharp turns at the end of the rows to change the direction of travel 180° (Saiz-Rubio et al. 2017).

• Finally, the interior space storing the electronic components and computing units must be cooled efficiently to avoid processing slowness from overheating. Many agricultural tasks take place in the summer when ambient temperatures are high and the sun’s radiation intense. The right location and choice of fans and ventilation grilles may effectively diminish the inside temperature of a robot and provide a safer environment for computers.

Figure 5.10 provides some examples of the mechanical components discussed above, such as a suspension system (a), a steering mechanism (b), cooling fans for the central computer of the VineRobot-II (c) and the open design of a robot that favours maintenance and assembly of new components (d) (Saiz-Rubio et al. 2017).

Fig. 5.10

Mechanical choices for agricultural robots: a suspension, b steering, c cooling and d frame

5.3.3 Fundamental Abilities: Navigation and Mapping

One of the most delicate and complex tasks entrusted to a robot is autonomous navigation. An operation that humans resolve effortlessly from childhood becomes a serious challenge for a machine when uncertainty is brought into the equation, as is the case in open agricultural fields and environments. An effective approach to cope with this challenge in row-structured agriculture is by dividing the auto-steering operation into two distinct stages: navigation between rows in a quasi-straight guidance, and headland turning to change rows after making a U-turn following a specific turning geometry. Figure 5.11 depicts both cases in a vineyard.

Fig. 5.11

Autonomous guidance of a vineyard robot: a inside-row guidance and b headland turning

While global positioning by GNSS technology is vital for field mapping and precision farming applications, autonomous guidance in orchards and vineyards cannot rely on satellite-based navigation exclusively because precise steering commands cannot be ensured with signal blocking from trees or multipath errors induced by nearby structures or vines. As a result, local perception provides the complementarity needed to ensure a richer understanding of a robot’s surrounding. Such perception is typically acquired by ultrasonic sensors, lidar rangefinders or any form of machine vision. Very often, the combination of various sensors, rather than just one, provides the level of accuracy required to guide a vehicle inside the tight space between adjacent rows, as shown in Fig. 5.11 (Buzet-sur-Baïse, France). Lidar and sonar have been extensively used to detect obstacles around a vehicle, but very often the guidance performance closest to human driving has been achieved with machine vision. When a camera is placed at the front of a vehicle, images with a vanishing point may be processed to find the optimal trajectory between crop rows, like a monocular camera coupled with a near infrared filter in Rovira-Más et al. (2005), which used the Hough transform to determine the central path. A serious disadvantage of monocular cameras for outdoor conditions is their strong dependence on changes in ambient illumination, which often results in lack of robustness if conditions differ markedly during the operational time, typically ranging from dawn to dusk. Stereoscopic vision, however, can circumvent this shortcoming because two identical lenses, mimicking human eyes, perceive a scene by comparing the relative position of the same features in two imaging sensors, therefore changes in illumination affect both sensors simultaneously in such a way that as long as there is enough light intensity to find textural changes, pixels will be correlated and their coordinates estimated. Furthermore, the resolution of stereo geometry gives the three coordinates of a point in space, i.e. the three-dimensional (3-D) representation of the scene ahead of the robot, which represents a description of reality richer than the information contained in two-dimensional (2-D) images acquired with monocular cameras. Figure 5.12 shows the navigation maps derived from various situations perceived with a compact off-the-shelf stereoscopic camera. A multiplicity of algorithms may be applied to these navigation maps to find the steering command that will guide the robot along the vineyard rows. A particular example of image processing and its associated control system for stereo-based 3-D perception in autonomous navigation can be found in Rovira-Más et al. (2015a).

Fig. 5.12

Automatic guidance between vineyard rows with stereoscopic vision: navigation maps

Even though the majority of working time occurs travelling along the rows, which is where information is retrieved from plants or soil and agricultural actions must be executed, turning at the headlands to change rows is necessary for a continuous operation without human intervention. Consequently, it will be necessary to develop and encode a reliable algorithm to engage one row after another with agility, which is not a trivial endeavour. To begin with, the guidance features provided by bounding rows in straight guidance will no longer be available. To make things worse, slippage increases in sharp turns, and a slight deviation when entering the following row might result in unfortunate collisions. For all these reasons, this is a delicate manoeuvre that necessitates a special formulation. The row spacing, for example, will have an effect on the geometry of the turn, not to mention the special cases of rows of variable length to fit irregular fields, boundary rows near roads with traffic, or uneven headlands in sloping terrain.

A practical approach to deal with the headland turn problem has been by dividing the turning sequence into a set of consecutive stages where different sensing technologies are fused in such a way that each stage is solved with the best information available in the vehicle (Subramanian and Burks 2007). The ultrasonic sensor network of Fig. 5.9 was used to enhance straight navigation and assist in headland operations by the robot of Fig. 5.11, with the additional assistance of two lateral sonars pointing at the canopy, one on the left side of the robot and the other on its right side, resulting in a total network of six encircling sonars. Figure 5.13 shows a schematic diagram of the six stages into which a complete turn was decomposed, and graphically depicts the work of the six-sonar network at the end of stage IV. As explained in Table 5.1, each turn involved the combination of stereo vision, sonar and dead-reckoning to achieve a turn every two rows. Further details on these operations can be found in Rovira-Más et al. (2016).

Fig. 5.13

Stages for executing headland turns by a vineyard robot

Table 5.1 Specifications for a multi-stage headland turning operation

Regardless of the navigation strategy chosen for an autonomous vehicle, a GNSS receiver will always provide valuable information for applications within the scope of robotics, precision farming and information technology (Rovira-Más et al. 2015b). The headland turning manoeuvre of Fig. 5.13, just to cite an example, uses GPS information to estimate the length travelled by the robot for stages III and IV that require dead-reckoning. In addition to navigation assistance, crop maps will benefit from global-based localization. However, the geodetic coordinates delivered by GNSS receivers through the NMEA code are not convenient for precision farming. Spherical coordinates such as latitude and longitude do not allow the use of Euclidean geometry, which is the basis for common calculations of distances and areas. The absence of a tangible origin of coordinates also complicates the creation and use of crop maps, whose final users are not typically experts in geographical systems. Earth sphericity can be neglected for relatively small areas such as vineyards, therefore UTM (universal transverse Mercator) or LTP (local tangent plane) coordinate systems are better adapted to robot-based mapping. The latter also allows end-users to choose the origin of the coordinate frame locally, what makes it ideal for users to correlate map zones within their own field. The LTP coordinate system, therefore, combines the advantages of global positioning with local coordinates East and North in a conventional Cartesian frame.

To make decisions based on objective data gathered from robotic platforms, as different sorts of data will be collected during the seasons, with diverse spatial resolution and measurement precision, a systematic way of correlating information in time and space will be necessary. An ordered division of field space into cells of meaningful size and agronomic significance allows the comparison of well-determined zones at a level of precision chosen by each user. However, the discretization of space into cells should not jeopardize the global–local advantages obtained with the LTP system. Fortunately, both approaches are compatible, and grids can be globally referenced in a Euclidean plane set to locate square cells by Cartesian coordinates East and North (Rovira-Más 2012). Moreover, this global-based grid approach allows for a real-time implementation as long as a GNSS receiver has been integrated properly in the mapping robot, as shown in Figs. 5.4 and 5.7. The raw data directly measured from the field by the onboard sensors are often too ‘noisy’ to make a map that can be read by growers or other machines. Geostatistics can be used to reduce the local noise in data reflected by marked changes over short distances (jumps). Based on the method of data processing chosen, maps will be available in real time, or alternatively, at the end of the mapping mission if the complete data set is needed to correct individual data points. In such cases, successive operations might be run immediately after mapping, leading to a quasi-real-time situation where maps are available from the field as soon as the robot has scanned the predefined area. An example following this approach is presented in Saiz-Rubio and Rovira-Más (2016).

5.3.4 Human–Robot Interaction in a User-Centred Design

Agricultural robots have to be designed with the premise that their future users are individuals used to handling tractors, harvesters, sprayers and other conventional equipment that is highly resistant, and straightforward to use and understand. Consequently, delicate, weak, highly-exposed, low-cost robots that work reasonably well indoors over firm and clean floors of research laboratories and unpolluted factories will never perform successfully and consistently in agricultural fields. Most agricultural robots are still at the research stages, and the complexity of handling and maintaining them is closer to experimental prototypes than commercial products. Efforts are currently being made to shrink this gap and make agricultural robots commercially available in less than a decade. The following paragraphs provide an overview of these secondary features that, without being central in the design of farm robots, are necessary to consider before deploying market-ready solutions.

The first point under discussion relates to transportation. A particular robotic solution may be integrated into a self-propelled platform such as a tractor or harvester, but in general a robot is designed to carry out a specific task in the field, and therefore must be carried from the storage building to the field and vice versa. This will restrict the size and weight of farm robots because the average user has to be capable of handling them without the need to purchase a new vehicle for this particular purpose. Conventional vans, utility vehicles, SUVs or pick-up trucks should suffice for just one operator to move the robot from one field to the next. In addition to space requirements, users must be able to load and unload farm robots without making any physical effort greater than lifting a reasonable payload, 10 kg for example, which essentially forces the robots to be self-propelling in the loading operations. This can be facilitated by a joystick through which full control of the steering mechanism and wheel motion is provided. These joysticks can be linked to the robot wirelessly, but for such a delicate operation where tolerances may be limited and collisions are likely, wired remote controls provide a safer solution. Once the robot has been downloaded from the transporting vehicle, the joystick will allow the robot to be placed in the first row selected to begin automated tasks. Figure 5.14 illustrates the process of transportation, unloading and placement of a robot as the preliminary steps to carry out automated tasks.

Fig. 5.14

Preliminary steps in automated operations: transportation, unloading and placement of robots

After placing the robot in the first row, automated operations can begin. A straightforward and unambiguous interface should let users select the main features for each particular mission, facilitating the initiation of automated tasks. This interface will comprise hardware-based and software-based interactions. The former may include the power switches connected to batteries, warning lights indicating robot status (Fig. 5.9), or the button enabling automatic mode; the latter will be compactly outlined within a graphic user interface (GUI) manipulated through a touchscreen monitor integrated into the robot, and optionally remotely transferred to a mobile terminal. Figure 5.15a provides an example of a GUI for a vineyard mapping robot. Notice that, in general, this command window offers three types of information exchange between the robot and the user:

Fig. 5.15

VineRobot-II design features: a graphic user interface and b solar panels

• (a) Visual information: real-time video, battery level, 2-D navigation map and crop parameters cell map.

• (b) Textual information: GPS data, row number and text messages.

• (c) Action buttons: save data, velocity, mode (manual or auto), number of rows to map, etc.

As technology advances and new materials become popular and available, user preferences evolve with time. Modern farmers demand innovative solutions at the same technological level reached by other production sectors. The introduction of robotics in rural areas could encourage young farmers to modernize their equipment under the context of digital agriculture, as long as market demands allow for the economic sustainability of their investments. However, sustainability is also being considered nowadays from an environmental point of view. The implementation of renewable energy and recyclable materials are receiving more attention every day among the manufacturers of agricultural equipment. Figure 5.15b shows a robot prototype with two plates of solar panels providing 60 W each (Saiz-Rubio et al. 2017).

5.4 Conclusions and Looking Beyond

A promising side effect of the successful introduction of robotics in commercial vineyards is the attraction that new technologies pose to young grape growers. The average age of farming populations in Europe and Japan is currently near retirement age, with very few professional farmers under 35 years old. The lure of electronics and automation will possibly help to counter the negative effect of an aging population in agriculture. This is one of the major problems faced by industrialized countries, especially with the potential demand for an increase of 100% in food with the growth in population expected in 2050. Figure 5.16b shows that there are many European farmers over 65 years old, which in many countries is the retirement age; and conversely, Fig. 5.16a shows the small number of farmers under 35 years old, the prototypical farmer who could give stability to the rural population in a rural renaissance induced by technology-based solutions.

Fig. 5.16

European farmers under 35 years old (a) and over 65 years old b (Eurostat)

In addition to the serious problem of an aging farming population, there are other issues for growers that make robotics attractive to viticulture. Among them, the shortage of skilled workers to prune vines in the winter, the lack of objective field data to maintain a wine of a certain quality consistently and its reputation over time, and the possibility of differential harvesting to avoid mixing grapes with different properties. Overall, there are many ways of improvement in viticulture through technology, but, on the other hand, there are also important challenges to overcome before reaching market readiness. The fact that robots are not widespread in vineyards worldwide suggests that these novel approaches remain difficult at the commercial and practical level. Technical challenges such as reliability and safety of autonomous vehicles operating several hours without human intervention, economic hurdles resulting from the need to use cutting-edge technology in products that must compete in price with other alternative solutions, and social barriers encountered when complex devices that produce unmanageable amounts of data have to be accepted and understood by an ever aging population all seem apparently insurmountable. However, recent progress in the fields of robotics and precision agriculture give much cause for optimism, and impressive innovations will soon reach the market and the global agronomic sector.