Keywords

8.1 Introduction

Food security is always one of the top priorities globally. As estimated by the United Nations, the global population will reach 9.7 billion in 2050 (United Nations, 2022). However, climate change makes food production more challenging by reducing farmable land and worsening the environment for animal husbandry. In addition, food production is facing issues of labor shortage and aging workforce (United Nations, 2022). Nowadays, few in the young generation are willing to work in agriculture, forestry, fishery, and animal husbandry because of the harsh working environments and disproportionate wages.

Past advances in farming have yielded new equipment and facilities designed to improve farming efficiency (e.g., tractors and greenhouses). However, the observation of farming or animal conditions still relies on manual observation. For example, farmers have to patrol in the field to check the growth condition of crops. In animal husbandry, farmers have to patrol regularly to monitor animal conditions. This is because the environments for crops and animal husbandry are usually complex. However, manual observation is slow and requires experience. More automatic monitoring approaches are needed.

In recent years, due to breakthroughs in computing speed, deep learning has become more popular as a method to solve complex machine vision problems in the fields of agriculture, forestry, fishery, and animal husbandry. The application of deep learning algorithms in machine vision is referred to as smart machine vision. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs; Rumelhart et al., 1985) are the common types of deep learning algorithms that are employed in smart machine vision. The use of smart machine vision is regarded as an automatic solution in the aforementioned fields.

This section introduces the workflow and applications of smart machine vision in agriculture, forestry, fishery, and animal husbandry. Firstly, different types of CNNs in various applications are introduced. Next, four components of smart machine vision applications are introduced. Last but not least, several examples of smart machine vision in agriculture, forestry, fishery, and animal husbandry are shown. These studies demonstrated how smart machine vision can help to resolve the food security problem.

8.2 Tasks of Smart Machine Vision

Smart machine vision applications can be categorized into static and dynamic tasks (Fig. 8.1). Static tasks include classification, localization and classification, semantic segmentation, and instance segmentation. Dynamic tasks are usually behavior recognition tasks. Typically, static tasks use images as the input. On the other hand, dynamics tasks use videos as the input.

Fig. 8.1
An image illustrates the static and dynamic tasks of smart machine vision. Static tasks include classification, localization and classification, semantic segmentation, and instance segmentation. Dynamic tasks include behavior recognition.

Tasks of smart machine vision

Full size image

Various types of CNNs are used for static tasks. For classification, CNNs containing convolution layers, pooling layers, and fully connected layers are used. These CNN models are usually referred to as backbone CNNs. Commonly used backbone CNNs include AlexNet, VGG, ResNet, etc. (Alzubaidi et al., 2021). For localization and classification, CNN models are usually composed of backbone CNNs, necks, and heads. Commonly used localization and classification CNN models include Fast R-CNN, YOLO, etc. (Liu et al., 2020). For semantic segmentation and instance segmentation, CNNs with encoder–decoder architectures are typically used. The commonly used semantic segmentation and instance segmentation CNNs include U-Net (Garcia-Garcia et al., 2018), YOLACT (Tian et al., 2021), etc.

Dynamic tasks are typically fulfilled using the combination of CNNs and RNNs. CNNs extract features from video frames, and RNNs determine the output by considering the features in consecutive frames of videos. A commonly used RNN is gated recurrent units and long short-term memory (Alzubaidi et al., 2021).

8.3 The Components of Smart Machine Vision

Typical smart machine vision applications in agriculture, forestry, fishery, and animal husbandry include four important components: image acquisition, machine learning, database, and user access (Fig. 8.2). Image acquisition is the first step in machine vision implementation. Images are collected by using cellphones manually or by using stationary cameras automatically. Typically, if the application requires only one image, cellphones are used for image acquisition. By contrast, if the application requires videos, stationary cameras are used for image acquisition.

Fig. 8.2
A flow diagram explains the machine vision in agricultural implementation. This includes image acquisition, database, user access, and machine learning.

Machine vision in agriculture implementation flow

Full size image

The component of machine learning includes five steps, namely image collection, image augmentation, model architecture selection, model training, and model performance evaluation. To train a deep learning model, it is recommended to acquire at least 500 images for each category. The images are next annotated. The annotated images are then split into training, validation, and test with a ratio of typically 8:1:1. Image augmentation (e.g., flipping and rotation) is subsequently applied to the annotated training images to generalize the images and improve the robustness of the model to be trained. A CNN model for a specific task (e.g., classification, localization and classification, semantic segmentation, and instance segmentation) is then chosen. The training of the model involves hyperparameter selection. Typical hyperparameters include learning rate and weight decay. Appropriate hyperparameters improve the performance of the model to be trained. After the model is trained, test images are applied to the trained model to evaluate the model performance. The aforementioned procedure completes the component of machine learning.

A database is usually used in smart machine vision applications too. The database is an essential component because the data for model training (acquired images and labels of images) are stored in the database. The database can be used to store the images uploaded by end users too.

Another essential component of smart machine vision is user access. Typically, a microservice is established to serve as a bridge between the internal system (i.e., the trained model and database) and end users. Through the microservice, the trained model and database can be accessed by both internal and end users. The end users can also provide new data through user access.

8.4 Examples of Smart Machine Vision in Agriculture, Forestry, Fishery, and Animal Husbandry

Smart machine vision has been applied to the fields of agriculture, forestry, fishery, and animal husbandry. This is especially an increasing trend. Smart machine vision alleviates the issue of labor shortage. With smart machine vision, work patterns are changed, and work loadings are reduced. Below are some examples of smart machine vision.

Crops are vulnerable to pests and diseases, environmental changes, and storage conditions, resulting in economic losses. The application of smart machine vision is an easier and faster solution to control the quality of crops. Numerous studies have applied smart machine vision to identify plant diseases and pests (Abade et al., 2021) and superficial damages (Li et al., 2020). These applications help farmers to reduce economic loss and increase production.

Forests play an essential role in food security and daily necessities (Sunderland et al., 2013). People who live near tropical forests can acquire food surrounding specific tree species and even on the tree. Also, different trees can be made into a wide variety of daily necessities. Studies were conducted to identify consumable wood species. Yang et al. (2019) differentiated between morphologically similar species in genus Cinnamomum (Lauraceae). The species C. osmophloeum yields cinnamaldehyde and is used as a herbal plant. Pelletier et al. (2019) and Schiefer et al. (2020) identified tree species and mapped tree species in a forest, respectively (Hamedianfar et al., 2022), which can help those living nearby to reliably acquire food and earn a living.

Fish is a major source of protein globally. However, the biological sustainability of oceans has been brought to attention in recent years. Smart machine vision was applied to identify species of marine organisms to prevent overfishing or inadvertent illegal fishing (Aguzzi et al., 2020). Also, the length and species of harvested fish, which is required by some fisheries management organizations, can be estimated and recorded using smart machine vision (Tseng & Kuo, 2020; Tseng et al., 2020). Aquaculture is another way to raise seafood. Certain studies evaluated the frequency of fish feeding using smart machine vision (Zhao et al., 2021). Shrimp body length was estimated for feeding management using smart machine vision (Lai et al., 2022).

Economic animals are another major source of protein. Smart machine vision can be applied to alleviate the need for patrols and manual observation in animal farming. Related studies include a monitoring system for detecting sick chickens (Ojo et al., 2022), an observation system for identifying the tail-biting behaviors of pigs (Chen et al., 2021), an automatic monitoring of newborn piglets tracking and lactating frequency of sows (Ho et al., 2021), and an inspection system for identifying lameness behaviors of cows (Mahmud et al., 2021).

8.5 Conclusion

Food production is now affected by labor shortage globally. To meet the demand of food, smart machine vision is applied in agriculture, forestry, fishery, and animal husbandry to develop automatic solutions that can replace human power. The whole process can be simplified as image acquisition, machine learning, database, and user access. With the application of smart machine vision, farmers can manage their fields efficiently, harvest richly, and thereby improve food security worldwide.