1 Introduction

Insect pests can induce crop diseases, thus affecting the growth of crops, or even leading to severe crop failure. It is of great significance for optimum yield and quality of agricultural production to investigate aspects of pest detection and recognition. Nowadays, computer vision technology inspired by the human visual system is widely applied to plant protection and agricultural management, and this technology makes great achievements in pest detection and recognition. Advanced image processing technology has greatly facilitated the agricultural pest detection and identification efficiency. Many methods of pest detection and recognition are investigated in recent years, such as computer-assisted estimation, cognitive vision, k-means clustering, sparse coding, support vector machine (SVM), and optimal deep residual learning [1,2,3,4,5,6,7]. However, most research based on computer vision is implemented using pest images obtained in indoor environments and is carried out under controlled laboratory conditions. It is relatively easy to process these images without obstruction or occlusion of leaves and other objects. The accuracy of this research depends on the image processing algorithms. In fact, the pest images obtained in outdoor environments are not as clear or appropriate as those acquired in the indoor environments. Besides, the sparse deployment of cameras in the farmland is not capable of covering the observing or monitoring scope required for pest detection and recognition. Then, the pest images tend to be of low magnification. Since the outdoor environment is complex, it is quite difficult to achieve clear detection and accurate identification of pests in a wide field of view (FOV) in the actual farmland background. In addition, the magnifications of the pest images obtained by the cameras with a wide FOV are often not high enough for image processing of pest recognition. These factors impose great challenges for agricultural pest detection and recognition.

In many imaging detection applications, it is required that optical systems are able to capture the high-resolution images in a wide FOV, such as space exploration, video surveillance, and endoscopes. High-resolution and wide FOV are two significant parameters for optical instruments. The former has the ability to obtain the detailed information of the target. The latter is essential for target searching and tracking. Zoom and multichannel systems are the practical methods to achieve both high-resolution and wide FOV [8,9,10,11,12,13,14,15]. For a compact system, liquid lenses and focus tunable lenses serve as the zooming and compensating components in modern zoom system design [8,9,10,11,12,13]. The multichannel systems are able to acquire the images of multiple channels with a common center FOV, simultaneously [14, 15]. These zoom and multichannel systems are generally heavier, bulkier and more expensive. Furthermore, the large amount of image data would make the rate of transmission and processing of images slower in these systems. The detection of targets in a specific area of interest (AOI) has received much attention in recent years due to its excellent performance in obtaining high-accuracy, improving efficiency and saving time in imaging applications. The foveated imaging system proposed by Martinez et al. is a variable resolution system, which can dynamically achieve local high-resolution images in a large FOV using an active optical element [16]. The active element is usually a liquid crystal spatial light modulator (SLM) or a deformable mirror (DM). Much research on foveated imaging systems using SLMs and DMs has been reported [16,17,18]. In these systems, the AOI aberrations are corrected by the active elements mentioned above, and the resolution of the unconcerned area is not necessarily high. The foveated imaging system can also be designed without an active element to correct the aberrations. The locally magnifying imager proposed by Parent et al. is an alterable magnification system achieved using the surface irregularity errors of a deformable mirror [19, 20]. The foveated imager developed by Niu et al. is able to dynamically image with a local high-magnification based on local focal length modulation [21]. This imager is designed to modulate the focal length of a local imaging area using a micro lens located near the first intermediary image plane. Each of these locally magnifying systems has a constant local magnification, and the magnifications are not greater than 1.4×.

In this paper, we propose a 4.5× local zoom system without reduction of the full FOV for insect pest images of local high-magnification in a wide FOV. This system has two imaging channels with the same image plane. One is designed as the local zoom imaging channel for pest recognition, while the other one is the peripheral imaging channel for searching pests in a wide FOV. The experimental results verify the basic principle of local zoom imaging of the 4.5× local zoom system. This system is well used for the imaging of aphids on plant leaves. The zoomed aphid images in a local scene of interest (LSOI) and the unzoomed aphid images in a wide FOV are captured by the established set-up. The local zoomed images can help pest recognition while the peripheral unzoomed images play an important role in expanding the system’s observing scope. Thus, the system can apply very well to agricultural pest detection and recognition with the sparse deployment of cameras in natural farmlands. The research result presented in this paper is also of great value in imaging for fine reconnaissance and searching in a wide FOV for space exploration, video surveillance and endoscopes.

2 Basic theory of local zoom imaging

2.1 Principle of local zoom imaging

The general “mechanically compensated” zoom system is typically composed of two fixed lens groups and two moving lens groups [22, 23]. The latter are two key lens groups. They are the variator for zooming and the compensator for keeping the image plane stationary [24]. Both of them are movable to achieve a zoom ratio. The motion of the variator is linear, while that of the compensator is nonlinear. Since the motion of the compensator is nonlinear, the mechanism of the system is usually arranged in a cam for two lens groups moving. It tends to be complicated and raises a high requirement in machining precision. The FOV of a general zoom system changes from the wide one to the narrow one when the focal length varies from the short one to the long one. This reduction of FOV leads to the loss of scene outside the narrow FOV at a long focal length. Thus, a general zoom system is not able to obtain images with the details of pests and the general information of peripheral scenes in a wide FOV.

To solve the above-mentioned problems, we propose a local zoom system without reduction of the FOV by employing a local zoom lens group, and achieve local zoom ratios of 1×–4.5×. This system is designed for the detection and recognition of insect pest images. The local zoom system is composed of a fixed imaging lens group, a laterally moving local zoom lens group and a fixed relay lens group, as shown in Fig. 1. The local zoom lens group is located between the imaging lens group and the relay lens group. The local zoom lens group consists of a variator for local zooming and a compensator for eliminating the axial shift of the image plane of the LSOI. In Fig. 1, the peripheral scene of the object to be imaged is shown in green, and the LSOI is shown in red. The peripheral scene marked with green light path is imaged by the imaging lens group and the relay lens group at the image plane without the local zoom lens group, and detected by the camera. This light path is the peripheral imaging channel. In Fig. 1, the LSOI is magnified at different magnifications of 2×, 3×, 4.5×. The light paths of three magnifications are marked in blue, yellow, and red, respectively. The LSOI passes through the imaging lens group, the local zoom lens group and the relay lens group, and is detected by the same camera. This light path is the local zoom imaging channel. The local scene is zoomed at different magnifications by shifting nonlinearly the positions of the variator and the compensator.

Fig. 1
figure 1

Layout of the local zoom system

Full size image

At the image plane, the blue, yellow and red areas represent images of the LSOI zoomed out with different zoom ratios of 2×, 3×, 4.5× by the local zoom lens group. The green area represents an image of the peripheral scene without being zoomed. The local magnification of the local zoom imaging channel is determined by the axial positions of the variator and the compensator. The variator and the compensator move along the z-axis paralleled to the optical axis for different local zoom ratios. The LSOIs of different lateral positions on the object to be imaged are zoomed by scanning the variator and the compensator synchronously in the x–y coordinate plane perpendicular to the optical axis.

2.2 System design strategy

The goal of our design is to achieve a single system with two imaging channels: one with a constant peripheral focal length and a wide FOV for searching insect pest, while the other one with different local focal lengths for fine recognition of insect pests in the region of interest. The two channels have the same image plane. The specifications of the 4.5× local zoom system are listed in Table 1.

Table 1 Specifications of a 4.5× local zoom system
Full size table

In the local zoom imaging channel, the power of the variator is negative while that of the compensator is positive. The variation of distance between the variator and the compensator causes a change in the local focal length and the local magnification. The focal length of the peripheral imaging channel keeps unchanged while the system is locally zooming. To obtain the 1×–4.5× zoom ratios, it is needed to determine the positions of the variator and the compensator for different local focal lengths. The focal lengths of the peripheral imaging channel and the local zoom imaging channel can be formulated as follows:

$${f^\prime }=f_{1}^{\prime } \cdot {\beta _4},$$
(1)
$$f_{i}^{\prime }=f_{1}^{\prime } \cdot {\beta _{2i}} \cdot {\beta _{3i}} \cdot {\beta _4},$$
(2)

where \(f_{1}^{\prime }\) is the focal length of the imaging lens group, and \({\beta _4}\) is the lateral magnification of the relay lens group. \({\beta _{2i}}\), \({\beta _{3i}}\) are the lateral magnifications of the variator and the compensator at different local focal lengths. \(i(1,2,3 \ldots n)\) denotes the number for different local zoom imaging channels. \({f^\prime }\) and \(f_{i}^{\prime }\) represent the focal lengths of the peripheral channel and the local zoom imaging channel, respectively. According to the Gauss formula, the local magnification is rewritten as the following equation:

$${M_i}=\frac{{f_{2}^{\prime }(f_{3}^{\prime } - L_{{3i}}^{\prime })}}{{f_{3}^{\prime }(f_{2}^{\prime }+{L_{2i}})}},$$
(3)

where \(f_{2}^{\prime }\), \(f_{3}^{\prime }\) are the focal lengths of the variator and the compensator, respectively. \({L_{2i}}\), \(L_{{3i}}^{\prime }\) are the object distance of the variator and the image distance of the compensator at different local focal lengths, respectively. The value of the local magnification is positive to ensure that the local magnified image is not inverted at the image plane. According to the Gauss formula, the distance between the variator and the compensator can also be worked out.

The paraxial analysis solution is implemented in this system using paraxial lenses. Then the parameters of the 4.5× local zoom system are figured out on the basis of the theory. Table 2 details the prescription of the 4.5× local zoom system. In Table 2, the air distances between the groups are given for different magnifications of 2×, 3×, 4.5×.

Table 2 Prescription for 4.5  × local zoom system
Full size table

3 Experimental set-up and results

In Fig. 2, the experimental set-up is built to validate the basic principle of the 4.5× local zoom system using a local zoom lens group. As shown in Fig. 2, the system consists of the imaging lens group with the focal length of 50 mm, two flat glasses with the local zoom lens group, two three-axis translation stages, the relay lens group with the magnification of − 1×, and the monochromatic CMOS camera (Basler acA1920-155um, 1/1.2″, 1920 × 1200, 5.86µm × 5.86 µm). The local zoom lens group has a clear diameter of 3 mm. The plano surfaces of the variator and the compensator are agglutinated on two flat glasses with UV adhesive. Two three-axis translation stages are used to laterally scan the local zoom lens group in the plane perpendicular to the optical axis and axially move the variator and the compensator along the z-axis parallel to the optical axis. The scanning of the local zoom lens group is designed for dynamic modulation of the local focal length for random FOVs. The variator and the compensator are kept coaxial during the scanning of random FOVs.

Fig. 2
figure 2

4.5× local zoom system experimental set-up

Full size image

The experimental results are displayed in Figs. 3, and 4. Figure 3 exhibits the images obtained by the set-up with different local magnifications. In these images, the circle edges of the local zoom zones are images of the element edges of the local zoom lens group. Figure 3a–c indicates the original images detected directly by the COMS camera. These images reveal the increase in the local magnification. It is observed that the brightness of the local zooming zones is worse than that of the peripheral ones. The higher the local magnification is, the less bright the local zooming zones become. This is due to the relative aperture of the local zoom imaging channel becoming smaller, while the local focal length is zooming and the pupil diameter remaining unchanged. This means the ability of the light-gathering power of the local zoom imaging channel drops. This channel presents less bright on the sensor. As the effective aperture of the local zoom imaging channel is limited by the elements of the local zoom lens group, it is hard to get a higher relative aperture. To increase the brightness and contrast of these images, we use the Multi-Scale Retinex (MSR) algorithm to enhance these images. Figure 3d–f illustrates the images in Fig. 3a–c after enhanced-processing. The enhanced images present the better contrast compared with the images in Fig. 3a–c.

Fig. 3
figure 3

Local zoom imaging experimental results for different local magnifications. a–c The local magnified images without processing. d–f The local magnified images with enhancement

Full size image
Fig. 4
figure 4

Local zoom imaging experimental results for different local magnifications using the grid pattern. Enhanced images with local magnifications of a 2×, b 3×, c 4.5×. The process of local zooming and that of scanning of LSOIs (see Visualization 1)

Full size image

To calculate the local magnification, we select a grid pattern with characters as the object to be imaged. Figure 4 shows the local zoomed images of the grid pattern. The images are enhanced using the MSR algorithm. Figure 4a–d demonstrates the images with the local magnifications of 2×, 3×, 4.5×. The local magnification is the ratio of a grid in the local zooming zone to one in the peripheral zone. The local magnification can be obtained easily by contrasting the grids in two zones. In addition, the experimental set-up can zoom different local scenes of interest dynamically by the scanning of the local zoom lens group. In Fig. 4, a real-time video (Visualization 1) displays the process of local zooming and that of scanning of LSOIs for the local zoom system.

As shown in Figs. 3 and 4, the resolution of the local zooming zones gets lower with the higher local magnifications. This is due to the spatial cut-off frequency of the local zoom imaging channel decreasing quickly with the smaller relative aperture at long focal length. Thus, the high frequency information is cut off and the resolution is lower. In Figs. 3 and 4, it is obvious that the four corners of images in Fig. 3 and the upper two corners of images in Fig. 4 are black. The black edges of images are because of the smaller size of images than the camera sensor size. In other words, the size of the selected object to be imaged is smaller than the field of view of the system. In the local zoom system, the image planes of two imaging channels are overlapped. From Figs. 3 and 4, it can be found that the information of the peripheral FOV around the local zooming zone is lost. The reason for the loss of the information is the light blocking of the element edges of the local zoom lens group. However, the lost information can be detected by scanning the local zoom lens group.

4 Imaging of aphids on leaves

The experimental set-up built as Fig. 2 is designed to image aphids on plant leaves. The camera used in the set-up mentioned in Sect. 3 is monochromatic. However, color images are required for pest detection and recognition. Thus, the camera is replaced by a color one (Basler alA1920-40gc, CMOS sensor, 1/1.2″, 1920 × 1200, 5.86µm × 5.86 µm). The aphid images in a wide FOV and these in one zoomed LSOI are captured by the set-up with a color CMOS camera, as shown in Figs. 5 and 6. As shown in Fig. 1, the object to be imaged is selected as plant leaves with aphids. Figure 5 shows the image of plant leaves captured in the peripheral imaging channel without being locally zoomed when the local zoom lens group is out of the full field of view of the imaging lens group. A fill light is used to brighten the leaves, due to insufficient light from the outside. These images captured in the peripheral imaging channel are used to determine the location of aphids. In Fig. 5, it can be seen that the aphids are on the local area of one leaf and the aphids are too small to be recognized. Thus, the local zoom lens group is used to magnify the aphids at different zoom ratios for its detection and recognition.

Fig. 5
figure 5

Image of plant leaves with aphids captured by the peripheral imaging channel without being local zoomed

Full size image
Fig. 6
figure 6

Images of plant leaves with aphids captured by the peripheral imaging channel and the local zoom imaging channel with different local magnifications. Original images with local magnifications of a 2×, b 3×, c 4.5×. Enhanced images with local magnifications of d 2×, e 3×, f 4.5×. The process of local zooming and that of scanning of the aphids in LSOIs (see Visualization 2 and Visualization 3, respectively)

Full size image

The local zoom lens group moves into the field of view of the imaging lens group driven by two 3-axis translation stages. The aphids in one LSOI are zoomed at different magnifications by the local zoom system in Fig. 6. The local magnifications of the aphids in the LSOI are 2×, 3×, 4.5×, respectively, in Fig. 6a–c and Fig. 6d–f. Figure 6a–c illustrates the original images detected directly by the color COMS camera. Due to worse brightness of the local zooming zone as mentioned in Sect. 3, these aphid images also need to be enhanced to increase the contrast of the LSOI. It will lead to the color distortion to enhance these color images of aphids by the MSR algorithm used in the grayscale images in Sect. 3. This is due to the changes of gray value of RGB color channels enhanced by the MSR algorithm. To correct this deficiency in color consistency, the Multi-Scale Retinex with Color Restore (MSRCR) algorithm is employed by introducing the color restoration function. The images are enhanced by the MSRCR algorithm as shown in Fig. 6d–f. The color of these images is well restored compared to that of those enhanced by the MSR algorithm. However, the color distortion still exists to some extent in these processed images. In Fig. 6, the characteristics of one aphid in the local zooming zones are more obvious compared with these of the aphids outside the local zooming zones. As shown in Fig. 6, the circle edges of the local zoom zones are very useful for the extraction of saliency characteristics of the aphids in the LSOIs. It is easy to detect and recognize the aphids using the local magnified aphid images. The local zoom lens group moves to different lateral positions driven by two three-axis translation stages for zooming the aphids in different LSOIs. Then, the aphids in one LSOI are zoomed by changing the axial positions of the variator and the compensator. In Fig. 6, two real-time videos (Visualization 2 and Visualization 3) show the process of local zooming and that of scanning of the aphids in LSOIs for the local zoom system.

Aphids on plant leaves are difficult to detect and recognize by cameras due to their tiny size. The cameras need to be capable of acquiring aphid images with high magnification. These cameras with long focal length and narrow FOV are generally selected. However, these cameras tend to be complicated, expensive, and densely deployed. From Figs. 5 and 6, the aphids on the leaf both in and out the LSOI are imaged at different local magnifications by the 4.5× local zoom system. The experimental results demonstrate the advantage of the 4.5× local zoom system in agricultural pest detection and recognition. The pest images captured by the peripheral imaging channel are used to determine the locations of pests. Then, the local zoom lens group will be scanned to zoom the pests in different LSOIs according to the determined locations. This system is able to achieve insect pest images of local high-magnification in a wide FOV and without reduction of the full FOV. Thus, it is possible to obtain local high-magnification pest images with the sparse deployment of the local zoom system.

5 Conclusions

In this paper, we propose a 4.5× local zoom system without reduction of the full FOV for insect pest images of local high-magnification in a wide FOV. The basic principle of the system local zoom imaging is presented in Sect. 2. This system has the peripheral imaging channel for searching pests in a wide FOV and the local zoom imaging channel for pest recognition. Two imaging channels have the same image plane. Then, the experimental set-up is built in accordance with the local zoom imaging basic principle. Finally, the set-up is used for the imaging of aphids on plant leaves. The MSR and MSRCR algorithms are adopted to increase the brightness and contrast of the local zoomed images. The enhanced images present a better contrast compared with those without processing. Real-time image processing algorithms can be introduced into the local zoom system for contrast enhancement in future research. The experimental results verify the basic principle of local zoom imaging of the 4.5× local zoom system. The system can be highly conducive to improving pest recognition accuracy in outdoor environments and simplifying imaging processing algorithm. This enables better detection and recognition of insect pests in complex environments.