1 Introduction

Machine vision systems are finding increasing use in the manufacturing and inspection fields due to their reliability, relatively low cost, and high throughput [1]. In implementing such systems, it is frequently necessary to focus a camera so as to obtain sharp images of the object of interest [2–8]. As a result, many autofocusing microscopes have been developed in recent years. Existing autofocusing microscopes can be broadly classified as either image-based or optics-based [9]. In microscopes of the former type, the focus position of the objective lens is adjusted by analyzing the image captured by an optical system [10–12]. Such an approach has a low cost and is therefore widely used for many industrial applications [13–27]. However, analyzing the image is highly complex, and thus the autofocusing procedure is rather time consuming [28]. Furthermore, depth of focus constraints limit the maximum focusing accuracy which can be obtained. Many proposals have been presented for resolving these problems by means of optics-based autofocusing methods [29–43]. However, to the best of the current authors’ knowledge, these methods achieve a satisfactory focusing accuracy over only a short linear autofocusing range.

Accordingly, this study develops a novel autofocusing microscope which provides both a large linear autofocusing range and a rapid response. The proposed microscope comprises two optical paths. The first optical path is used to achieve a highly precise focusing accuracy over a short linear autofocusing range, while the second path is used to achieve a less-precise focusing accuracy over a longer linear autofocusing range. The two optical paths are combined using a self-written autofocus-processing algorithm to achieve a large linear autofocusing range and a rapid response without any loss in the focusing accuracy. The proposed microscope is characterized numerically and is then verified experimentally using a laboratory-built prototype.

The remaining of this article is organized as follows. Section 2 describes the structure of a conventional optics-based autofocusing microscope with centroid method. Section 3 introduces the proposed optics-based autofocusing microscope and compares its output and motion characteristics with those of the conventional microscope with centroid method. Section 4 discusses the experimental characterization of the proposed autofocusing microscope using a laboratory-built prototype. Finally, Sect. 5 presents some brief concluding remarks.

2 Design of conventional optics-based autofocusing microscope with centroid method

2.1 Conventional autofocusing method with centroid method

Figure 1 presents a schematic illustration of the optical path within a conventional optics-based autofocusing microscope with centroid method. When the laser beam strikes point A on the sample surface (located at the focal plane 1 of the objective lens), it is reflected from the surface and is then incident on the CCD sensor at point A′. In accordance with basic geometric principles, the following equation can be obtained:

$$ \tan \alpha = d/f_{1} , $$
(1)

where α is the maximum incidence angle of the laser beam, d is the radius of the collimated laser beam, and f 1 is the focal length of the objective lens. When the sample surface is moved from plane 1 to plane 3 (corresponding to a defocus distance of +δ), the laser beam strikes point C on the sample surface and is then reflected. The reflected laser beam intersects plane 1 at point A1 and is then incident on the CCD sensor at point A1′. From basic geometric principles, the following equation can be obtained:

Fig. 1
figure 1

Schematic illustration of optical path in conventional optics-based autofocusing microscope with centroid method

Full size image
$$ \overline{\text{AA}}_{1} = 2\delta \tan \alpha . $$
(2)

Furthermore, the displacement Δ of the incident point on the CCD sensor can be obtained as

$$ \Updelta = K\overline{{{\text{AA}}_{1} }} ,$$
(3)

where Δ is the distance between point A′ and point A1′, and K is the total magnification of the objective lens and achromatic lens. K can be represented as

$$ K = {{f_{2} } \mathord{\left/ {\vphantom {{f_{2} } {f_{1} }}} \right. \kern-\nulldelimiterspace} {f_{1} }}, \, $$
(4)

where f 2 is the focal length of the achromatic lens. Substituting Eqs. (1), (2), and (4) into Eq. (3), the displacement distance Δ can be obtained as

$$ \Updelta = \delta \frac{{2df_{2} }}{{f_{1}^{2} }}. \, $$
(5)

Equation (5) shows that the displacement Δ and defocus distance δ are linearly related.

Figure 2 illustrates the shape of the laser spot on the CCD sensor given various defocus distances. Note that in the figure, points (X c, Y c) and (x centroid, y centroid) represent the positions of the geometrical image center and the centroid of the image captured by the CCD sensor, respectively. The image centroid coordinates can be expressed as follows:

$$ x_{\text{centroid}} = \frac{{\sum {\sum {\left( {x - X_{c} } \right)} } P_{ij} }}{{\sum {\sum {P_{ij} } } }}, \, $$
(6)
$$ y_{\text{centroid}} = \frac{{\sum {\sum {(y - Y_{c} )} } P_{ij} }}{{\sum {\sum {P_{ij} } } }}, $$
(7)

where i and j are the row number and column number of the CCD sensor, respectively, and P ij denotes the intensity of the pixel located at the intersection of row i and column j. In the design of the conventional optics-based autofocusing microscope with centroid method, the geometrical image center (X c, Y c) is assumed to be located at the origin of the coordinate frame xy and to remain constant. As a result, Eqs. (6) and (7) can be reformulated as follows:

Fig. 2
figure 2

Schematic representation of laser spot on CCD sensor given different values of the defocus distance

Full size image
$$ x_{\text{centroid}} = \frac{{\sum {\sum x } P_{ij} }}{{\sum {\sum {P_{ij} } } }}, \, $$
(8)
$$ y_{\text{centroid}} = \frac{{\sum {\sum y } P_{ij} }}{{\sum {\sum {P_{ij} } } }}. $$
(9)

From Eqs. (5), (8), and (9), it is seen that if the image intensity P ij remains constant over the entire CCD image, a linear relationship exists between the centroid of the image captured by the CCD device and the defocus distance δ. Exploiting this linear relationship, an autofocusing capability can be achieved by driving the objective lens using a position feedback signal derived from the centroid coordinates of the detected image [44].

2.2 Structure of conventional optics-based autofocusing microscope with centroid method

Figure 3 illustrates the basic structure of the conventional autofocusing microscope with centroid method constructed for comparison purposes in this study. As shown, the light beam emitted from a laser light source (Hitachi HL6501MG, 658 nm) is expanded and collimated through an expander lens. The collimated laser beam is bisected by a knife (As shown in Figs. 1 and 2, the knife is used to reshape the collimated laser beam into a semicircle beam, so the shape of the laser spot on the CCD sensor varies in accordance with the defocus distance δ.) and is then passed through two beam splitters (BSs). The first BS is designed to achieve an equal reflection and transmission of red light, while the second BS is designed to achieve a high reflection of red light and a high transmission of the remaining visible light (i.e., wavelengths other than 658 mm). The laser beam emerging from the second BS is passed through an objective lens and is incident on the sample surface. Following reflection from the sample surface, the laser beam returns through the objective lens and two BSs, passes through an achromatic lens, and is incident on a CCD sensor (Guppy F-146C, referred to hereafter as CCD1).

Fig. 3
figure 3

Structure of conventional optics-based autofocusing microscope with centroid method

Full size image

In accordance with basic geometric optics principles, the shape of the laser spot (or centroid (x centroid, y centroid) of the image) on the CCD sensor varies in accordance with the defocus distance δ. In this study, the variation of coordinates (x centroid, y centroid) was calculated using a self-written autofocus-processing algorithm. An autofocusing capability was then achieved by using a position feedback signal derived from the computed image centroid coordinates to dynamically adjust the position of the objective lens via a linear motor. Real-time images of the sample were captured during the focusing process using an infinity-corrected optical system (Navitar Zoom 6000).

3 Design of proposed optics-based autofocusing microscope

3.1 Structure of proposed optics-based autofocusing microscope

Figure 4 illustrates the structure of the proposed autofocusing microscope. As shown, the light beam emitted from the laser light source (Hitachi HL6501MG, 658 nm) is expanded and collimated by means of an expander lens and is then bisected by a knife. The light beam is then passed through two BSs. As in the conventional autofocusing structure, the first BS is designed to reflect 50 % of the red light and to transmit 50 % of the red light, while the second BS is designed to reflect most of the red light and to transmit the remaining visible light. The light emerging from the second BS is passed through an objective lens and is incident on the sample surface. The laser light reflected from the sample surface passes back through the objective lens and two BSs and is then incident on a third BS, where it is split into two separate optical paths. In one optical path (designated as Optical Path I), the light beam emerging from the BS is passed through an achromatic lens and is then incident on CCD1. Meanwhile, in the second optical path (designated as Optical Path II), the light beam passes through another achromatic lens and is then incident on a second CCD sensor (Guppy F-146C, referred to hereafter as CCD2).

Fig. 4
figure 4

Structure of proposed optics-based autofocusing microscope

Full size image

From Eqs. (5), (8), and (9), it follows that for a constant displacement Δ of the incident point, the defocus distance δ changes in accordance with changes in the focal length f 2 of the achromatic lens. Therefore, constructing the proposed autofocusing microscope, CCD1 and CCD2 are identical components, but the focal lengths of the achromatic lenses in Optical Paths I and II, respectively, are different. Specifically, the focal length of the achromatic lens in Optical Path I is greater than that of the achromatic lens in Optical Path II, i.e.,

$$ \Updelta = \delta_{\text{I}} \frac{{2df_{{2{\text{I}}}} }}{{f_{1}^{2} }}, \, $$
(10I)
$$ \Updelta = \delta_{\text{II}} \frac{{2df_{{2{\text{II}}}} }}{{f_{1}^{2} }}, \, $$
(10II)
$$ f_{{2{\text{I}}}} > f_{{2{\text{II}}}} , \, $$
(11)
$$ K_{\text{I}} = {{f_{{2{\text{I}}}} } \mathord{\left/ {\vphantom {{f_{{2{\text{I}}}} } {f_{1} }}} \right. \kern-\nulldelimiterspace} {f_{1} }}, \, $$
(12I)
$$ K_{\text{II}} = {{f_{{2{\text{II}}}} } \mathord{\left/ {\vphantom {{f_{{2{\text{II}}}} } {f_{1} }}} \right. \kern-\nulldelimiterspace} {f_{1} }}. \, $$
(12II)

Note that Eqs. (10I) and (10II) refer to Optical Path I and Optical Path II, respectively, and \( f_{{2{\text{I}}}} \) and \( f_{{2{\text{II}}}} \) are the focal lengths of the achromatic lenses in Optical Path I and Optical Path II, respectively. Optical Path I results in a greater total magnification K I, and therefore provides the means to accomplish autofocusing over a short linear range but with a high focusing accuracy. By contrast, Optical Path II yields a lower total magnification K II, and therefore provides the means to accomplish autofocusing over a longer linear range but with a lower focusing accuracy. Comparing Figs. 3 and 4, it can be seen that the proposed autofocusing microscope reduces to the conventional optics-based microscope with centroid method if Optical Path II is not activated. However, in implementing the proposed microscope, Optical Paths I and II are both activated and are combined via a self-written autofocus-processing algorithm to achieve autofocusing over a long linear distance with a short response time and with no loss in focusing accuracy.

3.2 Output and motion characteristics of conventional autofocusing microscope with centroid method and proposed autofocusing microscope

Figure 5a illustrates the basic output characteristics of the conventional autofocusing microscope with centroid method (please refer to user’s manual of ATF4 auto focus and scanning sensor, WDI Wise Device Inc.) shown in Fig. 3. As shown, a linear autofocusing range exists within which the centroid coordinates of the image captured by the CCD sensor vary linearly with the defocus distance δ. Within this linear range, the position of the objective lens can be easily adjusted so as to achieve a precise focusing effect by means of a linear motor driven by a feedback signal derived dynamically in accordance with changes in the centroid coordinates. However, as shown in Fig. 5b, if the initial defocus distance δ 1 is located outside of this linear range, the autofocusing procedure should commence by shifting the objective lens through a distance L, where L is equivalent to half the linear range. If the resulting defocus distance δ2 is still located outside the linear range, the objective lens should be shifted through a further displacement L. This procedure is repeated until the defocus distance (e.g., δ3) falls within the linear range, at which point the objective lens can be moved directly to the point of maximum focus. Figure 6 presents a flow chart showing the basic steps in the conventional autofocusing procedure with centroid method. From Figs. 5 and 6, it is seen that the objective lens must be shifted multiple times if the initial defocus distance is located far from the linear autofocusing range. Thus, while the structure shown in Fig. 3 enables the point of maximum focus to be reliably determined, the response time may be too long for many applications (e.g., TFT array laser repair and LED substrate laser scribing).

Fig. 5
figure 5

a Output characteristics and b motion characteristics of conventional autofocusing microscope with centroid method

Full size image
Fig. 6
figure 6

Flow chart of autofocus-processing algorithm for conventional optics-based autofocusing microscope with centroid method

Full size image

Figure 7 illustrates the output and motion characteristics of the proposed autofocusing microscope. As shown, both optical paths yield a linear autofocusing range. As \( f_{{2{\text{I}}}} > f_{{2{\text{II}}}} \) (i.e., \( K_{{2{\text{I}}}} > K_{{2{\text{II}}}} \)), the linear range associated with Optical Path II is greater than that associated with Optical Path I. If the initial defocus distance δ 1 is located outside of the linear range of Optical Path I, but within that of Optical Path II, the objective lens is shifted through a distance equal to the defocus distance via a linear motor in accordance with the calculated centroid coordinates (x centroid, y centroid) of the image detected by CCD2 in Optical Path II. However, Optical path II has a low focus accuracy. Therefore, the position of the objective lens is moved to δ2, outside of the depth of focus of this system. However, if δ2 is located within half the linear range of Optical Path I, the objective lens is moved directly to the point of maximum focus. Figure 8 presents a flow chart showing the basic steps in the proposed autofocusing procedure. From Figs. 7 and 8, it can be seen that irrespective of the initial defocus distance δ, the objective lens need be shifted no more than twice in arriving at the point of maximum focus. In other words, of the two autofocusing methods presented in Figs. 5 and 7, respectively, the proposed autofocusing microscope has a larger linear autofocusing range and a more rapid response, but maintains an equivalent focusing accuracy.

Fig. 7
figure 7

Output and motion characteristics of proposed autofocusing microscope

Full size image
Fig. 8
figure 8

Flow chart of autofocus-processing algorithm in proposed optics-based autofocusing microscope

Full size image

3.3 Numerical characterization of proposed autofocusing microscope

A series of ray-tracing simulations were performed to verify the focusing performance of the proposed autofocusing microscope and to determine suitable values of the major design parameters (i.e., d, f 1, f 2, K, and so on). Figure 9a, b illustrates the detailed structure of the proposed autofocusing microscope and the optical model constructed using commercial ZEMAX software. Table 1 summarizes the selected values of each design parameter.

Fig. 9
figure 9

a Detailed structure of proposed autofocusing microscope, b ZEMAX optical model of proposed autofocusing microscope

Full size image
Table 1 Design parameters of proposed autofocusing microscope
Full size table

Figure 10 presents the simulation results obtained for the laser spot shapes on CCD1 and CCD2, respectively, given different values of the defocus position. Meanwhile, Fig. 11 illustrates the simulation results obtained for the variation of the image centroid with the defocus distance in the two optical paths. Figures 10 and 11 show that a linear relationship exists between the centroid coordinates (x centroid, y centroid) of the detected image and the defocus distanceδ. In other words, the numerical results are consistent with the theoretical analysis presented in Sect. 3.1. The relationship between the image centroid coordinates (x centroid, y centroid) and the defocus distance δ in Optical Path I and Optical Path II is given as follows:

$$ {\text{centroid }}\left( {x_{\text{centroid}} ,y_{\text{centroid}} } \right) = - 6.93\delta , $$
(13I)
$$ {\text{centroid }}\left( {x_{\text{centroid}} ,y_{\text{centroid}} } \right) = - 3.18\delta . $$
(13II)

(Note that Eqs. (13I) and (13II) were obtained via a linear curve fitting technique performed using Microsoft Office Excel.) It is noted that the ratio of the slope of the linear curve in Eq. (13I) to that of the linear curve in Eq. (13II) is equal to 2.18 (= 6.93/3.18).

Fig. 10
figure 10

Simulation results for shape of laser spot on CCD sensor given different values of the defocus distance

Full size image
Fig. 11
figure 11

Simulation results for variation of image centroid position with defocus distance

Full size image

4 Experimental characterization of prototype autofocusing microscope

The validity of the proposed autofocusing microscope was verified by constructing a laboratory-built prototype equipped with a human machine interface (HMI) written in LabVIEW. Figure 12 presents two photographs of the laboratory-built prototype, while Fig. 13 shows experimental images of the laser spot on the two CCD sensors given different values of the defocus distance. Finally, Fig. 14 illustrates the variation of the image centroid position with the defocus distance in the two optical paths. Figures 13 and 14 show that in both optical paths, the image centroid position varies linearly with the defocus distance δ over a certain defocus range. In other words, the experimental results are consistent with the theoretical results presented in Sect. 3.3. In Fig. 14, the linear relationships between the image centroid coordinates (x centroid, y centroid) and the defocus distance δ in Optical Path I and Optical Path II are given, respectively, by

$$ {\text{centroid }}\left( {x_{\text{centroid}} ,y_{\text{centroid}} } \right) = - 6.99\delta , $$
(14I)
$$ {\text{centroid }}\left( {x_{\text{centroid}} ,y_{\text{centroid}} } \right) = - 3.26\delta . $$
(14II)

(Note that Eqs. (14I) and (14II) were obtained by applying a linear curve fitting technique to the experimental results presented in Fig. 14 using Microsoft Office Excel.) The ratio of the slope of the linear curve in Eq. (14I) to that of the linear curve in Eq. (14II) is 2.14 (= 6.99/3.26). It is noted that this experimental value is in good agreement with the theoretical value of 2.18 given in Sect. 3.3.

Fig. 12
figure 12

a Photograph of laboratory-built prototype, b magnified view of region A in (a)

Full size image
Fig. 13
figure 13

Experimental results for shape of laser spot on CCD sensors given different values of the defocus distance

Full size image
Fig. 14
figure 14

Experimental results for variation of image centroid position with defocus distance

Full size image

However, in Optical Path I of Fig. 14, it is interesting to note that a nonlinear autofocusing range exists outside the linear autofocusing range. This is because when the defocus distance δ is located outside of this linear range, the reflected laser beam grows bigger, and a certain part of the reflected laser beam is incident on the region outside the CCD chip of CCD1, as shown in Fig. 13. As a result, CCD1 can not capture the total intensity of the reflected laser beam, and the centroid of the image captured by CCD1 has a nonlinear situation.

To demonstrate the feasibility of the proposed autofocusing microscope, an autofocusing experiment was conducted using a mirror sample and an initial defocus distance of δ = 400 μm. For comparison purposes, the experiment was also performed using the conventional autofocusing structure with centroid method shown in Fig. 3. The experimental results obtained using the conventional and proposed autofocusing microscopes are shown in the video sequences presented in Figs. 15 and 16, respectively. In Fig. 15, it is seen that in the conventional autofocusing microscope with centroid method, the objective lens is moved from the initial defocus distance of 400 μm to a distance of 153.2 μm and is then moved through two further displacements to the final focusing position (yielding a focusing accuracy of less than 2 μm). By contrast, Fig. 16 shows that in the proposed autofocusing microscope, the objective lens is moved from the initial defocus position of 400 μm to a new position of 31 μm and is then moved directly to the final focusing position (again, with a focusing accuracy of less than 2 μm). Note that the objective lens can be moved directly via a linear motor (piezo-electric motor, HR8 series, Nanomotion Inc.) featuring a closed-loop control scheme based upon a feedback signal generated with an optical encoder (resolution of 0.1 μm). Therefore, the final focusing position was measured by using a position feedback signal from the optical encoder. In other words, compared to the conventional method with centroid method, the proposed autofocusing microscope yields an equivalent focusing accuracy, but has both a larger linear autofocusing range and a more rapid response.

Fig. 15
figure 15

Dynamic testing video of conventional autofocusing microscope with centroid method and mirror sample

Full size image
Fig.16
figure 16

Dynamic testing video of proposed autofocusing microscope with mirror sample

Full size image

5 Conclusions

This study has presented a novel optics-based autofocusing microscope with both a large linear autofocusing range and a rapid response. The proposed microscope comprises two optical paths. Optical Path I achieves a precise focusing accuracy over a short linear autofocusing range, while Optical Path II achieves a lower focusing accuracy, but over a longer linear autofocusing range. The two optical paths are combined using a self-written autofocus-processing algorithm to achieve a rapid autofocusing capability over a large linear autofocusing range without any loss in the focusing accuracy. The performance of the proposed microscope has been demonstrated using a laboratory-built prototype. The experimental results have shown that the proposed microscope has a larger linear autofocusing range and a faster response time than a conventional autofocusing microscope with centroid method, but achieves a comparable focusing performance (less than 2 μm). As a result, the proposed microscope provides an ideal solution for a wide range of optical inspection and industrial applications.