A novel feature-based tracking approach to the detection, localization, and 3-D reconstruction of internal defects in hardwood logs using computer tomography

Abstract

A novel feature-based tracking approach based on the Kalman filter is proposed for the detection, localization, and 3-D reconstruction of internal defects in hardwood logs from cross-sectional computer tomography (CT) images. The defects are simultaneously detected, classified, localized, and reconstructed in 3-D space, making the proposed scheme computationally much more efficient than existing methods where the defects are detected and localized independently in individual CT image slices and the 3-D reconstruction of the defects accomplished via correspondence analysis across the various CT image slices. Robust techniques for defect detection and classification are proposed. Defect class-specific tracking schemes based on the Kalman filter, B-spline contour approximation, and Snakes contour fitting are designed which use the geometric parameters of the defect contours as the tracking variables. Experimental results on cross-sectional CT images of hardwood logs from select species such as white ash, hard maple, and red oak are presented.

1 Original contribution of the work

The paper describes the design and implementation of a novel feature-based tracking approach for the detection, localization, and 3-D reconstruction of internal defects in hardwood logs from cross-sectional computer tomography (CT) images. With declining trends in hardwood forest resources and increasing costs of hardwood logs, improving the lumber value yield via detection and localization of internal log defects prior to lumber production has become imperative for many sawmills. Although the technical feasibility of CT scanning of logs in real time is fast approaching reality, the computational methods for analyzing the resulting CT images reliably and in real time are still a major challenge. In most conventional approaches, the internal log defects are detected and localized by analyzing individual CT image slices independently and the 3-D reconstruction of the defects accomplished via correspondence analysis across the various CT image slices. The resulting computational complexity makes the conventional approaches unsuitable for real-time applications. In the proposed scheme, defect detection, defect localization, and 3-D defect reconstruction are integrated within a Kalman filter-based feature tracking framework by exploiting spatial coherence across successive CT image slices. Robust techniques for defect detection and classification are proposed. Defect class-specific tracking schemes based on the Kalman filter, B-spline contour approximation, and Snakes contour fitting are designed which use the geometric parameters of the defect contours as the tracking variables. Robust techniques for extraction and characterization of the external log surface are also designed. Since the internal defects and the external log surface are simultaneously detected, classified, localized, and reconstructed in 3-D space, the proposed scheme is shown to be a major improvement over conventional approaches in terms of both computational efficiency and detection accuracy. Experimental validation of the proposed scheme on cross-sectional CT images of hardwood logs from select species such as white ash, hard maple, and red oak is also presented.

2 Introduction

The value of hardwood lumber is determined by the quantity, size, and types of internal log defects such as knots, cracks, decay, and other anomalies of tree growth that eventually appear on the lumber surfaces. Depending on the nature of the hardwood end utilization, each log must be sawed to minimize the presence of these internal defects on the resulting lumber surfaces. In order to achieve this goal, the internal defects within the log must be accurately identified and localized prior to the sawing of the log. In most sawmills, however, logs are processed into lumber with little or no information about the internal log defects and with inaccurate or incomplete geometric data about the logs. The logs are sawed by a sawmill operator who uses only his best judgement based on external log inspection and knowledge of lumber grades to make sawing decisions. Sometimes the sawmill operator unknowingly chooses the best sawing pattern for a log. However, many times his choices are far from optimal and less lumber or lumber of a lower grade is produced. Furthermore, boredom and fatigue of the sawmill operator can affect the accuracy of the lumber processing, resulting in suboptimal lumber production where the potential value of logs is wasted.

Production of lumber is essentially a destructive process. As the log is sawed, new information is divulged on the quality of the wood inside which may suggest a different and better sawing pattern. However, since each step in the sawing process is irreversible, the loss in the value yield has already been incurred and cannot be rectified. Haygreen [1] has reported a low conversion efficiency of about 35% for conventional sawmills. With the rising costs of hardwood logs accounting for over 80% of total production costs [2], improving the lumber value yield from hardwood logs has become imperative for many sawmills. Given the inherent limitations of external log inspection, it is reasonable to assume that future gains in lumber value yield will be possible only by internal log scanning [2–10].

Identification and localization of internal log defects are estimated to lead to potential gains of about 15–18% in lumber value, representing a savings of over $2 billion for the hardwood lumber industry in the United States [10, 11]. Forest products-based economies are increasingly dependent on getting the highest value wood products from a declining forest resource base. This results in disproportionate harvesting pressure on high-demand hardwood species such as hard maple, black walnut, white ash, and red oak that exhibit the greatest differences in value between the highest and lowest lumber grades [12]. Environmental concerns and the ecological need for maintaining biodiversity in forest ecosystems underscore the need to utilize as many hardwood species for wood products as possible, to improve the efficiency in converting low-grade logs into high-value lumber products, to reduce unnecessary wastage, and to conserve valuable hardwood forest resources. One way of achieving these goals is by identification and localization of internal defects in hardwood logs and using this information to optimize the processing of the resulting lumber.

Studies of computer axial tomography (CAT or CT) and magnetic resonance imaging (MRI) [also known as nuclear magnetic resonance (NMR) imaging] for internal log defects [2, 6–8, 10, 13–15] have demonstrated that the CT and MRI technologies available today can be used successfully to image the internal features of logs. CT scanners which are essentially solid state (i.e., with a minimum of moving parts) can scan at rates exceeding 30 slices per second. Thus, the technical feasibility of scanning logs in real time is fast approaching reality. However, the computational methods for analyzing the CT images for internal defects reliably and in real time, and exploiting the knowledge of the internal defects to determine optimal lumber processing strategies remain a challenging and open research topic.

In this paper we describe the design and implementation of a computer vision system for the detection, localization, and 3-D reconstruction of internal defects in hardwood logs from cross-sectional CT images. Robust techniques for defect detection and classification are proposed. In contrast to traditional methods, where the defects are detected and localized independently in individual CT image slices and the 3-D reconstruction of the defects accomplished via correspondence analysis across the various CT image slices, the proposed system integrates defect detection, defect localization, and 3-D defect reconstruction within a Kalman filter-based feature tracking framework. The defects are simultaneously detected, classified, localized, and reconstructed in 3-D space, making the proposed system computationally much more efficient than existing systems. Defect class-specific tracking schemes based on the Kalman filter are designed and implemented. The Kalman filter [16] uses the geometric parameters of the defect contours identified in the individual CT image slices as the tracking variables. The pixels corresponding to the outer boundary of the log cross-section are extracted in each CT image slice and collated across all the CT image slices to generate the external surface of the log. The identified and localized internal defects and the external surface of the log are reconstructed in 3-D space, resulting in a virtual 3-D model of the log along with its internal defects. The 3-D model of the log and internal defects could subsequently be used by a computer-based lumber production planning system to determine an optimal sawing strategy. Alternatively, the system could be used as a decision aid by sawyers and machinists in a sawmill where the detection and visualization of the internal defects and computer simulation of key machining operations would assist in determining an optimal lumber production strategy for a given hardwood log. The system could also be used as an interactive training tool for novice sawyers and machinists whereby they could practice various sawing strategies on virtual logs before working on real logs.

The remainder of the paper is organized as follows. Section 3 provides a brief review of previous work. Section 4 provides an overview of the proposed computer vision system. Section 5 provides a detailed scheme for the detection, identification, and localization of internal defects in a single CT image slice using a combination of structural and spectral features. The procedures for the detection, identification, and localization of knots, holes, and cracks in a single CT image slice are detailed in Sects. 5.1, 5.3, and 5.4, respectively. Section 6 describes the 3-D reconstruction of the internal defects using Kalman filter-based tracking algorithms. The 3-D reconstruction procedures for knots, and for cracks and holes are presented in Sects. 6.1 and 6.2, respectively. Section 7 presents experimental results on real CT image data from hardwood logs. Section 8 concludes the paper with an outline for future work.

3 A brief review of previous work

Computer axial tomography (CT or CAT) and MRI [also known as NMR imaging] represent two potentially viable techniques for acquiring cross-sectional images of logs. Although MRI is a more recent innovation, CT technology is approaching the speed necessary for real-time production use in sawmills [2]. MR images are characterized by the fact that wet portions of the object being imaged appear as relatively light regions in the resulting image, whereas dry portions of the object appear as relatively dark regions in the image. Hence MRI techniques are particularly well suited for detecting internal features of logs, such as knots, reaction wood, wet wood, and gum spots that are characterized by varying moisture content in the underlying wood [3, 17].

In CT images, the grayscale value of a pixel is directly proportional to the X-ray absorption which is then correlated with the material density at the pixel location [2, 8, 15, 18]. Knots and moisture pockets are noted to have higher material density and/or higher moisture content than surrounding clear wood and are often characterized by pixels with very high grayscale values in the CT image. Holes and decay pockets are void areas that are filled with air or decayed wood, respectively, and hence characterized by low material density. Consequently, the CT image pixels corresponding to holes and decay pockets have very low grayscale values. Cracks are also filled with air, and like holes and decay pockets, are characterized by pixels with very low grayscale values in the CT image. The major difference is that holes and decay pockets usually have circular cross-sections and are typically short in length whereas cracks are usually thin and long.

Gray-level thresholding and binarization techniques have been used extensively in internal defect identification in cross-sectional CT images of logs. As depicted in (1), the grayscale value of a pixel F(i,j) is used to classify the pixel as belonging to one of the major internal defect classes such as knots, moisture pockets, holes, decay pockets and cracks [4–7, 10, 19–22].

$${\rm Defect}\_{\rm Class}(i,j)=\left \{ \begin{array}{ll} {\hbox{Knots, moisture pockets}} & {\hbox{if}\; F(i,j) > T_1}\\ {\hbox{Cracks, holes, decay pockets}} & {\hbox{if}\; F(i,j) <T_2}\\ {\hbox{Normal wood}} & {\hbox{otherwise}} \\ \end{array} \right. $$

(1)

In (1), the threshold values T ₁ and T ₂ are determined by analyzing the grayscale histogram over all the CT image slices. The grayscale thresholding operation is followed by region growing that groups spatially contiguous pixels with the same class label into defect regions. As can be seen from (1), a grayscale-based classification scheme cannot distinguish between knots and moisture pockets or between cracks, holes, and decay pockets. Neural networks have been used to integrate grayscale features within a local 2-D and 3-D pixel neighborhood for internal defect classification [23–25]. However, since only gray level values are used as inputs, the neural network is reported to have problems distinguishing between knots and moisture pockets, both of which are characterized by high grayscale values in the CT images [24, 25]. Consequently, more recent approaches use shape and texture features in addition to pixel grayscale values to identify and localize the major internal defects [13, 14, 26–29]. Zhu et al. [29] use the basic 3-D geometric features of the defects in conjunction with the Dempster–Schafer theory of evidential reasoning for defect classification. Butler et al. [22, 26] impose the spatial coherence constraint in their defect classification algorithms. Samson [30] presents a geometrical model to describe knots within logs and on the surface of lumber beams sawn from those logs. Samson [30] also presents an algorithm to compute the effect of the presence of knots in the conversion of logs into structural lumber. Bhandarkar et al. [13, 14] use the 3-D geometry of the reconstructed defects for the purpose of classification. In particular, the 3-D shape parameters are used to distinguish between knots and moisture pockets and between cracks and holes/decay pockets.

The results of internal defect identification and localization can be used to reconstruct a 3-D model of the log along with its internal defects [13, 14, 21, 31–34]. Software programs that simulate various machining operations such as sawing and veneering on the virtual 3-D log reconstruction have been described in the literature [13, 14, 35–38]. Likewise, programs for automated grading of virtually produced lumber (using the aforementioned simulators) and physical lumber produced by sawmills have also been reported in the literature [4, 39–44]. These programs have been used to estimate the improvements in value yield recovery for lumber production using internal log scanning which are reported to be in the range of 40–60% [14, 36, 45].

The above review covers the most relevant developments in the use of CT and MRI technologies for internal defect detection and identification in hardwood logs. The review paper by Pham and Alcock [46] covers the wider area of automated visual inspection of logs and lumber and includes both external scanning using optical sensors and internal scanning using CT and MRI technologies.

4 Overview of the proposed defect detection system

The proposed computer vision system for identification, localization, and 3-D reconstruction of internal defects in hardwood logs via analysis of cross-sectional CT images is motivated by two major limitations of most existing systems. First, most existing computer vision systems identify internal log defects using thresholding algorithms that are based on the analysis of the pixel-level grayscale histogram computed from the input CT or MR image. However, it has been observed that such pixel-level thresholding or binarization methods are often inadequate in terms of their ability to discriminate the defect-containing areas in the input image from the defect-free areas even with fairly sophisticated analysis of the grayscale histogram [13, 14, 35–38]. In many CT images, the grayscale values of the pixels comprising a knot are usually close to those of pixels comprising the bright rings in the annular ring structure of the log. Consequently, if the value of threshold T ₁ in (1) is chosen to be too high, some pixels comprising a true knot might be mistakenly classified as non-knots. Conversely, with a very low value for threshold T ₁, many pixels comprising the bright rings will be misclassified as comprising a knot. Figure 1a shows an example CT image containing two knots (i.e., the two homogeneous bright areas). With a threshold value T ₁ = 185, the region classified as a knot in the upper right corner of the image contains too many pixels which actually belong to the ring structure (Fig. 1b). Simultaneously, some pixels which actually comprise the knot are classified as non-knots. A similar problem exists in the case of detection of cracks. A very low value for threshold T ₂ in (1) results in fragmentation of the crack, whereas too large a value for T ₂ results in the inclusion of pixels comprising the grayscale valleys between successive bright rings in the log ring structure. Figure 2a shows a CT image containing a crack. With a threshold value T ₂ = 50, the binarized image (Fig. 2b) can be seen to include several pixels comprising the grayscale valleys between successive rings in the log ring structure. Fragmentation of the crack can also be observed in Fig. 2b. These problems are further exacerbated when the log is too wet or too dry. When the log is too wet, there is a greater overlap between the grayscale values of pixels comprising the ring structure and those of pixels comprising the knots. Conversely, when the log is too dry, the grayscale values of pixels comprising the holes and cracks are almost the same as those of pixels comprising the grayscale valleys between successive rings in the log ring structure. The above examples clearly demonstrate the need to combine structural or shape information along with spectral or grayscale information within the defect detection and identification process.

Fig. 1

An example CT image and the result of knot detection using (1): a original CT image, and b binary image

Fig. 2

A example CT image and the result of crack detection using (1): a original CT image, and b binary image

Second, in most existing computer vision systems, the detection, identification, and localization of internal log defects are done independently in each CT or MR image slice. Simple correspondence analysis across multiple CT image slice is performed to reconstruct the defect in 3-D space [13, 14]. The geometrical parameters pertaining to the defect shape, size, and location are used to establish the correspondence across multiple image slices. However, this technique is computationally inefficient and unsuitable for real-time application since the spatial coherence of the defect along the axial direction is not exploited. In addition, since only two successive slices are used to establish the correspondence, the reconstruction is prone to error in the case of defects with complex 3-D shapes. Clearly, a technique that more closely integrates the processes of defect detection, identification, and 3-D reconstruction is needed.

The proposed system addresses both the aforementioned shortcomings. Defect detection and identification is achieved by integrating the spectral and structural properties of potential defect-containing regions in a CT image slice. Knots and rings share the same grayscale values but differ in their structural properties. Rings can be modeled as narrow strip-like regions in a local window, whereas knots can be modeled as ellipsoidal regions with bounded eccentricity. Analysis of the pixel grayscale density in a local window centered at each pixel within the defect-containing region is used to distinguish between the knots and rings. Since the above technique tends to cause erosion of the knot boundaries, a morphological operator is used to recover the boundaries of the knots. Since cracks are typically thin and long, they are also modeled as narrow strip-like regions in a local window. The grayscale valleys between successive rings also exhibit the same grayscale values as the cracks. However, the fact that a crack is usually perpendicular to the rings and grayscale valleys is exploited [47]. The points of intersection of a crack with the rings and grayscale valleys, termed as fork points, are detected and grouped to extract the line segments comprising the crack. False crack lines are removed via analysis of orientation of these line segments.

The processes of detection, identification, and reconstruction of the log and its internal defects are integrated within a Kalman filtering framework. A Kalman filter [16] is used to track cracks and holes continuously from one CT image slice to the next. A dynamic contour extraction technique based on Kalman Snakes [48] is used to reconstruct the knots and the outer boundary of the log in 3-D space. The Kalman filter is used to predict the locations of the defects in successive slices. This results in much faster detection, identification, and localization of the defects since only a local search within a fairly small neighborhood of the predicted defect locations is entailed. For example, when the approximate location and shape of a knot contour in the current CT image slice are known, the Kalman Snakes method can be used to predict the location and shape of the new contour in a succeeding CT image slice using local information, thus speeding up the overall computation. The overall flowchart for the proposed defect detection system is given in Fig. 3.

Fig. 3

The flowchart of the proposed defect detection system

5 Defect detection in a single CT image

The two primary goals of the proposed computer vision system are (1) to integrate structural and spectral features in the processes of detection, localization, and 3-D reconstruction of internal defects and (2) to integrate the processes of detection, localization, and 3-D reconstruction of internal defects within a single Kalman filter-based feature tracking framework. Defect detection and localization within a single CT image slice is performed in order to initialize the Kalman filter-based tracking algorithm. After a defect is detected and localized within a CT image slice, the tracking algorithm is used to detect and localize the defect in successive CT image slices and also reconstruct it in 3-D space.

5.1 Detection of knots

Since a simple pixel-level binarization or thresholding scheme typically misclassifies the bright annular ring areas as knots, a new binarization method is proposed to separate the knot-containing regions from the ring areas. Knots and rings share similar grayscale values but differ in their structural properties. Rings can be modeled as narrow strip-like regions in a local window, whereas knots can be modeled as ellipsoidal regions with bounded eccentricity. The ring areas can be eliminated via analysis of the pixel grayscale density in a local window centered at each pixel within the defect-containing region as outlined in (2).

$$K(i,j)=\left \{ \begin{array}{ll} 1 & {\hbox{if}\; \frac{1}{M^2}\sum\limits_{(x,y)\in W(i,j)}F(x,y)>T_{\rm k}}\\ 0 & {\hbox{otherwise}}\\ \end{array} \right. $$

(2)

where K(i,j) is the resulting binary image delineating the knots, W(i,j) is a window of predetermined size M ×  M centered at pixel (i,j), F(i,j) is a CT image slice of the log, and T _k is a predetermined threshold used to classify a pixel as a knot pixel. The binary image resulting from the application of (2) to the CT image slice in Fig. 1a is shown in Fig. 4a. A comparison of Figs. 1a and 4a shows that the application of (2) results in the erosion of the knot boundaries. Consequently, a morphological dilation operation [49] is used to recover the knot boundaries. The morphological dilation operation is represented as $K\oplus B$ where $\oplus$ is the dilation operator, K is the binary image resulting from the application of (2), and B is the dilation operator mask of size M ×  M depicted in Fig. 4b. The knots extracted after applying the dilation operation are shown in Fig. 4c.

Fig. 4

Result of knot detection: a result of analysis of local graylevel pixel density, b dilation mask, and c extracted knots

5.2 Detection of the outer log boundary

The outer boundary of the log cross-section in a single CT image slice can be simply detected using the binarization technique given in (2). However, the size of the window W used for the detection of the outer log boundary is much smaller (3 × 3 is typically adequate). Moreover, the threshold value T used for detection of the outer log boundary is also much smaller; typically it is slightly larger than the grayscale value of a void area in the CT image.

5.3 Detection of holes

Holes are actual void areas and appear as dark regions in the CT images with graylevels similar to those of the background. A simple thresholding scheme outlined in (3) is used to classify the CT image pixels as holes.

$$ K(i,j)=\left \{ \begin{array}{ll} 1 & {\hbox{if}\;F(i,j)<T_{\rm h}}\\ 0 & {\hbox{otherwise}}\\ \end{array} \right. $$

(3)

Since holes are usually small in size and approximately round in shape, false holes, typically caused by small cracks or grayscale valleys between successive rings, are removed by using a combination of morphological erosion and dilation operations on the thresholded result [49]. The mask size for the erosion and dilation operators in the case of detection of holes is smaller than the mask size of the dilation operator used in knot detection. A 3 ×  3 mask size is used for the erosion and dilation operators for hole detection. Figure 5 depicts the procedure to detect holes. Figure 5a is a CT image slice which contains a crack and several holes. After thresholding, holes are retained in the resulting binary image. However, a number of non-hole regions are also detected (Fig. 5b). After application of the erosion operator (Fig. 5c), the nonhole pixels are removed, but the holes are eroded. The dilation operator is then applied to restore the holes to their true size (Fig. 5d).

Fig. 5

Result of hole detection: a input CT image, b thresholded image, c removal of cracks and valleys using erosion, and d restoration of holes using dilation

5.4 Crack detection

A crack, as it appears in a CT image of a log, is usually long and thin. A straightforward grayscale-based binarization of the image results in either fragmentation of the detected cracks or too many misclassifications of regions denoting the grayscale valleys between the annular rings as cracks (as shown in Fig. 2b). Since both the grayscale valleys and cracks are narrow and long, the grayscale density-based binarization technique (2) is not able to separate them. However, cracks are typically perpendicular to the grayscale valleys and the local direction of a grayscale valley can be estimated by approximating the grayscale valleys by concentric circles centered at the centroid of the log cross-section. This property is exploited in the proposed crack detection scheme.

The proposed crack detection scheme is summarized as follows. The local linear structures resulting from cracks and valleys are first detected. Since the lines defining the cracks and the lines defining the local structure of the valleys intersect each other, the points of intersection are represented as fork points within a local window. By detecting and localizing these fork points, the actual crack features can be detected and localized. In order to reject false crack features, which are usually caused by the presence of random noise between the rings, it is determined whether or not the detected crack feature is parallel (within a certain angular threshold) to the local structure of the valleys. If the detected crack line is indeed parallel to the local structure of the grayscale valleys, then it is discarded else it is retained as an actual crack feature.

In order to detect the edges associated with the crack features, four directional Sobel-like operators are used. These operators perform a combination of direction-sensitive smoothing and edge detection for linear features. These operators are observed to reliably detect and localize the linear features in a CT image, even in the presence of noise. They can detect crack features without causing them to be fragmented. Additionally, the grayscale valleys between the rings are detected as well. A sample image resulting from the application of these operators is shown in Fig. 6a. The resulting image is binarized using a simple thresholding technique and a thinning (i.e., skeletonization) algorithm described in [50] is used to skeletonize the binary image (Fig. 6b). Fork points in the thinned image are detected and localized using a local 7 ×  7 window. Fork points that are distributed along the same crack feature are grouped together using a greedy clustering algorithm that is based on connectivity and mutual proximity of the fork points. The clustering algorithm exploits the fact that fork points on the same crack feature should be spatially proximate and connected to each other. Groups of fork points that are parallel to the local ring or valley structure are discarded. Figure 6c, d shows the results of fork detection and fork grouping, respectively.

Fig. 6

Result of crack detection: a output of the Sobel-like operators, b result of thinning the binary image, c result of fork detection, and d result of fork grouping

The groups of fork pixels that are retained are characterized by line segments that describe the crack feature. The RANSAC algorithm [51] is used to perform line fitting since it is robust to the presence of outliers in the input data. The RANSAC algorithm randomly samples a subset of points from the given set of data points to perform least-squares line fitting. The data points whose distances from the fitted line are above a certain threshold are considered as outliers, whereas the remaining data points are considered to be inliers. If the number of inliers is above a certain percentage of the total number of data points, the least-squares line fitting is performed again but the input data points are restricted to the inlier set. If the inlier set is not large enough, the above procedure is repeated on a new subset of the input data points.

In our case, the RANSAC-based line fitting procedure is not restricted to the fork pixels since the number of candidate fork pixels in a group is typically not large enough to ensure robust line fitting. Hence, in addition to the fork pixels within a group, the pixels between the fork pixels in the skeletonized (i.e., thinned) image are considered as well. In order to achieve this, the candidate fork pixels within a group are ordered. The bounding box for the group of candidate fork pixels is computed. If the width of the bounding box is larger than the height, the candidate fork pixels in the group are ordered along the x axis, otherwise they are ordered along the y axis. If {p _i } is the ordered set of candidate fork pixels within the group, then for each pair of successive fork pixels (p _i , p _i+1), a local window can be defined by using the pixels (p _i , p _i+1) as the corner points of the window. The 1-pixels in the skeletonized image obtained from within all the windows defined for every pair of successive candidate fork pixels together with the fork pixels are used in the line fitting procedure. Figure 7 depicts the result of the RANSAC-based line fitting procedure.

Fig. 7

The result of the RANSAC-based line fitting procedure

The extraction of the actual crack line segment in the binary image is based on the parameters of the fitted line. The primary reason for doing so is that the line segment representing the actual crack line may be longer than the line segment defined by the fork pixels. This is so because the ring structure is weak in the vicinity of the center of the log cross-section, causing fork pixels to go undetected. Thus, a 1-pixel in the binary image is deemed to belong to the actual crack line segment, if it is both, close to the fitted line and also spatially connected via a path consisting entirely of 1-pixels to the fork pixels within the group. An iterative depth-first search procedure detailed in [52] is used to localize the actual crack pixels.

6 Three-dimensional defect reconstruction using the Kalman filter

Internal defects within the log are observed to exhibit spatial coherence across several successive CT image slices, i.e., the appearances of an internal defect in successive CT image slices share similar attributes, such as location, shape, size, and grayscale distribution. Once an internal defect is detected in an image slice, its attributes in successive slices can be predicted. Thus, when the attributes of an internal defect in previous image slices are known, it is much more efficient to predict its attribute values in succeeding image slices and then compute the actual attribute values by using the predicted values as the starting point. Defect correspondence between successive slices is automatically established in this scheme thus precluding the need to perform correspondence analysis independently for each pair of successive image slices. The result is a computationally more efficient defect extraction and 3-D defect reconstruction procedure. The Kalman filter is adopted as the prediction scheme for this purpose. Computationally efficient algorithms for detection of knots, holes, and cracks are designed to take advantage of the predicted defect attribute values resulting from the Kalman filter. Continuous defect tracking and 3-D construction across CT image slices are achieved by iteratively repeating the process of Kalman filter-based prediction and fast defect extraction.

Knots, holes, and cracks exhibit very different geometrical attributes in a 2-D image slice and hence have different Kalman filtering models. A knot can be approximated by a homogeneous region bounded by a convex contour. The convexity assumption enables efficient geometric computation. A crack is usually long and thin and is approximated by a line segment. A hole usually has a smaller cross-sectional area and can be represented by a rectangular bounding box. In addition, the outer boundary of the log cross-section is also detected and tracked across CT image slices in order to virtually reconstruct the entire log in 3-D space.

6.1 Three-dimensional reconstruction of knots and the exterior log surface

The knots and the outer boundary of the log cross-section can be simply encoded by using the positions of their centroids and enumerating the pixels on their respective bounding contours. However, the raw contour pixels cannot be used directly as the tracked/predicted variables in the Kalman filter model for two reasons:

• 1. The number of contour pixels for a single knot defect may vary significantly in different image slices. This makes formulation of the model a difficult task.

• 2. It is computationally inefficient to use too many tracked/predicted variables in the Kalman filter.

In addition, the complexity of geometrical computation for a nonconvex contour is much higher than that for a convex contour. This is an important consideration in a lumber production planning system, where a significant amount of geometrical computation is entailed in determining the contour(s) of intersection of a knot (or the exterior log surface) and the sawing plane. Thus, it is desirable to encode a knot defect or the exterior log boundary in a single CT slice using a convex hull defined by a small number of points. A B-spline contour approximation algorithm can be used to determine the control points of the convex hull. The control points are then used as the tracking parameters in the Kalman filtering model. In this section, the Kalman filter model for 3-D reconstruction of knots and the outer surface of the log is described.

The Kalman filter model for the reconstruction of knots is depicted in Fig. 8 and is described as follows:

• 1. Initially, a knot defect area is detected in a single CT image using the technique described in Sect. 5.1.

• 2. The contour of the knot defect is extracted and its convex hull is computed. M B-spline control points are then used to approximate the convex hull.

• 3. The Kalman filter is applied to predict the “velocity” of the knot defect. Here, the term velocity denotes the rate at which the shape of the knot changes across successive CT image slices. Let d _x denote the velocity component along the x axis and d _y the velocity component along the y axis. Let s denote the scale parameter in the time interval [t, t + 1] such that s = 0 denotes the fact that the object size is unchanged in the time interval [t, t + 1], s > 0 denotes that the object size has increased and s < 0 denotes that the object size has shrunk in the same time interval. If at time t, the centroid of the convex hull is given by (c _x , c _y ), and the velocity by (d _x , d _y , s), then for a point (x, y) on the convex hull at time t, its position (x′, y′) at time t + 1 can be computed using (4).

  $$\begin{aligned} x' &= x+d_x+s(x-c_x) \\ y' &= y+d_y+s(y-c_y) \\ \end{aligned} $$

  (4)

• 4. The predicted velocity (d _x , d _y , s) is used to estimate the new position of the predicted convex contour using (4). The updated convex contour is obtained by using the predicted convex contour to initialize the Snakes contour fitting algorithm. The Snakes contour fitting algorithm is used to search for the actual boundary of the knot in the new CT image slice.

• 5. Steps 2, 3, and 4 are repeated until all the CT image slices are processed or the size of the knot is too small to be classified as a valid knot.

Fig. 8

An outline of the Kalman Snakes-based tracking method

Since the above algorithm uses a combination of Kalman filter-based prediction and contour fitting using Snakes, it is termed as the Kalman Snakes algorithm. The Kalman Snakes algorithm for reconstruction of the exterior log surface is similar, except for some differences in how the convex hull points are generated. The algorithm for convex hull generation is given in the following subsection. The details of the B-spline contour approximation algorithm and Kalman filter-based prediction can be found in [55].

6.1.1 Convex hull extraction

Given N 2-D contour points, there exists a subset consisting of M of the N contour points such that these M contour points define a convex contour and each of the original N contour points is either enclosed by this convex contour or is a vertex of the convex contour. The M contour points described above constitute the convex hull of the original N contour points. Graham’s algorithm [53] allows for the efficient computation of the convex hull. Graham’s algorithm can be briefly described as follows.

The top most point from the original point set is chosen as the starting point. All other points in the original point set are sorted in clockwise order based on their angles formed with the topmost point.Graham’s algorithm iteratively adds a new point p _i from the sorted point array in to the convex point set; and checks whether the point p _i-1 added prior to p _i violates the convexity relationship with respect to points p _i and p _i-2 where p _i-2 is the point added prior to point p _i-1. Point p _i-1 is deleted if it violates the convexity relationship. The details of Graham’s algorithm can be found in [53]. Figure 9 depicts an example of a set of the original points where point p ₀ is chosen as the starting point. The other points are sorted and sequentially added to the convex point set. Points p ₂ and p ₇ are removed because they do not comply with the convexity relationship. The complexity of Graham’s algorithm is O(N log N). In order to reduce the computational complexity, we resample the contour points (increase the sampling interval) in order to reduce N before Graham’s algorithm is applied.

Fig. 9

An example to illustrate Graham’s convex hull finding algorithm

It is empirically observed that the error resulting from approximating the contour of a knot defect in a single CT image slice with its convex hull is relatively insignificant since most knot contours are very close to being convex in shape. The outer boundary of a log cross-section, as it appears in a single CT image slice, is also close to being a convex contour in shape. However, since the outer boundary is large, there is a possibility of treating large void areas as normal wood when approximating a slightly nonconvex outer boundary by its convex hull. Therefore, it is necessary to check the difference between the original outer boundary and its convex hull approximation using the following procedure. Let P denote an ordered sequence of the contour points belonging to the outer boundary of the log cross-section. The contour points in P that belong to the convex hull are marked. All possible subsequences in P, such that the endpoints of each subsequence are convex hull points and all other points in the subsequence are nonconvex hull points, are generated. Each such subsequence of points defines a region which is included in the convex hull but does not belong to the interior of the actual contour. The convex hull of each such region, with an area above a predefined threshold, is also stored as part of the contour data in addition to the convex hull of the actual contour. For example, consider the sequence of points p ₀, p ₁,...,p ₈ in Fig. 9. If the original order of the contour points is p ₀, p ₁, p ₂, p ₃, p ₄, p ₅, p ₆, p ₇, p ₈, then based on the aforementioned algorithm, the subsequences that need to be stored are p ₁, p ₂, p ₃ and p ₆, p ₇, p ₈ (Fig. 10). On the other hand, if the original order of the points is p ₀, p ₁, p ₃, p ₂, p ₄, p ₅, p ₆, p ₇, p ₈, then based on the aforementioned algorithm, the subsequences that need to be stored are p ₄, p ₂, p ₃ and p ₆, p ₇, p ₈ (Fig. 11).

Fig. 10

Order of the original contour points p ₀, p ₁, p ₂, p ₃, p ₄, p ₅, p ₆, p ₇, p ₈

Fig. 11

Order of the original contour points p ₀, p ₁, p ₃, p ₂, p ₄, p ₅, p ₆, p ₇, p ₈

6.1.2 Active contour modeling using Snakes

Snakes is an active contour modeling method used to represent and extract deformable contours in an image. Given an initial contour, Snakes evolve to the actual contour by minimizing an energy function, such as the one given in (5). Usually, the Snakes energy function is a sum of two terms, the first representing the internal energy and the second the external energy. The internal energy term imposes a smoothness constraint on the object shape whereas the external energy terms pull the points on the Snakes contour towards the actual object boundary. The internal energy term, which includes the sum of the distances between successive Snakes contour points and the sum of the second-order differences of the Snakes contour points, ensures smoothness and compactness of the object shape. The external energy term in (5) contains the image gradient, which tends to pull the Snakes contour points towards the image pixels with high gradient values.

$$\begin{aligned} E_{{\rm s}}& = \sum \alpha|\dot{x}(s)|+\beta|\ddot{x}(s)|+E_{\rm ext}(x(s)) \\ &=\sum \underbrace{\alpha|v_i-v_{i-1}|+\beta|v_{i-1}-2v_i+v_{i+1}|}_{{\rm internal}\, {\rm energy}}+\underbrace{-\gamma|\nabla I(v_i)|}_{{\rm external}\,{\rm energy}} \\ \end{aligned}$$

(5)

If the image gradient values are very low in the vicinity of the Snakes contour points, there are no external forces to pull the Snakes contour points towards the edge points. Hence, in this situation, the Snakes contour points, although smooth, provide a very poor fit to the actual edge pixel data. For example, the interior of knot defects is usually homogeneous and strong image gradient values exist only along the outer boundary of the knots. Thus, a Snakes contour that is initialized with image pixels within the interior of a knot would not converge on the outer boundary of the knot. In a more complicated situation, a small image gradient value within the interior of the knot could draw the Snakes contour points towards the interior of the knot from which it may never recover. There are many methods proposed to tackle the Snakes contour fitting problem within homogeneous image regions. Xu and Prince [54] have proposed a gradient vector flow technique, which iteratively propagates the gradient value from a pixel with high gradient value to the neighboring pixel locations, thus producing a gradient vector flow. This technique solves the problem caused by the presence of locally homogeneous regions when performing Snakes contour fitting, but is computationally intensive as several iterations are needed to establish the vector flow over the entire image. Moreover, when a hole exists within the interior of a knot resulting in strong gradient values around the hole, the gradient vector flow algorithm only makes the situation worse.

A knot region in a single CT image slice can be extracted by using a binarization technique, which allows for a much simpler solution to the problem caused by the presence of homogeneous regions in the image. If a point belongs to the interior of the knot, a force can be designed to move this point away from the centroid of the knot thus causing it to approach the actual outer boundary of the knot. If a point does not belong to the knot, a force can be designed to move this point towards the centroid of the knot thus bringing it closer to the outer boundary of the knot. The redesigned external energy equation E _ext = sign(T _k − D(v _i ))|v _i  − C| has this property. Here C is the centroid of the knot, $D(v_i)=1/M^2\sum_{(x,y)\in W(v_i)}F(x,y)$ is the grayscale density at Snakes contour point v _i measured in a local window W (v _i ) of size M  ×  M centered at v _i , F(x,y) is the image intensity function, T _k is the binarization threshold for knots and sign(x) = 1 if x  ≥  0; and sign(x) =  − 1 otherwise. If D(v _i )  ≥  T _k, then v _i is a knot point and larger |v _i  − C| values will result in lower energy since sign(T _k − D(v _i )) =  − 1. On the other hand, if D(v _i ) < T, then v _i is not a knot point and smaller |v _i  − C| will result in lower energy since sign(T _k − D(v _i )) = 1. Note that the computation of the grayscale density D(v _i ) is fairly robust to the presence of small holes within the interior of the knot as long as the overall area covered by the holes within the window W(v _i ) is a small fraction of the window size M ². The redesigned energy function can be expressed as follows:

$$E_{\rm s}=\sum\underbrace{\alpha|v_i-v_{i-1}|+\beta|v_{i-1}- 2v_i+v_{i+1}|}_{{\rm internal}\,{\rm energy}}+\underbrace{{\rm sign}(T_{\rm k}-D(v_i))|v_i- C|}_{{\rm external}\,{\rm energy}}$$

(6)

in which T _k is the chosen binarization threshold for knots, D(v _i ) is the grayscale density value at Snakes contour point v _i , and C(x,y) is the centroid of the Snakes contour which is a good approximation to the centroid of the knot. This redesigned Snake contour fitting technique is termed as Binary Snakes fitting in this paper.

The Binary Snakes fitting technique can be used to extract both knots and the outer log cross-section boundaries in a single CT image slice. The only difference is that in the case of the outer log boundaries the binarization threshold is much smaller than the one used for knots and the window size M can be just 1 since the grayscale difference between the interior and exterior of the log cross-section is typically quite significant. Figure 12a,b depict the results of Binary Snakes contour fitting for knot extraction on two CT image slices.

Fig. 12

Experimental results of Binary Snakes contour fitting for knot extraction: a CT image slice 28, and b CT image slice 34

6.1.3 Kalman Snakes

The Snakes contour fitting algorithm converges to the actual boundary in much fewer iterations if the initial Snakes contour points are chosen close enough to the actual boundary. Therefore it is natural to combine the Kalman filtering algorithm with the Snakes fitting algorithm, where the Kalman filter is used to predict the initial locations of the Snakes contour points in the new CT image slice given the locations of the boundary points in previous CT image slices. A hybrid velocity- and contour-based Kalman filter model proposed by Koller et al. [56, 57] is used in this paper. The model can be described using the following equations:

$$\xi^-_{k+1} = \xi^+_k+q_k $$

(7)

$$Z_k = H_k\xi^-_k+v_k $$

(8)

$$X^-_{k+1} = X^+_k+H_k\xi_k $$

(9)

$$ Z^X_k = X^-_k+v_k $$

(10)

Equation (7) represents the velocity transition function, (8) the velocity measurement function, (9) the contour transition function, and (10) the contour measurement function. Each of the two pairs of (7) and (8), and (9) and (10) constitute a standard linear Kalman filter model. Variables with the superscript “ − ” denote the prior estimations or predicted values. Variables with superscript “+” denote the post estimations or the predicted values after adjustments. The various variables in (7)–(10) are explained as follows:

ξ _k ::

The velocity vector at time k. ξ _k  =  (d _x d _y s)^T where d _x is the speed or velocity component along the x axis, d _y the speed or velocity component along the y axis and s is the scale parameter as described in (4).

q _k ::

Random additive noise in the velocity transition function. It is modeled as zero-mean Gaussian white noise that follows the distribution $\mathcal{N}(0,Q_k)$ A time-invariant (i.e., stationary) noise model is assumed, i.e., Q _k  = Q ₀.

v _k ::

Random additive noise in the velocity measurement function. It is modeled as zero-mean stationary Gaussian white noise, i.e., v _k ∼ N(0, R _k ) where R _k  = R ₀.

X _k ::

The points along the contour of an object at time k. X _k  = (x ₁,y ₁,...,x _n ,y _n )^T, which represents the vector of the B-spline control points in Kalman Snakes model.

H _k ::

The measurement matrix given by $H_k=\left( \begin{array}{ll} I_2 & (\hat X_{1,k}^+-C_k)\\ \vdots &\ddots \\ I_2 & (\hat X_{n,k}^+-C_k)\\ \end{array} \right)$ where $I_2=\left( \begin{array}{ll} 1&0\\ 0&1\\ \end{array} \right).$

C _k ::

The centroid of the object at time k.

Z ^X _k ::

The control points of the actual contour obtained from the Snakes fitting algorithm.

Z _k ::

The actual velocity measurement used for velocity predictiongiven by Z _k  = Z ^X _k  − X ⁻ _k .

The Kalman filtering algorithm can be explained as follows:

• 1. Given the adjusted velocity prediction (postprediction) ξ ⁺ _k at time k, predict the velocity at time k + 1 using ξ ⁻ _k+1  = ξ ⁺ _k  + q where initially, ξ ⁺₀  =  (0 0 0)^T.

• 2. Predict the positions of the contour control points X ⁻ _k+1 using ξ ⁻ _k+1 : X ⁻ _k+1  = X ⁺ _k  + H ⁺ _k ξ ⁻ _k+1 .

• 3. Compute the prior-covariance of the velocity transition function: π ⁻ _k+1  = π ⁺ _k  + Q.

• 4. Based on the predicted contour control points, compute the actual contour control points Z _k+1 at time k + 1 using the Snakes fitting technique and compute the error as $\tilde{Z}_{k+1} = Z_{k+1}-X_{k+1}^-.$

• 5. Compute the postcovariance for the transition function as (π ⁺ _k+1 )⁻¹ = H _k+1′R ⁻ H _k+1 + (π ⁻ _k+1 )⁻¹ where H′ is the transpose of H.

• 6. Compute the postcovariance of the measurement function: (Π ⁺ _k+1 )⁻¹ = R ⁻¹ + (Π ⁻ _k+1 )⁻¹.

• 7. Compute the postprediction for the velocity: $\hat{\xi}_{k+1}^+=\pi_{k+1}^+(H_{k+1}'R^{-1}\tilde{Z}_{k+1}+ \pi_{k+1}^{-1}\hat{\xi}_{k+1}^-).$

• 8. Compute the postprediction for the contour control points: $\hat{X}_{k+1}^+=\Pi_{k+1}^+(R^{-1}\tilde{Z}_{k+1}+ (\Pi_{k+1}^-)^{-1}\hat{X}_{k+1}^-).$

For further details on the Kalman filter, the interested reader is referred to [55].

6.2 Three-dimensional construction of holes and cracks

The Kalman filter-based model for 3-D reconstruction of cracks and holes is given by the following equations:

$$\hat{\xi}_{k+1}^- = \hat{\xi}_k^++q_k$$

(11)

$$Z_k = \hat{\xi}_{k}^-+v_k $$

(12)

where $\hat{\xi}$ is the tracking variable and Z _k is the measured value of the tracking variable. The contour of a hole is encoded by its bounding rectangle. The velocity of the center point of the rectangle ξ = (ξ _x , ξ _y )^T is used as the tracking variable in the tracking model for holes. Given a prediction $\hat{\xi}_{k+1}^-,$ the new center point of the hole is computed. A rectangular image region centered on the predicted center point is then subject to binarization using (3) and the new location of the hole is obtained exactly using the technique for hole detection in a single CT image slice. Thus, the 3-D reconstruction of a hole is achieved by tracking the hole region across successive CT image slices. The need for explicit correspondence analysis between hole regions extracted independently in successive CT image slices is thus averted.

The orientation angle θ of the line segment representing a crack is used as the tracking variable ξ in the Kalman filter-based tracking model for a crack. After a crack is detected and extracted in the previous CT image slices, its orientation angle in succeeding CT image slices is predicted using (11). A fast crack localization scheme is used to extract the crack in succeeding CT image slices using the predicted orientation. For continuous crack detection, the image is first binarized using the scheme described in [52]. The crack detected in the previous CT image slice is represented by the parameters (ρ, θ, x ₀, y ₀,  l) where (x ₀, y ₀) is the crack center and l the length of the line segment describing the crack. The crack line is described by the equation x cos θ + y sin θ = ρ. The actual orientation angle of the crack in a succeeding CT image slice may be slightly different from the predicted value of θ. The following search algorithm is used to determine the value of parameter θ.

For each predicted value of θ, a rectangle with height l, width δ and orientation θ and centered at point (x, y) is generated as shown in Fig. 13a. Note that δ is a predetermined small value. The image in the interior of the rectangle is projected along orientation θ, as shown in Fig. 13b, c. The projected values (the sum of the pixel graylevel values) along the orientation θ are stored in array p(x) where 0 ≤  x  <  δ. Let $f(\theta)=\hbox{max}_{0 \leq x < \delta} p(x).$ A local search is performed in the range $\left[ \theta - \Delta \theta, \theta + \Delta \theta \right]$ and the optimal value θ* is chosen such that $\theta*= \hbox{arg max}_{{\theta' \in [\theta - \Delta \theta, \theta + \Delta \theta]}} f(\theta').$ Given the new orientation θ* and $x* = \hbox{arg max}_{0 \leq x < \delta} p(x),$ the line corresponding to the updated crack can be determined as $(\rho', \theta') = (x_{0}\,\hbox{cos}\,\theta* + y_{0}\,\hbox{sin}\,\theta{*} - x', \theta*)$ (Fig. 14). Next, the center of the crack line and its length are updated using the following procedure. A new rectangle with parameters (l ₁, δ₁, x ₁, y ₁, θ*) is chosen as shown in Fig. 15a where l ₁ is the length of the new rectangle such that l ₁ > l, δ₁ is its width, and (x ₁, y ₁) is its center point which is the projection of point (x ₀, y ₀) on the line (ρ′, θ′) such that (x ₁, y ₁) =  (ρ′ cos θ′ − ρ₀ sin θ′,ρ′  sin θ′  +  ρ₀ cos θ′), where ρ₀  =  x ₀ sin θ′ − y ₀ cos θ′. Next, the image within the interior of this rectangle is projected along the direction perpendicular to line (ρ′, θ′) as shown in Fig. 15b. The largest continuous area in the projection corresponds to the new crack. However, in some cases, the crack could be broken, therefore all the segments with length larger than a threshold, where the threshold is chosen to be larger than the gap between the annular rings, are connected together to localize the crack. The two endpoints (x ₁, y ₁) and (x ₂, y ₂) of the line segment can be obtained from the locations of the two endpoints of the projection. Thus, the center point and the length of the updated crack can be computed and the updated parameters of the crack in the new CT image slice obtained.

Fig. 13

Crack localization in the new slice: a rectangle defining the crack, b image of the crack in the interior of the rectangle, and c projected pixel graylevel values along the orientation θ

Fig. 14

Updating the crack parameters in the new slice

Fig. 15

Determination of the crack length in the new slice: a Rectangle defining the crack in the new slice, and b projection of the pixel graylevel values along a direction perpendicular to the crack line

The Kalman filter-based tracking method described above for the detection and 3-D reconstruction of crack defects is much faster than detecting crack defects independently in individual CT image slices and then computing the 3-D reconstruction by establishing defect correspondence across the CT image slices. This is so because in the Kalman filter-based tracking approach, it is not necessary to perform all the steps involved in crack detection and localization (i.e., thinning the binarized image, extracting the fork points, grouping the fork points, performing RANSAC-based line fitting to the grouped fork points, and performing a search to determine the crack length) in subsequent CT image slices once they have been performed for the first CT image slice in the image sequence. Moreover, by performing the projection to obtain the length of the cracks, some false cracks can be removed if their length is too short.

After the defect detection and localization is performed in the new CT image slice using the Kalman filter-based tracking method described above, the defect areas that are detected and localized are removed from further consideration. The procedures for defect detection and localization in a single CT image slice (i.e., without defect tracking) are applied to the remainder of the image to search for new defects.

6.3 Removal of false defects and insertion of missing defects

Spatial coherence is exploited to remove false defects and account for missing defects in the CT image slices. If a defect is detected only in one CT image slice, and no corresponding defect is detected in k previous or k succeeding CT image slices, where k is a predetermined threshold (in our case k = 2), then the defect is deemed to have been caused by random noise and is removed from further consideration. Likewise, when the size of a defect is small, it is possible that it is not detected in a single CT image slice. If a defect is detected in CT image slices i − 1 and i + 1 but not in CT image slice i, then it is necessary to verify whether or not the defect exists in CT image slice i. The approximate position of the defect in CT image slice i is computed via interpolation between its positions in CT image slices i − 1 and i + 1. The existence of the defect at the interpolated position in the CT image slice is verified by using the defect detection and localization procedure for a single CT image slice (i.e., without tracking) but with relaxed threshold values. For example, in the case of a hole, the detection and localization procedure described in Sect. 5.3 is used but the value of threshold T _h in (3) is chosen to be slightly higher. If a hole is detected with the relaxed threshold value, then it is deemed to be a real hole.

The aforementioned techniques for removal of false defects and insertion of missing defects by exploiting spatial coherence have been verified using the available CT image data sets. It was empirically observed that detection of false knots or insertion of missing knots is typically not an issue since knots tend to be fairly large and distinct. Holes, on the other hand, could be missed if they are small in diameter. However, after exploiting spatial coherence it was possible to restore missing hole defects. In the case of crack defects, the Kalman filter-based tracking technique described previously was observed to be robust enough to detect, localize, and compute a 3-D reconstruction of a crack, once detected and localized in previous CT image slices. In the case of false cracks and holes, verifying spatial support from previous and/or succeeding CT image slices was found to be very effective in their removal.

7 Experimental results

The proposed Kalman filter-based tracking scheme for detection, localization, and 3-D reconstruction of internal defects in hardwood logs from CT image data was subject to experimental verification and validation. Experiments were conducted on four sets of log data, from three popular hardwood species found in the United States, namely, white ash, red oak, and hard maple, and were labeled as Ash1, Ash2, Maple, and Oak, respectively. The cross-sectional CT images of the hardwood logs were captured using a Toshiba TCT 20AX CT scanner (a third generation CT scanner) with a pixel resolution of 0.75 mm by 0.75 mm, an intensity resolution of 8 bits per pixel (i.e., 256 gray levels) and an image size of 316 × 316 pixels. The scanning of a 4 m log resulted in 224 cross-sectional CT images. All the programs were run on a 2.0 GHz Pentium 4 Xeon workstation with 1.5 GB RAM and 1.0 MB of cache memory.

Figure 16 shows the results of Kalman filter-based tracking and contour fitting using Binary Snakes for detection and localization of knots over a continuous sequence of CT image slices. Figure 17 depicts the resulting 3-D reconstruction of the knots. Likewise, Fig. 18 shows the results for detection and localization of cracks and holes over a sequence of CT images. Holes are represented by their bounding rectangles. Rectangles with same color in different image slices correspond to the same hole. A crack is modeled as a line, but for the sake of clarity a rectangle is used to mark its locations (Fig. 18). A semitransparent view of the log showing its internal defects is depicted in Fig. 19.

Fig. 16

Results of Kalman filter-based tracking and contour fitting using Binary Snakes contour fitting for detection, localization, and 3-D reconstruction of knots

Fig. 17

Three-dimensional reconstruction of the knots

Fig. 18

Results of Kalman filter-based tracking and contour fitting using Binary Snakes contour fitting for detection, localization, and 3-D reconstruction of cracks and holes

Fig. 19

A semitransparent view of the log showing its internal defects

Table 1 summarizes the defect detection performance of the proposed scheme for over 224 cross-sectional CT image slices of hardwood log data Ash1 for each of the three major internal defect types: knot, hole, and crack. The detection rate, false positive rate, and false negative rate of the proposed scheme are computed by comparing the results of the proposed scheme with those obtained from a human expert grader examining the physically sawn lumber. Also, the performance of the previous scheme [13, 14] that detected and localized defects in each CT image slice independently (i.e., slice by slice without tracking the defects across multiple CT image slices) was compared with the performance of the proposed scheme (Table 1). Although the Kalman filter-based tracking scheme did not result in any improvement in the detection rate or the false positive rate for knots, it did improve the detection rate for cracks from 94 to 98% and the false positive rate from 12 to 2% when compared to the scheme that processed and analyzed each CT image slice independently. Likewise, the proposed Kalman filter-based scheme did not improve the detection rate for holes but did improve the false positive rate from 12 to 3%. Tables 2, 3, and 4 summarize the performance of the proposed Kalman filter-based tracking scheme for hardwood log data Ash2, Maple, and Oak, respectively. In all cases, it can be seen that the proposed Kalman filter-based tracking approach does improve upon the performance of the scheme that processes and analyzes each CT image slice independently in terms of both, defect detection rate and false positive rate.

Table 1 The detection rate for the three major defect types on log data Ash1

Table 2 The detection rate for the three major defect types on log data Ash2

Table 3 The detection rate for the three major defect types on log data Maple

Table 4 The detection rate for three major defect types on log data Oak

Table 5 compares the average processing time per CT image slice of the proposed Kalman filter-based tracking scheme and the scheme that processes and analyzes each CT image slice independently (i.e., slice by slice) on the hardwood log data Ash1. In order to ensure a fair comparison, the detection time for the outer log boundary and each of the three major internal defect types, i.e., knot, crack, and hole are measured and tabulated independently. Note that the 3-D defect reconstruction in the proposed method is done automatically by using the integrated detection-tracking scheme, whereas the slice-by-slice scheme needs extra time to compute the defect correspondences in order to reconstruct the 3-D defect model. The time used to compute the defect correspondences varies with the different number of defects and the methods used to compute the correspondences as tabulated in our previous work [13]. Note that the time taken to compute the defect correspondences is not included for the slice-by-slice method in Table 5, whereas in the case of the integrated detection-tracking scheme, the 3-D model is established automatically.

Table 5 The average processing time per CT image slice in milliseconds

8 Conclusions and future work

In this paper, a novel feature-based tracking approach was proposed and implemented for the detection, localization, and 3-D reconstruction of internal defects in hardwood logs from cross-sectional CT images. In contrast to traditional methods, where the defects are detected and localized independently in individual CT image slices and the 3-D reconstruction of the defects accomplished via correspondence analysis across the various CT image slices, the proposed system integrated defect detection, defect localization, and 3-D defect reconstruction by incorporating a Kalman filter-based feature tracking scheme. The defects were simultaneously detected, classified, localized, and reconstructed in 3-D space making the proposed scheme computationally much more efficient than existing methods. Robust techniques for defect detection and classification were proposed and implemented. Defect class-specific tracking schemes based on the Kalman filter were designed which use the geometric parameters of the defect contours as the tracking variables. The geometric parameters of the defect contours are computed using a combination of B-spline contour approximation and an improvised Binary Snakes contour fitting procedure termed as Kalman Snakes. Robust techniques for extraction and characterization of the external log surface were also designed. Experimental results on cross-sectional CT images of hardwood logs from select species such as white ash, hard maple, and red oak demonstrated that the proposed scheme was a significant improvement over the traditinal schemes that processed and analyzed each CT image slice independently in terms of both detection accuracy and processing speed. The results of defect detection, defect localization, and 3-D defect reconstruction could be used for automatic planning of log sawing and lumber production strategies, and as an interactive graphical tool for the training of novice sawyers. These research directions will be pursued in the near future.