1 Introduction

For the last decades, the Maintenance, Repair and Overhaul (MRO) industry has traditionally been using welding processes such as Gas Tungsten Arc (GTA) welding to perform repairs of metallic parts. However, such welding processes are usually operated manually, which give inconsistent results since they are highly dependent upon the welder’s skills. Moreover, GTA welding repairs usually induce high residual stresses and important deformations due to a relatively large heat-affected zone, and the bonding between the weld and substrate can potentially be quite weak [1,2,3,4]. The range of materials and the types of defect that can be consistently repaired by GTA welding are therefore limited.

To circumvent those issues and extend the range of applicability of the repair process, the welding process may be replaced by an additive manufacturing process such as Laser Metal Deposition (LMD). This process is very well suited for surface repair applications since it allows for a limited heat input and a precise control of the deposition zone while being relatively cost-effective and easily automatable compared to other common welding and coating processes (GTA/EB/PTA welding, Plasma/HVOF thermal spraying) [2].

While there has been a significant amount of work done on repairs with an LMD process, most studies focus on the material and process aspects. For example, Pinkerton [2] focuses on repairing slots with H13 steel and statistically analyzing the material and mechanical performance of the deposit. Pinkerton [5] reviews the theory and applications of LMD, including case studies of industrial repairs. Graf [6] studied the refill of grooves by LMD with stainless steel and a titanium alloy, and Petrat [7] researched process parameters and rebuild strategies for repairing a gas turbine burner with Inconel 718.

There are comparatively fewer studies focusing on automatizing the repair process, using either an additive manufacturing process such as LMD [3, 4, 8, 9] or other manufacturing processes such as welding and/or CNC machining [10, 11]. Improving the automation of the repair process may involve scanning the part for building its 3D digital model to automatically localize the defect region, extract the repair volume, and generate suitable toolpath trajectories.

Most publications on repair automation may be divided into two categories, depending on whether or not the nominal Computer-Aided Design (CAD) model of the part to be repaired is available and used in the repair methodology. For example, Bremer [10], Jones [12], and Zhang [13] recover the repair volume of a defective part by comparing the nominal CAD model to range data.

However, relying on nominal CAD models for performing surface repairs has a few drawbacks. Indeed, the CAD model is not always available due to confidentiality issues, and when the CAD model is in fact available, it usually does not correspond exactly to the real part, notably for aluminum alloy castings [8, 9, 11]. Also, requiring the knowledge of the full geometry of a part to perform a localized surface repair may seem like overkill.

Thus, several studies have been dealing with non-CAD-based automated repair, and they mostly focus on the repair of a specific type of part such as turbine blades. For instance, to generate a defect-free model of a defective blade, Gao [14] uses polygonal modeling and Gao [8] combines polygonal modeling and surface fitting. Chen [15] combines piecewise Hermite interpolation and linear compensation. Gao [9] reconstructs a nominal CAD model based on a surface extension approach and cross-section curves (CSC). Yilmaz [11] performs a sweep between CSCs located before and after a defect. Piya [4] and Wilson [3] use a digitized mesh model and a prominent cross-section approach to extract the repair volume by CSC lofting and a Boolean operation. Tao [16] adapts the CSC lofting approach in order to handle defective blade tips.

Such non-CAD-based methods, reviewed by Wu [17], apply to a specific type of part such as turbine blades, for which the general shape and function are known a priori. However, those methods cannot be directly applied to parts of arbitrary geometry with unknown shape or function a priori. Moreover, a significant amount of user intervention is usually necessary to digitally construct the repair volume, often relying on the use of third-party commercial CAD/CAM software [3, 4, 8, 9, 11, 14] at some stage of the repair process. Also, those methods typically require a full 3D scan of the part, which can be a rather challenging task for large and intricate geometries. They also necessitate a very clean range dataset, which generally involves a pre-processing step that is tedious to perform manually and is hard to automate fully. Indeed, many scanning errors such as spurious peaks and missing data typically occur for concave geometries made of reflective materials like metals, those errors being notably caused by secondary reflections, dead angle, and occlusion effects [18, 19].

The main contribution of this paper lies on a new methodology for performing local surface repairs by LMD on parts of arbitrary shapes without relying on a nominal CAD model and with minimal user intervention during the process. This methodology, dubbed InterSAC (Intersection SAmple Consensus), involves a modified RANSAC (RANdom SAmple Consensus) algorithm run on local range measurement data, followed by an intersection operation carried out either analytically or numerically. This allows to extract the repair volume and find its edges accurately and robustly without any prior knowledge of the part, provided the defect has been machined into a surface cavity of canonical shape. The InterSAC algorithm combines several modifications that tailor the RANSAC approach to a repair application by improving on its accuracy, repeatability, and theoretical runtime. Such an approach is robust enough to remove the need for major pre-processing of the range data by the user, or the need for performing multiple scans and fusing the range data to filter out scanning errors, as in [20]. Once the repair volume has been extracted, scan paths can be generated for the refill of the cavity by an additive manufacturing (AM) process such as LMD [21, 22].

The present paper shall focus on the identification and intersection steps of the method.

This paper is structured as follows. In Section 2, the InterSAC methodology is explained in details and the modified RANSAC algorithm is specified. To illustrate the method, an experimental procedure is described in Section 3 and its results are given in Section 4. Section 5 discusses the findings, and a conclusion and perspectives are given in Section 6.

2 Repair methodology

2.1 Method outline

The present method partially automates the repair of a defect by an AM process such as LMD. It is applied to external or internal defects that have been machined into a surface cavity. This cavity is to be refilled in such a way as to recover the local geometry of the part.

The objectives of the present method are thus to automatically detect and refill the machined cavity without the need to rely on a nominal CAD model and with minimal user intervention during the repair process. This implies scanning the repair area, locating and segmenting the cavity, extracting its edges, calculating its volume, and generating suitable scan paths. These paths, represented as analytical or numerical 3D curves, can then be turned into machine code and fed into the LMD process controller to perform the refill of the cavity. A general approach for repairing metallic parts by an AM process is summarized in Fig. 1.

Fig. 1
figure 1

General repair method

Full size image

The defective metallic part is first scanned by some imaging techniques, such as radiography or CT scans. The scan data is then inspected for defect identification, either manually by an operator or automatically by machine vision algorithms as in the work of Mery [23]. Toolpaths are then generated, manually or automatically, for machining the defect area into a cavity [24,25,26]. In those cases where the machining toolpaths can be easily recycled into scan paths for refilling the cavity, the repair operation can readily be performed once the AM process parameters have been optimally set [27, 28]. However, path strategies for refilling the cavity by an AM process such as LMD are rather specific [21, 22] and usually differ greatly from the machining strategy. Indeed, the process constraints are not the same, and the difference in effector size is substantial. Hence, in most repair applications, a specific scan path strategy has to be used for the AM refill process. To this end, this paper presents a method called “InterSAC” (Intersection SAmple Consensus) for robustly extracting the repair volume based on 3D range data. Once the 3D volume to be refilled is known, scan paths can be generated for performing the repair operation.

For the InterSAC method to be applicable, there are two major assumptions. First, it is assumed that the general shape of the cavity to be refilled is amenable to an analytic representation. In practice, this can be achieved by machining the defect accordingly. Second, it is assumed that the original local geometry of the part in the cavity area corresponds to the shape of the external surface of the part in the neighborhood of the cavity (otherwise called “neighboring surface”). The neighborhood of the cavity is therefore assumed to be sufficiently smooth and regular to be represented as an analytic surface. Particular surface features that were originally present in or near the cavity area but have been machined out may be rebuilt separately in a subsequent step, on top of the refill as performed in the present method.

The InterSAC method starts by scanning the cavity area with a non-contact range measurement sensor, such as a 2D laser triangulation sensor, to obtain a 3D digital representation of the cavity and its neighboring surface. It is not necessary to scan the part in its entirety as only local range data information is required for the present method. For good performance, the range data should include the whole cavity as well as a good portion of the “neighboring surface.” However, the present method is robust enough to be also applied to partial range data where the cavity area and/or the neighboring surface are partially truncated.

The user also selects a canonical, i.e., primitive, surface that best represents the geometry of the machined cavity (sphere, cylinder, cone, paraboloid, ellipsoid, hyperboloid…), as well as a surface that best represents the neighboring surface (sphere, cone, cylinder, paraboloid, ellipsoid, hyperboloid, torus, quadric, superquadric, conic, bivariate polynomial…). If the type of neighboring surface is unknown, the user may choose a type of surface of sufficiently high degree, so that it may correctly approximate the actual neighboring surface.

Regarding the nature or type of surfaces and shapes used to represent the cavity and its neighboring surface, the InterSAC method can in principle handle almost any analytic surface that is amenable to an implicit representation (i.e., of the form F(x, y, z) = 0). Explicit representations of the form z = F(x, y) can also be used since they can trivially be converted into implicit representations. However, the use of parametric surfaces is more challenging for a RANSAC-based identification due to independent parameters and is left to future work.

Once the user has initially set the geometrical models for the cavity and its neighboring surface, the RANSAC-based algorithm then proceeds to estimate the model parameters based on the range data.

Because there are two different geometric models to be identified within the range data, two identification steps are performed in a sequential manner. In the second identification step, the InterSAC algorithm only samples data points corresponding to the outliers of the firstly identified model. The inliers of the first model are therefore discarded from the sample dataset for the second identification step. This serves to reduce the number of data points available for randomized sampling, thereby reducing the theoretical number of random samples to be drawn.

The preferable order for the two shape identification steps depends on several factors outlined in Section 2.2. As a rule of thumb, assuming a dataset composed of 50% of inliers for the first model and 50% of inliers for the second model, it is theoretically best to start with the simpler model (i.e., with less parameters to be identified).

Once the parameters of the two geometrical models are estimated, the 3D repair volume is then obtained by performing an intersection operation between the two geometrical models, which yields their 3D intersection curve and surface. The 3D intersection curve defines the surface edge between the cavity and its neighboring surface. Below the surface edge, the bottom of the 3D repair volume is defined by the geometrical model of the cavity bounded by the 3D intersection curve. Above the surface edge, the top of the repair volume is defined by the intersection surface, which corresponds to the neighboring surface bounded by the 3D intersection curve. Hence, with the knowledge of the parameters of the two geometrical models and their 3D intersection curve, a 3D repair volume can be formed in such a way as to recover the local geometry of the metallic part in the repair area. As noted earlier, it is assumed that the original surface geometry of the part in the cavity area corresponds to the shape of the neighboring surface. If a particular surface feature was in fact originally present in or near the repair area but was integrally or partially machined out to form the cavity, it could be rebuilt by LMD in a subsequent step on top of the 3D repair volume as obtained in the present method.

The repair volume extraction by an intersection operation is illustrated in Fig. 2 for the particular case of a spherical cavity machined on a flat workpiece, respectively represented by a sphere and a plane. Their intersection curve is a circle, their intersection surface is a disk (i.e., flat geometrical objects oriented in 3D space), and the repair volume is a spherical dome.

Fig. 2
figure 2

Repair volume extraction by intersection of a spherical cavity and its neighboring plane—intersection circle in red

Full size image

With geometrical models of high degree, the intersection curve and surface may not be flat (e.g., spherical cavity on a toroidal surface). Also, an analytic solution is not always known, so that a numerical procedure may be required (see discussion in Section 5).

The algorithm finally generates the scan paths and the machine code for the refilling process. If the refilling by LMD is carried out successfully, the local shape of the metallic part is fully recovered.

2.2 InterSAC algorithm

In this section, the identification steps of the method are specified. This approach is based on a modified RANSAC algorithm that combines several previously known modifications (including MSAC, NAPSAC, and LO-RANSAC) that tailor the RANSAC approach to the repair application by improving on its accuracy, repeatability, and theoretical runtime. Given the probabilistic nature of the algorithm, the theoretical runtime here refers to the number of iterations theoretically required for the algorithm to succeed with a given confidence level.

The original RANSAC algorithm [29] aims at robustly estimating model parameters within a dataset containing both model inliers and outliers. It is based on an iterative procedure that randomly samples a subset of the data in each iteration and estimates the corresponding model parameters. The size of the subset is chosen to be as small as possible (e.g., 2 points for a line, 3 points for a plane, 4 for a sphere…). The data points that fall within a certain distance, i.e., tolerance, of the geometrical model are called inliers, and together they form the consensus set. From one iteration to the next, the algorithm retains the model parameters with the largest consensus set or lowest cost. Several stopping criteria exist, but it is usually set to stop after a certain number of iterations or after a sufficiently good model has been found (i.e., with more inliers than a threshold). Because the model parameters of the best consensus set have been estimated with only a sample of the inliers, a final least-square fitting step on all the inliers is usually added, as per the so-called gold standard form of the RANSAC algorithm, so that the entire consensus set participates in the estimation of the final model parameters.

While fairly simple in its principle and quite robust in practice, RANSAC suffers from a few drawbacks. Indeed, the results are not repeatable from one run to the next due to the random sampling process, which is an undesirable effect in the context of an industrial repair process. Also, the original RANSAC method does not necessarily find the optimal set of inliers. And when it does find a near-optimal set of inliers, the algorithm may still continue running as the stopping criterion is usually based on a statistical hypothesis. Moreover, RANSAC assumes that all inliers yield good model parameters. However, inliers are usually affected by noise, so that all subsets of inliers do not give the exact same model parameters. In other words, some subsets of inliers are better than others at estimating the model parameters.

For an industrial repair process, it is essential that the results of the InterSAC method be both accurate and repeatable. To this end, the RANSAC algorithm is modified to yield a RANSAC-like algorithm that is tailored to this application.

In [30], the cost function of RANSAC is modified with no extra computational burden by using an M-estimator.

This MSAC method (M-estimator SAmple Consensus) keeps a constant error value for outliers, as in RANSAC, but it also considers a non-zero error value for the inliers based on how well they fit the estimated model. This MSAC cost function better handles the presence of noise and helps selecting the best model parameters.

Furthermore, to remedy the lack of repeatability of the original RANSAC algorithm, a local optimization step can be used to help find the optimal consensus set and the corresponding optimal model parameters, i.e., the consensus set yielding the lowest cost over the entire dataset. Finding the optimal model parameters and consensus set, which are usually unique, serves to make the algorithm more repeatable and accurate.

Depending on the authors [31,32,33], the local optimization step may involve resampling, rescoring, and pruning the consensus set, either during or at the end of the iteration process.

In the present paper, the local optimization step according to [34] is implemented, where the best consensus set is optimized in a final step, at the end of the main iterative process. Other local optimization methods have since been published [32, 33], but the current local optimization step already gives satisfactory results (see Section 4), and implementing the aforementioned improvements could drastically increase the runtime and complexity of the algorithm for marginally better repeatability and accuracy.

Also, since the RANSAC-based algorithm of the present method is aimed at a purely geometrical problem based on gridded data points, the random sampling process may be slightly improved upon by using an approach similar to that of NAPSAC (N-Adjacent Points SAmple Consensus) [35]. This approach assumes that if a data point is an inlier, then chances are that its close neighbors are inliers as well. Thus, each sample set is formed by randomly picking a first data point and completing the sample set by drawing data points in the neighborhood of that first data point. The choice of the distance between the first data point and a neighboring point is based on a random variable that follows a discrete Gaussian distribution, or any other suitable discrete probability distribution. This semi-random sampling procedure increases the chances of forming a sample set with inlier data points only. Hence, compared to original RANSAC, a lesser number of draws is needed to ensure with a given probability that we will draw a sample set that is solely made of model inliers. The theoretical runtime of the RANSAC-based algorithm, i.e., the number of iterations theoretically required for a given confidence level, is therefore diminished by using a NAPSAC-like sampling instead of a purely random sampling.

The use of a NAPSAC-like sampling is all the more advantageous for the purposes of the present method since no preliminary ordering step is necessary to compute the distance between the first data point and its neighbors. Indeed, the range data is already ordered on a grid and not in a point cloud as in the original NAPSAC. Hence, using a NAPSAC-like sampling process in the InterSAC method when using gridded data represents no significant increase in complexity or computational cost, yet it decreases the theoretical runtime or number of iterations.

Overall, by adding an M-estimator, a local optimization step and a NAPSAC-based sampling, the RANSAC has been tailored to the specific problem at hand by improving on its repeatability and accuracy.

Furthermore, since there are two different geometric models to be identified, i.e., the cavity and its neighboring surface, the identification step is performed twice in a subsequent manner. A choice must therefore be made as to which model shall be identified first. While the particular order has no impact on the accuracy or repeatability of the InterSAC method, its theoretical runtime can be significantly reduced by choosing this order judiciously.

For example, whenever the range data contains more points from the neighboring surface than the cavity, and the size of the sample set of the neighboring surface (3 non-collinear points for a plane) is less than that of the cavity (4 non-coplanar points for a sphere), a random (as in the original RANSAC) or quasi-random (as in NAPSAC) sampling process is more likely to draw a sample set of inliers belonging solely to the neighboring surface than to the cavity. Hence, in that case, the first identification step can theoretically be terminated earlier, i.e., less samples need to be drawn, if we choose to first identify the planar neighboring surface rather than the spherical cavity. The second identification step being based on the entire dataset minus the inliers of the firstly identified model, a lot of outliers of the second model are ruled out. Hence, in that second step, the probability of drawing a sample set of all inliers is now much higher than if it had been performed in the first identification step. It therefore makes sense to keep the most challenging identification step for last. More generally, in the event of a range dataset consisting of 50% of inliers for a first model and 50% for a second model, it is more advantageous to first identify the model of lesser degree or complexity, i.e., with fewer parameters and therefore a smaller associated sample size.

3 Experiments

In order to validate the methodology and algorithm described in Section 2.2, cavities are machined in a workpiece that is subsequently scanned by a 2D high-speed laser profilometer. The resulting range data is then used to test the InterSAC method.

The workpiece is mounted on the table of a vertical machining center, either in a flat or in an inclined position at about 27.3° ± 0.5° with respect to the horizontal plane. Both inclinations are tested in order to assess the robustness of the repair method when range data is perturbed by occlusions, secondary reflections, and geometric deformations during the laser triangulation measurement process [19].

The 2D high-speed laser profilometer is mounted along the Z-axis of a machining center. The laser profilometer performs the scan in continuous mode along the Y-axis at a constant speed of 200 mm/min. 2D range profiles are captured every 20 to 40 ms, i.e., every 0.067 to 0.133 mm, along the Y-axis. On the X-axis, along which the 2D laser profile extends, data points are spaced out by 0.100 mm. The range dataset is then pre-processed and fed into the RANSAC-based algorithm, developed in a MATLAB environment, in order to extract the repair volume and generate suitable refill trajectories.

4 Results

This section presents the range dataset of the laser triangulation measurement process, the repair volume, and edge feature extracted from the range data and the generation of scan path trajectories. The results presented hereafter will be focusing on the specific case of a spherical cavity machined on a flat workpiece inclined at 27.3° ± 0.5° with a 10 ± 0.005 mm radius and a depth of 6 ± 0.010 mm. Similar results can however obtained with spherical cavities of various radii, depth, and inclinations. Tolerances for radius and depth have been deduced from machine and cutting tool tolerance specifications whereas the inclination tolerance comes from an estimate of the measurement tolerance for the angle.

4.1 Laser scanning

Because of the reflectivity of the workpiece and the concave shape of the cavity, spurious peaks or gaps appear near the surface edge of the cavity, as illustrated in Fig. 3. Such adversarial effects in laser range scanning are for example described in [15, 19].

Fig. 3
figure 3

Laser scanning of spherical cavity radius = 10 mm and depth = 6 mm without inclination (a) and with 27.3° inclination (b)

Full size image

To suppress those erroneous peaks from the range data, a median filter could for example be used as it presents good edge preserving and noise removal properties [36]. However, those spurious peaks are not localized enough to be filtered out completely without significantly modifying the geometrical information contained within the range data.

Other techniques for removing errors such as spurious peaks rely on the fusion of range data obtained from multiple scans along various directions and orientations. The position of the spurious peaks and other errors being dependent upon the position of the CCD sensor of the range scanner, various checks can be performed on the range datasets to identify and remove incoherent measurements [20].

4.2 Cavity segmentation

According to the InterSAC method, the spherical cavity and its neighboring planar surface are identified by feeding the range data into a RANSAC-based algorithm described previously in Section 2.2. Considering the average grid spacing of the range data is around 0.1 mm, a tolerance of 0.05 mm is chosen for selecting model inliers in the InterSAC method.

4.2.1 Neighboring surface identification

In this experiment, the surface around the cavity is approximately planar. Hence, the goal is to identify the optimal normalized parameters [a b c d] that define the best fitting plane according to the implicit plane equation:

$$ ax+ by+ cz+d=0 $$
(1)

Because a unique plane passes through any 3 distinct non-collinear points, the sample size for the RANSAC-like procedure on a plane is set to 3. After 50 iterations, this particular run of the algorithm has identified 95,748 plane inliers out of 118,818 range data points, as illustrated in Fig. 4 where the best plane estimate is presented on the right, and an example of a plane estimate during the iteration process is reproduced on the left. The inclination angle is estimated at 27.38°, representing an error of about 0.3% with respect to the experimental inclination angle (27.3°) and is within the estimated tolerance of ± 0.5°.

Fig. 4
figure 4

Identification of planar surface (plane inliers in red) radius = 10 mm and depth = 6 mm with 27.3° inclination iteration plane (a) and final plane (b)

Full size image

Overall, it is observed that most of the plane inliers have been identified, and that the normalized plane parameters [a b c d] = [−0.46 − 0.01 0.89 11.29] yield a plane that correctly fits the neighboring surface of the cavity, despite the presence of spurious peaks in the range data.

4.2.2 Cavity identification

The center (x S, y S, z S) and radius R of a sphere are uniquely defined by the following implicit equation:

$$ {\left(x-{x}_{\mathrm{S}}\right)}^2+{\left(y-{y}_{\mathrm{S}}\right)}^2+{\left(z-{z}_{\mathrm{S}}\right)}^2={R}^2 $$
(2)

Hence, the goal is to find the center (x S, y S, z S) and radius R of the sphere that best fits the spherical cavity. Because a sphere is uniquely defined by 4 non-coplanar points that are not 3-by-3 collinear, the sample size for the RANSAC-like procedure on a sphere is set to 4. After 100 iterations, this particular run of the algorithm has identified 18,862 sphere inliers out of 118,818 range data points, as illustrated in Fig. 5 where the best sphere estimate is presented on the right, and an example of a sphere estimate during the iteration process is reproduced on the left.

Fig. 5
figure 5

Spherical cavity identification (sphere inliers in red) radius = 10 mm and depth = 6 mm with 27.3° inclination iteration sphere (a) and final sphere (b)

Full size image

Once again, it is observed that most of the sphere inliers have been identified, and that the sphere parameters (x S, y S, z S ) = (43.46,  122.11, 15.60)0 and R = 10.006 mm yield a sphere that correctly fits the bottom of the cavity, despite the presence of spurious peaks in the vicinity of the surface edge of the cavity.

4.2.3 Cavity segmentation and edge extraction

Cavity segmentation

Now that the parameters of the sphere and the plane have been estimated, their intersection can be computed analytically to yield the depth, volume, and edges of the cavity to be refilled.

The non-null and non-singular intersection of a plane and a sphere is a circle lying in 3D space, and the corresponding intersection surface is the associated disk. Since the implicit form of the equation of a circle in 3D space is not unique, it is more conveniently expressed in its parametric form.

Once the intersection circle is known (Fig. 6), the geometry of the spherical dome constituting the spherical cavity is entirely defined, and its depth and volume can be computed analytically.

Fig. 6
figure 6

Intersection circle (in yellow) for a spherical cavity radius = 10 mm and depth = 6 mm with 27.3° inclination. Perspective view (a) and top view (b)

Full size image

To test out the accuracy and repeatability of the algorithm, it is run five times using the range data of the spherical cavity of radius 10 mm and depth 6 mm inclined at 27.3°. The same sampling process is used each time, with for instance the same number of samples drawn for the plane (50 iterations) and the sphere (100 iterations).

It is observed in Table 1 that each of the five runs gives a sphere radius within at most 0.066 mm of the theoretical radius (10 ± 0.005 mm), a depth within 0.089 mm of the theoretical depth (6 ± 0.010 mm), and a volume within 24.68 mm3 of the theoretical volume (904.78 mm3). Relative errors are at most 0.66% for the radius, 1.48% for the depth, and 3.30% for the volume. Hence, the method systematically yields a rather accurate estimate of the radius, depth, and volume of the cavity.

Table 1 Results of spherical cavity segmentation for five different runs of the algorithm (theoretical values: radius = 10 mm, depth = 6 mm, volume = 904.78 mm3)
Full size table

To further assess the repeatability of this method, ten test runs are performed to compute the average and standard deviation of the results (Table 2). It is found that the method is quite accurate as, for example, it yields an average estimated radius with a 0.182% relative error on the theoretical radius. Also, the method is quite precise as the spread of the results is rather tight (e.g., 95% of the time, the estimated radius is within 0.05 mm of the average estimate).

Table 2 Averaged results of spherical cavity segmentation for ten different runs of the algorithm (theoretical values: radius = 10 mm, depth = 6 mm, volume = 904.78 mm3)
Full size table

Edge extraction

The edge feature extraction is critical to the proper calculation of the refill trajectory. In the present method, the intersection circle represents the theoretical surface edge of the cavity.

If the edge of the cavity were detected through more conventional means, for example by convolving an edge filter kernel as done in intensity image processing, the resulting edge would likely require user intervention on a case to case basis. Indeed, spurious peaks occur near edges in the range data, which can lead to deformed edge features. Moreover, edge extraction through a filtering process (Scharr, Sobel-Feldman, Canny, Prewitt…) requires the setting of various thresholds as well as performing other related tasks such as edge thinning (non-maxima suppression) and edge closing operations to obtain a clean continuous edge. Thus, extracting edge features with such operators is far from trivial to automate, especially in the presence of spurious peaks in the range data. Hence, in those cases where the cavity and its neighboring surface can both be represented as analytic surfaces, it is preferable to obtain the edge feature by an intersection operation, especially when the intersection curve has a known analytical solution.

Moreover, because the range scanning process can yield rather inaccurate measurements near edges [37, 38], it is more advantageous to calculate the edge feature by leveraging the information contained in the bulk of the shapes (cavity and neighboring surface) rather than relying solely on range data points near edges. Indeed, as illustrated in Fig. 7, the error on the edge location obtained with a 3 × 3 2D Scharr operator is an order of magnitude higher than with an intersection process as presented in this paper. Experimentally, the error on edge location is typically comprised between 0.1 and 0.5 mm throughout the closed edge contour, depending on the inclination and size of the spherical cavity. Indeed, as shown in Fig. 8, the edges are usually smoothened and deformed by the range scanning process, especially at higher inclination angles, so that the range data does not accurately represent the local geometry near sharp edges. For instance, in Fig. 8c, the edge area is quite distorted and is not as rounded as expected from the geometry of the actual cavity. The InterSAC method therefore allows to deduce the actual edge location and geometry more accurately by exploiting the overall geometry of the cavity, and not just the range data points near the edges.

Fig. 7
figure 7

Comparison of edge obtained with intersection circle (in white) and Scharr operator (contour plot) for a spherical cavity without inclination and no mat paint (radius = 10 mm). Top view (a) and close-up view of red square area (b)

Full size image
Fig. 8
figure 8

Edge deformation effect with laser range scanning for spherical cavity of radius 10 mm and depth 6 mm. Photograph of cavity (a). Range data in flat position (b). Range data at 27.3° orientation (c)

Full size image

To further demonstrate the validity of the method as well as its improved edge accuracy compared to edge detection methods, final results for two other cases are presented in Fig. 9 (radius 5 mm, depth 3 mm) and Fig. 10 (radius 10 mm, depth 3 mm). Although the error on edge accuracy presented in Fig. 10 is less pronounced, the overall increase in edge feature accuracy allows in turn an increase in accuracy of the trajectories for the LMD repair process and thus ultimately, a better refill performance.

Fig. 9
figure 9

Segmentation result for spherical cavity of radius 5 mm and depth 3 mm. General view (a). Focus on edge discrepancy due to range scan deformation (b, c)

Full size image
Fig. 10
figure 10

Segmentation result for spherical cavity of radius 10 mm and depth 3 mm. General view (a). Focus on edge discrepancy due to range scan deformation (b, c)

Full size image

5 Discussion

The InterSAC method of the present study has so far been applied to a spherical cavity machined on the surface of a flat workpiece. The corresponding sphere and plane were successfully identified by the RANSAC-based algorithm. Their intersection curve, i.e., a 3D circle, was solved in its parametric form. The sphere that fits the cavity and the 3D circle together define the repair volume in a unique manner. It is a spherical dome of a particular 3D orientation, the volume of which being known analytically.

The approach taken in the present method will be extended to other geometric models of higher degree, provided they can be expressed analytically in an implicit or explicit form. Hence, the method can in principle handle most algebraic surfaces, natural primitives (planes, spheres, cones, cylinders, tori…), quadrics, superquadrics, cubics, bivariate polynomials, and so on. Regarding parametric surfaces (ruled surfaces, Bézier, B-spline, NURBS…), their identification through the use of a RANSAC-based algorithm is not straightforward due to the presence of independent parameters and will require further study.

To apply the method to such implicit or explicit surfaces, the size of the sample set in the RANSAC-based algorithm must be adjusted accordingly. Also, the calculation of model parameters based on the sample set cannot always be done analytically, as in the case of a plane or a sphere. The use of a non-linear minimization algorithm such as the algorithm of Levenberg-Marquardt may thus be necessary to obtain the model parameters, as done in [37].

Regarding the intersection operation, the analytic expression of the 3D intersection curve is not always known, whether it be in implicit, explicit, or parametric form. Thus, an intersection algorithm may be required to numerically obtain the intersection curve. Many intersection techniques have been developed over the years for computing the intersection curve of many types of surfaces including implicit surfaces such as quadrics [39,40,41]. Such techniques may adopt marching methods, decomposition methods, lattice methods, algebraic methods, geometric methods, or else. They may yield an approximate, or sometimes even exact [38], parametrization of the intersection curve, or a discrete set of points that extends along the theoretical intersection curve. Under this purely numerical form, an approximate continuous intersection curve can be obtained by interpolation. If previous approaches fail, the InterSAC method as presented could be slightly modified to perform the second identification step on the entire dataset rather than on the outliers of the firstly identified model. The data points that belong to both models may be considered as “intersection inliers.” If the model tolerance is well chosen, those “intersection inliers” should approximately form a discretized 3D intersection curve, which may then be rendered continuous by interpolation. Once the intersection curve is computed numerically, either in parametric or interpolated form, the repair volume can be constructed as before by combining the intersecting surfaces and their intersection curve.

As the degree of the surfaces increases, calculating the intersection volume, i.e., the repair volume, analytically may no longer be possible, and numerical integration may be required. This may however be a cumbersome process, especially when the integration boundaries, i.e., the intersecting surfaces and intersection curve, are given in implicit form. One approach would be to sample points on the surface boundary of the repair volume and use a 3D convex hull algorithm such as Quickhull [39] to compute its approximate volume. To obtain the boundary points directly, the intersecting implicit surfaces could be converted to an explicit or parametric form. If such a conversion cannot be done, the intersecting implicit surfaces of the repair volume can be polygonized [40,41,42] within a domain defined by the intersection curve. The accuracy of the volume estimate can be adjusted by changing the number of boundary points used in the convex hull algorithm.

Whenever the geometric model of the cavity and/or of its neighboring surface are not known by the user or are hard to identify visually, the user may choose a type of geometric surface that is general and flexible enough to approximate the shape of the cavity and/or of its neighboring surface. For instance, the user may select a bivariate polynomial as a reasonable approximation to the neighboring surface in the immediate vicinity of a spherical cavity. If the model parameters computed by the RANSAC-based algorithm lead to a locally good approximate surface, then a reasonable approximation of the repair volume and surface edge may be obtained. A post-repair machining step may be used to recover the exact geometry of the original part.

Within the range scan data, multiple areas may correspond to the type of cavity or neighboring surface initially selected by the user. Alternately, several cavities may be present in the range data. While the original RANSAC algorithm does not natively handle the detection of multiple instances of a given model, the InterSAC method can be extended to handle such cases. Indeed, by removing identified inliers from the sample pool of the subsequent identification step, multiple instances of a model can be identified in sequence. More identification steps are added in series until enough models have been found or an insufficient amount of sample points is left. The right surfaces to consider are selected at the end of the identification steps, either directly by the user or with the aid of supplementary heuristics (e.g., approximate orientation of the normal vector for planar surfaces).

Once the repair volume has been constructed and the refill volume calculated, scan paths have to be automatically generated for performing the refill of the cavity with an additive manufacturing process such LMD.

6 Conclusion and perspectives

The method presented in this paper robustly locates and segments a cavity of canonical, i.e., primitive, shape based solely on a local laser scan of the cavity area. Thus, it does not require any a priori information regarding the geometry of the part, such as its CAD model.

Despite noise, occlusions, secondary reflections, and edge deformation in the range data due to the reflectivity of the workpiece and the concavity of the repair area, no filtering or other major pre-processing tasks are required. Indeed, thanks to the robustness of the RANSAC-based algorithm, the repair volume can be directly segmented using the raw range data. Moreover, the intersection operation allows to extract the edge feature more accurately by avoiding the reliance on range data near the edges, which are known to usually be deformed or inaccurate with most non-contact range sensors. In addition, this method is also robust to range sensor orientation as the method can handle partially occluded range data. Only one laser scan sweep is therefore needed in practice.

Also, in the present method, user intervention is only required at the beginning of the repair method for heuristically selecting the surfaces to be identified. Beyond that, the repair becomes a hands-off process. Hence, the present method significantly contributes to the improvement of the automatization of the repair process.

Future work will include the extension of the InterSAC approach to surfaces of higher complexity, as well as the development of an automated scan path generation algorithm.