1 Introduction

The application of structural health monitoring (SHM) methods for real-world use cases is not possible without a method of quantifying quality and reliability. SHM systems need to be evaluated based on a performance index. This index can be defined in different ways, based on physical quantities. Especially important are performance parameters, describing detection rates for exemplary damages and accuracies for processes evaluating the location of the damages.

Based on the methods of quantifying the quality of non-destructive testing (NDT) methods, this paper used the concept of probability of detection (POD). The POD describes the probability to find a damage of a given size with a given confidence level. The evaluation of the performance index a 90|95, which is the damage size which can be detected with a probability of 90 % at a confidence of 95 %, is based on statistical methods. Traditionally, the underlying data are generated with a number of experiments, conducted with several probes and multiple NDT engineers. The procedure is described in detail, e.g. in [1]. The effort for this procedure is high. Nevertheless, the effort is even higher for SHM systems. Their inherent feature of having the measurement system directly connected to the structure leads to much higher expenditures regarding the measurement equipment. The number of probes has to be equal to the number of necessary SHM systems. Moreover, the SHM system, fixed to the structure, exhibits the dependency of the defect location regarding the quality of defect detection, especially for anisotropic materials and real-world load-carrying structures.

Because of these reasons, it is necessary to find alternative solutions to evaluate the performance. One opportunity is based on artificial realizations of the system response at its structure with necessary defects. This is known as model-assisted probability of detection (MAPOD). Examples of previous works on this topic for conventional NDT can be found in [2–5]. In this proposal, we show two possible approaches, which are both based on numerical simulations. The investigated method of acousto-ultrasonics uses a network of piezoelectric transducers to introduce lamb waves. They interact with the structure and a possible defect before being captured by another piezoelectric transducer. The effect of damage is visible for a path between actuator and sensor within the network of transducers. The two methods, shown, permit a MAPOD based on a single path. While one method is using a 3D model, the other method is based on a 2D model, representing the cross section of the structure for a given path. The aim of this paper is to show the application of these two approaches for a SARISTU use case, namely delamination caused by barely visible impacts, and compare the physical results as well as the resulting MAPOD-based performance indices. In a first step, the two models are explained shortly, and afterwards, the underlying statistical procedures to calculate the POD are given. The following description of MAPOD for the SARISTU use case shows the practical applicability including a detailed explanation of the results. The final conclusion also implies possible further steps to go beyond path-based MAPOD towards structure-based MAPOD.

2 Description of the Used Numerical Models

To describe the travelling of the lamb waves and their interaction with damage, two different approaches have been used. While the first method describes the travelling wave in all dimensions [spectral finite element (FE) method], the second approach (CIVA model) is limited to a cross section of the structure on the path between the actuating and the sensing transducer. To model the wave, this approach uses semi-analytical methods, while the first approach is based on numerical spectral FE methods in time domain. To describe the actuating and sensing of lamb waves with the help of piezoelectric transducers (PWAS—piezoelectric wafer active sensors), different methods exist. Most common is the use of a (varied) pin-force model to be able to describe the phenomena without using a detailed description of multiphysics within the piezoelectric element. A varied version has been used for both numerical models within this paper. The two models, used, are explained shortly in the following subsections.

2.1 Spectral Finite Element Method

With the spectral FE method, the wave field within a platelike structure can be calculated in three dimensions. The structure is represented by FEs, and the PWAS, including their structural effects, are modelled, too. To overcome the disadvantages of the FE method for platelike structures, such as a necessity of a very high number of elements and therefore degrees of freedom, this method uses first-order plate theory and FE nodes, distributed according to the Gauss–Lobatto–Legendre polynomials. The general approach of using spectral FEs in time domain is described in [6]. For the calculation of anisotropic layered structures, element matrices for stiffness, mass, damping, etc., are calculated, which describe the properties over the whole plate thickness. A big advantage of this method is based on the bandlike structure of the used matrices, which results in a noticeable reduction of computational costs. The used spectral element is explained in detail in [7]. Its functionality as well as different applications can be also found in [8].

The delamination is implemented via a separation between two layers. The separation is realized with two different elements over the thickness. For both, separate element matrices are calculated. The nodes within the delamination area are not connected. A material penetration is checked and prevented within the calculation via adding additional contact forces in case of penetration. A detailed explanation of the used method to model a delamination is given in [9, 10].

2.2 CIVA Model

To provide efficient tools for predicting NDT guided waves, simulation tools are being developed at CEA LIST; they are based on semi-analytical (SAFE) [11] or hybrid semi-analytical/numerical techniques [12]. Both approaches lie on a modal decomposition and post-processing of these modes to account for transducer diffraction effects and flaw scattering. SAFE involves a FE computation in the guide section, allowing the computation of both wave numbers and modal displacements in the section as being the eigenvalues and eigenvectors (respectively) of a quadratic system of equations; this system is the discrete form of a variational problem in the guide section. As it is restricted to the section, it is computationally very efficient. The propagation is otherwise accounted for by means of analytic propagators in the guiding direction normal to the section considered. The computation of modal amplitudes emitted by a transducer is performed under the assumption that piezotransducers can be modelled as sources of normal or tangential stresses over their active surface. The module available in CIVA allows one to calculate the modes, the beam field and the interaction either with a crack oriented perpendicularly to the direction of guide (available in the first release: CIVA 10 GWT), or with a CAD-defined defect (available in the second release: CIVA 11 GWT). In the former case, the computation is based on a mode-matching technique that does not require the meshing of the zone surrounding the defect. To extend the capabilities of the first modules, hybrid techniques have been developed and integrated in the second release (CIVA 11 GWT). They can take into account the scattering by several defects or discontinuities such as a stiffener arbitrarily positioned in a part. For this approach, non-local phenomena (guided propagation) are modelled by modal decomposition in homogeneous portions of the piece using the SAFE method, while local phenomena are handled using FE modelling. The problem of interaction with any perturbation of the waveguide is written in the form of a scattering matrix. This matrix links an input vector constituted by the modal coefficients of the incoming wave, to an output vector constituted by those of the outgoing wave. To deal with arbitrary flaw shapes, guide inhomogeneities or junctions between several guides, a FE scheme has been developed with the goal to limit the computation zone to a minimal size for efficiency. The computation relies on the use of artificial boundary conditions with transparency. Radiation conditions at infinity are brought back to the artificial boundaries by building an operator coupling the FEs inside the FE zone to modal solutions in guides. An original mixed formulation has been derived with the unknowns being the displacement field in the bounded domain and the normal component of the normal stresses on the artificial boundaries. The scattered field is then projected on modal solutions in guides through the use of biorthogonality relations. This method that we call “Hybrid Modal-FE method” has been established for 2D and 3D waveguides in Cartesian coordinates. The theoretical details of this method are provided in [12]. In the official version of CIVA (11 GWT), only the 2D (dealing with the scattering of lamb wave in isotropic guides) and axisymmetric version are implemented. In composite parts, specific attention must be paid to deal with a multilayered structure with anisotropic properties of plates considered here. Contrary to GW propagation in isotropic materials, the behaviour of GW in anisotropic materials depends on the direction of propagation. Standard formulae of the SAFE method to account for the θ dependency can be found in the literature [13] and have been implemented in CIVA.

3 Calculating the Probability of Detection

Different approaches for calculating POD curves and resulting performance indices are known, which can be distinguished in two main groups. The approaches within this paper represent one each. The first approach is based on the evaluation of a linear behaviour between a quantity representing the damage size and some quantity representing the measured output of the SHM system. The analysis is performed according to the military handbook MIL1823 [1]. Other methods especially regarding the calculation of confidence intervals exist (see, e.g. [14–17]).

3.1 POD Based on Linear Regression Curve

Within this application, the quantity, which represents the damage size, is the area of the delamination. In the following, this is called a. To describe the measured output of the SHM system, a damage index was used. This damage index is calculated by subtracting the correlation coefficient CC from one. CC is calculated as correlation coefficient of the baseline signal from the undamaged structure and the signal from the damaged structure.

The use of other damage indices is possible, and a detailed description can be found in [18]. The damage index is called â from now on to be consistent with the MIL1823. Given that â over a on logarithmic scales results in a linear dependency. For fixed values of a, values of â show some deviation around the regression line (see Fig. 1a). The SHM system identifies a delamination if a measured â is above a threshold â thr. The value of â thr can be calculated from the noise floor of â, when there is no damage. As for the case of numerical calculations, the noise floor has to be added in a separate step and needs to be given, and â thr was evaluated from transducer paths distant to the damage area for the case of very small introduced delamination. When the regression line and the distribution of the scatter around the regression line are described mathematically with slope, intercept and scattering, these values can be used for calculating the POD curve. It describes the POD of a damage size a to be detected with the SHM method that is providing the value â. From the POD curve, the a 90|95 value can be extracted as performance index. It gives the damage size a at which 90 % of the damages of this size can be detected with a confidence level of 95 %. The procedure is presented in Fig. 1b. The calculation of the confidence bounds is based on the delta method. For the calculations of the POD curve, the software R with the package mh1823 POD is used [19].

Fig. 1
figure 1

POD based on linear regression curve. a Linear regression between a and â. b POD curve generated from linear regression curve

Full size image

4 Application of MAPOD for CFRP Structures

Within the SARISTU project, a platelike exemplary structure was used to show the possibilities of numerical approaches of path-based MAPOD. This structure exhibits 11 layers of carbon fibre-reinforced plastics (CFRP) made from M21 matrix with T800S fibres. The set-up is shown in Fig. 2.

Fig. 2
figure 2

Set-up of the SARISTU platelike structure to verify MAPOD concepts. The figure shows the geometry with its dimensions and locations of the transducers in (a) and the layup with orientation of the different layers in (b)

Full size image

To verify the models, used for MAPOD, the results can be compared to an experimental set-up, where a damage between PWAS 3 and PWAS 7 was introduced. The delamination can be considered as ellipse shaped with 19 and 28 mm as half-axes.

4.1 MAPOD Based on Linear Regression Curve and Spectral Element Method

To be able to use the SEM model for calculation of MAPOD, the similarity between experimental results and numerical results has to be secured. Figure 3 shows that for PWAS 6 as actuator, experimental and numerical time domain signals result in similar damage indices. As the damage is located central between PWAS 3 and PWAS 7, only the path 6–4 is directly affected by the damage, while for all other paths, reflections might have an effect on the receiving signal. While the damage index for numerical and experimental data on the direct path (6–4) is very much the same, the experimental data of all other paths result in slightly higher damage indices, compared to the damage indices calculated from the numerical data.

Fig. 3
figure 3

Comparison of damage indices for 3D SEM simulation and experiment. While for paths, which are not directly crossing the delamination, the experimental data result in slightly higher damage indices, for the direct path, simulation and experimental results show an excellent agreement

Full size image

To generate data for a path-based MAPOD, the path between PWAS 3 and PWAS 7 was selected. Between these two transducers, five locations are distributed evenly (x = 150, 200, 250, 300, 350). For all five locations, damages of eight different extents have been chosen. The ellipse-shaped delaminations exhibit all but one a ratio of 2:3 for their half-axes. The path-based damage index was calculated. As damage size, the area of delamination was used. On a double logarithmic scale, a linear regression between damage size and damage index is shown in (Fig. 4). As expected, the damage index is not the same regardless of the damage location. Nevertheless, it is not always a special location, which exhibits the smallest or highest damage index for all damage sizes. Therefore, the location is taken as a disturbance parameter, influencing the scattering of the data.

Fig. 4
figure 4

Linear relation between damage index and damage size. For eight different damage sizes at five different locations, distributed evenly between the transducers, the damage index has been evaluated. With a double logarithmic scale, linear behaviour can be seen. An effect of the location is noticeable, but not independent of damage size

Full size image

Based on these 40 simulations, a calculation of MAPOD was carried out with mh1823 POD software [19]. As no threshold from experimental data was given, â thr was calculated from 15 measurements, using the paths 3–1, 3–2, and 3–4 for all five damage locations and the smallest damage. This results in â thr of 0.0006. Based on this value, the regression line and the scattering, an a 90|95 value of approximately 200 mm2 is defined in the POD curve (see Fig. 5). This value is very low and substantiates the delamination detection possibilities with the used method of acousto-ultrasonics. It gives an estimation for damages, which are located on the direct path between two transducers.

Fig. 5
figure 5

MAPOD with performance index a 90|95 for the path between PWAS 3 and PWAS 7. Based on the numerical results, the calculated MAPOD gives an a 90|95 value of approx. 200 mm2

Full size image

The described calculations show a general approach on a conceptual level with an exemplary use case. For further improvement, a more detailed analysis of the threshold â thr based on the noise level for the used measurement equipment needs to be performed. A shift of â thr to higher values leads to a shift of the POD curve as well as of the a 90|95 value to the right, respectively, higher damage sizes.

4.2 MAPOD Based on Linear Regression Curve and 2D Hybrid Modal-Finite Element Method

A calculation of MAPOD was carried out with CIVA software based on the 2D Hybrid Modal-Finite Element method. Damages of eight different extents have also been chosen that would give the same area of the ellipse-shaped delaminations define in the previous MAPOD calculations. The delamination is located at the interface between the second and third layer. The uncertain parameters used in the MAPOD calculation are the defect location between the transducers and the density of the material. The locations of the delamination defect between PWAS 3 and PWAS 7 are described by a uniform distribution for x in the interval [150, 350], identical to the procedure in A. The density values are defined by a normal distribution with a standard deviation of 0.1 around a mean value of 1.58 g cm−3. For each defect length, 5 draws of uncertain parameters taken in their corresponding distribution give a total of 40 simulations performed at 200 kHz for a bandwidth of 40 %. The computation time for one simulation over the 72 frequency steps is about 6 min on an Intel® Xeon® CPU X5690@3.5GHZ 2 cores RAM 24 Go. The path-based damage index was calculated as the signal sum of squared differences (SSSD) algorithm given in [18] that focus on the whole signal and the differences between healthy and (potentially) damage state. On a double logarithmic scale, a linear regression between damage size and damage index is shown in (Fig. 6). As expected, the damage index varies regardless of the damage location and increases with the damage length.

Fig. 6
figure 6

Linear relation between SSSD damage index and damage length from the 2D CIVA model. For eight different damage sizes at uniformly distributed location between the two transducers, and a normal distribution of the density of the material, the damage index has been evaluated. With a double logarithmic scale, linear behaviour can be seen

Full size image

In the previous SEM-MAPOD calculation, the damage index threshold is calculated from the measurements on indirect paths where the differences between the safe and damage samples arise from the wave reflections on the defect. In the 2D hybrid modal-finite element method modelling used in CIVA modelling, such reflections cannot be simulated because only direct path are taken into account in this 2D description. So, we define the threshold as the minimum value of the linear regression line that finally corresponds to the threshold defined in the SEM-MAPOD calculation (even if the value cannot be compared because of the different model and damage index algorithm). Based on this value, the regression line and the scattering, an a 90|95 value of approximately 15.5 mm is defined in the POD curve (see Fig. 7). Assuming that the damage length is an axis of an ellipse-shaped delamination to compare to the SEM-MAPOD result, one would find a damage size of a 90/95 ~ 500 mm2. One advantage of the 2D CIVA-MAPOD calculations is the low computation time allowing to increase the number and the width of the uncertain parameter distributions.

Fig. 7
figure 7

MAPOD with performance index a 90|95 for the path between PWAS 3 and PWAS 7. Based on the numerical results from CIVA model, the calculated MAPOD gives an a 90|95 value of approx. 15.5 mm

Full size image

5 Conclusion

Model-assisted POD is a promising method to enable the estimation of detection rates for given damage sizes and types. Within this paper, two methodologies for path-based MAPOD are presented. Both are based on reliable modelling tools using numerical simulations. With two different basic approaches, the elaboration of POD curves for respective structures and application scenarios is described in detail. While one approach makes a detailed analysis of the generated wave field possible, but takes more time for computation, the other approach achieves lower computational costs via simplification within two dimensions. The presented methodologies can be used to assist in the conception of SHM systems as well as in following certification, necessary for acceptance tests. Therefore, they build one step on the way to use SHM systems in industrial applications.

For a detailed description of the performance of on SHM system, additional methods, which do not only include a path-based MAPOD, but give a structure-based MAPOD, build the next step. Therefore, it is necessary to develop enhanced statistical methods, including a statistical description of the location parameter. The path-based MAPOD builds an excellent fundament for the required future work.