Introduction

Coccidiosis is a protozoal infection of the intestinal tract which especially affects poultry causing significant economic losses. Prevention of this infection is important because most of the damage occurs before clinical signs become evident [1]. Prerequisites are high hygiene standards, but the ultimate prevention requires chemoprophylaxis; therefore, 11 different coccidiostats are authorized in the European Union (EU) to be used as feed additives (Regulation (EC) No. 1831/2003) [2].

EU Member States are obliged to monitor eggs and feed samples for the presence of the authorized coccidiostats because unintended carry-over from feed produced with the highest authorized dose of the coccidiostats into the afterwards produced non-target feed can occur. Maximum residue levels (MRLs) for the presence of coccidiostats in food (Commission Regulation (EU) No. 610/2012) [3] and maximum levels (MLs) in feed (Commission Regulation No. 574/2011) [4] have been set within the EU in order to protect consumers and animals.

It is known that in some countries only a limited amount of samples is analysed because of the high cost of the analysis. For example, in the UK, approximately 250 egg samples are analysed annually for lasalocid from the ten billion eggs consumed. The accuracy of this low amount of samples in representing the quality of eggs and egg products is debatable [5]. Therefore, there is a need to increase the number of monitored samples, and a fast and inexpensive screening method could be beneficial if applied prior to a more expensive and reliable confirmatory method. However, the number of available screening methods is very limited [6] in contrast to the amount of described confirmatory methods [7–12].

Recently, a multiplex flow cytometric immunoassay (FCIA) for the simultaneous detection of six coccidiostats in feed and eggs was published [13] with sensitivities in accordance with the EU maximum (residue) levels (M(R)Ls). In order to apply this multiplex method in routine monitoring, validation is needed. The key requirement for this validation is to prevent false-negative results; therefore, the β-error at the level of interest should be lower than or equal to 5 % [14]. Unfortunately, the Commission Decision 2002/657/EC [14] does not give a practical approach regarding the immunoassay screening methods, but the new European Union Reference Laboratory (EURL) guideline does [15]. The present validation study was performed according to this Commission Decision, taking into account general guidelines on validation of qualitative methods and according to the EURL guidelines [16].

The present study describes the validation of the FCIA for coccidiostats with the levels of interest set at 0.5 M(R)L and M(R)L for all six coccidiostats in eggs and feed to set accurate cut-off levels and determine the rates of false positives. The cut-off level is the response or signal from a screening test which indicates that a sample contains an analyte at or above the level of interest, and different approaches were used to determine them at M(R)L levels. The rate of false positives was calculated for the blank samples and the samples contaminated at 0.25 and 0.5 M(R)L, at a false-negative rate of 5 %, using t statistics. Additionally, the robustness of the method was investigated by varying the amount of organic solvent (acetonitrile or methanol) and the pH during the extraction and assay procedure.

Materials and methods

Materials

The materials used in the FCIA were previously described [13]. Blank eggs were supplied by CER (Marloie, Belgium) and RIKILT, Wageningen UR (Wageningen, the Netherlands), and taken from local stores (Albert Hein, Jumbo, and a local chicken farm, all in Wijchen, the Netherlands). Poultry feed samples were supplied by MasterLab B.V. (Boxmeer, the Netherlands) and RIKILT, Wageningen UR.

Instrumentation

The measurements were performed on a flow cytometer (FlexMAP 3D®) and on a planar imaging detector (MAGPIX®), both obtained from Luminex (Austin, TX, USA).

Methods

Multiplex coccidiostat assay

The preparation of the coccidiostat–protein conjugates and the polyclonal antisera as well as the optimization of the assay protocol and the specificity of the final FCIA was described previously [13].

Robustness

The robustness was evaluated by altering three selected conditions in the extraction and assay procedure, which were the amount of organic solvents (acetonitrile in the extraction and methanol in the assay) and the pH. The influence on the assay results was tested by taking three different probable variations for each condition.

Precision

For the precision of the assay, four different circumstances were tested. The intra-assay (1 day) coefficient of variation (CV%) was calculated by measuring one blank sample ten times and by the measurement of seven different blank and spiked samples. For the determination of the inter-assay CV% (3 days), one blank sample was measured ten times, and 20 different blank and spiked samples were measured, the measurements spread over 3 days.

Method validation

For the single-laboratory validation, 20 confirmed blank chicken eggs and 20 confirmed blank laying hen feed samples were collected [13]. The validation study was conducted following a nested design where three (or four) concentrations were measured for each sample, namely 0 (blank sample), 0.25 M(R)L, 0.5 M(R)L and M(R)L. The measurements were repeated over 3 days. Aliquots of each blank sample were spiked at the selected concentrations with a mixture of five coccidiostats (narasin, lasalocid, monensin, nicarbazin and diclazuril). The high cross-reaction of one of the antibodies for narasin and salinomycin makes it impossible to add both coccidiostats to the same sample. Additional aliquots of each of the blank samples were therefore also spiked at M(R)L and 0.5 M(R)L with salinomycin only.

Results and discussion

The presented FCIA is in a competition assay format where the coccidiostats in the sample compete with the coccidiostats on the bead surface for the rabbit antibody binding sites. The binding of the antibody with the coccidiostat on the bead is detected by an anti-rabbit-phycoerythrin (PE) conjugate. The PE is a fluorescent reporter protein whose concentration is expressed in median fluorescence intensity (MFI). The FCIA gives inversely related responses to the analyte concentrations. The MFI value is corrected for daily inter-plate fluctuations by calculating the percentage of relative binding (%B/B0) from the maximum response (B0) which is obtained from a blank sample.

The FCIA for coccidiostats was validated for egg and feed independently, because the regulatory levels differ for both matrices. Five of the coccidiostats used in this FCIA have no structural resemblance, and therefore, validations had to be carried out for all five. However, narasin and salinomycin are structurally related, and therefore, either one could have been selected as a representative for the validation. However, in this study, both coccidiostats were tested separately to get a good insight in the performance of the assay.

The screening target concentrations were set at the M(R)Ls for narasin, lasalocid, diclazuril, nicarbazin (DNC), monensin and salinomycin, which in egg are 2, 150, 2, 100, 2 and 3 μg/kg and in feed 700, 1,250, 10, 1,250, 1,250 and 700 μg/kg, respectively [13].

Robustness

Three deliberately introduced variations, which reasonably could occur during routine assay performance were tested for their influences on the assay outcomes. The variaton tested were the amounts of organic solvents (acetonitrile in the extraction or methanol in the assay) and the pH. There were mainly minor influences on the assay results compared to the normal test conditions. The monensin assay was an exception because changing the methanol concentration from 10 to 8 or 12 % could lead to a false non-compliant result.

Taking into account the worldwide shortage of acetonitrile, alternative trials were carried out with other organic extraction solvents. Ethyl acetate gave the best results, but the recoveries were nevertheless slightly lower, and acetonitrile remained the preferred solvent.

Precision

The intra- and inter-assay coefficient of variation (CV%) on repeated analyses of one blank sample should ideally be below 5 %. All assays fulfilled this criterion.

The precision (within-day (S r) and between-day repeatability/intermediate precision (S Int)) was also calculated on the %B/B0 by means of analysis of variance (ANOVA) on the blanks and on the samples spiked at the different concentration levels and analysed over 3 days. Table 1 displays the values determined for S r and S Int and for each coccidiostat in both matrices.

Table 1 Precision data calculated on the %B/B0 for the single-laboratory validation of coccidiostats in eggs and laying hen feed
Full size table

In real-life conditions, different samples are measured which increased the CV%. The highest CV%s were measured for feed samples which can be explained by the differences in composition.

Cut-off levels

A screening method has to discriminate suspicious (non-compliant) samples from the total amount of samples. Setting up the cut-off level implies the finding of the appropriate balance between the false-positive (false non-compliant) and false-negative (false-compliant) measurements. According to the Commission Decision [14], the false-negative rate should be less than 5 %. The false-positive rate is not fixed, but should be as low as possible because all positive screened samples have to be confirmed by a more expensive confirmatory method. Unacceptable high numbers of false-positive samples would lead to unacceptable high costs.

In our study, the calculations in the validation were based on the analysis of 20 blank chicken eggs or 20 laying hen feed samples and the same samples spiked at the levels of interest which were set at the regulatory M(R)L values for each coccidiostat. The cut-off levels were determined by three methods, a statistical assessment and two rapid estimations.

Statistical assessment of the cut-off values (method 1)

The calculation of the cut-off values and the assessment of the rates of false positives, based on these cut-off values, were carried out using t statistics and ANOVA calculations (Table 2) according to Eqs. (1) and (2)

Table 2 Cut-off values and rate of false-positive results in eggs and laying hen feed
Full size table
$$ \mathrm{Cut}\hbox{--} \mathrm{off}\;\mathrm{value}=\mathrm{mean}+t\;\mathrm{value}\left(\beta =5\%\right)\times \mathrm{total}\;\mathrm{standard}\;\mathrm{deviation} $$
(1)

where:

  • The mean is the mean of the results of all measurements for one target coccidiostat at the target M(R)L level

  • The t value is derived from a one-sided t test (β = 5 %), given in statistical t tables

  • The total standard deviation derived is calculated using ANOVA

The rate of false positives is determined (Eq. (2)) as being the one-tailed Student's t distribution of the probability that the response of a negative sample is by chance below the cut-off value. In such a case, the negative would be wrongly classified as positive.

$$ t\;\mathrm{value}=\frac{\left({\mathrm{mean}}_{\mathrm{negative}\;\mathrm{samples}}-\mathrm{cut}\hbox{--} \mathrm{off}\;\mathrm{value}\right)}{\mathrm{standard}\;{\mathrm{deviation}}_{\mathrm{negative}\;\mathrm{samples}}} $$
(2)

In this formula, the mean values and the corresponding standard deviations of the response from the negative samples are used along with the cut-off value calculated with Eq (1). The probability corresponding to this t value gives the rate of false-positive results. The rates of false positives of the blank samples were below 1 % (Table 2).

Thus, the method is absolutely fit for purpose when differentiating blank samples from samples with levels at or above the M(R)Ls. The only exception was diclazuril in feed. There were two causes that could be addressed. Firstly, the ML for diclazuril in feed (10 μg/kg) is much lower than the MLs of the other coccidiostats measured in this assay (ranging from 700 to 1,250 μg/kg). In the second place, there was a bigger standard deviation between the different samples in feed than in egg.

In a further step to evaluate the assay performance, the blank samples were also spiked at 0.25 MRL and 0.5 MRL (egg) and 0.5 ML (feed) for each coccidiostat.

In the egg samples, it is surely demonstrated that the method is highly specific (the number of samples wrongly classified as positive is non-significant). The best results were obtained for narasin and salinomycin, because at 0.5 MRL, the rate of false positives was still below 5 %. Second best results were obtained for DNC, since above 0.25 MRL but below 0.5 MRL, the rate of false positive was below 5 %. Less favourable results were obtained for lasalocid, diclazuril and monensin, since the concentration at which the rate of false positive was still below 5 % cannot be deducted from the experiments. This unknown concentration lies between 0 and 0.25 MRL.

In feed, similar analyses can be made for false-positive results of samples containing a coccidiostat above 0 but below the MRL. For all compounds, the concentration at which the false-positive rate is still below 5 % cannot be deducted from the experiments. It lies between 0 and 0.5 MRL. It can therefore not be stated that the likelihood of wrongly classifying samples with a coccidiostat even up to 0.5 MRL is below 5 %.

In summary, the usefulness of the FCIA for official control depends on the situation on the market and on the cost of the screening and confirmatory method. If most samples found in real practice are real blanks with only a small portion of samples exceeding the limit, the current multiplex screening test is worth to be applied and will deliver reliable results.

Rapid estimation of the cut-off values—routine application

In practice, for end users and helpful during the extension of a multiplex immunoassay, there is a need for getting a rapid and reliable estimation of the cut-off values. In this frame, we compared the values determined when using the two calculation methods described in annex 1 and 2 of the EURL guidelines for validation of screening methods [16]. Furthermore, the 0.5 M(R)L level for each coccidiostat was also examined because of the low number (n = 20) of analysed samples and to comply with the approach described in the guidelines.

Method 2 (annex 1 [16])

The first rapid approach to determine the cut-off level was to examine the variation of responses in the 20 blank samples and in those spiked. The cut-off levels could be set at the highest value (%B/B0) of the spiked samples (at M(R)Ls) if there was no overlap with the lowest value of the blank (Table 3). These cut-off levels could be set for all assays, except for diclazuril in feed. Consequently, it was concluded that the diclazuril assay did not fulfil the validation criteria in feed.

Table 3 Overview of calculated cut-off, lowest blanks and threshold (T) values (%B/B0)
Full size table

Method 3 (annex 2 [16])

The second rapid method used a statistical approach, which took into account the β-error of 5 %. The threshold values (T) and the cut-off values (FM) were calculated for each coccidiostat using Eqs. (3) and (4).

$$ \mathrm{Cut}\hbox{--} \mathrm{off}\ \mathrm{value}=\mathrm{mean}\left(\mathrm{spiked}\;\mathrm{samples}\right)+1.64\times \mathrm{standard}\;\mathrm{deviation}\left(\mathrm{spiked}\;\mathrm{samples}\right) $$
(3)
$$ \mathrm{Threshold}\;\mathrm{value}=\mathrm{mean}\left(\mathrm{blanks}\right)-1.64\times \mathrm{standard}\;\mathrm{deviation}\left(\mathrm{blanks}\right) $$
(4)

For all assays, the cut-off values were below the thresholds (Table 3), except for diclazuril in feed; therefore, the false-negative (false-compliant) rate is at or below 5 % according to the EURL guidelines [16]. For salinomycin and monensin in feed, one false-compliant result was obtained for each, which is still in accordance with the EURL guidelines [16]. Diclazuril in feed again did not pass any of the validation criteria.

Comparison of the cut-off levels

The different calculated cut-off values were compared after measurements of 20 blank and spiked samples at M(R)L (Fig. 1). The three calculated cut-off values (M(R)L) were almost identical except for salinomycin and monensin in feed. Method 2 was better for the salinomycin and monensin assay, because no false-compliant result was observed (Fig. 1). It could be concluded that a rapid estimation was as reliable as the determination using t statistics and ANOVA calculations in method 1.

Fig. 1
figure 1

Comparison of the different calculated cut-off values and the measurements of the blank and samples spiked at M(R)L

Full size image

Special attention should be paid to narasin and salinomycin because both are detected in the same assay. According to the results (Fig. 1), the salinomycin assay gave the highest cut-off value and was selected, although this would increase the false non-compliant results if only narasin is present.

Method 2 was chosen as the fastest and easiest way to set the cut-off levels (Table 3). These levels are fully comparable to the values calculated with the other two methods. Such a method is useful when the MRL is changing, as happened recently for nicarbazin in egg [3], and very beneficial when other tests are added to this multiplex assay.

A ring trail will be performed to verify these preliminary cut-off levels. Another option to obtain real cut-off levels could be to monitor cut-off levels at 0.5 M(R)L for 1 year.

Comparison between instruments—transferability

During the validation, the measurements were carried out on the FlexMAP 3D®, a flow cytometer-based analyser, and the MAGPIX®, a smaller, much less expensive, easy to handle and transportable imaging-based analyser. Subsequently, the results of both analysers were compared (Fig. 2). Very good correlations (r ranging from 0.994 to 0.9994) were found between the two different analysers, implying that both are suited for the same screening purpose.

Fig. 2
figure 2

Comparison of the responses (MFIs) obtained with the FlexMAP 3D® and MAGPIX® analysers within 1 day during the validation of the multiplex FCIA in egg

Full size image

Conclusions

The single-laboratory validation of the FCIA was successful for the six coccidiostats in eggs and for five coccidiostats in feed in which the diclazuril assay could not fulfil the validation criteria. The cut-off levels obtained with method 2 for narasin/salinomycin, lasalocid, diclazuril, nicarbazin (DNC) and monensin in eggs were 60, 32, 76, 80 and 84 (%B/B0), respectively. In feed, these levels were 70, 64, 72 and 78 (%B/B0) for narasin/salinomycin, lasalocid, nicarbazin (DNC) and monensin, respectively. These levels are almost identical with the statistical (reference) approach and the other rapid approach (3).

The data show that the easiest method (2), looking at the extremes of the blanks and spikes, is as reliable as the other two more complicated calculation methods. This is a real advantage and time saving for end users and helpful during the extension of a multiplex immunoassay.

Furthermore, a very good correlation (r ranging from 0.994 to 0.9994) was observed between the two different analysers, FlexMAP 3D® and MAGPIX®, demonstrating adequate transferability.