Introduction

With recent technological developments, digital PCR (dPCR) is becoming widely available and has the potential to become the reference measurement procedure for absolute quantification of nucleic acids in molecular biology. Digital PCR enables direct quantification without the need for a standard dilution series and, in theory, offers an increased accuracy compared to quantitative PCR [1, 2]. However, the first generation of digital PCR platforms still needs careful validation before being used broadly in clinical diagnostics.

To date, the droplet digital PCR system (ddPCR, Bio-Rad) is the most widely used dPCR platform. By using microfluidics, it can generate up to 20,000 droplets from an initial 20-μL PCR mix with each droplet representing an isolated endpoint PCR reaction. Subsequently, fluorescently labeled probes enable a simple readout of the endpoint fluorescence in each droplet and allow the classification of droplets as positive or negative [3, 4]. As a result, absolute quantification can be performed by assuming a Poisson distribution, and the concentration of nucleic acid templates in the total reaction can be calculated based on the frequency of positive to negative droplets, the total number of droplets, and the droplet volumes [4, 5].

ddPCR data analysis

Most dPCR platforms use fluorescent probes or intercalating dyes which provide a fluorescent signal for the droplets that contain amplified PCR product. Accordingly, positive droplets will have a higher fluorescence amplitude compared to the background fluorescence of the negative droplets. Despite the simplicity of this principle, there is no well-defined indication for the exact position of the threshold value to distinguish positive from negative droplets. This particularly holds true for assays that produce droplets where no clear differentiation is observed between negative and positive droplets, as is often the case in a heterogeneous pool of DNA templates, i.e., HIV and/or other microorganisms.

Another important aspect to consider when positioning the threshold is the droplets that fall between the negative and positive population, often referred to as “rain.” The origin of this phenomenon still remains a topic of discussion (Fig. 1a, b). This rain may arise from coagulation of multiple droplets that result in a higher background fluorescence. Another reason can be a suboptimal PCR amplification due to variation in PCR efficiency between droplets caused by a non-uniform distribution of the PCR mix components or by small differences in sequence composition between individual DNA target sequences. The latter condition will certainly hold true for absolute quantification of viral or bacterial genomes which may be characterized by a high frequency of sequence variation (e.g., HIV). Hence, both coagulating droplets as well as target sequence variation may play a role in the formation of rain.

Fig. 1
figure 1

Examples of QuantaSofts’ ddPCR fluorescent readout. A Positive sample showing positive droplet population, negative droplet population, and positive rain (droplets between the arrows). B Negative template control showing negative rain (droplets between the arrows). C QuantaSofts’ automated threshold determination: The threshold (straight line) falls in the negative droplet population resulting in false-positive droplets (arrows)

Full size image

A final consideration regarding the current generation of the ddPCR platform arises from false-positive droplets that sporadically occur. These false-positive droplets are sometimes indistinguishable from the true-positive droplets and have been observed in several studies [3, 6–8].

ddPCR data analysis tools

The ddPCR setup invokes a different methodology of analyzing data compared to existing quantitative real-time PCR (qPCR). Currently, only a few methods have been described for threshold setting in ddPCR reactions [7, 9]. According to the instructions provided by the manufacturer of the ddPCR QX100 platform (Bio-Rad), it is possible to set a manual threshold, allowing the investigator to make a visual interpretation of the background, rain, and false-positive droplets, but this may also introduce investigator specific bias. In addition, automated systems for threshold settings are also available in the QuantaSoft software that comes with the ddPCR platform. However, this threshold setting is not always accurate as the threshold often falls within the population of negative droplets or is not possible to calculate (Fig. 1c). Moreover, the exact method of threshold calculation is not disclosed by the manufacturer, making interpretation impossible.

Apart from the QuantaSoft software, first attempts to enable a more robust automated threshold setting were made by Strain et al. and more recently by Jones et al. [7, 9]. These methods use clustering algorithms to identify the positive and negative droplet populations and do not use one threshold but define the 95/99 % confidence intervals for the negative and positive population. Droplets that fall within the intervals are allocated to the respective population, and rain is discarded and excluded from further calculations.

It must be noted that the exclusion of rain by these clustering approaches may cause an underestimation of the true concentration if this rain is resulting from suboptimal PCR reactions of target template molecules. This could be the case if the PCR reaction is partly inhibited or when minor mismatches occur between the primer/probe sequences and the target molecules. To investigate this hypothesis, a series of ddPCR reactions were performed with HIV DNA amplicons containing a mismatch in the probe-binding region (see Electronic Supplementary Material (ESM): ESM 1 and ESM 2). This resulted in intermediate fluorescence when a mismatch was present, proving that rain may contain true-positive droplets and is not always an artifact. Accordingly, Dreo et al. recently showed that the clustering method can result in discarding true-positive droplets when testing low-target concentrations of Erwinia amylovora in plant material [10]. Therefore, a single threshold may still be preferred since the rain arising from coalescence can be assessed by running multiple (>3) negative template controls (NTC) containing template-free genomic DNA. This was also proposed by Dreo et al., and in this context, a new method was described where a global manual threshold (MTg) is defined as the averaged fluorescence signal in the NTCs plus six times the standard deviation [10].

A final consideration concerning available data analysis methods is that both clustering and MTg methods assume a normal distribution of the fluorescent amplitude of the droplet fluorescence and do not correct for possible shifts in baseline fluorescence between NTCs [7, 9]. This may have a major influence on the correct allocation of droplets in case the distribution of fluorescent amplitudes does not follow a normal distribution.

New ddpcr data analysis tool: ddpcRquant

Here, a new method of threshold setting is proposed for the quantification of DNA templates by ddPCR. The method is based on extreme value theory and resolves the described problems. The method does not make assumptions about the distribution of the droplet fluorescence. Consequently, a deviation from normality of the negative droplet fluorescent amplitudes will not influence the correctness of the method. Moreover, it results in a single threshold and does not discard any of the droplets.

In addition, small shifts in baseline fluorescence of the negative droplet population can be often observed (ESM 3). QuantaSoft and all other algorithms do not account for these small shifts. This can possibly lead to an inaccurate quantification since a threshold value is calculated based on the NTC and transferred to all samples. Consequently, discrepancies in baseline fluorescence can have an influence on the number of positive droplets (ESM 3). Therefore, a baseline correction to account for these small shifts has to be included.

Material and methods

ddPCR datasets

A set of 26 ddPCR runs was used as raw data for all experiments performed in this article. These runs mainly included HIV-related absolute quantification assays on patient samples, which were obtained in previous studies. These are assays to quantify HIV 2-long terminal repeat circles [11], total HIV DNA [12, 13], HIV unspliced cell-associated RNA [14], HIV multiply spliced cell-associated RNA [15], and the woodchuck hepatitis post-transcriptional regulatory element (WPRE) from lentiviral vectors [16]. PCR cycling was performed with the QX100™ system (Bio-Rad, Pleasanton, CA), as previously described [6, 17]. QuantaSoft absolute quantification by single and multiple well data analysis methods was used.

Evaluation of the distribution of the negative droplet population

Normality of the negative droplets of 154 NTCs from 26 assays was assessed statistically by the Anderson-Darling test using the R package nortest (version 1.0.-3) and visually by constructing QQ plots.

Development of the ddpcRquant method for automated ddpcr data analysis

The ddpcRquant method provides an automated analysis of one-dimensional/single-color ddPCR data by calculating a single threshold value based on the negative template controls for each assay in a ddPCR run. The script/package is written in the R language [18] and is made available at http://www.ddpcrquant.ugent.be.

Briefly, the ddpcRquant workflow is designed to use the combined data of multiple replicate NTCs of an assay to model the extreme values by extreme value theory (see results and ESM 4 for a detailed description). Subsequently, the threshold is set as a predefined percentile of the fitted extreme value distribution. Next, this threshold is used to classify the negative and total droplets for each sample. Finally, absolute quantification of DNA template in the ddPCR mix is performed assuming a Poisson distribution, resulting in following formula:

$$ C=- \ln \left(\frac{N_{\mathrm{neg}}}{N}\right)\times \frac{1000}{V_{\mathrm{d}}}\times D $$

with C is the concentration (copies/μL stock), N neg is the negative droplets, N is the total droplets, V d is the 0.91 μL, and D is the dilution factor (volume mix (μL)/volume sample (μL)), as described earlier [1, 3].

Results

Droplet populations of ddPCR are not normally distributed

QQ plots suggest a substantial deviation from normality in most NTCs. Although to be interpreted with care due to the large sample size, p values from the Anderson-Darling test confirmed that the fluorescence readouts from a large number of NTCs did not likely follow the normal distribution: 128 p values were smaller than 10−6, five were larger than 0.001, and only two p values were larger than 0.05. Falsely assuming normal distribution can lead to a threshold too close to the negative droplet population and inaccurate quantification results (Fig. 1c).

Design of ddpcRquant for automated ddPCR data analysis

With the knowledge gained from the previous experiments, a new automated method for one-dimensional ddPCR data analysis is proposed. The ddpcRquant workflow sets a threshold based on the negative template controls which are performed with each ddPCR assay (Fig. 2). The script was designed to use the combined data of replicate NTCs in case more than one is performed per assay. Consequently, the selected threshold is applied to the samples and the concentration is calculated.

Fig. 2
figure 2

Overview of the ddpcRquant workflow. (Step1) Input and automatization process showing the required input information from the head file to automatically allocate the correct amplitude file to the individual wells. (Step2-3) The different steps in NTC preprocessing resulting in a calculated threshold based on extreme value theory. (Step4) Sample processing that applies the calculated threshold and Poisson statistics to return the concentration in each sample. (Step5) Output generation: Different plots and a summary file are generated and returned to the user together with an HTML output file containing all information

Full size image

Step1: input and automating process

The QuantaSoft software that comes with the ddPCR automatically generates a head file with .csv (comma separated value) extension which contains the metadata of each well in a ddPCR run (assay, sample, etc.). In addition to the head file, a second file is required per well which includes the fluorescent amplitude data for each separate droplet (Fig. 2 Step1). These files need to be manually exported from the QuantaSoft software (ESM 5). These amplitude files, together with the head file, are used as input for the script.

For automated data analyses purposes, the script uses four information parameters (well, assay, sample, and type assay) stored in the headfile.csv to automatically locate NTCs and samples per assay and fetch the corresponding raw fluorescence data/amplitude files per well. Therefore, a consistent annotation is required (ESM 6).

Step2: NTC preprocessing

Because small shifts in baseline fluorescence can be observed, the NTC preprocessing comprises a baseline correction before merging fluorescence data of multiple NTCs per assay (Fig. 2 Step2 and ESM 3). For each sample, the Robertson-Cryer mode estimator (also known as the half sample mode) [19, 20] is calculated and subtracted from all fluorescence intensities. After this baseline correction, the data points of the individual NTCs of an assay are merged.

Step3: NTC processing and threshold setting

The merged NTC data set is used for the estimation of the threshold using methods relying on extreme value theory. The droplet fluorescence intensities are randomly assigned to k groups of equal size. The maximum fluorescence intensity within each of these groups is calculated. The k group maxima are then used to estimate the parameters of a generalized extreme value distribution using maximum likelihood (R evd package [21]). The 0.995 percentile is then considered to be the threshold. Note that this percentile can be adjusted by the end-user. The process of random grouping and parameter estimation is repeated 100 times, after which the average of the 100 thresholds is taken to be the final threshold.

Step4: samples preprocessing and processing

The fluorescence intensities are baseline corrected by calculating the Robertson-Cryer mode estimator for the fluorescence intensities below a certain cutoff c and subsequently subtracting the estimated mode from the fluorescence intensities [19]. This cutoff c is taken to be the average of the Robertson-Cryer estimates of the NTCs plus the threshold determined in step 3. Alternatively, c can be specified by the user.

By comparing the baseline-corrected intensities to the threshold of step 3, the droplets are classified into positive or negative droplets. The number of negative and positive droplets is used to calculate the concentration of DNA template in the ddPCR mix as previously described [3, 4] (Fig. 2 Step4).

Step5: output

As a summary, a .csv file per assay is generated containing information of the well, assay, name, type, concentration (positive templates per μl ddPCR mix), upper and lower CI, number of positive, and negative and total droplets. Next, plots are returned for distribution fitting of the NTC, the data plot of the NTC with calculated threshold, and the sample data plots with the threshold (Fig. 2 Step5). As a total overview, all data output is saved in an HyperText Markup Language (HTML) file that is opened automatically upon ending of the analysis.

Comparison of ddpcRquant to QuantaSoft on an example dataset

In order to compare the method of ddpcRquant to the QuantaSoft software provided by the manufacturer (Bio-Rad), 1 of the 26 HIV quantification data sets was used as an example [13]. Here, both the single and combined well-automated analyses of QuantaSoft were compared to the ddpcRquant algorithm (Fig. 3, ESM 7).

Fig. 3
figure 3

Comparison of QuantaSoft auto-analysis methods (single and combined well analysis) and ddpcrQuant on an example dataset of an HIV total DNA quantification assay. Overview of the calculated concentration (A) and number of positive droplets (B) in the NTC and samples

Full size image

Processing of the combined NTCs shows that in the automated analysis methods of QuantaSoft (Single and Combined Well auto analysis), the number of positive droplets from four merged NTCs was respectively 31 and 26 droplets, whereas the ddpcRquant analysis results in 6 positive droplets (Fig. 3b, ESM 7). Consequently, sample processing/quantification by the QuantaSoft analysis methods shows an overall higher concentration level of template (Fig. 3a, ESM 7). In addition, in some samples, the concentration was not possible to be calculated by the QuantaSoft methods in comparison with ddpcRquant.

Discussion

Considerations for extreme value theory

Extreme value theory studies the distribution of a series of maxima of large samples. In the current context, the droplets from an NTC can be distributed over a fixed number of groups so that each group still contains a large number of droplets. Further calculations will only require the largest intensity from each group. Provided that the number of observations in each group is large, extreme value theory implies that the maxima are distributed as a generalized extreme value (GEV) distribution, whatever distribution of the original observations (known as the parent distribution) (Fisher–Tippett theorem). This is asymptotically a nonparametric theory because the GEV holds whatever the parent distribution of the droplet intensities that maxima were sampled from, as long as the number of observations in each group is sufficiently large. Hence, our method does not impose any distributional assumptions on the droplet intensities.

A second argument in favor of extreme value statistics is that setting a threshold as a percentile of the intensity parent distribution is difficult because the percentile is typically in the far end of the tail. In particular, with a parametric fit of a hypothesized parent distribution (e.g., a normal distribution), the correctness of the percentile is very sensitive to the distributional assumption, and with a nonparametric density estimate, the far end percentile cannot be estimated accurately [22].

In applying the extreme value methodology, the user should be aware of setting certain parameters and how they influence the interpretation of the obtained results. This is discussed extensively in ESM 4. In summary, the user should consider the number of groups, number of NTCs, and the tradeoff between specificity and sensitivity.

Our methodology furthermore corrects for NTC-to-NTC (or sample-to-sample) variability in baseline fluorescence levels by calculating a robust estimate of the mode which is to be used as a normalization constant. This robust mode estimator results in stable estimates of the extreme value distribution parameters and threshold when pooling data from several NTCs (ESM 4 Fig. 6).

Comparison of ddpcRquant to QuantaSoft

The major reason for the observed discrepancy in concentration levels is that the calculated threshold by QuantaSoft is placed too close to negative droplet population (ESM 8, ESM 9 red arrows). This can be caused by: (1) no correction for the observed shift in baseline/background fluorescence between NTCs/samples (ESM 8 pink arrow) and (2) assuming a normal distribution of the negative droplet population without performing normality testing/distribution fitting experiments. Indeed, distribution fitting showed that the droplet population is not normally distributed (ESM 10).

In the output generated by ddpcRquant, these problems are resolved and the threshold is placed in a data driven manner, without making assumptions on the distribution of the negative droplets (ESM 11).

Conclusion: applicability and limitations of ddpcRquant

The ddpcRquant algorithm is developed for single color analysis and is applicable to high and low level detection. Furthermore, the ddpcRquant method accounts for two observed problems with current data analysis methods: (1) baseline shift between samples and (2) the requirement of the normal distribution. By implementing a baseline correction and extreme value theory, these issues are resolved.

In addition, the type of probe and the composition of the sample (e.g., DNA, cDNA, eluted DNA, and direct lysate) can affect both the distribution as well as the amount of background fluorescence. Our method is able to cope with these situations, provided that the NTCs are run in the same background (e.g., DNA, lacking the specific template) as the samples.

Finally, we emphasize that ddpcRquant is a data-driven method to assist the end-user determine a threshold and perform absolute quantification. Nonetheless, the end-user is still encouraged to evaluate the threshold setting and responsible for the interpretation/quality control of the ddPCR experiment.