Introduction

BRCA1 and BRCA2 (BRCA) are DNA repair genes involved in the homologous recombination repair pathway of double-strand breaks (Connor et al. 2016). Recent clinic- and population-based cohorts found pathogenic germline mutations in BRCA in 1 to 5 % of pancreatic ductal adenocarcinoma (PDAC) patients (Grant et al. 2015; Holter et al. 2015; Shindo et al. 2017). Mounting evidence suggests that BRCA-associated PDAC has a distinct genomic landscape, characterized by an increased number of insertions, deletions, structural rearrangements, and an “immunogenic” signature (Connor et al. 2016).

The identification of germline BRCA mutations has several potential benefits for patients with PDAC and their families. First, mutation carriers may have a unique response to platinum-based chemotherapies (Golan et al. 2014; Waddell et al. 2015) and poly-ADP-ribose polymerase inhibitors (Kaufman et al. 2015), so these patients should be considered for precision therapy clinical trials (i.e., NCT02184195). Second, relatives of probands who carry BRCA mutations should undergo genetic testing because relatives who also carry the mutations face an elevated risk of several malignancies and can benefit from intensive screening for these malignancies and from prophylactic mastectomy and salpingo-oopherectomy (National Comprehensive Cancer Network 2017).

Different strategies to identify germline BRCA mutations in PDAC have not been systematically compared. The National Comprehensive Cancer Network (NCCN) (National Comprehensive Cancer Network 2017) and the Ontario Ministry of Health and Long-Term Care (MOHLTC) (Holter et al. 2015) provide guidelines for BRCA testing that can be applied to PDAC. BRCAPRO (Berry et al. 2002) is a Bayesian statistical model to identify BRCA mutation carriers that has been validated in numerous breast and ovarian cancer populations and performs comparably to other statistical models (Fischer et al. 2013). The NCCN guidelines have recently been updated to specifically address PDAC probands; neither the MOHLTC guidelines nor BRCAPRO account for the association between PDAC and BRCA mutations.

In our experience, the decision to send a PDAC patient for germline BRCA testing is often based on whether a genetic counselor estimates that the probability of the proband testing positive exceeds some threshold; for example, 10% is specified in the MOHLTC guidelines. The accuracy of estimates of genetic counselors in PDAC has never been tested. More broadly, few studies have assessed the accuracy of genetic counselor estimates in any population (Euhus et al. 2002).

In this study, we assessed the predictive characteristics of the NCCN and MOHLTC guidelines and the estimated probabilities from BRCAPRO and genetic counselors in a prospective clinic-based cohort of probands with PDAC.

Methods

Our Research Ethics Board approved this study (REB #: 16-0082-E).

Participants

Patients with PDAC were prospectively recruited for BRCA sequencing from the Wallace McCain Centre for Pancreas Cancer at the Princess Margaret Hospital in Toronto from August 2012 to October 2015. All patients with an incident diagnosis of PDAC within three months of potential enrolment into the study were invited to participate, regardless of family or personal history of cancer. Only incident cases were included to minimize survivorship bias. Two hundred and ninety-six of these probands were included in an earlier publication describing BRCA mutation prevalence (Holter et al. 2015).

Invitations were sent to participate in a survey by email to members of the Canadian Association of Genetic Counsellors (CAGC) and the National Society of Genetic Counselors (NSGC). Members were asked to respond only if they managed PDAC patients as part of their clinical practices. Genetic counselors who agreed to participate were randomly assigned to 1 of 20 surveys using a randomly generated link on an intermediate website. CAGC members were invited to participate on November 14, 2016. NSGC members were originally invited on November 29, 2016; however, responses were invalid because of an error in the intermediary site affecting randomization and survey participation. The error was corrected and the survey was resent on January 4, 2017. As an incentive to complete the survey, the NSGC members were provided with the opportunity to enter a raffle for a prize. Demographic data were optional. Respondents were given two weeks to answer the survey.

Instrumentation

DNA was extracted from blood mixed with EDTA using the Qiagen Pure-gene kit (Qiagen, Hilden, Germany). The first 297 probands underwent BRCA testing as previously described using Sanger sequencing and multiplex ligation-dependent probe amplification (Holter et al. 2015). The subsequent 187 probands underwent BRCA testing using targeted next-generation sequencing. We designed a custom research platform, named the Cancer Relevant Genes panel, using Agilent SureSelect chemistry (Agilent Technologies Inc., Santa Clara, CA) that included BRCA1 and BRCA2. Briefly, germline DNA samples were sheared, barcoded, and hybridized with biotinylated RNA strand probes. Batches of 16 samples were pooled for paired-end sequencing for 600 cycles generating 300 base-pair reads on an Illumina MiSeq using a V3 sequencing cartridge (Illumina Inc., San Diego, CA). The Burrows-Wheeler Aligner (Li and Durbin 2009) aligned the sequence reads to the reference human genome. The HaplotypeCaller module of GATK (McKenna et al. 2010) identified single-nucleotide variants, insertions, and deletions. Variants were considered if they had at least 20-fold depth of coverage and the alternate allele was present in at least 25% of the reads. All pathogenic variants identified using next-generation sequencing were confirmed using Sanger sequencing.

Progeny (Progeny, Delray Beach, USA) was used to create de-identified pedigrees for each proband. Pedigrees were randomly grouped into 20 surveys. Novi Survey (Novi Systems, Boston, USA) was used to generate the surveys and collect responses. Each of the 20 surveys began with 20 to 25 different pedigrees, which were presented with the self-reported ethnicity of the proband. Respondents were asked to provide their estimated probability that each proband carried a pathogenic BRCA mutation.

Data Analysis

Two authors (R.G. and S.H.) independently assessed all probands according to the 2017 NCCN and MOHLTC guidelines. Discrepancies were resolved through a third author (S.G.). For both guidelines, an additional primary cancer in the proband was treated as a cancer in a close blood relative. Information on prostate cancer pathology was not recorded, so for the NCCN guidelines, probands were assessed twice using two different assumptions: (i) all prostate cancers had Gleason scores 7 or higher, or (ii) all prostate cancer had Gleason scores under 7. For the MOHLTC guidelines, criterion 13, a “pedigree strongly suggestive of hereditary breast/ovarian cancer, i.e., risk of carrying a mutation for the individual being tested is > 10%,” was ignored because there was no guidance how to operationalize this in PDAC probands.

The characteristics of carriers and non-carriers of mutations in BRCA were compared using the Fisher exact test for categorical variables. The sensitivity, specificity, positive predictive value, and negative predictive values were calculated for each guideline. Comparisons were made using McNemar’s test.

BRCAPRO (Berry et al. 2002), implemented in BayesMendel (Chen et al. 2004) (version 2.1), generated estimates for the probability each proband carried a BRCA mutation using the standard recommended input parameters. The estimates of genetic counselors were averaged for each proband.

Discrimination was assessed using the area under the receiver operator curves. Areas under the curve were generated and compared using the bootstrap method implemented in the “pROC” package (Robin et al. 2011) (version 1.8). Calibration of the models was assessed graphically by comparing the observed and expected number of mutation carriers within quintiles of prediction estimates. Calibration of the models was tested using the Hosmer and Lemeshow goodness of fit test.

All analysis was performed in R (version 3.1.1). Statistical significance was defined as P < 0.05.

Results

The cohort included 484 probands with PDAC (Table 1). Twenty-two probands (4.5%) carried pathogenic BRCA mutations. There was no difference in the rate of mutations detected by direct sequencing and multiplex ligation-dependent probe amplification compared with next-generation sequencing (14/297 versus 8/187, respectively, P = 1.0).

Table 1 Characteristics of the cohort of probands with pancreatic ductal adenocarcinoma
Full size table

Mutation status was significantly associated with ethnicity and family history of breast cancer (Table 1). Of the 57 probands who identified as Ashkenazi Jewish, two patients carried the founder mutation c.185delAG (rs386833395) in BRCA1 and five carried the founder mutation c.6174delT (rs80359550) in BRCA2. There were no other pathogenic mutations found in Ashkenazi Jewish probands. All pathogenic mutations in probands with other ancestries were unique within the cohort and predicted to inactivate their respective protein.

One hundred and nineteen genetic counselors completed the survey (Table 2). Each pedigree was evaluated on average 5.9 times (standard deviation 2.3). When asked for a free text answer on what factors contributed towards recommending that probands undergo BRCA testing, responses included family history (66 respondents), age of onset (35), meeting NCCN guidelines (13), patient preferences (11), other risk factors such as smoking and diabetes (8), meeting unspecified guidelines (7), ancestry (7), treatment implications (4), pathological characteristics (4), and insurance coverage (3). Thus, guidelines were a key factor in referring a proband for BRCA testing for at least 20/119 (19.6%) respondents.

Table 2 Characteristics of the genetic counselors who completed the survey
Full size table

The guidelines varied widely across predictive characteristics (Table 3, Fig. 1). Sensitivity was the same for the NCCN guidelines whether all prostate cancers were assumed to have Gleason scores of 7 or more, or if all prostate cancers were assumed to have Gleason scores of < 7 (0.864, 95% confidence interval (0.651–0.971)) for both), but specificity was improved when all prostate cancers were assumed to have Gleason scores < 7 (0.712 (0.668–0.753) versus 0.634 (0.588–0.678), P < 0.001). Thus, we assumed all prostate cancers had Gleason scores < 7 in Table 3. Only 16/484 (3.3%) probands met the MOHLTC guidelines for genetic testing. The NCCN guidelines had higher sensitivity than the MOHLTC guidelines (0.864 versus 0.227, P < 0.001) but lower specificity (0.712 versus 0.976, P < 0.001). Both guidelines had positive predictive values over 0.1 and negative predictive values over 0.95.

Table 3 Sensitivity, specificity, positive predictive value, and negative predictive value with the 95% confidence intervals for different guidelines and strategies to select pancreatic ductal adenocarcinoma patients for BRCA1 and BRCA2 testing
Full size table
Fig. 1
figure 1

Receiver operating curves for genetic counselor and BRCAPRO estimates of the probability that pancreatic cancer probands carry a pathogenic germline BRCA mutation. AUC is the area under the curve. NCCN and MOHLTC are the NCCN and MOHLTC guidelines for BRCA testing, respectively

Full size image

Estimates from genetic counselors and BRCAPRO had similar discrimination (area under the curve 0.755 and 0.775, respectively, P = 0.702) (Fig. 1) and were moderately correlated (correlation = 0.388). Genetic counselors generally overestimated the probability that each proband carried a mutation (P = 0.008) (Fig. 2). BRCAPRO severely underestimated the probability that each proband carried a mutation (P < 0.001) (Fig. 2). The expected number of mutation carriers in the cohort based on genetic counselor and BRCAPRO estimates were 42/484 (8.7%) and 1/484 (0.2%), respectively.

Fig. 2
figure 2

Expected versus observed proportions of carriers of pancreatic cancer probands within quintiles of estimates by genetic counselors and BRCAPRO. Note that the points for the first, third, and fourth quintiles for the BRCAPRO predictions overlap with expected proportions less than 0.001 and observed proportions of approximately 0.02

Full size image

There was no group of genetic counselors that had superior estimates when comparing discrimination across genetic counselors who saw more than one PDAC patient per month versus less, who had practiced for over 5 years versus less, or members in the CAGC versus NSGC (data not shown).

Discussion

BRCA-associated PDAC has a characteristic genomic landscape (Connor et al. 2016) that may be associated with a unique response to conventional and investigational therapies (Golan et al. 2014). Relatives of BRCA carriers with PDAC can benefit from more intensive cancer screening and prophylactic cancer strategies (National Comprehensive Cancer Network 2017). There is accumulating evidence that BRCA-associated PDAC has a favorable response to platinum-based therapies (Golan et al. 2014; Waddell et al. 2015). Therefore, clinicians may consider BRCA status when considering between two standard first-line chemotherapies for metastatic disease: FOLFIRINOX (Conroy et al. 2011), which contains oxaliplatin, a platinum-based chemotherapy; and gemcitabine and nab-paclitaxel (Von Hoff et al. 2013), which does not. Patients with BRCA-associated PDAC should be also considered for trials of targeted therapies (i.e., NCT02184195, which evaluates the PARP inhibitor olaparib), especially given the evidence for PARP inhibitors in BRCA-deficient breast (Robson et al. 2017) and ovarian (Mirza et al. 2016) cancers.

Recent clinic- and population-based studies demonstrate that BRCA mutations cause a clinically relevant proportion of PDAC (Grant et al. 2015; Holter et al. 2015; Shindo et al. 2017) but the optimal approach to identify carriers is unclear. In this study, we compared the performance of the NCCN and MOHLTC guidelines, and estimates from genetic counselors and BRCAPRO, for the prediction of germline BRCA mutations in a prospective clinic-based cohort of incident probands with PDAC. We found that the NCCN guidelines had substantially higher sensitivity when compared with MOHLTC guidelines (0.864 versus 0.227, P < 0.001), but lower specificity (0.712 versus 0.976, P < 0.001). The NCCN guidelines retained a positive predictive value of over 10%, a commonly cited threshold for genetic testing that is incorporated as criterion 13 in the MOHLTC guidelines. Estimates from genetic counselors and BRCAPRO had similar ability to discriminate between carriers and non-carriers, but the estimates from BRCAPRO drastically underestimated the likelihood that a proband carried a mutation.

In our cohort, approximately 20 % of genetic counselors cited clinical guidelines to be a key factor when deciding whether to recommend patients with PDAC for genetic testing; however, the performance of the 2017 NCCN guidelines was untested in pancreatic cancer prior to our study. Expert panels created the NCCN and MOHLTC guidelines based on consensus interpretations of the literature without external validation. Reassuringly, our results support the application of the NCCN guidelines to probands with PDAC.

Pancreatic cancer patients face a dismal prognosis, so the window of opportunity for genetic testing is narrow. Guidelines are important because they can be expeditiously applied by any member of the healthcare team to identify patients who should be referred for genetic counseling and/or testing.

Most genetic counselors cited family history and other clinical attributes when asked for factors that contribute towards recommending that a proband undergoes mutation testing. The performance of genetic counselor estimates has not been extensively studied. Euhus et al. (Euhus et al. 2002) found that BRCAPRO had better discrimination than eight experienced genetic counselors based on estimates from a cohort of breast and ovarian cancer patients. Our study is the first to assess genetic counselor estimates in PDAC and includes the largest number of genetic counselors to date, enabling comparisons between groups of genetic counselors. We found that genetic counselors had similar discrimination but superior calibration compared with BRCAPRO. Thus, our results support genetic counselors performing a risk assessment for probands with PDAC before recommending BRCA sequencing.

Study Limitations

Our study is subject to several limitations. First, our cohort was comprised of incident probands with PDAC prospectively recruited at a multidisciplinary PDAC clinic for germline BRCA sequencing, without consideration of family history or ancestry. Caution should be made when generalizing our findings to population-based patients, although our multidisciplinary clinic sees about 30% of all PDAC in the large province of Ontario, Canada. Second, we cannot estimate the response rate of genetic counselors to our survey. To make our results clinically relevant, we asked that only genetic counselors who manage PDAC patients respond. We do not know how many genetic counselors in the CAGC and NSGC manage PDAC patients, but we assume it is a minority of these professionals. We found no differences between subgroups of genetic counselors, suggesting that response rates are unlikely to substantially bias our results. Third, our survey may have been subject to recall bias and other cognitive biases, which cannot be corrected in our study design. Finally, the Gleason score and stage of the prostate cancers in our cohort were unavailable. Gleason score and stage are strongly associated with BRCA mutation status (Pritchard et al. 2016). In practice, the Gleason score of relatives may be unknown to patients. We found equal sensitivity but improved specificity for the NCCN guidelines when prostate cancers were all assumed to have Gleason scores <7. These assumptions affected the classification of 36/484 (7.4%) probands. It is probable that we have slightly underestimated the sensitivity and specificity of the NCCN guidelines in these probands.

Practice Recommendations

We have estimated the sensitivity, specificity, negative predictive value, and positive predictive value of the NCCN and MOHLTC guidelines in a clinic-based cohort of pancreatic cancer patients. Genetic counselors can use this information during counseling sessions to help patients decide whether to undergo BRCA testing. If a patient does not meet the NCCN guidelines, the negative predictive value is 0.991, implying that there is less than a 1% chance that they will carry a mutation. Patients may decide against BRCA testing in this situation, particularly if they are paying for the testing themselves with limited financial resources. In contrast, patients who meet the NCCN and MOHLTC criteria have a 12.5 and 31.3% chance of testing positive, respectively. Patients in this situation may be more likely to decide to have BRCA testing, considering that a positive test may help to tailor their chemotherapy (Golan et al. 2014) and prevent cancer in their relatives (National Comprehensive Cancer Network 2017).

Our data argue against using BRCAPRO in pancreatic cancer patients because the algorithm severely underestimates the probability of carrying a BRCA mutation in this population. The clinical impression of genetic counselors is more accurate, with the caveat that there is a bias towards higher estimates.

Research Recommendations

The accuracy of BRCA mutation predictions in pancreatic cancer will depend on whether the association between PDAC and BRCA mutations is incorporated. The NCCN guidelines and estimates from genetic counselors incorporate the association between PDAC and BRCA mutations, whereas the MOHLTC guidelines and BRCAPRO do not. To improve accuracy, we recommend that the MOHLTC guidelines and BRCAPRO be updated to account for the association between PDAC and BRCA mutations. Similarly, criteria for BRCA testing for other public and private healthcare providers should be updated to include pancreatic cancer.

Conclusion

In conclusion, our findings support the use of the NCCN guidelines and estimates from genetic counselors for predicting BRCA mutation status in PDAC probands. Genetic counselors, clinicians, and policy makers should consider these strategies when recommending BRCA sequencing for patients with PDAC.