Introduction

Next generation sequencing (NGS) technology has made it possible to examine the genome more thoroughly than ever before. This technology has lowered the cost of genetic testing, which enables clinicians to order panel tests that include a growing number of genes. Increased utilization of panel testing has raised concern for accurate interpretation of variants, including variants of uncertain significance (VUS) that are often encountered when multiple genes are sequenced. In one study, 29 genes were sequenced in 1062 people and found that 41% of individuals had at least one VUS and 11.4% had two or more VUS (Lincoln et al. 2015). When 20,000 genes were analyzed using whole exome sequencing (WES), VUS were found in 95% of the study population (Maxwell et al. 2016). Therefore, understanding the effects of genetic variants and how the interpretation of variants influences clinical care becomes of increasing importance.

In 2015, the American College of Medical Genetics and Genomics (ACMG) recommended a 5-tiered system for classifying variants: pathogenic (P), likely pathogenic (LP), VUS, likely benign (LB), and benign (B) (Richards et al. 2015). According to these guidelines, laboratory classification of a variant should be based on multiple lines of evidence, including population data, variant databases, segregation data, scientific and medical literature, and in silico predictors (Richards et al. 2015). Some experts attest that the most useful sources of information include allele frequency, conservation data, co-segregation, and the mutation type (Amendola et al. 2015). Based on the information obtained from many of the aforementioned sources, the pathogenicity of a variant can be determined with varying degrees of certainty.

Even with this guideline for variant classification, healthcare providers must determine the clinical utility of the results and their impact on patient management. The ACMG states that variants classified as either LP or P are clinically actionable, as these terms imply a greater than 90% certainty that the variant in question is truly disease-causing (Richards et al. 2015). In such circumstances, risk-based management decisions can be made based on pathogenicity, including prophylactic surgery and increased surveillance for tumors.

A variant interpretation discrepancy occurs when the same variant is classified differently by two or more different laboratories or sources. These differences can be due to how the data is interpreted, such as weighing evidence differently and setting varying thresholds for pathogenicity. Additionally, some researchers have access to information that others do not, such as a laboratory with an undisclosed internal variant database (Amendola et al. 2015).

In order to analyze the prevalence of discordant classification in cancer genes, one study looked at agreement between databases for variants found in their subjects via WES. Classifications were discordant in 16% of cases involving an autosomal dominant cancer gene and 23% of cases involving an autosomal recessive cancer gene. There was a 4% rate of disagreement in clinical actionability for autosomal dominant conditions and a 9% rate for autosomal recessive conditions (Maxwell et al. 2016). Another study found that 26% of cancer variants found via recruitment into the Prospective Registry of Multiplex Testing (PROMPT) database had conflicting interpretations when compared to the ClinVar database. In addition, 11% of variants had classifications that differed in clinical actionability (Balmaña et al. 2016). Having up to 26% discordance in variant calling is an issue for clinicians in the cancer genetics setting, who may receive different results depending on the source’s interpretation. This is especially troubling in up to 11% of cases when the classification impacts clinical actionability.

Variant classifications can differ between sources, and clinicians may have difficulty applying discrepant variant interpretations in a clinical setting. This study identified the strategies that clinical genetic counselors used to understand variant results, determined current counseling practice when there are discordant interpretations, and shed light on the concerns that arise when counseling variant interpretation discrepancies.

Methods

Participants

This study surveyed practicing cancer genetic counselors, specifically members of the National Society of Genetic Counselors (NSGC) Cancer Special Interest Group (SIG). An email was sent to Cancer SIG members explaining the basis of the study and inviting them to complete the survey (Supplementary Material 1). Participation in the survey constituted consent to the study, and counselors could opt to discontinue at any point in the survey. The inclusion criteria consisted of (1) being a board certified or board eligible genetic counselor working primarily in oncology, (2) spending more than 50% of their time in a clinical setting, and (3) having attended an accredited genetic counseling master’s program. These criteria excluded participants who had different experience and expertise than required for this study. Student members of the SIG were also excluded.

Procedures and Instrumentation

The survey was created by the authors using Qualtrics software (2015) available through the University of Texas Health Science Center at Houston. It was distributed via email to members of the NSGC Cancer SIG in June 2016. Two reminder emails were sent in July and August 2016. The survey was closed on August 31, 2016. The survey took approximately 15–20 min for each participant to complete, and all answers were anonymous.

The survey was a semi-structured questionnaire with 32 questions. There were seven demographic questions that collected information about schooling and work setting. The next section evaluated the counselor’s strategies for assessing variant results. In this section, participants were asked questions about researching variants. Specifically, they were asked the lines of evidence they used and how often they researched variants independently. A variant database was defined as an online source containing classifications of individual variants and phenotypic data associated with the variant. Counselors were then asked about their current counseling practice regarding discrepancies in variant interpretation, including how often they identified a discrepancy and how they managed them in a clinical setting. There were two scenarios involving discovery of these discrepancies, followed by questions about how participants would handle these situations. The scenarios were developed with input from clinical cancer genetic counselors, and they were designed to compare how respondents approached variant classification discrepancies with differences in clinical actionability and type of syndrome. In scenario A, the genetic counselor tested an unaffected patient who had a VUS in either the BRCA1 or BRCA2 gene but then learned the patient’s unaffected sister had the same variant which was classified as P through testing at another laboratory. In scenario B, a patient with colon cancer had an LP variant in a mismatch repair gene associated with Lynch syndrome. The patient’s unaffected son had testing at another laboratory that showed he had the same variant, but it was classified differently. Lastly, there were questions regarding counseling concerns when there are discrepancies in variant interpretation. At the end of the survey, the counselors had the option to input their email address to enter a drawing for a gift card. All research protocols met the requirements of the University of Texas Health Committee for the Protection of Human Subjects, and this study was assigned approval number HSC-MS-16-0436.

Data Analysis

STATA 14 software was used to analyze statistics (StataCorp 2015). Primary outcomes included descriptive analyses. Results were reported as frequencies with percentages. Secondary explorative analysis was performed, using Fisher exact tests or t tests based on the nature of the data, to compare differences between groups. All comparative tests were considered significant at type I error rate of 5%. Free text responses were reviewed for similarities and unique considerations.

Results

Demographics

There were 281 responses to the survey, which represented 33% of the largest mailing to 849 counselors. Thirty-three respondents did not complete the demographic information and therefore were excluded from the analysis. There were 24 respondents who met exclusion criteria, including current students and genetic counselors that did not work primarily in a clinical oncology setting. This left a total of 224 responses that were included. Of these, 60 (27%) were partially complete, and any answered questions were incorporated in the analysis. The demographic information of the respondents is reported in Table 1.

Table 1 Demographics
Full size table

Strategies for Assessing Variant Results

Participants were asked a series of questions about how they approach a variant without a known discrepant interpretation. A large majority (96%) of counselors indicated that they conduct their own research on genetic testing results that report a variant. The most common source used in research was variant databases, with 83% of respondents indicating that they use them “always” or “most of the time.” About 36% of respondents utilized functional studies either through literature review or asking if the reporting laboratory had performed such studies. Counselors were able to fill in a free response to clarify other sources used to gather variant information. Nine of these (5%) reported that they obtain information from other labs in addition to the performing laboratory. Eight respondents said they spoke with the lab involved (4%). One respondent stated, “I call the lab GC and get whatever data they can give me.”

The respondents were asked if their confidence in the classification of a variant depended on the performing laboratory. Of the 213 respondents that answered this question, 178 (83%) of counselors affirmed that their confidence does depend on the performing laboratory. To clarify, one respondent stated, “Not all labs - or classification systems - are created equally. Whichever lab was ‘better’…is the classification I would use for medical management.” Another participant further defined “better” as that with a more extensive dataset, specifically saying, “If [lab name excluded] was involved, I may tell the patient I would lean towards their classification since they have the largest dataset.”

Frequency and Time Researching Variant Interpretation Discrepancies

Of 180 total respondents to this portion of the survey, 167 (93%) had seen a variant interpretation discrepancy in practice and 111 (62%) had come across a discrepancy on three or more occasions in the last 3 years. A total of 143 (78%) counselors discovered a discrepancy by searching the variant in a variant database. Additionally, counselors reported that they learned of a discrepancy by either using different laboratories to test relatives or testing two unrelated patients with the same variant at different laboratories.

Those that indicated they refer to variant databases “always” or “most of the time” when researching variants were more likely to discover a discrepancy in a database (p < 0.0001). However, this same group was not statistically more or less likely to discover discrepancies overall than those who refer to variant databases less often (p = 0.518). There was also no statistical difference in the likelihood of identifying a discrepancy based on the type of testing most frequently ordered (p = 0.254). Of our 213 respondents for this question, 186 indicated that they order panel testing most often (87%), 4 ordered syndrome-specific testing most often (2%), and 23 said they order the two about equally (11%).

Of 181 counselors that answered questions in this portion of the survey, 98 (54%) reported taking 1–15 min to research a variant with no known discrepant interpretation. When asked how long it takes to research a variant with discordant interpretations, 40 (24%) counselors indicated that it takes 46–60 min, 39 (23%) counselors selected 16–30 min, and 31 (19%) selected 31–45 min. There was a significant difference in the time spent researching a non-discrepant variant interpretation versus a discrepant variant interpretation (p = 0.001), with the majority of counselors spending more time researching discrepant variant interpretations (Fig. 1). A total of 52 (33%) counselors spent 45 min or more of extra time researching a discrepancy compared to researching a non-discrepant variant interpretation. The amount of time a counselor spent following up on a discrepancy was not dependent on the number of discrepancies they had previously discovered (p = 0.482).

Fig.1
figure 1

Time spent researching a discrepant variant versus a non-discrepant variant

Full size image

In Fig.1, the gray bars represent the number of counselors that spent less or equal time researching a variant interpretation discrepancy versus a non-discrepant variant. The black bars represent counselors who spent more time researching a variant interpretation discrepancy. Counselors spend more time researching a discrepant variant interpretation than a non-discrepant variant interpretation (p = 0.001)

Scenarios

Scenarios were utilized to assess how respondents approached counseling sessions involving discrepancies in variant interpretation. In scenario A, a variant in either the BRCA1 or BRCA2 gene is classified by one laboratory as P and by another laboratory as VUS. Most counselors indicated that they would explain both classifications, explain how each classification was reached, emphasize that they are not the same, and discuss clinical management for each.

In scenario B, the lab that tested the patient’s son classified the variant as P, while the lab that tested the patient classified the variant as LP. The same four responses were selected most often for this question as for the question in scenario A. For the majority of responses, those that selected a specific approach for the discrepancy in scenario A involving the BRCA1 and BRCA2 genes were statistically more likely to choose that same approach for the discrepancy in scenario B involving a mismatch repair gene. In scenario B, four free responses (2%) pointed out that LP variants have the same management recommendations as P variants. One respondent asserted, “Although the management recommendations would be the same from my perspective, I would note that one lab does not feel the evidence is quite as strong, and it may be unlikely, but is possible that in the future interpretation and recommendations could change.” In the second question in scenario B, where the son’s results are now classified as VUS rather than P, one respondent detailed their strategy by expressing that before the session they “would retest through the other lab to have an amended report reflecting the conflict in interpretation. You have to be safe rather than sorry.”

Regarding the counseling session in scenario B, counselors were likely to respond similarly to the second question as to the first follow-up question by explaining both classifications, how each classification was reached, emphasize that the classification are not the same, and discuss clinical management for each. In general, 73% of respondents chose to emphasize that the classifications were not the same and 90% of respondents said they would discuss clinical management for each when comparing LP and VUS. This is compared to the 26% that chose to emphasize that the two classifications were not the same and 36% that chose to discuss clinical management when comparing LP and P (Fig. 2).

Fig. 2
figure 2

The percentage of respondents that chose each approach to the counseling session with and without a difference in clinical actionability. More counselors chose to emphasize that the classifications were not the same and to discuss clinical management for each when there is a difference in actionability

Full size image

Both scenarios allowed counselors to write “other” responses to the questions. One participant noted, “I wish I knew the answer to the question. I don't know what to do in cases like this.” Additional responses similarly illustrated that even though there was consistency in responses to the standardized questions, counselors were uncertain about how to approach these complex situations. Many of the responses emphasized the importance of family history in deciding how to approach a counseling session involving a discrepancy and how to manage the patient’s care. For example, a respondent clarified that they “discuss the result in the context of family history and whether prophylactic surgery could be recommended based on family history alone.” Some respondents said that they would consider other clinical information besides family history, including microsatellite instability (MSI) and immunohistochemistry (IHC). One respondent noted, “I would probably use additional clinical information like IHC to determine how I felt about my patient's result.” As well as considering clinical information, the free responses pointed out a focus on psychosocial support for the patient. One counselor explained that they “discuss psychosocial aspects of having discrepant test results and how that has impacted the patient.”

Concerns When Counseling a Discrepancy in Variant Interpretation

When asked to identify their concerns about the genetic counseling process involving variant interpretation discrepancies, 162 of 164 (99%) counselors selected at least one. The most frequently selected concern was lack of data sharing (90%), followed by lack of a central database (72%), and lack of educational resources (60%). Some of the free responses also indicated that lack of communication between the labs involved was a concern.

Ideal Features of a Centralized Database

Participants were asked to indicate who should input information into a centralized database, and 150 (95%) selected laboratories only or a combination of laboratories and another source. No participants chose patients as the sole source of information for a centralized database. A total of 96 respondents (59%) indicated that a non-profit should own such a database. Forty-four counselors (27%) chose government as the ideal owner of a centralized database. In the free response section, 13 respondents indicated that they were unsure who should own the database (8%). One participant highlighted the difficulties with this resource by stating, “Public databases will only make and have made variant classification more challenging for the busy GC.”

Desired Resources

Lastly, counselors chose resources that would help in counseling when they discover a discrepancy in variant interpretation (Fig. 3). The most common response selected by 148 (89%) counselors was additional support from the laboratories involved with the discrepancy, followed by 145 (88%) that chose practice guidelines from a major society/organization, 121 (74%) that chose continuing education opportunities, and 95 (58%) that chose functional studies.

Fig.3
figure 3

The number of respondents that desired each resource to aid in counseling about a discrepancy in variant interpretation

Full size image

Discussion

Frequency and Time Considerations

This study investigated how cancer genetic counselors approached discrepancies in variant interpretation and shed light on concerns about counseling these discrepancies. A large majority (93%) of cancer genetic counselors saw at least one variant classification discrepancy. Therefore, it is likely that clinical cancer genetic counselors will encounter the issue of variant interpretation discrepancies in practice. There was no difference in the number of discrepancies discovered by those who order panel testing, syndrome-specific testing, or both about equally. This indicated that cancer genetic counselors did not avoid encountering discrepancies by ordering syndrome-specific panels.

Previous studies reported variant classification discrepancies at a frequency of 16–26% (Balmaña et al. 2016; Maxwell et al. 2016). In our study, most respondents (62%) discovered an average of one discrepancy per year. This appears to be less than what would be expected from the previously reported frequencies. However, other studies were looking at results that included large numbers of genes on pan-cancer panels or WES. Our study did not inquire on the specific size of panels ordered by participants, so it is possible that these larger tests are not always ordered by the respondents. As expected, the more genes that are sequenced, the likelihood of discrepancies in variant interpretation increases (Lincoln et al. 2015). Additionally, counselors ordering hereditary cancer testing may not independently investigate the classification of all variants seen on a report. While a large portion of our study population indicated that they had researched results in the past, certain classifications were potentially more likely to trigger further research. For example, a counselor may be more likely to look into a VUS compared to an LP finding. Furthermore, if a patient had an LP result that explained a suggestive clinical presentation or family history, then this result may not be further investigated. This might also be a reason that fewer variant discrepancies were encountered by the respondents in this study.

Researching a variant with discrepant interpretations was more time-consuming than researching a non-discrepant variant. A third of counselors reported that they spend 45 min or more of extra time researching a variant with conflicting interpretations. Time spent on researching a discrepancy is concerning as it is time taken away from clinical, research, or academic work. The study also noted that the time a counselor spent on researching a variant with discrepant interpretations was not dependent on prior exposure or experience with discrepancies. The length of research time could vary depending on the amount of information that is available on that specific variant. In the survey, no single factor was identified as impacting the amount of time spent researching a discrepancy in variant interpretation. Therefore, research time appeared to be impacted by a combination of factors, the specific discrepancy, and additional influences that were not addressed in this survey.

Implications for the Counseling Session

When it comes to patient care involving discrepancies in variant interpretation, counselors were consistent in their approach in both survey scenarios. There were no fluctuations due to differences in clinical actionability of the classifications or the gene being tested. Despite the consistency in standardized responses, counselors emphasized in the free text that they did not always know how to approach these situations. When confronted with a discrepancy in clinical actionability, counselors were more likely to emphasize that the classifications were not the same and to discuss management based on each classification.

The free text fields indicated that many counselors adapted the session to the individual patient based on the personal and/or family history of cancer, rather than modifying the session to the syndrome involved or the clinical actionability of the result. Genetic counselors indicated that they would tailor recommendations by synthesizing information from test results, personal history, and family history, ultimately providing patients with a personalized risk assessment. Overall, the scenarios demonstrated that patient care goes beyond test results to include individualization based on family history, psychosocial situation, and personal history including pathology results.

Databases and Counseling Concerns

Approximately 87% of respondents reported utilizing variant databases to research variants, which is likely due to the fact that many of these databases were readily accessible online, such as OMIM and ClinVar (Richards et al. 2015). Survey participants also reported that variant databases were the most common way to discover a discrepancy in variant interpretation. However, those that referred to variant databases more often were not more likely to discover discrepancies than counselors who used them less often. Counselors who did not use databases were not avoiding discovery of discrepancies.

Our study showed that very few counselors were satisfied with resources currently available to evaluate discrepancies in variant interpretation, given that almost all respondents selected at least one concern when counseling about discrepant results. The most commonly selected concerns were lack of data sharing and lack of a central database. Lack of data sharing has been previously discussed as a hindrance to reconciling discrepancies in variant interpretation. Several papers have encouraged data sharing to improve consistency between laboratories (Balmaña et al. 2016; Harrison et al. 2017; Lincoln et al. 2015).). Additionally, certain professional societies, such as ACMG, the American Medical Association and the National Society of Genetic Counselors, have also given position statements that support sharing genetic data (ACMG Board of Directors 2017; American Medical Association 2013; National Society of Genetic Counselors 2015). However, some laboratories and researchers may feel that they have a proprietary right to the accumulated data and that sharing data means that others can make monetary gain from their work (Savage 2017).

Variant databases were available to clinicians, but each database had limitations. A research study by Yang et al. (2017) showed that there was limited data curation in ClinVar: older classifications tended to conflict with updated classifications, nomenclature differed for variants within the database, and submitters varied in their level of credibility, which may not be apparent to the user. It is difficult to determine who should be responsible for curating the information in a centralized database, as reflected by the free responses indicating that counselors were not sure who should own the ideal database. Counselors desired a database that is similar to existing databases, but with careful curation to ensure updated information, standardized nomenclature, and credibility of variant contributors.

The study respondents were asked about the ideal variant database including data input and curation. The majority of counselors indicated that the laboratory alone or the laboratory combined with another source, such as a non-profit or the government, should input variant data into a centralized database. Based on these responses, the ideal database consists of laboratories submitting information into a central database that is owned by a non-profit organization without a potential conflict of interest.

Laboratories and Other Resources

Most counselors (83%) indicated that their confidence in a classification depends on the laboratory providing it. Furthermore, additional support from the laboratories was the most commonly selected resource that would help when addressing a variant with discrepant interpretations. This highlights the importance of the relationship between the performing laboratory and ordering clinician to instill trust and give support. Survey respondents expressed the desire for laboratories to communicate effectively about the methodology and interpretation of the results, as well as share variant data with other laboratories.

Following additional support from the laboratories involved, practice guidelines from a major society/organization and continuing education opportunities were the second and third most commonly desired resources, respectively. Currently, no professional societies have issued guidelines addressing discrepancies in variant classification, including the guidelines on interpretation of sequence variants from the ACMG. Little research has been done on the clinician’s responsibilities regarding variant interpretation discrepancies, so creation of national guidelines is a difficult task. However, future practice guidelines would help create consistent patient care when variant interpretation discrepancies are discovered in a clinical setting.

Additionally, more than half (58%) of counselors chose functional studies as a resource that could assist in resolving discrepancies in variant interpretation, but only 36% of respondents reported utilizing these studies in variant assessment. This inconsistency can be attributed to the fact that functional studies are not available for many variants. The lack of availability is likely due to the fact that they require a high monetary and temporal investment (Simpson and Smith 2017). Additionally, functional studies may not always accurately reflect the clinical significance of a variant, and there are currently no criteria to define a well-established functional study. Despite these challenges, functional studies are known to be valuable for interpretation of variants, and performing these studies is of importance to advance knowledge of hereditary cancer (Amendola et al. 2015; Imyanitov et al. 2004; Richards et al. 2015; Starita et al. 2017).

Study Limitations

The respondents to this survey may represent a skewed sample due to selection bias. Genetic counselors who have seen a variant interpretation discrepancy in practice or who conduct their own research on variants may have been more likely to take the survey.

The survey itself was created by the investigators and was not validated. In addition, some responses had a small sample size, which limited the ability for statistical comparisons between groups. For example, there were a small number of respondents that order syndrome-specific testing most often (n = 4, 2%), which made comparisons between that group and the group that ordered panel testing most often (n = 186, 87%) less robust.

Research Recommendations

These results demonstrate a need for additional resources to aid counselors in addressing discrepancies in variant interpretation. While respondents noted that having a central database owned by a non-profit with variant information from testing laboratories was desirable, additional research could help further define the ideal variant database. This could include informing the creation of a new database or outlining options to consider when updating current databases. As more genes are added to panels, a future study could be conducted to evaluate if counselors indeed see an increase in the number of variant interpretation discrepancies encountered in practice over time. Additionally, assessing patient experiences and perceptions when they are told that their variant is interpreted differently by two laboratories would be beneficial.

Conclusions

This investigation is the first to address genetic counseling practices regarding variant interpretation discrepancies in hereditary cancer and observes that most genetic counselors have seen discrepancies in practice. Genetic counselors desire further resources to aid in addressing these discrepancies, including a centralized database with current and unbiased information, support and data-sharing from the genetic testing laboratories, practice guidelines from a major organization, continuing education opportunities, and functional studies. Ultimately, more support and collaboration is needed to resolve discrepancies in variant interpretation and enhance clinical care.