Introduction

Requests for second-opinion consultation by subspecialty radiologists is common in radiology departments across the country, particularly in tertiary and quaternary care referral centers [1,2,3,4,5,6]. Although referred patients arrive with reports for imaging studies performed at a variety of outside centers, many clinicians continue to request formal second-opinion interpretations from their institution’s subspecialty radiologists. Second opinions have always been in place in diagnostic radiology departments; however, given the increase in complexity and the number of images in cross-sectional studies, many departments are adapting the practice of the official interpretation of secondary studies [1,2,3,4,5,6]. With increasing emphasis on reducing unnecessary repeat studies and ultimately curbing the cost of imaging, formal policies regarding second-opinion consultation continue to be defined [7].

Multiple studies have published discrepancy rates as high as 25% between radiologists with different levels of training, including radiologists trained in musculoskeletal imaging [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28]; however, no studies specifically address the factors that may lead to discrepancies between second-opinion and original radiology interpretations. The purpose of our investigation is to determine factors that may lead to significant discrepancies in second-opinion consultation of orthopedic oncology patients, and particularly if musculoskeletal fellowship training can decrease clinically significant discrepancies.

Materials and methods

Institutional review board approval

The study protocol was approved by the institutional review board and informed consent was waived because of the retrospective nature of this study. Patient confidentiality was maintained in accordance with HIPAA guidelines.

Patient population

An institutional database was searched for secondary interpretations placed on outside cross-sectional imaging studies, as requested by an orthopedic oncology service, from January 2014 to December 2017. The appropriate consecutive patients were identified using a radiology database search engine, Primordial (Primordial Inc., San Mateo, CA, USA). Patients were included in the analysis if the original radiology report was available at the time of the secondary interpretation. Additionally, electronic medical records were reviewed to document the patient’s demographic information and indication for the original imaging study.

All subjects had second-opinion interpretations performed by one of seven fellowship-trained musculoskeletal radiologists with 25, 22, 15, 9, 7, 4, and 2 years of experience, after review with musculoskeletal radiology fellows and the orthopedic oncology clinical team (surgeon, non-operative orthopedic physician, physician assistant, and orthopedic surgery house-staff) at a multidisciplinary conference. All secondary consults referred by the orthopedic oncology service are reviewed at the weekly multidisciplinary conference, where each case is presented by the orthopedic oncology team. The musculoskeletal attending radiologists, musculoskeletal radiology fellows, and the orthopedic oncology team review all relevant imaging studies and discuss diagnostic considerations and treatment options. After a consensus is reached, a formal consultation report is generated for every case, which describes the imaging findings, differential diagnosis, and diagnostic or treatment recommendations.

Comparison of primary and secondary reports was performed using a previously published seven-point scale [10]: I (no discrepancy), II (failure to detect a clinically insignificant abnormality), III (clinically insignificant difference in interpretation), IV (difference in imaging follow-up recommendation), V (equivocal initial interpretation with subsequent definitive subspecialty interpretation), VI (clinically significant difference in interpretation), and VII (failure to detect a clinically significant abnormality). Clinical significance was defined by whether a discrepant interpretation resulted in a change in diagnosis (e.g., benign to malignant), change in treatment (e.g., surgical to medical), or a change in follow-up (e.g., further imaging or no additional follow-up necessary).

An online search was also performed for each primary radiologist to record if a fellowship in musculoskeletal imaging was completed and document years of post-residency experience. Additionally, primary radiologists were characterized as private or academic, with academic radiologists defined as those working in an institution with an accredited residency or fellowship program. As a marker of practice volume, the number of Medicare patients billed in 2015 was recorded using the 2015 Medicare Provider Utilization and Payment Data [29]. As a marker of practice complexity, the average hierarchical condition category (HCC) for each radiologist was recorded, also using 2015 Medicare Provider Utilization and Payment Data. HCC is an index reflecting Medicare beneficiaries’ number and the severity of comorbidities and is used by Medicare as a marker of patient complexity [30].

Statistical analysis

Version 9.4 of SAS software (SAS Institute, Cary, NC, USA), was used for data analysis and statistics. Multivariate logistic regression analysis was performed with adjustments using both patient characteristics (age, sex, anatomical body part imaged) and radiologist characteristics (completion of musculoskeletal fellowship, years of experience post-residency, academic or private setting, average hierarchical condition category, and number of unique Medicare patients billed in 2015). p values less than 0.05 are considered statistically significant.

Results

A total of 933 patients had secondary interpretations of cross-sectional studies requested by the orthopedic oncology service, from January 2014 to June 2017. Three hundred and sixty-two cases (38.8%) were excluded from the study, as 359 had no primary reports available at the time of the secondary interpretation and an additional 3 studies were deemed to be technically inadequate. Five hundred and seventy-one subjects (61.2%) had primary reports available and are included in the analysis of the results.

The average subject age was 48.6 ± 18.9 years and the most common indication for initial imaging was “pain” (64%). Two hundred and seventy-four subjects (47.9%) were male and 297 subjects (52.0%) were female. Of the 571 subjects included, 184 cases were initially interpreted by an outside fellowship trained musculoskeletal (MSK) radiologist and 387 cases were initially interpreted by a non-MSK fellowship trained (non-MSK) radiologist. There were 236 unique radiologists that generated the original interpretations (average of 2.42 cases per radiologist) and no single radiologist interpreted more than 7 unique cases. In the group of MSK radiologists, 16.7% worked in academic settings compared with 20.5% of the non-MSK radiologists (p > 0.05).

The rate of clinically significant discrepancy (categories VI and VII) was 9.2% when initially interpreted by an MSK radiologist compared with 27.9% when initially performed by a non-MSK radiologist (p < 0.05). Table 1 demonstrates the number of patients within each discrepancy category. When the initial interpretation was performed by an MSK-trained radiologist, the rate of clinically significant discrepancy was 9.9% for private practice MSK radiologists compared with 5.8% for academic MSK radiologists (p = 0.45).

Table 1 Comparison of discrepancies: initially interpreted by musculoskeletal (MSK) vs non-MSK radiologist
Full size table

An example of a category VI discrepancy was an initial interpretation of a lower extremity mass that was thought to be consistent with hematoma; however, secondary interpretation raised the suspicion of a soft-tissue malignancy, which was eventually biopsied and proven to be an undifferentiated pleomorphic sarcoma (Fig. 1). Category VII discrepancies were less common, and include a case in which the primary interpretation failed to detect an incomplete fracture and incorrectly attributed the resultant bone marrow edema to an infiltrative lesion (Fig. 2).

Fig. 1
figure 1

A 66-year-old woman with a right shin mass. Axial T2-weighted fat-saturated and axial T1-weighted images demonstrate a T2 hyperintense heterogeneous mass abutting the anterior cortex of the tibial diaphysis with fluid extending along the superficial fascia, pathologically proven to be a pleomorphic sarcoma. Outside interpretation incorrectly concluded that the mass was consistent with a hematoma

Full size image
Fig. 2
figure 2

A 69-year-old woman with right hip pain. Coronal TI-weighted and coronal STIR images demonstrate marrow edema within the right ilium, and an incomplete fracture line along the lateral right iliac cortex (white arrow) with surrounding soft-tissue edema. Outside interpretation failed to identify the fracture line and incorrectly attributed the resultant bone marrow edema to an infiltrative lesion

Full size image

The average post-residency experience for MSK radiologists was 10.6 ± 6.1 years compared with 19.1 ± 8.2 years for non-MSK radiologists (p < 0.05). There was no statistically significant difference in the average number of Medicare patients billed per year for MSK radiologists (2697 ± 43) compared with non-MSK radiologists (2,380 ± 398, p > 0.05). Additionally, there was no statistically significant difference in the average HCC for MSK radiologists (1.72 ± 0.04) compared with non-MSK radiologists (1.68 ± 0.03, p > 0.05).

Pathological results were available in 128 of the 571 cases (22.4%) with concordance of secondary consultations in 119 out of 128 cases (92.9%). Outside MSK radiology interpretations were concordant in 34 out of 41 cases (82.9%), whereas outside non-MSK radiology interpretations were concordant in 52 out of 87 cases (59.7%, p < 0.05). Concordance was defined as diagnostic agreement between radiology and pathology for benign, malignant, or infectious lesions.

In the multivariate logistic regression analysis, after adjustment by both patient characteristics (age, sex, anatomical body part imaged) and radiologist characteristics (completion of musculoskeletal fellowship, years of experience post-residency, academic or private setting, average hierarchical condition category, and number of unique Medicare patients billed in 2015), the likelihood of clinically significant discrepancies were greater for primary interpretations by non-MSK radiologists compared with MSK radiologists (OR = 1.36; 95% CI = 1.23–2.49).

Discussion

Given that large discrepancy rates can have significant morbidity in oncology patients, we sought to analyze factors that may lead to radiological discrepancies in second-opinion consultations, and specifically if musculoskeletal fellowship training can decrease clinically significant discrepancies. The rate of clinically significant discrepancies in orthopedic oncology was 27.9% when initially interpreted by a non-MSK radiologist compared with 9.2% when initially interpreted by an outside MSK radiologist (p < 0.05). When the initial interpretation was performed by an MSK-trained radiologist, the rate of clinically significant discrepancy was 9.9% for private practice MSK radiologists compared with 5.8% for academic MSK radiologists (p = 0.45). Although the difference in discrepancy rates between outside MSK radiologists was not statistically significant, the result may be due to a small sample size of outside academic interpretations (34 cases) that went on to acquire second-opinion interpretations from our institution’s radiologists.

In the cohort of primary radiologists, 33% cases were initially interpreted by an outside fellowship-trained MSK radiologist and 67% of cases were interpreted by a radiologist who had not completed an MSK fellowship. After adjustment of both patient and radiologist characteristics, multivariate logistic regression analysis shows that the likelihood of clinically significant discrepancies was greater for initial interpretations by non-MSK radiologists compared with MSK radiologists (OR = 1.36; 95% CI = 1.23–2.49). The results indicate that subspecialty-trained radiologists can help to generate more accurate diagnoses and ultimately lead to a lower rate of clinically significant discrepancies. Using publicly available information online, further analysis shows that 98% of all practices in this study employed an MSK fellowship-trained radiologist. Given that nearly all the practices had an MSK specialist available, the large number of non-MSK radiologists interpreting orthopedic oncology cases may reflect modern work practices and the pressure to get through cases as quickly as possible. However, a high rate of discrepancy by non-MSK radiologists is not surprising given the challenges in interpreting complex musculoskeletal studies with limited training.

Numerous published manuscripts have assessed interpretational discrepancy rates among radiologists [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28]; however, only two previous studies have evaluated the rate of discrepancies in orthopedic oncology [9, 10]. Specifically, the first study [9] demonstrated a 36.3% rate of discrepant interpretations in orthopedic oncology patients and the second study [10] showed a 22.2% rate of clinically significant discrepancies. When a definitive pathological diagnosis was possible, the second-opinion consultations were accurate in 82–93% of examinations compared with 64% of cases interpreted by the original radiologist, similar to our results [9, 10]. Although both studies concluded that patients would benefit from second-opinion consultations, no published research to date has evaluated the factors that may lead to discrepant reads. By considering the characteristics of the original radiologists, our analysis identifies fellowship training as the most essential aspect of providing accurate interpretations for orthopedic oncology cases. Additionally, the data support the sentiment that fellowship-trained radiologists may be relied upon in complex oncology cases, even when these cases are interpreted outside of tertiary or quaternary referral centers.

Several limitations must be acknowledged in this single-center retrospective study. The analysis only includes patients who had outside reports available at the time of interpretation, introducing the potential for selection bias. Next, we relied on the multidisciplinary consensus diagnosis to determine the accuracy of the original interpretation, as pathological diagnosis was not available in many cases. Although it is possible that the consensus diagnosis may be incorrect, previous studies have shown that second-opinion interpretations for orthopedic oncology were accurate in up to 93% of examinations when a pathological diagnosis was available [9, 10]. Additionally, as both MSK and non-MSK radiologists were compared with the same set of multidisciplinary subspecialists, we believe that the consensus diagnosis is a reasonable substitute for pathological confirmation. Finally, Medicare data were used to compare practice volume and practice complexity, which may not be generalizable to radiologists with a different distribution of patients.

Conclusion

By completing formal fellowship training, radiologists have an opportunity to use their extra training to add valuable insight into the diagnostic work-up and potentially improve patient outcomes. The results suggest that even in practice settings outside tertiary referral centers, fellowship-trained radiologists generate a lower discrepancy rate in an orthopedic oncological patient population. Specifically, with accurate interpretation or appropriate referral, subspecialty radiologists can prevent unnecessary invasive interventions or be the first to suggest more aggressive therapeutic procedures.