Radiologists communicate their findings through the dictated radiology report. This report’s format has remained essentially unchanged over the past several decades and is usually composed of free-form transcribed dictation organized within a minimalist structure [1, 2]. This includes subdivisions for the type of examination, a brief clinical history, the indication for the exam, findings (the “body” of the report), and a brief impression. The body of such a report typically consists of unstructured, non-standardized paragraphs describing the imaging findings created and organized solely at the individual radiologist’s discretion.

This format has been criticized as being responsible for reports that are inconsistent, unclear, and/or incomplete [2, 3]. These limitations can result in miscommunication of relevant findings, which can negatively impact patient care, and frustrate referring physicians [3, 4]. Such negative outcomes could be even more likely in circumstances where comprehensive, precise, and detailed descriptions are needed with standardized terminology. An example would be pancreatic cancer (PC), where accurate staging for planning chemotherapy, radiation therapy, and/or surgery is particularly dependent on detailed descriptions of the extent of tumor vascular involvement.

An alternative to solve these limitations is that of “structured” reporting. In such circumstances, the radiologist follows a closely defined template, oriented to a particular disease. This guides the radiologist to provide all needed information in sufficient detail and may include instructions on the use of standardized terminology to improve clarity. Organizations such as American Pancreatic Association (APA), the National Comprehensive Cancer Network (NCCN), and the Radiological Society of North America (RSNA) have created reporting templates for several tumors including PC [5–9]. Another benefit of the use of such vetted templates would be to speed the incorporation of new staging criteria into radiology reports.

What is currently not known is to what degree structured reporting would improve radiology reporting over free-form reporting. How complete and precise are free-form radiology reports? Additionally, how quickly are changes in staging or verbiage for a given disease incorporated by radiologists into their free-form reports? Such information would be useful since broad implementation of structured reporting would result in major changes for how radiologists practice.

The aim of our project was to try to answer these questions by retrospectively analyzing the completeness and clarity of free-form radiology CT reports for baseline PC staging. The completeness and clarity of these free-form reports were evaluated by comparing them against the individual elements of the 2009 RSNA primary pancreatic mass radiology template. We chose the 2008–2010 time interval for two reasons. First, we wanted to evaluate the rate of incorporation of new information into free-form reports. Starting in 2006, articles in the surgical oncology literature began recommending the use of standardized terminology for describing vascular involvement to help stratify patients into the categories of resectable, unresectable, and, a new category, “borderline” resectable disease, as a means to choose appropriate therapies [10–13]. By 2008, such verbiage and staging criteria were being used routinely at our institution pancreatic multidisciplinary meetings by surgeons, oncologists, radiation oncologists, and gastroenterologists. Secondly, subsequent to 2010, our institution took initial steps toward creating disease-specific reporting templates which would have confounded our analysis of free-form reports.

Methods

This retrospective study was approved by our institution’s IRB, and a waiver of written consent was granted. One thousand two hundred and ninety dictated reports of CT studies of the abdomen or abdomen and pelvis, performed with pancreas protocol multiphasic CT, for baseline staging of pathologically proven PC, were retrieved by a search of institutional databases and evaluated by one of two radiologists with focused expertise in PC. Nine hundred ninety-five reports were excluded due to verbiage indicating the presence of metastatic disease, leaving 295 reports of potentially resectable or locally advanced pancreatic adenocarcinoma for the analysis. Reports where radiologists indicated the presence of metastatic disease were excluded, because in such circumstances radiologists could justifiably minimize the description of vascular involvement, which would have confounded our study.

These 295 free-form radiology reports were originally generated by either a pancreas specialist (abdominal radiologist with expertise in PC staging and active member of the pancreas multidisciplinary conference in our institution) or a general abdominal radiologist between August 2008 and December 2010. The completeness and clarity of the reports were assessed by comparing them against the elements of the 2009 version of the RSNA CT oncology primary pancreas mass dictation template. The reports were therefore scored for the description of the primary tumor size, location, internal features (i.e., solid, cystic, and mixed), contrast enhancement (hypodense, hyperdense, isodense, etc.), local extension to adjacent organs (duodenum, adrenal, stomach, spleen, and lesser sac), and vascular involvement of the superior mesenteric artery (SMA), celiac axis (CA), hepatic artery (HA), superior mesenteric vein (SMV), portal vein (PV), and splenic vein (SV). The reports were also evaluated for the description of CBD involvement (dilatation), pancreatic ductal involvement, nodal disease (yes or no, and the location such as periportal, celiac, SMA, peripancreatic, and retroperitoneal), and distant metastasis (possible sites), as specified in the RSNA CT oncology primary pancreas mass template with the following modifications.

Regarding the primary mass, an exception was made that reports were scored for whether there was a comment regarding the presence or absence of biliary obstruction rather than a precise description of the level of obstruction. Regarding the description of vascular involvement, the reports were scored regarding whether they used “standardized” terminology or “non-standard” terminology. Standardized terminology was defined as the use of either the term “degrees of involvement” of a given vessel’s circumference or “abutment” (up to 180° contact with the vessel wall circumference) or “encasement” (greater than 180° of tumor direct contact with the vessel wall circumference). These terms are used and defined in NCCN guidelines and RSNA CT 2009 oncology primary pancreas mass template [8, 14]. Any other language was classified as “non-standard.” Vessels not commented on in the report were assumed to be not involved. Regarding nodal involvement, we added scoring for descriptions of adenopathy and locations rather than simply statements of the presence or absence of lymphadenopathy. For distant metastases, scoring was added for comments indicating specifically review of the liver, lung, peritoneum, and/or bone rather than statements indicating only the presence or absence of distant metastases. Our database record form is displayed in Fig. 1, and illustrates how the information in the free-form reports was scored and entered into our database (Fig. 1).

Fig. 1
figure 1

Structured report data entry form

Full size image

Additionally, the study radiologists evaluated and scored the reports for whether they could stage the patient as resectable (this included borderline resectable) or locally advanced based solely on the report, or whether they would have needed to revisit the actual images. The reports were scored as resectable (including borderline resectable) or unresectable based on the extent of vascular involvement defined by relevant articles in the surgical oncology literature and NCCN guidelines, version 2013 [8, 12–15]. Resectable was defined as clear fat planes around the CA, SMA, and HA and no involvement/distortion of the SMV/PV. Borderline resectable was segmental involvement/distortion/occlusion of SMV/PV with intact vessel proximal and distal, allowing for safe vascular resection and graft reconstruction, GDA encasement with either short segment encasement or abutment of the HA without the involvement of the CA and abutment of the SMA. Unresectability criteria included encasement of the SMA and CA. Table 1 summarizes the staging/resectability criteria utilized. The radiologists also scored whether they could comprehensively understand tumor involvement of vasculature based solely on the report. This additional measure was done because a description of tumor involvement sufficient for determining stage might be insufficient for understanding the full extent of disease, e.g., a description indicating tumor encasement of the CA that ambiguously describes SMA involvement would be sufficient to classify a patient as unresectable, but would provide an incomplete picture of the extent of vascular involvement.

Table 1 Resectability criteria
Full size table

Descriptive statistics were used to analyze the frequency of the presence of standard report elements and the presence of other common elements not in the standard report template. Fisher’s exact tests were used to analyze differences in completeness and clarity of the reports by year and type of Radiologist.

Results

Of the 295 radiology reports, 89 (30.2%) had been read by general abdominal radiologists and 206 (69.8%) by pancreas specialists. General radiologists read 44 reports in 2008–2009 and 45 in 2010, and specialists read 81 in 2008–2009 and 125 in 2010. The percentage of reports read by general radiologists vs. pancreas specialists did not change significantly between 2008–2009 and 2010, although specialists read more reports in 2010 than in 2008–2009. A total of 125 reports were read in 2008–2009 and 170 in 2010.

Frequency of reporting of specific pancreatic tumor features

Table 2 shows the frequency of reporting of features of the primary tumor. The location of the tumor was the most commonly reported finding, given in 93.9% (277/295) of reports, while size of primary tumor was noted in 69.8% (206/295). In contrast, imaging features of the primary tumor were given in 27.5% (81/295) of reports. There was no statistically significant difference in the frequency of reporting of various features when comparing 2008–2009 with 2010 except for characterization of the primary tumor (cystic, solid, etc.), which was more likely to be provided in 2010, with 32.5% (55/170) in 2010 vs. 20.8% (26/125) in 2008–2009 (p = 0.0345).

Table 2 Completeness of free-form radiology reports for baseline multiphasic CT for patient presenting with pancreatic cancer
Full size table

Frequency and nature of reporting of details of tumor involvement of vasculature

Table 3 shows the frequency of reporting of arterial involvement using standardized terms. Regarding the involvement of arterial structures, standardized terms (“abutment” or “encasement” or specific degrees of circumferential involvement, i.e., “180°”) were used in 45.8% (135/295) of reports. Non-standard terms, mixed use (standard and non-standard), or no comment of any kind regarding arterial involvement were the case in 13.9% (41/295), 7.8% (23/295), and 9.5% (28/295) of reports, respectively. “Negative” was used in 22.7% (67/295) of reports and one report (0.34%) simply stated “locally advanced.” As shown in Table 3, pancreas specialists were statistically significantly more likely to use standardized terms.

Table 3 Frequency of use of standard terms to describe tumor involvement of vessels
Full size table

Regarding the description of venous structure involvement, standardized terms were provided in 51.2% (151/295) of reports. Non-standard terms, mixed use (standard and non-standard), and no comment were present in 20.3% (60/295), 6.8% (20/295), and 10.8% (32/295) of reports, respectively.

Thirty-nine different non-standard terms were used to describe tumor involvement of vascular structures, with the more commonly encountered variants being “narrowing” (N = 38), “involved” (N = 29), “extends along” (N = 16), “infiltrates” (N = 12), and “compress” (N = 6). Table 4 gives the frequency of each non-standard term.

Table 4 Frequency of other common terms
Full size table

When evaluating changing usage of terminology over time, standardized terminology was statistically significantly more frequent in 2010 (53.5%, 91/170) than in 2008–2009 (39.2%, 49/125) (p = 0.01).

Regarding the description of variant arterial anatomy, 15.6% (46/295) of reports described such a variant, while 76.3% (225/295) of reports had no comment on variant arterial anatomy, and 8.1% (24/295) said “no variant.” Specialists were more likely to note such a variant (p = 0.0004). Overall, there was no statistically significant difference in reporting of such variants when comparing reports from 2008–2009 with those from 2010.

Ability to determine staging or understand comprehensively tumor involvement of vasculature based solely on the free-form radiology report

Utilizing solely the information provided in the text of the radiology report, 61% (180/295) of cases were interpreted by our study’s radiologists as resectable/borderline resectable and 18.6% (55/295) as locally advanced. In 20.3% (60/295) of all reports, the study radiologists could not determine the stage/resectability status based on the free-form radiology report alone. There was no statistically significant difference when analyzing reports from 2008–2009 vs. those from 2010. Pancreas specialists had a statistically significantly smaller percentage of reports that would have required image review (16.5 vs. 29.21% for general radiologists, p = 0.0353).

Additionally, in 36.9% (109/295) of all reports, the study radiologists indicated that they would have to review the actual CT images in order to fully understand tumor vascular involvement, even when assuming that unreported vessels were uninvolved. There was no statistically significant difference when analyzing reports from 2008–2009 vs. those from 2010. Specialists had a statistically significantly smaller number of reports that would require actual image review (23 vs. 46% for general body radiologists, p = 0.0184).

Frequency of reporting of potential metastatic/nodal sites

Table 5 summarizes the frequency that positive or negative comments were made, or whether none was made, for various potential sites of metastatic disease. The frequency with which a comment of some kind (positive or negative) was made for the status of a given possible metastatic site was variable, ranging from approximately 95% for the liver to 40% for the peritoneum and 14% for the lungs. These findings were not significantly different between 2008–2009 and 2010 except for comments regarding the absence or possibility of bone metastases (p = 0.0125), which were less likely to be commented upon in 2010, and the absence of possibility of liver metastases (p = 0.0065), in which radiologists were more likely to say “negative” in 2010. In all cases, although with variable levels of statistical significance, specialists were more likely to comment on the status of a potential metastatic site than general radiologists.

Table 5 Reporting of nodal involvement and presence or absence of metastatic disease
Full size table

With regard to nodal sites, in 94.2% of reports some comment was made regarding whether there was, or was not, possible metastatic nodal disease. In 57% of reports, radiologists provided additional details beyond simply commenting on the presence or absence of adenopathy (e.g., nodal morphology, internal necrosis, and reference to specific nodal basins).

Discussion

We sought in this project to learn the potential value of structured reporting in PC staging by evaluating the quality of conventional free-form radiology reports. Our findings indicate that in up to 20% (60/295) of free-form reports for initial staging of PC, the information provided was insufficient to determine if a patient had resectable (including borderline resectable) or locally advanced disease. Furthermore, in order to fully understand the degree of vascular involvement, review of the CT images was deemed necessary in 36.9% (109/295) of reports.

Our study showed that radiologists with focused expertise in PC performed better than general radiologists in terms of defining stage and resectability. Specialists had a higher proportion of reports in which the resectability status could be clearly defined as resectable or locally advanced by the report alone (p = 0.03). They also had a statistically significantly higher proportion of reports in which the degree of vascular involvement could be understood without revisiting the CT images (p = 0.01). We believe that this was because of their greater expertise and more frequent communication with surgeons/oncologists. Also, the content of their reports had greater clarity for referring clinicians, since pancreas specialists used standardized terms more frequently than general radiologists (p = 0.0004). These findings suggest that channeling more of these cases to specialists may improve the overall quality of radiology reports, providing a more complete and clear description of relevant imaging findings. However, it is important to consider that such an approach may be challenging when simultaneously trying to produce reports in a very timely manner, since subspecialists may not always available in all practices in different institutions. Furthermore, even though pancreas specialists performed better than general radiologists, our data showed that in approximately 16% (34/206) of pancreas specialists reports, the resectability status could not be defined based on the report alone, but required the review of the CT images.

Our data also showed that reports in 2010 had a greater proportion of standardized terms than those in 2008–2009 (p = 0.04). We interpret the greater use of standard terminology in 2010 to indicate that it takes time for new knowledge and/or terminology to be incorporated into the daily clinical practice of radiologists. We would note, however, that even though there was improvement in 2010, the improvement was moderate and even specialists may have benefited from a structured template.

Based on our findings, we believe that adherence to a structured template for PC staging would improve the completeness and clarity of radiology reports even among pancreas specialists, and we speculate that it would also help disseminate crucial concepts and terminology more quickly. These templates serve as a “checklist” for essential items that must be included in a report to ensure completeness. They also promote the use of a standardized vocabulary to avoid ambiguity and ensure clarity, improving their clinical utility. This practice could simplify patient stratification for trials, treatment planning, and communication with referring clinicians and, ultimately, improve patient care. Standardized reporting could also potentially limit unnecessary repeat CT imaging as patients move between institutions.

Our findings are in agreement with the existing literature that highlights the potential benefits of a structured reporting system. In a recent study, Schwartz et al. evaluated the content, clarity, and clinical usefulness of 330 randomly selected, free-form, and structured radiology reports of body computed tomography and concluded that structured reports were superior to free-form radiology reports in both content and clarity [3]. Sundaram et al. concluded that the use of structured reporting in cardiac CT is a valuable tool to help radiologists communicate their findings with clarity and completeness, consistently generating comprehensive reports [16]. Steele et al. found that radiology residents felt the need for a more structured approach to learning how to effectively report radiological findings [4]. Brook et al. investigated CT structured reporting for pancreatic cancer and concluded that structured reports outperformed free-form reports for staging, surgical planning, and resectability evaluation [9]. We would note that a novel feature of our study was the evaluation of “general” radiologists vs. pancreas specialists. It is probable that such “specialists” would do better, but direct evaluation of their reporting has not been previously described to provide some measures as to whether specialists would also benefit from template reporting. We conclude here that they likely would.

Our findings suggest that the benefits of adopting structured reporting for pancreatic cancer staging outweigh the potential disadvantages of implementing such practice such as overcoming radiologists’ resistance to change, difficulty in adjusting to the rigidity of form of structured templates, possible lack of clinical productivity, and the inherent challenges of making such a practice uniform across different institutions.

We would also note that our study had several limitations. First, it was a retrospective chart review and due to its nature subject to selection and information bias. Second, we reviewed only the written report, and not the images themselves. For this reason, we cannot comment on the accuracy of the reading radiologist’s review of the images. Third, we assumed that vessels not commented on in the report were not involved by tumor, which may not have been the case. Lastly, the RSNA CT Oncology Primary Pancreas Mass template created in December 2009 was updated in July 2012, which is a more detailed template. We did not use the updated version since it was not available at the time our study was started. However, we would note that we added scoring for nodal disease description and anatomic specification of sites of metastatic disease, to provide a more comprehensive assessment of staging descriptors than those terms used in the 2009 RSNA Template.

In conclusion, we believe that our study shows limitations for free-from CT radiology reporting for PC staging in regard to completeness and clarity. We believe that adherence to a structured report template would have improved these radiology reports by ensuring completeness and promoting the use of a standardized terminology. We believe that such an approach would make radiology reports much more clinically useful for referring clinicians involved in the care of PC patients and could reduce the need for repeat examinations as patients move between institutions.