Introduction

A voiding cystourethrogram (VCUG) is often part of the imaging evaluation for children with urinary tract infection (UTI) or prenatal hydronephrosis. The VCUG has the potential to provide a tremendous amount of information about the anatomical and functional status of the urinary tract, including identification of vesicoureteral reflux (VUR). One might expect that certain elements of the report would be consistently reported in all cases. However, our clinical experience has been that there is substantial variability both within and among institutions, which suggests that there is little standardization of the information documented in these reports.

Inconsistent reporting of findings is a common problem in radiology  —  reports are not all created equal. Studies have demonstrated wide variation in reporting standards for procedures as diverse as chest radiography, abdominal CT, mammography and sonography [1–4]. Variations occur in description of the technique, clinical findings, terminology, final interpretations, and recommendations for patient management or further imaging. Itemized reports or templates have been proposed to address such inconsistencies [5, 6], including a VCUG report template put forth by the Radiological Society of North America (RSNA) [7]. However the use of such tools regarding the consistency of VCUG reports in clinical practice has not been investigated.

We sought to assess the completeness of VCUG reports arising from one of three types of institutions: free-standing pediatric hospitals, pediatric hospitals within a hospital, and non-pediatric facilities. Every report was assessed against a set of standard demographic, technical, anatomical and functional parameters that should ideally be components of every VCUG report. We hypothesized that there would be significant variation in the completeness of VCUG reports comparing free-standing pediatric hospitals to other institutions.

Materials and methods

Standardized assessment tool

Pediatric radiologists and urologists from Hospital A created a standardized assessment tool used to measure the completeness of VCUG reports. This instrument incorporated 26 variables related to demographics (e.g., age, gender, study indication), technical information (e.g., catheterization technique with or without subsequent urine culture, single or multiple voiding phases), anatomical information (e.g., bladder capacity, International Reflux Study reflux grade, renal collecting system duplication) and functional information (e.g., bladder emptying) (see Appendix). Reports from all facilities were compared to this ideal standard.

Selection of cases and controls

The aim of this study was to compare the documentation of VCUG report findings among free-standing pediatric hospitals, pediatric hospitals within a larger general hospital, and non-pediatric facilities using our assessment tool. Our hospital (Hospital A, a free-standing pediatric hospital with a dedicated pediatric genitourinary radiology team) was used as the control group. We determined a priori that the study period would be Jan. 1, 2007, through Dec. 31, 2011. After approval from the institutional review board, we identified reports of VCUGs performed at outside institutions during the study period for patients referred to Hospital A for care; these reports were identified according to the Health Insurance Portability and Accountability Act through a retrospective review of medical records and our VUR and prenatal hydronephrosis databases. We similarly used radiology billing and medical records to identify all VCUG studies performed at Hospital A during the same period. A matched cohort was identified to control for specific patient characteristics and study findings that impact VCUG technique and subsequent reporting, because such factors may differ systematically among institutions. Age (<12 months or ≥12 months) was matched because infants often have cyclic VCUGs [8], which increases the complexity of VCUG reporting. We also matched on gender (because of differences in urethral anatomy), study indication (UTI vs. prenatal hydronephrosis), and presence or absence of VUR (reports from studies with VUR obviously include information that does not appear in studies without VUR, e.g., VUR grade). After categorizing the reports according to these criteria, each group of outside VCUG reports was matched with a twice-larger group of internal (Hospital A) VCUG reports randomly selected from all internal reports with similar values for matching criteria. There were 152 outside VCUG reports and 304 internal (Hospital A) VCUG reports for analysis (total: 456 reports).

Categorization of institutions and radiologists

The Children’s Hospital Association (CHA) Web site was used to categorize the hospital where each VCUG originated as a free-standing pediatric hospital (n = 3), a pediatric specialty hospital or a pediatric hospital within a larger general hospital (n = 11) [9]. If an institution was not present on the CHA Web site it was classified as a non-pediatric facility (n = 24). Another potentially significant contributor to the VCUG report quality is whether the report was issued by a pediatric radiologist or non-pediatric radiologist. The Check Board Certification tool within the American Board of Radiology Web site was used to identify whether the attending radiologist had pediatric subspecialty certification [10], in which case he or she was classified as a pediatric radiologist. For those without pediatric subspecialty certification and those not identified by the ABR tool, the Web site of the radiologist’s employer was searched to identify whether the radiologist had completed a pediatric radiology fellowship or his or her research and clinical focus was dedicated solely to pediatric radiology. This ensured that radiologists exempted from pediatric subspecialty certification were classified appropriately.

Statistical analysis

Each VCUG report was assessed with the standardized assessment tool. The reports were abstracted by a single reviewer for consistency. For quality control a 10% subset of reports was reviewed by a second reviewer and the results compared; concordance was identified for more than 99% of items. Using the quality measurement tool, one point was assigned for each item on the tool, for a maximum total of 26 points (20 points for reports in children without VUR). Each item was given equal weight. A score was assigned by dividing the number of points achieved for each report by the total number of possible points (20 or 26). This score represented the percentage of variables assessed in each VCUG report and was the primary outcome.

Descriptive statistics were used to display relationships between significant hospital and provider characteristics and the percentage of variables assessed. A multivariate linear regression was developed to analyze the association between the outcome and the type of institution where the report was generated and whether the report was read by a pediatric radiologist. Matching characteristics were controlled for in the model. Generalized estimating equations were used to account for correlations among reports generated within the same institution. The functional form of the outcome was investigated, and diagnostic checks and sensitivity to outliers of the fitted model were performed to verify model assumptions. All statistical tests were two-sided, and P < 0.05 was considered significant. SASTM software v. 9.2 (SAS Institute Inc., Cary, NC) was used for the analysis.

Results

Table 1 displays the characteristics of the 152 outside hospital reports and 304 internal (Hospital A) VCUG reports, each categorized according to hospital type.

Table 1 Characteristics of VCUG studies
Full size table

There was wide variation in specific VCUG report items (Table 2). No report assessed all items. Overall, the mean (SD) percentage of items assessed was 67+/−14%. Demographic characteristics were universally reported, but many other items were reported very infrequently, including clinical factors like collecting system duplication status and whether the VUR occurred during the filling or voiding phases of micturition. Although current protocols emphasize the importance of minimizing radiation exposure [11], less than half of all VCUGs (58% of free-standing pediatric hospitals, 14% of pediatric hospitals within a general hospital, and 2% of non-pediatric facilities) reported the radiation exposure associated with the procedure.

Table 2 How frequently was each element of the VCUG assessed and reported?
Full size table

In univariate analyses, we observed a substantial difference in VCUG report quality among institutions; free-standing pediatric hospitals assessed 74+/−7% of items compared to pediatric hospitals within a general hospital, which assessed 61+/−10% of items (P < 0.0001), and non-pediatric facilities, which assessed 48+/−11% of items (P < 0.0001). The average number of variables assessed by a pediatric radiologist was 72+/−9% compared to 46+/−10% when the report was read by a non-pediatric radiologist, P = 0.02 (Table 3). After controlling for the type of radiologist reading the report, gender, study indication, age at time of VCUG, and whether the report was positive or negative for VUR, a report generated at a non-pediatric facility compared to one generated at a free-standing pediatric hospital had on average 17% (95% CI: 14.5–19.7%) fewer variables assessed, P < 0.0001 (Table 4). Similarly, a report generated at a pediatric hospital within a hospital compared to one generated at a free-standing pediatric hospital had on average 9% (95% CI: 6–12.5%) fewer variables assessed, P < 0.0001. Finally, a report that was read by a pediatric radiologist compared to one read by a non-pediatric radiologist had on average 12.1% (95% CI: 9.1–15.3%) more variables assessed, P < 0.0001.

Table 3 Univariate associations between percentage of items assessed in VCUG reports and the type of hospital and report characteristics
Full size table
Table 4 Multivariate analysis of the association between the percentage of items assessed in VCUG reports and the type of hospital and report characteristics
Full size table

Discussion

We observed great variation among facilities in the completeness of VCUG reports, with a higher proportion of complete reports being observed at pediatric specialty centers. If the primary aim of the VCUG is to simply assess for the presence or absence of VUR, all centers performed exceedingly well, with 100% reporting of this parameter. However, the VCUG is a study that is able to assess much more than the presence or absence of VUR. If assessed and subsequently reported in the VCUG, a Christmas tree-shape bladder with bladder diverticula, a low bladder capacity, and a funneled urethra could indicate a neurogenic bladder. Similarly a dilated prostatic urethra could suggest posterior urethral valves as the reason for secondary vesicoureteral reflux. In cases where VUR is identified, important information such as the reflux grade, the bladder volume at the onset of reflux, whether the reflux occurred during the filling or voiding phases of the study, and an assessment of the presence or absence of periureteral diverticula provide important prognostic information to guide the clinician’s counseling and management plan [12–15]. Here we show that a significant number of all VCUG reports lack these important anatomical and functional assessments but that reports generated at free-standing pediatric hospitals or pediatric hospitals within a general hospital had more complete reports than those generated at non-pediatric institutions, as did those read by a pediatric radiologist.

Several reasons could explain the large variation in VCUG reporting. Multidisciplinary conferences at pediatric tertiary centers may facilitate communication and collaboration between pediatric radiologists and pediatric urologists, thus helping radiologists understand the type of information of greatest importance to other clinicians. Moreover, these hospitals likely have a higher volume of these procedures, which increases the radiologists’ familiarity with the technique and the important non-reflux-related anatomical and functional components of the VCUG report.

The frequency of omissions in key components of the VCUG at all three types of institutions in this study demonstrates that there is room for improvement at all institutions. Items consistently omitted from reports included whether a cyclic study was performed, which can improve the diagnostic accuracy of a VCUG [8], and whether the VUR occurred during the filling or voiding phase, which can help prognostication [14]. Another potential goal for improved reporting at all institutions is the description of the collecting system as single or duplicated (among patients with VUR). Should definitive management be necessary, knowing whether the collecting system is duplicated has implications for the pediatric urologist’s surgical approach. Even in cases where a cyclic study was performed or the phase at which VUR first occurred was noted or the duplication status was assessed, failure to include these details explicitly in the report forces the clinician to guess as to whether the omission means that the finding was not present or was not assessed. This is a clear opportunity for quality improvement.

These findings are relevant within the broader context of the health care system because payers such as insurance companies increasingly treat radiology as a commodity. This is based on the premise that for a particular diagnostic study comparable reports can be generated at any facility. Payers therefore incentivize patients to undergo diagnostic imaging at lower-cost general radiology facilities instead of specialized tertiary-care centers [16, 17]. The current findings suggest that, given the wide variation in reporting, the decline in quality associated with such a strategy is measureable and may be substantial. The completeness of VCUG reports as measured by content was significantly higher at pediatric hospitals and when pediatric radiologists generated the reports. Thus the adoption and promulgation of standardized reporting templates, in addition to advancing knowledge in the nuances of pediatric radiology techniques at non-pediatric facilities, could improve the quality of VCUG reports and possibly the quality of the study itself. Until these measures are undertaken, pediatric patients may benefit from having studies performed at high-volume specialized pediatric centers.

The low rate of reporting of radiation exposure is concerning for a number of reasons. Medical radiation exposure among children is increasingly recognized as a significant public health issue [18, 19]. There is evidence that simply reporting exposure levels during diagnostic imaging procedures can result in lowering of subsequent exposure levels [20], perhaps related to a so-called Hawthorne effect, wherein the act of measuring the parameter leads to changes in behavior among the observed [21]. Although in this study we measured only the reporting of radiation exposure, and not the levels themselves, other investigators have found that imaging equipment at general adult and even some pediatric facilities may not be routinely adjusted for pediatric patients or different pediatric age groups, resulting in higher-than-necessary exposure levels [18, 19, 22, 23].

Furthermore an incomplete VCUG report can lead to increased radiation exposure by requiring repeat imaging. Unlike CT scans and MRI studies in which all of the imaging data are recorded (and all images can be reexamined or reinterpreted), most radiology departments only store a few representative VCUG images at study completion. The report is therefore the only opportunity for complete documentation of the observed findings, other than the selected saved images themselves. For example, if a VCUG report for an infant boy does not include a statement regarding the urethra, and urethral images were not saved, then a repeat study may be required, thereby exposing the child to unnecessary radiation. Thus strategies to improve the completeness of VCUG reports could not only help the radiographic assessment of the child with VUR but also improve the quality of the report and reduce the harm from diagnostic radiation in these children.

The results of this study should be interpreted in light of its limitations. Most significant among these is the relatively selective nature of the VCUG reports that were available for review. The great majority of pediatric hospital reports came from our own facility, while the outside reports came from a more diverse set of facilities. This raises issues of selection, particularly if the outside studies are more likely to have certain characteristics, such as presence of VUR. We attempted to address this by explicitly matching for several characteristics that appeared to be differentially distributed between outside and internal (Hospital A) reports and by using generalized estimating equations to account for the correlation in reports generated at particular hospitals. Another limitation is that the 26-item quality-assessment tool was devised by our own pediatric radiologists and pediatric urologists, based on technical and clinical factors that we thought important in a VCUG report. The choice of items therefore reflects the clinical biases and preferences of the study authors and their colleagues. While these decisions were somewhat subjective, the included items were not chosen simply because they were routinely reported at our institution; indeed, it turns out that our institution performed poorly for several items, including catheter size, bladder emptying during voiding, the phase of VUR onset and other items. Also, in requiring key items not included in the RSNA template [7], the tool used in this study is more stringent. For example, our tool assessed whether the International Reflux Study VUR grade, the phase of VUR onset (filling or voiding), and upper tract drainage after voiding were reported, among others. Even given these differences in report measurement tools, the discrepancies in reporting between free-standing pediatric hospitals and non-pediatric institutions are of such large magnitude that changes in the quality assessment tool are not likely to substantially change the results. Although inclusion of items queried at our study may lead to a longer VCUG report, it is our opinion that they also produce a report that provides a more complete picture of the anatomical and functional characteristics of the upper and lower urinary tracts. The aim of this study was not to devise an ideal VCUG report template but rather to highlight the variation in VCUG reporting among institutions. Such a task should be left to a consensus decision among key stakeholders from the radiologic, nephrourologic and general pediatric communities. Finally, this study focused on the content of the VCUG report itself, not on the accuracy or reliability of the clinical findings. We cannot state whether the findings of the diagnostic study (e.g., VUR grade) were more or less accurate at certain facilities compared to others. However, clinical accuracy has been studied extensively within the radiology literature, and substantial variation has been observed among institutions in numerous settings [24–27]; there is little reason to expect different results in the VCUG context.

Conclusion

Documentation of VCUG findings varies significantly, with more complete reports observed at pediatric centers. Quality improvement strategies such as template-driven VCUG reports could help to improve the completeness of VCUG reports, regardless of facility. In the absence of such systematic change, our findings suggest that the quality and completeness of VCUG reports is significantly dependent on both the interpreting radiologist and the center at which the study is performed.