Introduction

The presence of air or gas in the interpleural space (i.e., the space between the lung and the chest) is called pneumothorax and causes the lung to collapse as a consequence of pressure changes. Although pneumothorax may be acquired as a consequence of an invasive procedure or a trauma, it is commonly spontaneous: primary spontaneous pneumothorax (PSP) occurs in patients without underlying lung disease and secondary spontaneous pneumothorax (SSP) occurs in patients with underlying lung disease. The treatment options and recommendations for PSP are usually related to the degree of lung collapse [1, 2]. Although suspected pneumothorax may be screened using chest X-rays, computed tomography (CT) is a more powerful tool to detect small pneumothorax [3, 4]. In rural teleradiology services and in some urban services, only conventional X-rays are available. Consequently, chest X-rays are still commonly used to quantify the size of the pneumothorax.

Several methods have been formulated for the quantification of pneumothorax in chest X-rays. The Rhea method [5] was proposed to determine the size of pneumothorax in upright patients. The Collins method [4] is similar to the Rhea method, but the pneumothorax size is calculated using a linear function adjusted with CT regression analysis (with the patient lying down). The Light index [1, 6] is a commonly used method in Europe [7]. To date, there is no consensus regarding the best method for pneumothorax size quantification [8–10] or treatment, which is usually based on the size of the pneumothorax [2]. However, there is also a lack of consensus on when a pneumothorax is “small” or “large”. Therefore, rather than evaluating the best method for quantifying the pneumothorax size, the objective of this evaluation was to assess the effects of the radiologist (with different experience), display (medical-grade grayscale display and consumer-grade color display), and display calibration (with or without calibration) on each of the methods.

In digital radiology services, the use of different display types may result in inconsistent image presentation. The use of inconsistent display types may be corrected using the display calibration method proposed in “Part 14: Grayscale Standard Display Function” of the Digital Imaging and Communication in Medicine (DICOM) standard [11, 12]. The calibration method is based on the Barten model [12] to ensure the presentation of images with equal perceived contrast to observers independent of the luminance range of the display. In this study, the observations were performed on each display by using both the DICOM Grayscale Standard Display Function (GSDF) calibration and the factory calibration.

Materials and Methods

Study Sample and Readers

The sample was obtained from our previous studies on film digitizer comparisons [13, 14]. The cases corresponded to digital chest X-rays (computed radiography [CR]) acquired at the Fundación Santa Fe de Bogotá (FSFB), Bogotá, Colombia, between November 2007 and June 2009. Cases were randomly selected without repetition and were included in the sample if a chest CT was available to establish the gold standard and to quantify the pneumothorax size.

This evaluation was part of a larger study that used a treatment-by-reader-by-case factorial design with 76 cases: 16 pneumothorax cases and 51 cases with other conditions included (i.e., nodules, interstitial opacities, or normal cases). For each display (i.e., treatment), the six radiologists (i.e., readers) observed each digitized chest X-ray film (i.e., case). The overall study was a multireader-multicase receiver operating characteristic (ROC) study [15–19]. For the ROC analysis, the following scale was defined: 0, definitely absent; 1, most likely absent; 2, cannot decide; 3, most likely present; and 4, definitely present. When the radiologist marked a case with a ROC score of 3 or 4, the software requests the observer to mark the required points to calculate the pneumothorax size, and the parameters used for calculation were stored in the database. According to the CT pneumothorax quantification, the distribution of cases was as follows: eight cases with pneumothorax size less than 25 %, four cases with pneumothorax size between 25 and 50 %, and one case with pneumothorax size greater than 50 %.

The readers were six hospital radiologists between 2 and 10 years of experience after board certification (two senior faculty radiologists, two junior faculty radiologists, and two fellow radiologists). Observers were selected by the chief of the Department of Radiology, according to their time availability and to balance their years of experience to achieve a moderate variability among observers; readers were treated as fixed effects in statistical analysis.

Observed Variables

In the observation of each case, for a given display with or without GSDF calibration, the radiologist marked specific points on the lung and hemithorax visualization software to calculate the size of pneumothorax. In the Collins method [4], as in the other methods, the size of the pneumothorax is a function of the interpleural distance (ID), which is expressed using the following equation:

$$ {\%}_{\mathrm{Collins}}=4.2+4.7\left(\mathrm{ID}\right), $$
(1)

where ID is calculated as the sum of three distances: the maximum apical interpleural distance (A), the interpleural distance at the midpoint of the upper half of the lung (B), and the interpleural distance at the midpoint of the lower half of the lung (C). Therefore, we used the following equation:

$$ {\%}_{\mathrm{Collins}}=4.2+4.7\left(A+B+C\right), $$
(2)

Using the Rhea method [5], the pneumothorax size is also a function of the interpleural distance. To apply the Rhea method, the average interpleural distance (AID) is used:

$$ \mathrm{AID}=\frac{A+B+C}{3}, $$
(3)

Using the AID, the corresponding pneumothorax size was determined from the nomogram presented by Rhea et al. [5]. To incorporate the nomogram in our software, a regression function was performed. The software enables the pneumothorax size to be calculated using both the Collins and Rhea methods with the same point selected by the radiologist because the two methods are functions of the same points A, B, and C (i.e., both are functions of the AID or the ID). The function used to calculate the Rhea method is given by the following equation:

$$ \begin{array}{l}{\%}_{\mathrm{Rhea}}=5+\frac{35}{4}\left(\mathrm{AID}\right)\hfill \\ {}=5+\frac{35}{12}\left(\mathrm{ID}\right),\hfill \end{array} $$
(4)

which can also be expressed in terms of A, B, and C:

$$ {\%}_{\mathrm{Rhea}}=5+1\frac{35}{12}\left(A+B+C\right), $$
(5)

In the Light index [1, 6], the pneumothorax size is a function of the average diameter of the hemithorax (a) and the average diameter of the lung (b). The average diameter is measured at the midpoint of the upper half of the lung and used in the following equation:

$$ {\%}_{\mathrm{Light}}=100-100\cdot {\left(\frac{b}{a}\right)}^3, $$
(6)

Reading Software

The AndesPACS software, which was developed for our previous study [13], was used to visualize the cases and to enter the observed data. The software allows image manipulation functions to be used according to the reader’s criteria, i.e., filters, zoom, brightness/contrast, window/level, and negative/positive. The radiologists were blinded by the software to each patient’s identity and condition. Tools for measuring the size of the pneumothorax were also included (see Fig. 1). The radiologist marked the points of interest in the lung and hemithorax for each method, and the software calculated and reported the size of the pneumothorax. In Fig. 1, the segments Ra, Rb, and Rc correspond to the interpleural distances A, B, and C used in the Collins and Rhea methods (Eqs. 2 and 5, respectively). The segments La and Lb correspond to the average diameter of the hemithorax (a) and the average diameter of the lung (b) in the Light index (Eq. 6).

Fig. 1
figure 1

Software tools for the quantification of pneumothorax size

Full size image

Displays and Calibration

Three differently priced displays were included in this study to evaluate the effects of the different technologies in the quantification of pneumothorax size. A 3MPx MD213MG (NEC Display Solutions, Tokyo, Japan) medical-grade grayscale display (referred to as 3MP) with a dot pitch of 0.21 mm, a spatial resolution of 2,048 × 1,536 pixels, a maximum luminance of 1,450 cd/m2, a 10-bit grayscale, and a cost of US$ 8,500 was selected. The following consumer-grade displays were used: (1) a Dell UltraSharp U2711 LCD (Dell Computer Corporation, Round Rock, TX, USA) (referred to as LCD) with a dot pitch of 0.23 mm, a spatial resolution of 2,560 × 1,440 pixels, a maximum luminance of 350 cd/m2, and a cost of US$ 862; and (2) a Dell Vostro 3750 laptop computer with an LED display (referred to as LED) with a dot pitch of 0.24 mm, 1,600 × 900 pixels, a maximum luminance of 220 cd/m2, and a cost of US$ 780.

The three displays were calibrated according to the GSDF. A Gossen Mavo-Monitor USB photometer (Gossen Foto- und Lichtmesstechnik GmbH, Germany) was used to determine the characteristic curve of each display. The display contrast and brightness were set before the calibration process by using an RP-133 standard pattern created by the Society of Motion Picture and Television Engineers (SMPTE) [20–23]. The maximum luminance for each display was set to 388 cd/m2 for 3MP, 134.4 cd/m2 for LED, and 178.1 cd/m2 for LCD without ambient light. The reflecting ambient luminance with the display turned off was added to obtain the final characteristic curve (Fig. 2). The final calibration was accomplished by using lookup tables in the visualization software as outlined by Thompson et al. [24].

Fig. 2
figure 2

Display characteristic curves. Solid lines represent characteristic curves without calibration; dashed lines represent transformed characteristic curves with calibration applied

Full size image

X-ray Film Capture

The chest CR cases were printed on 35 × 43-cm films by using an Agfa Drystar 5503 digital film printer (Agfa HealthCare NV, Belgium) at 508-dpi resolution and 14-bit contrast. Each film was digitized in our previous study [20] by using an iCR-612SL film digitizer (iCRcompany, Torrance, CA) at 375 dpi (6,488 × 5,248 matrix) in 8-bit grayscale. The exposure option for “normal” films was selected, and the corresponding images were stored in DICOM format without compression.

Procedure

In each reading session, the radiologist verified the contrast and luminance settings of the display with the RP-133 pattern at a controlled ambient luminosity (20 lx). The images were interpreted over a 6-month period in 4-h sessions by each radiologist. The order of the cases was random for each display, but a 76-case interval was incorporated between two observations of the same case by a radiologist to avoid recall. As a result, it was possible for the radiologist to observe both pathological and normal cases, and the quantification of pneumothorax size was not performed for certain cases (false-negative cases). For this reason, the statistics presented in this paper were obtained only from pneumothorax cases in which the sizes where calculated by the six radiologist, reducing the sample to 13 pneumothorax cases from the original 16 cases in the overall study.

Data Analysis

Statistical analysis was performed using IBM SPSS Statistics 19 software (SPSS Inc., USA) and Stata 12.1 software (Stata Corp., College Station, TX, USA). To evaluate the effects of radiologist, display, and GSDF display calibration (i.e., factory calibration and DICOM GSDF calibration) in the quantification of pneumothorax size, an analysis of variance (ANOVA) with repeated measures was performed using SPSS. To evaluate the effects of the experience of radiologists, repeated measures ANOVA—with radiologist nested in the experience factor—was performed using Stata. All statistics were calculated for a significance level of 0.05.

Results

ANOVAs were performed for the three quantification methods to evaluate the effects of different factors involved in clinical teleradiology or PACS applications, including radiologists, radiologists experience, displays, and display calibration. The mean pneumothorax sizes observed for the display and calibration interactions for each quantification method are shown in Table 1. Similar mean values were obtained within each pneumothorax quantification method. No differences were noted for the calibration (p = 0.072) and display (p = 0.053) factors and for the display and calibration interaction (p = 0.0182) in using both the Rhea and Collins methods. The differences using the Light index were also no significant for the calibration (p = 0.666) and display (p = 0.665) factors, and for the display and calibration interaction (p = 0.712).

Table 1 Pneumothorax size for the display and calibration interactions for each quantification method
Full size table

Using the 1,404 readings, an overall ANOVA was performed including the pneumothorax size quantification method factor in order to evaluate differences between the three methods. The overall mean for the Rhea method, Collins method, and Light Index were 25.1, 36.6, and 40.1, respectively. Although large and significant differences were found among the three quantification methods (p = 0.003), no differences were observed for both the calibration and the display factors, as shown in Table 2. All the paired comparisons between quantification methods were also significant.

Table 2 Pneumothorax size grouped by quantification method, display, and calibration
Full size table

For the three assessed methods (see Table 3), no differences were found for the display factor, the calibration factor, and the interaction between them. In contrast, significant differences were observed among radiologist (p values of 0.039 for Rhea and Collins methods, and 0.046 for the Light index). In regard to the performance of the observers as a function of years of experience, the radiologist factor was nested within the “experience” factor (with two radiologists in each of the three experience levels). None of the experience factors were significant: 0.34 for the Rhea and Collins methods, and 0.51 for the Light index.

Table 3 Analysis of variance for the main factors and their interactions in the three methods
Full size table

Discussion

Quantifying pneumothorax by using the Rhea method results in lower values than those quantified using the Collins method or the Light index. Although differences were observed for individual radiologists, no effects were found for the radiologist experience, the display, the calibration factors, and for the display and calibration interaction, which were the main factors of interest in our study. The results of this study suggest that for the evaluated quantification methods in clinical conditions (with different radiologists using different types of calibrations and displays), the estimation of pneumothorax size is consistent when quantification is performed according to the radiologist’s preferred method.

Both the Collins and Rhea methods are linear functions based on the same parameter (i.e., interpleural distance) and with similar constants and slopes, as noted in Eqs. 2 and 5. Consequently, similar results were obtained for the effects for the evaluated factors in the statistical analysis.

Our findings of lower estimates in using the Rhea method are consistent with the results of Kelly et al. [9] obtained in 57 pneumothorax cases. Kelly et al. obtained lower estimates with the Rhea method than with the Collins method. Noppen et al. [8] performed a study in 18 patients with PSP and observed a strong correlation of the Light index with the amount of air present in the pleural space. In contrast, Hoi et al. reported on lower estimates by using the Light index and poor agreement between the methods in a study performed in 27 patients by using two observers. However, we obtained lower estimates with the Collins method than with the Light index. This discrepancy may have occurred because Hoi et al. selected the Collins method as the “true” estimation size for the subgroup analysis, and the two methods were thus averaged for the overall analysis. Our study was conducted with less pneumothorax cases, but we performed the analysis by using a treatment-by-reader-by-case factorial design with repeated measures with 1,404 estimated pneumothorax sizes, which may provide more accurate results [25, 26], even thought if there were less cases included with respect to the mentioned studies.

Conclusion

This study focused on factors affecting the quantification of pneumothorax size in a digital radiology environment when the pneumothorax size quantification method was selected according to the radiologist’s preferred criteria, rather than evaluate which of the methods was most accurate. The study was performed in conditions very similar to the clinical conditions: cases presented to the radiologists include normal cases and pathologic cases with nonexclusive different conditions present (i.e., nodules, interstitial opacities, and pneumothorax); the software used and the reading luminance conditions were very similar to that used in our hospital. In addition, the radiologists were blinded to the patient’s condition.

We conclude that once the radiologist selects a method to quantify the pneumothorax size, the obtained value is independent of what type of display and calibration is used.

This evaluation was performed from data of a larger study—with a sample including normal cases or other conditions—in which the radiologist were not forced to quantify the pneumothorax size for each observed case (even if the case is a true pneumothorax). As a result, false negatives were produced, and for these readings, missing values were produced (false-positive cases were not included in the data set). These missing values reduced from 1,728 possible pneumothorax quantifications (for 16 pneumothorax cases) to perform the full repeated measure analysis to 1,404, reducing the observed power.