1 Introduction

Prior to treatment, patient-specific quality assurance (QA) is the identification of any potential errors in the treatment planning and delivery processes to ensure accurate treatment for intensity-modulated radiation therapy (IMRT) and volumetric-modulated arc therapy (VMAT). This pretreatment patient-specific QA consist of checking for agreement between the actual distribution delivered by the linear accelerator and the planned dose distribution. Various measurement-based QA techniques using two-dimensional (2D) diode, ion chamber array, radiographic film, and electronic portal image device (EPID) have been utilized for pretreatment patient-specific QA [1,2,3,4]. In general, the dose distribution analysis is evaluated using the gamma index method between the dose distribution measured using a measurement devices and the predicted dose distribution derived from treatment planning system (TPS) [5]. Portal dosimetry is a method of measuring the fluence map using the EPID integrated to the linear accelerator (LINAC) and comparing it with the predicted dose using the portal dose image prediction (PDIP, Ver. 13.7, Varian Medical Systems, Palo Alto, CA) algorithm. Patient-specific QA procedures for 2D ion-chamber array detector and portal dosimetry are similar [6]. ArcCHECK™ (Sun Nuclear Corporation, Melbourne, FL) with helical diode arrays is essentially invariant with the gantry angle due to detector arrangement. Li et al. reported that ArcCHECK™ for IMRT/VMAT patient-specific QA was sufficient to evaluate the dose response to changes in repetition rate, monitor unit (MU), and field size [7]. Recently, software-based patient-specific QA systems with independent dose calculation algorithms have been commercialized, such as Mobius3D™ (Varian Medical Systems, Palo Alto, CA) and SunCHECK™ (Sun Nuclear Corporation, Melbourne, FL). Lee et al. reported the dosimetric performance of Mobius3D™ for patient-specific prescription dose verification compared to other QA systems [8]. They concluded that Mobius3D™ could be used interchangeably with phantom-based dosimetry systems commonly used as patient-specific prescription dose verification tools. Another system, SunCHECK™ developed by Sun Nuclear Corporation, has different types of patient-specific QA methods, such as 2D dose analysis using EPID (2D PerFRACTION™) and 3D dose analysis using log files (3D PerFRACTION™). The 2D analysis of SunCHECK™ is similar to the PDIP provided by Varian, but the 3D analysis can reconstruct the delivered patient 3D dose using a dose calculation algorithm independent of the treatment log file. A previous publication have described the sensitivity of the PerFRACTION™ to detect a variety of errors including dose, multi-leaf collimator (MLC) position, intensity modulation, and position of a movable couch rail [9]. Sait et al. reported a study on the validation of PerFRACTION™ software based on a 3D EPID for patient-specific QA, and they concluded that with PerFRACTION™, actual treatment quality could be determined in relation to machine, attachment, patient, and setup variations arising in practice [10].

In this study, we compared patient-specific QA results with other QA systems for the clinical utility of PerFRACTION™ for various treatment sites. To evaluate the QA performance of PerFRACTION™, the correlation trend between the plan complexity and the gamma passing rate (GPR) obtained using the PerFRACTION™ were also evaluated and compared with those for other QA systems. To the best of my knowledge, this study is the first to focus on a comparison of correlations between GPRs and plan complexity indices as well as gamma analysis using commercial PerFRACTION™.

2 Materials and methods

2.1 System specification and commissioning

The SunCHECK™ (Sun Nuclear Corporation, Melbourne, FL) platform consists of two kinds of quality assurance (QA) programs: SunCHECK Patient™ and SunCHECK Machine™. SunCHECK Patient™ includes DoseCHECK™ and PerFRACTION™. The server-based web application is accessed from clients with network access to an application server. DoseCHECK™ independently calculates the 3D dose distribution with Sun Nuclear Dose Calculator (SDC), which use collapsed cone convolution/superposition (CCC) algorithm, thereby enables to compare the 3D dose calculated by TPS. PerFRACTION™ performs phantom-less pre-treatment QA by comparing 2D dose distribution acquired with EPID images (Fraction 0™), also enables in vivo dosimetry by reconstructing the dose using only log file or both log file and EPID images (Fraction n™). SNC Machine™ performs TG-142 based imaging and machine QA by automatically capturing QA files although this feature was not installed in our institute.

For modeling SDC, beam data such as beam profile, percentage depth dose (PDD), output factor, reference dose, and computed tomography (CT) to electron density (ED) curve are required. The beam model configured by the treatment machine information is used within PerFRACTIONTM and DoseCHECKTM to calculate both 2D and 3D dose distribution.

2.2 Treatment plan for patient cases

This study composed of clinical cases of brain, head and neck (H&N), lung, and prostate treatment to evaluate various sites. A total 46 patients (10 brain, 10 H&N, 14 prostate, and 12 lung cases) who underwent VMAT were treated with TrueBeam equipped with a high-definition (HD) 120 MLC (Varian Medical Systems, Palo Alto, CA). All VMAT plan were generated using the Eclipse TPS (Ver. 13.7, Varian Medical Systems, Palo Alto, CA). Table 1 shows the parameters used to create treatment plans for various treatment sites.

Table 1 Parameters used to create a treatment plan for various treatment sites
Full size table

2.3 Patient-specific QA systems

2.3.1 Portal dose imaging prediction (PDIP)

All EPID images were acquired with an amorphous silicon (a-Si) 1200 EPID detector attached to portal vision extract E-arm, which is remotely positioned with high accuracy and reproducibility. The detector has an active imaging area of 40 × 40 cm2 with a respective resolution of 1190 × 1190 pixels, corresponding to 0.336 mm/pixel. PDIP was performed using the EPID and a portal dose imaging prediction algorithm (Ver. 13.7, Varian Medical Systems, Palo Alto, CA) implemented in the Eclipse. Verification plans for portal dosimetry were generated according to our institutional protocol. The distance from the source-to-imager was kept at 100 cm when acquiring the delivered fluence.

2.3.2 ArcCHECK™ measurement

The ArcCHECK™ (SunNuclear Corporation, Melbourne, FL, USA) composed of 1386 n-Si diode detectors having active area of 0.8 cm × 0.8 cm is an acrylic (PMMA) cylindrical phantom with a density of 1.18 g/cm3. The detectors form 21 helical continuous rings at 1 cm intervals, with 66 detectors on each ring. ArcCHECK™ dosimeters were calibrated according to the manufacturer protocol. The verification plans were calculated on the homogeneous virtual ArcCHECK™ using 2.5 mm 3D dose calculation grid size. Couch top was included into the calculation as an ROI. Before each measurement session, a background correction and a 10 × 10 cm2 open field, source to axis distance (SAD) 100 cm was delivered and used for dose calibration following the manufacturer’s recommendation.

2.3.3 2D PerFRACTION™ analysis

2D PerFRACTION™ analysis is a phantom-less 2D absolute dose verification method using EPID. The EPID image can be used to determine 2D dose distribution that can be compared with the dose distribution in that plane calculated with the predicted 2D dose distribution. The conversion matrix to convert each pixel value to dose was acquired by calibration process using a set of rectangular open fields of various sizes and positions delivered in air according to the manufacturer protocol. EPID images measured at source-to-imager distance 100 cm were required in the calibration process for all energies. The EPID images for verification plan are converted into a 2D planar dose using conversion matrix, and compared with a predicted 2D planar dose calculated at 5 cm depth of the virtual water phantom using SDC.

2.3.4 3D PerFRACTION™ analysis

For 3D PerFRACTION™ analysis, log file information can be used to reconstruct 3D dose distribution using SDC. Log file produced by treatment machine contains information such as the actual jaw positions, MLC positions, MU fraction, and couch positions. After the treatment plan is approved, export all DICOM files including the CT images, RT plans, and RT structures from Eclipse to SunCHECK™. Then, 3D dose distribution based on delivered machine parameters from log file was reconstructed to reflect machine delivery variation. Then, the delivered 3D dose distribution obtained through SDC and log file was compared with the calculated 3D dose distribution.

2.4 Modulation complexity indices

To investigate the modulation indices for each VMAT plan, the modulation complexity score (MCS) and the leaf travel modulation complexity score (LTMCS) were calculated based on previous studies [9, 11]. The MCS is defined as a sum over all segments of the aperture area variability (AAV), leaf sequence variability (LSV), and normalized monitor unit (MU) value. The AAV is defined as the apertures area of opposing leaves in the single control point (CP) per the maximum area in the arc:

$${\mathrm{AAV}}_{\mathrm{CP}}= \frac{\sum_{a=1}^{A}\left({\langle {\mathrm{pos}}_{a}\rangle }_{\mathrm{leftbank}}- {\langle {\mathrm{pos}}_{a}\rangle }_{\mathrm{rightbank}}\right)}{\sum_{a=1}^{A}\left({\langle {\mathrm{max}(\mathrm{pos}}_{a})\rangle }_{\mathrm{leftbank}\in \mathrm{arc}}- {\langle \mathrm{max}({\mathrm{pos}}_{a})\rangle }_{\mathrm{rightbank}\in \mathrm{arc}}\right)}.$$

In the above equation, pos and A means to the MLC position and the number of leaves in the arc, respectively.

The LSV is defined for each control point as the difference in position between adjacent MLC leaves. The positional variations are relative to the maximum possible changes in the CP (posmax(CP)).

$${\mathrm{pos}}_{\mathrm{max}}\left(\mathrm{CP}\right)= {\langle \mathrm{max}\left({\mathrm{pos}}_{n\in N}\right)-\mathrm{min}\left({\mathrm{pos}}_{n\in N}\right)\rangle }_{\mathrm{leafbank}}$$
$${\mathrm{LSV}}_{\mathrm{CP}}= {\left(\frac{\sum_{n=1}^{N-1}\left({\mathrm{pos}}_{\mathrm{max}}- \left|{\mathrm{pos}}_{\mathrm{n}}-{\mathrm{pos}}_{n+1}\right|\right)}{\left(N-1\right) \times {\mathrm{pos}}_{\mathrm{max}}}\right)}_{\mathrm{leftbank}} \times {\left(\frac{\sum_{n=1}^{N-1}\left({\mathrm{pos}}_{\mathrm{max}}- \left|{\mathrm{pos}}_{n}-{\mathrm{pos}}_{n+1}\right|\right)}{\left(N-1\right) \times {\mathrm{pos}}_{\mathrm{max}}}\right)}_{\mathrm{rightbank}}$$

where N is the number of moving leaves in the jaws.

The MCS calculated by using a mean value between adjacent CP of LSV and AAV. The mean value is weighted by the relative MU and then summed over all CP in the arc as follow:

$$\mathrm{MCS}= \sum_{i=1}^{I-1}\left[\frac{\left({\mathrm{AAV}}_{{\mathrm{cp}}_{i}}+ {\mathrm{AAV}}_{{\mathrm{cp}}_{i+1}}\right)}{2} \times \frac{\left({\mathrm{LSV}}_{{\mathrm{cp}}_{i}}+ {\mathrm{LSV}}_{{\mathrm{cp}}_{i+1}}\right)}{2} \times \frac{{\mathrm{MU}}_{{\mathrm{CP}}_{i, i+1}}}{{\mathrm{MU}}_{\mathrm{arc}}}\right].$$

The MCS has values ranging from 0 to 1. As the MLC modulation increases, the MCS decrease. (MCS = 1: no MLC modulation).

The LTMCS is defined as the combination of average leaf travel (LT) and MCS. It is calculated using the simple formula such as LTMCS = LT × MCS. The LT is also obtained as follow:

$$\mathrm{LT}=\frac{1000\mathrm{ mm}-{\mathrm{LT}}_{\mathrm{mean}}}{1000\mathrm{ mm}}.$$

The LT indicated a value of overall all in-field MLC motions of the considered plan in the range from 0 to 1, where 1 is obtained from static MLC (no movement of MLC). Thus, LTMCS is also ranged from 0 to 1, and it increased with decreasing the degrees of MLC modulations and the MLC motions.

2.5 Data analysis

To quantify the difference between the calculated and measured dose distribution, global gamma analysis was performed to determine the agreement using a 2% dose difference and a 2 mm distance to agreement (2%/2 mm) as well as 3%/3 mm for the criteria. The threshold value was 10% that means doses less than 10% of the maximum dose were not considered for the gamma analysis. The gamma passing rate (%GP), defined as the percentage of points satisfying the condition of gamma index (GI < 1) less than 1, was calculated for both criteria [12, 13]. In addition, the correlation between plan complexity and gamma index analysis was evaluated in patient-specific QA using the various QA systems.

The correlations between MCS and LTMCS and gamma passing rates using four QA systems were analyzed by calculating Spearman’s rank correlation coefficients (rs) with the corresponding p values. According to the Evans guidelines, the absolute rs values equal to or larger than 0.2 and smaller than 0.4 indicate weak correlations (0.2 ≤ rs < 0.4); The absolute rs values equal to or larger than 0.4 and smaller than 0.6 indicate moderate correlations (0.4 ≤ rs < 0.6); The absolute rs values equal to or larger than 0.6 and smaller than 0.8 indicate strong correlations (0.6 ≤ rs < 0.8); The absolute rs values equal to or larger than 0.8 indicate very strong correlations (rs ≥ 0.8) [14, 15].

3 Results

3.1 Gamma passing rates of the four QA systems

Figure 1 shows the comparative example of gamma analysis in each QA system with treatment plan from TPS for one patient with prostate cancer. Figure 2 shows the box plots of GPRs for 2%/2 mm and 3%/3 mm criteria from four QA systems with respect to treatment planning system for brain, H&N, prostate, and lung treatment sites. The GPRs of dose distributions obtained from four QA systems for all treatment cases were more than 98% for the 3%/3 mm criterion compared to those acquired from TPS. According to each case, all QA systems had almost similar GPRs. In addition, all GPRs of PDIP and 2D/3D PerFRACTION™ for the 2%/2 mm criterion were more than 96% for all treatment cases except for the GPRs of ArcCHECK™. ArcCHECK™ alone had slightly lower with GPRs of 94% to 95% in all cases for the 2%/2 mm criterion. The GPRs of PDIP and 2D/3D PerFRACTION™ were nearly identical for both 2%/2 mm and 3%/3 mm criteria.

Fig. 1
figure 1figure 1

Comparative example of gamma analysis in a PDIP, b ArcCHECK™, c 2D PerFRACTION™, and d 3D PerFRACTION™ with treatment plan from TPS for one prostate cancer patient

Full size image
Fig. 2
figure 2

Box plots of gamma passing rates for a 2%/2 mm and b 3%/3 mm criteria from four QA systems with respect to treatment planning system for brain, head and neck, prostate, and lung treatment sites

Full size image

3.2 Plan complexity indices

Table 2 indicates the mean and standard deviation of the degree of modulation for 10 brain, 10 H&N, 14 prostate, and 12 lung VMAT plans. Scores for MCS and LTMCS range from 0 to 1, with high values corresponding to lower complexity plans. Of the four treatment sites, the brain had high MCS and LTMCS and the H&N case had low MCS and LTMCS. Mean MCS were 0.321, 0.153, 0.309, and 0.288 for brain, H&N, prostate, and lung cases. In addition, the mean LTMCS were 0.326, 0.151, 0.318, and 0.185, respectively.

Table 2 Mean and standard deviation of modulation degrees on VMAT plans for 10 brain, 10 head and neck, 14 prostate, and 12 lung patients
Full size table

3.3 Correlation between the GPRs and the complexity indices

Figure 3 shows the mean rs and the corresponding p values of GPRs for plan complexity indices, such as MCS and LTMCS, representing the degree of modulation for each treatment site. For 10 brain cases, the mean rs between GPRs using PDIP and 2D/3D PerFRACTION™ and plan complexity indices were weakly correlated for the 3%/3 mm criterion and moderately correlated for the 2%/2 mm criterion. On the other hand, the mean rs between GPRs using ArcCHECK™ and plan complexity indices were moderate correlations for the 2%/2 mm and weak correlations for 3%/3 mm criteria. The trend of mean rs was similar for both criteria in PDIP and 2D/3D PerFRACTION™ except for ArcCHECK™. For 10 H&N cases, only the mean rs between GPRs using PDIP and plan complexity indices were higher than 0.6 for both criteria, indicating strong correlations, and the mean rs between GPRs using ArcCHECK™ and 2D/3D PerFRACTION™ and plan complexity indices were less than 0.5 for both criteria, indicating moderate correlations. The trend of mean rs was similar for both criteria in ArcCHECK™ and 2D/3D PerFRACTION™ except for PDIP. For 14 prostate cases, the mean rs between GPRs using four QA systems and plan complexity indices for both criteria were weak correlations ranging from 0.2 to 0.4 except for 0.546 between GPRs using ArcCHECK™ and MCS and − 0.582 between GPRs using 2D PerFRACTION™ and LTMCS for 2%/2 mm. In addition, the trend of mean rs was similar for both criteria in PDIP and 2D/3D PerFRACTION™ except for ArcCHECK™. For 12 lung cases, the mean rs between GPRs using all QA systems and plan complexity indices were significantly similar to those for brain case. The corresponding mean rs between GPRs using PDIP and 2D/3D PerFRACTION™ and plan complexity indices exceeded 0.3 for the 3%/3 mm criterion and exceeded 0.5 for the 2%/2 mm criterion. As in the brain cases, the mean rs between GPRs using ArcCHECK™ and plan complexity indices were moderate correlations less than 0.5 for the 2%/2 mm and weak correlations less than 0.1 for 3%/3 mm criteria.

Fig. 3
figure 3

Spearman’s rank correlation coefficients and corresponding p values between plan complexity indices (MCS and LTMCS) and gamma passing rates for a 2%/2 mm and b 3%/3 mm criteria from four QA systems on each treatment site

Full size image

4 Discussion

In this study, the usability of PerFRACTION™ in clinical practice was investigated compared to the performance of other QA systems for pretreatment patient-specific QA. We also evaluated the GPR analysis and correlation between GPR and plan complexity for the four QA systems.

From the GPRs analysis performed here, the mean GPRs obtained for each treatment site from the 46 VMAT plans, were almost similar good agreement in all QA systems for both the 2%/2 mm and 3%/3 mm criteria, expect for those obtained with ArcCHECK™ using 2%/2 mm criterion for some cases. Among the dose distributions measured using ArcCHECK™, GPRs analyzed with 2%/2 mm criterion showed a lower agreement with the dose distribution calculated by TPS compared to other QA systems, which was observed in brain and lung cases with small target sizes. The poor agreement in ArcCHECK™ to the detailed criteria may be due to the setup uncertainty of the system and insufficient resolution to detect MLC systematic errors when using the array detector. However, all GPRs between the dose distribution obtained from ArcCHECK™ with prediction distributions provided by the TPS to the same spatial locations were also over 94% for the 2%/2 mm criterion, satisfying the patient-specific QA protocol tolerance (> 90%) at our clinic. In addition, our results were evaluated to have similar performance to other QA systems for patient-specific QA through direct GPR comparison between 2 and 3D distributions obtained using PerFRACTION™ and dose distributions calculated using TPS.

Another characteristic of this study was to compare and investigate the correlation between the GPRs obtained from each QA system, although not a large number of patients, with various treatment sites. This was to evaluate the sensitivity of each QA system by comparing the correlation of the gamma analysis result on the plan complexity according to the degree of modulation for the treatment plans. With respect to the trend of mean rs between plan complexity indices and GPRs indicated in Fig. 3, the rs values for all QA systems were observed to weak or moderate correlations level within suggestive evidence area (0.005 < p value < 0.05) [16]. In significant evidence area (p value < 0.005), overall rs were ranged with strong or moderate correlation level except one prostate case using 2D PerFRACTION™. In H&N cases, the rs between GPRs of PerFRACTION™ and plan complexity index was the opposite trend to that of the PDIP. This is probably due to the insufficient number of cases used for rs analysis and difference in dose verification algorithms of PDIP and PerFRACTION™. It is thought that statistically significant results can be obtained if correlation analysis is performed on more cases.

Although no very strong correlation (rs > 0.8) was observed in this study, the trend of mean rs between GPRs for both criteria and plan complexity indices between PDIP and 2D/3D PerFRACTION™ as in the GPRs analysis was significantly similar for brain, prostate, and lung cases with lower complexity compared to H&N case. The trend of mean rs for 2D/3D PerFRACTION™ for H&N cases with high complexity was similar to that of ArcCHECK™ and slightly lower correlation was observed than that of PDIP.

2D PerFRACTION™ and PDIP are similar in the way of measuring the fluence using the same EPID for pretreatment patient-specific QA. Due to this reason, the obtained GPRs using 2D PerFRACTION™ had good agreement with those of PDIP for both criteria. In addition, the mean rs between GPRs and plan complexity indices using 2D PerFRACTION™ were similar to those of PDIP expect for H&N case with significantly larger target size and higher complexity. These results show that 2D PerFRACTION™ is well commissioned.

3D PerFRACTION™, which can analyze the GPRs in 3D dose distribution generated by MLC log file, is a time-efficient method to track the accuracy of dose delivery depending on the verification process. In this way, the consistency of daily treatment can be measured. This lack of patient-specific QA for daily treatment has long been realized, but most radiotherapy departments have so far no practical solution for this issue [17]. Therefore, the clinical use of 3D PerFRACTION™ can be an important solution for daily patient-specific QA. Based on the results of this study, both QA systems, 2D/3D PerFRACTION™, tended to be nearly consistent for the GPRs and mean rs. A similar study using in-house software performed by Defoor et al. showed that the EPID and MLC log file methods had good agreement in QA results for H&N, lung, and prostate patients, although the use of EPID had higher standard deviation than that using log files [18]. Their result is similar to our result. When these both systems were used as an appropriate combination in clinical practice, PerFRACTION™ will be able to perform not only patient-specific QA before treatment but also delivery accuracy in each fractional treatment.

A limitation of this study is that it did not proceed in various cases and many patients. In a future study, our clinic will use PerFRACTION™ to perform patient-specific QA on many patients and a variety of cases. Some statistically significant trends were not found in ArcCHECK™ about the mean rs, which indicates the correlation between GPRs and plan complexity (MCS and LTMCS), but it is predicted that more statistically meaningful results could be obtained if more patient data for various treatment sites are applied.

5 Conclusion

With the results of comparing GPRs obtained from the 46 treatment plans and the correlation trends between GPRs and plan complexity indices, this study showed that the performance of 2D/3D PerFRACTION™ for pretreatment patient-specific QA was almost comparable to that of PDIP, although there was small difference from ArcCHECK™ for some cases. Thus, we found that the PerFRACTION™ is a suitable QA system for pretreatment patient-specific QA in a variety of treatment sites. If the combined QA process of 2D/3D PerFRACTION™ is performed in clinical practice, it is considered to be an efficient QA method to track the accuracy of dose delivery during treatment period as well as pretreatment patient-specific QA.