Introduction

Quality assurance (QA) of a magnetic resonance imaging (MRI) equipment can be considered as a twofold process. The equipment performance is typically measured during the acceptance inspection and at regular intervals throughout the lifetime of a system. Similar inspection and measurements are preferably done just before the end of a warranty. Between these comprehensive evaluations, the system stability and image quality conformance should be followed with less comprehensive checks. Apart from the checks performed by the scanner itself, users may execute specific tests e.g., daily, weekly, or monthly. The extent and interval of these measurements vary according to the department policies, machine setup, and available resources. These may consist from a simple one image inspection to a multi-parameter test series. Often, the required tests can be adopted from a manufacturer’s recommended maintenance program, but it is also important to fully understand how the tests are performed and what the expected results are [1–5].

In our experience, a single image acquired before the first patient of the day offers a low effort and useful system check. Error messages, artifacts seen in a gross visual inspection of the image or other scanning anomalies, can alert the user and hasten the response to solve the underlying problem. When this daily QA check is repeated with identical imaging parameters and phantom setups, the data can be used to derive the various time series of the scanner performance. With manual processing and bookkeeping, an objective daily quality control can quickly become challenging in a multi-scanner environment due to manpower requirements. A computer-driven analysis has a potential to improve the efficiency of the QA process substantially. Fully automated image processing and result output can positively impact the QA process by decreasing reaction time, improving repeatability, and by offering novel performance evaluation methods. The useful variables that can be followed depend on the selected phantom, imaging parameters, and the analysis methodology [6–9].

Previous studies have presented various solutions for automatic QA of MRI with different aspects. The stability of functional magnetic resonance and diffusion imaging has been studied before in e.g., [10, 11]. Similar methods have been applied to follow the quality of research data from PACS almost on-line [12]. Also, automatic QA systems have been designed to speed up phantom image analysis [13]. A fully automatic QA process has been presented before for CT scanners by Nowik et al. [14]. However, to our knowledge, there are no scientific works or commercial products introducing networked MRI QA analysis process that works automatically in a multi-scanner environment.

In this study, we have developed a daily MRI QA workflow that can measure multiple scanner performance parameters with minimal manual labor required. The daily QA system is built around a phantom image taken by the scanner operator (e.g., MR technologist) at the beginning of day. The image is acquired with a consistent phantom setup and standardized imaging parameters. Recorded parameters are processed into graphs available to everyone involved in the MRI QA process via a web-based interface. In-house made (or open source) software enables development and evaluation of new algorithms and methods for quality assurance. This kind of software can be easily extended by the user and employed to test new quality assurance algorithms.

Materials and Methods

General System Setup

The automatic QA system workflow begins with an initial phantom image scanning, which was performed similarly on all the scanners. Each of the geographically sparsely located imaging sites was housed from one to three scanners. The acquired images were sent to a DICOM server dedicated to image processing. The accumulated daily QA images were processed daily, and the results were uploaded to an interactive web page residing in the hospital district intranet. The process is presented in Fig. 1.

Fig. 1
figure 1

The flowchart of the automatic MRI QA process

Full size image

Imaging Process

Fourteen MRI scanners from three different vendors were included in the QA program of which three were mobile units and one was an extremity scanner. Three of the statically installed scanners had a field strength of 3T whereas the rest of the scanners were 1.5T. A QA image was taken daily before the first patient study of the day by using a standard head coil and a cylindrical disc or sphere-shaped general QA phantom provided by a manufacturer. The alignment of the phantom was kept as consistent as possible from day to day. The exact combination of the head coil in use and the phantom model varied between imaging sites. The diameter of the phantoms varied from 13 to 20 cm.

The imaging sequence for the daily QA image was a standard spin echo (SE) sequence with parameters presented in Table 1. The resulting images included a round signal producing area in the middle and a background area void of signal as presented in Fig. 2. The daily QA protocol has been followed already for a decade with only small alterations in the imaging sequence or coil-phantom combinations.

Table 1 The daily QA sequence parameters
Full size table
Fig. 2
figure 2

The ROI setup for SNR (a), image intensity uniformity (b), image ghosting (c), and geometric distortion measurement (d)

Full size image

To assess the effect of the selected imaging sequence, a parallel QA data with multiple sequence types was obtained from one of the scanners over the period of 2 months. The scanner was a stationary 1.5T whole-body scanner used daily for clinical patient studies. The same phantom and image slice position were applied for this series. In addition to the standard SE protocol, the daily QA image was acquired by using a gradient echo (GRE) and echo planar imaging (EPI)-based sequences (Table 2).

Table 2 The parameters for the GRE and EPI sequences employed to study the effect of the sequence type to the results
Full size table

Analysis of the Images

After the scanning, images were sent to a QA DICOM server on a dedicated image processing computer. The image analysis was run daily, and the results were kept up-to-date. The in-house analysis software was written in C++ using the Insight Segmentation and Registration Toolkit (National Library of Medicine, US) to take an advantage of a well-established medical image processing library. The extracted variables were signal-to-noise ratio (SNR), image intensity uniformity, image ghosting, and geometrical accuracy. Additionally, the center frequency of the image was recorded from the DICOM header of the image. The methods of defining regions of interests (ROIs) and the QA parameter calculations are described in the following chapters.

Definition of the Regions of Interest

The ROIs were resized and positioned automatically. First, the central signal area was traced using a Hough transform-based circle detection algorithm. After this, the rectangular background noise ROIs were placed on the top and bottom of the image in the frequency encoding direction and signal ghosting ROIs to the left and right sides of the image in the phase encoding direction, respectively. The appropriate margins were kept between the rectangular ROIs and both the signal producing area and the outermost edge of the image. The background ROI widths were set to 75%, and the ghosting ROI heights were set to 50% of the respective image dimension. An example of the ROI placement is shown in Fig. 2.

Signal-to-Noise Ratio

SNR in the daily QA image was calculated according to the method 4 in NEMA signal-to-noise standard [15], which is an intended method for SNR calculations in a single image. The signal was defined to be a mean signal level in circular ROI which has an identical center and 80% of radius of the signal producing area. The noise is determined by calculating the standard deviation in the combined area of the rectangular background noise ROIs. Finally, SNR was calculated by

$$ \mathrm{S}\mathrm{N}\mathrm{R}=0.66*\frac{\mathrm{signal}}{\mathrm{noise}} $$
(1)

The multiplier 0.66 is an oft-used factor for compensating the theoretical Rician distribution of a magnitude image to correspond that of an underlying Gaussian distribution.

Image Intensity Uniformity

The image intensity uniformity was calculated by using three comparable methods of which two are presented in the NEMA guidance for image uniformity measurements [16] and one in the IEC standard 62464–1 [17]. The area used for the signal uniformity determination was identical with the signal area in SNR calculation.

In the method introduced by NEMA, the uniformity is calculated by

$$ {\mathrm{Uniformity}}_{\mathrm{NEMA}}=1-\frac{S_{\max }-{S}_{\min }}{S_{\max }+{S}_{\min }}, $$
(2)

where S max and S min are referring to the maximum and minimum intensities in signal ROI, respectively. Additionally, the image may be filtered with a kernel

$$ \left[\begin{array}{lll}1\hfill & 2\hfill & 1\hfill \\ {}2\hfill & 4\hfill & 2\hfill \\ {}1\hfill & 2\hfill & 1\hfill \end{array}\right] $$
(3)

to minimize the effects of noise on the measurement. According to the IEC standard 62464–1 [17], the image uniformity is calculated by

$$ {\mathrm{Uniformity}}_{\mathrm{IEC}}=1-\frac{{\displaystyle {\sum}_{1=1}^N\left(\left|{S}_i-S\right|\right)}/N}{S} $$
(4)

where S i is an individual pixel value inside the signal ROI, S is the mean value of all pixels in the signal ROI, and N is the total number of pixels in the signal ROI.

Image Ghosting

The image ghosting was calculated as presented in the IEC standard 62464–1 [17]. The signal ROI placement was identical to that of the SNR calculation. Signal ghosting was defined as the strongest intensity inside the ghosting ROIs, as presented in Fig. 2, after the image has been filtered with a 5 by 5 averaging kernel. Finally, the ghosting percentage is calculated by

$$ \mathrm{Ghosting}=100\%*\frac{I_G}{S}, $$
(5)

where I G is the highest intensity within the ghosting ROIs after the image filtration and the S is the mean signal intensity in the signal producing area.

Geometrical Distortion

The geometrical distortion of the image was calculated by measuring the largest distance of the signal producing area in the x and y direction as presented in Fig. 2. Before the calculation, the phantom image is threshold filtered with a level corresponding to half of the mean signal level in the signal producing area. The pixel dimensions were extracted from the DICOM header.

Manual Analysis Comparison

The correctness of the automatic analysis was tested by comparing the SNR, image intensity uniformity, and image ghosting results from a limited data set with a manual analysis. The data set was obtained using 1.5T scanner over a period of 3 months with no detected faults in the equipment. The image analysis software ImageJ (National Institutes of Health, Bethesda, MD) was used in the manual analyses with corresponding methods and ROIs as presented before for the automatic analysis. The analysis was done by an experienced QA physicist familiar with the used methods. For the image intensity uniformity, the unfiltered NEMA [16] approach was used. All ROIs were determined by hand while the window and level functions of the software were used to enhance the visibility of the background noise.

The Presentation of the QA Results

The automatic QA analysis was producing a comma separated value (CSV) text file where each line consisted of results from a single QA image. The CSV file was parsed, and the results were presented as plots on a web page. An interactive tool was also provided for the user to look into measured data and link a data point to the corresponding image (Fig. 3).

Fig. 3
figure 3

An interactive tool that enables the visualization and analysis of daily QA data

Full size image

The results were presented in multiple formats to benefit different user groups. Static plots were generated to give a quick and easy overview of the system progression for the operators who performed the daily QA image acquisition. More in-depth analysis capability for QA specialists was implemented as a platform-independent browser tool which can be used to view images producing abnormal results and tools for quick linear fitting for the selected portion of the data. Static time series images were dynamically generated on the server side using a PHP graph library PHPlot [18], and the browser-based plotting tool was created by using an open source JavaScript charting library dygraphs [19]. For scientific and debugging purposes, MATLAB-based viewer was built to provide a way of verifying where the ROIs were placed and a platform for the rapid improvement and development of new methods and measurements.

Results

The automatic QA workflow was producing day by day image quality data from all the scanners in the department regardless of geographical location, manufacturer, or model. Typical result plots are shown in Fig. 4. Two examples of findings are presented in Figs. 5 and 6.

Fig. 4
figure 4

Typical long-term image quality parameter plots for a static magnet

Full size image
Fig. 5
figure 5

An example of a ghosting finding in the daily QA data. The reason for the artifact was later identified to a faulty fan in the gradient modulator unit

Full size image
Fig. 6
figure 6

An example of image intensity uniformity finding in the daily QA data. The periodical appearance of the DC offset artifact can be clearly detected

Full size image

A comparison of imaging sequences is presented in Fig. 7. During the test period of 2 months with the image, QA parameters obtained with all the sequences suggested a stable operation of the scanner with no irregular events. There are some visible differences on the characteristics of the data produced by each sequence. The numerical value for SNR is the highest with the SE sequence and lowest with the EPI sequence. Signal uniformity is very similar with all of the sequences. Ghosting in the EPI sequence seems to be approximately seven times higher in comparison with other sequences. The coefficients of variation of SNR, image intensity uniformity, and image ghosting with each sequence are presented in Table 3.

Fig. 7
figure 7

A comparison of the spin echo, gradient echo, and echo planar imaging sequences

Full size image
Table 3 The coefficient of variation of SNR, image intensity uniformity, and image ghosting measured by three different sequences
Full size table

The differences in results between manual and automatic analysis are presented in Table 4. The mean values of image ghosting were 21% higher when the automatic analysis was used instead of the manual. Also, the standard deviation of the SNR was 32% higher and the standard deviation of the image ghosting 49% lower with the automatic analysis compared with the manual.

Table 4 Comparison between the means and standard deviations (sd) of image quality parameters obtained with automatic and manual analyses
Full size table

Discussion

The presented automatic MRI QA analysis workflow provides an efficient and scalable way to carry out daily QA. The quality parameter graphs are updated on a daily basis, and thus up-to-date information about the equipment is available to all whom it might concern. However, the greatest value of the system is the ability to assess the long-term development of the image quality parameters. This provides both general knowledge on the stability of the equipment and background information for decision making when an anomaly of a single value is assessed. It is also important to know if the performance of the scanner has remained constant since the installation of the equipment or perhaps declined over time. Also, the automated system allows the retrospective investigations of the old images if new measurements are implemented.

The effect of the chosen imaging sequence on the results was studied on one scanner using GRE- and EPI-based sequences adjacent to the standard SE sequence. During the test period, there were no faults in the system which would reveal more on the characteristics of each sequence. However, it is likely that in the case of an abnormal event in the system, there would be sensitivity differences in how each sequence is responding to the fault in question. For example, instability in a gradient system should be first noted in the EPI sequence due to inherited dependence on accurate gradient control [20]. The increased coefficient of variation in the image ghosting measurement with GRE sequence and image intensity uniformity with GRE and EPI sequences also suggests a high sensitivity to small changes in the image acquisition process.

It could be possible to gain additional information, if different kinds of an image or multiple images would be acquired instead of the single slice used in the current protocol. One could obtain more accurate SNR estimation for example by using double image acquisition [15]. The issues associated with the SNR measurement have been thoroughly discussed by Dietrich et al. [21]. However, with fixed imaging protocol, the single image method can be valid for detecting notable variations or a long-term drift in the scanner. We believe that a greater limitation of our procedure comes from the size of the phantoms which mostly are relatively small compared to the full fields of view. This may reduce the sensitivity of the geometrical distortion and image uniformity measurements [22]. The main purpose of the manuscript is to present general modifiable and extendable framework for daily QA. The exact methods and parameters to measure scanner performance can be chosen to match institutional QA program.

The QA values obtained by the automatic analysis were compared with the manually attained results. The bias in the ghosting measurement is most likely present due to variation in the highest ghosting intensity detection. In the automatic analysis, the algorithms are detecting optimum areas in the image while in the manual analysis the detection is done visually. Also, the standard deviation in the automatic ghosting measurements is lower suggesting more repeatable ROI placement. As small deviations were expected, the automatic procedure was verified and found suitable for the long-term monitoring.

The monitored values described here are dependent on the exact combination of the specific phantom and scanner. Thus, the results cannot be directly applied for assessing the absolute performance differences between scanners. The phantom shapes ranged from cylindrical to spherical and with wall material from rigid to relatively soft plastic. Also, the composition of the liquid inside the phantom was not consistent, especially between 1.5T and 3T systems.

The time series graphs are reviewed by QA physicists regularly or when there is a suspected fault in a scanner. An automatic detection of abnormal results would further decrease the need for human input to the QA process. Published acceptance levels on the QA parameters can be used either directly or as a guiding basis for daily QA data. These parameters include limits for acceptable signal uniformity, ghosting, and geometrical distortion [16, 17]. However, the published limits are only valid for defined sequences and phantoms. For the purpose of daily QA, it is often practical to use a readily available phantom and simple image acquisition protocol. To achieve maximum applicability, acceptable limits have to be set individually for each scanner and parameter based on the longitudinal observations. In the process, both short-time deviation characteristics and long-term drift properties should be taken into account.

Conclusions

The presented automatic MRI QA system provides an efficient tool for following the short- and long-term stabilities of MRI scanners. The amount of manual work required is almost independent of the number or the type of the scanners. The data can be analyzed and accessed from any standard computer equipped with a network connection. The system provides an easy to approach data managing for QA tasks and allows more sophisticated statistical or image processing methods to be developed and implemented. This work offers a starting point for addressing the specific QA needs of individual MRI equipment and imaging techniques, or even those of various other imaging modalities.