1 Introduction

In recent years, neural networks have been applied in computed tomography (CT) imaging. Several studies on the applications of such networks have been published, and their potential in solving several problems in the field of CT imaging has been extensively evaluated. Such networks have been used in applications such as spectral distortion correction [1, 2] for photon-counting X-ray CT, dual-domain learning for two-dimensional [3, 4] and three-dimensional [5,6,7] low-dose CT reconstruction, sparse-view [8] and limited-angle [9] CT reconstruction, noise suppression in the sinogram domain [10] and image domain [11], dual-energy imaging with energy-integrating detectors [12] and photon-counting detectors [13], and CT artifact reduction [14, 15]. However, neural networks have not yet been widely used in practice owing to their confidence scores. Hence, researchers have adopted objective and subjective image quality assessments to evaluate the feasibility of using neural networks in CT imaging [16, 17]. However, traditional metrics are preferred for objective assessments. The commonly used contrast-to-noise ratio measures the region of interest (ROI) clarity, signal-to-noise ratio measures the noise level, noise power spectrum (NPS) measures the noise correlation, and modulation transfer function measures the spatial frequency [18, 19]. Neural networks are normally nonlinear, nonstationary, and unexplainable; however, to the best of our knowledge, limited research on theoretically tractable methods has been conducted.

Subjective assessments are commonly used in the field of CT imaging. Radiologists are invited to observe and score images obtained from various methods [20, 21]. Because subjective assessments are time-consuming and laborious, researchers study model observers to simulate the evaluation behavior of radiologists. The assessment of image quality carried out by a radiologist, that is, a human observer is modeled as a classification problem solved by hypothesis testing. The likelihood ratio is used as the decision variable to obtain an ideal observer (IO). Because the IO is intractable, a linear approximation of the IO is assumed to obtain a Hotelling observer (HO). Combining the human-eye model with HO, researchers have obtained the most widely used channelized Hotelling observer (CHO) [22]. The CHO agrees well with human observers [23, 24]; however, it requires knowledge of the image mean and covariance. Recently, neural networks have been introduced to explore nonlinear model observers that enable better approximation of the human observer or better detectability for auxiliary diagnoses [25,26,27]. However, these methods only target traditional reconstruction methods [28, 29]. The application of these methods to neural network reconstruction has not yet been explored.

Noise propagation through a reconstruction neural network is necessary for assessing the performance of the network. Covariance prediction can reveal the uncertainty in inference, thus providing a safer answer to the CT reconstruction problem. Furthermore, it can be used in the calculation of model observers and subjective assessments.

The covariance estimation of neural-network outputs is currently of significant interest. Abdelaziz et al. [30] studied the uncertainty propagation of deep neural networks for automatic speech recognition with the assumption that the output was Gaussian. Lee et al. [31] used a Gaussian process equivalent to a deep fully connected neural network to obtain an exact Bayesian inference under the assumption that the parameters and layer outputs follow an independent and identical distribution. Tanno et al. [32] simultaneously estimated the mean and covariance of high-resolution outputs from low-resolution magnetic resonance images based on the assumption of a Gaussian distribution. However, for CT imaging, covariance estimation of the neural network output has not yet been studied.

In this study, we propose a new analytical noise propagation method, that is, covariance estimation, particularly for a convolutional neural network (CNN) in CT imaging of noisy projections. With a trained CNN ready for inference, we propagate the noise layer by layer. For linear layers, the output covariance can be calculated accurately. For nonlinear activation layers, we perform a Taylor expansion to obtain a linear approximation, which enables linear propagation of noise through the nonlinear layers. Because a CNN is a stack of linear and nonlinear layers, its covariance estimation is a combination of layer noise propagations. We validate the results of the covariance estimation method by comparing the results with those of statistical estimations using noisy projection and reconstruction samples with different phantoms, noise levels, and activation functions.

2 Methods

2.1 Physical model for CT imaging

A simple model for data acquisition in a CT scan is formulated as

$$I = I_{0} e^{ - p} = I_{0} e^{{ - \int {\mu {\text{d}}r} }} ,$$
(1)

where \(I\) represents detected photons, and \(I_{0}\) represents incident photons. Projection \(p\) is a line integral of the linear attenuation coefficient \(\mu\). Usually, the noise distribution of \(I\) is assumed to be Poisson or Gaussian, and the noise distribution of \(p\) can be approximated as Gaussian with mean \(\overline{p}\) and variance \(\exp (\overline{p})/I_{0}\), that is, \(p\sim\mathcal{N}(\overline{p},\exp(\overline{p})/I_{0} )\).

Analytical, iterative, and neural network methods can be used to reconstruct a linear attenuation coefficient map \({{\varvec{\upmu}}}\) from its projection \({\mathbf{p}}\). A CNN is a commonly used reconstruction method for CT imaging. A CNN typically consists of five types of basic layers: convolution, activation, batch normalization, pooling, and up-sampling layers. The overall operation of the network is a cascade of these layers, that is,

$${{\varvec{\upmu}}} = \Phi ({\mathbf{p}}) = \varphi_{L} \left( {\varphi_{L - 1} \left( { \cdots \varphi_{l} \cdots \left( {\varphi_{2} \left( {\varphi_{1} ({\mathbf{p}})} \right)} \right)} \right)} \right),$$
(2)

where \(\varphi_{l}\) denotes the operation function of one layer, and \(\Phi\) denotes the overall function of the neural network. Evidently, the noise in \({\mathbf{p}}\) will result in a noisy \({{\varvec{\upmu}}}\), even though several networks are used to reduce noise. We discovered that the noise propagation through \(\varphi_{l}\)’s to output \({{\varvec{\upmu}}}\) can be studied step-by-step if a network model is ready to serve for inference, that is, network parameters are set. The key lies in sorting out the covariance estimation through the five basic layers constituting the entire CNN.

2.2 Covariance propagation through basic layers of a CNN

Let vector \({\mathbf{x}} \in {\mathbb{R}}^{M \times 1}\) be an arbitrary layer input, and let \({\mathbf{y}}\) be the corresponding layer output. This section presents the covariance estimation of \({\mathbf{y}}\) from \({\mathbf{x}}\).

2.2.1 Convolution layer

For a convolutional layer, its output \({\mathbf{y}}_{{{\text{conv}}}} \in {\mathbb{R}}^{N \times 1}\) can be expressed as a linear combination of inputs:

$${\mathbf{y}}_{{{\text{conv}}}} = \sum\limits_{i = 1}^{C} {{\mathbf{W}}_{i} {\mathbf{x}}_{i} + b_{i} } ,$$
(3)

where \(C\) denotes the number of input channels, \({\mathbf{W}} \in {\mathbb{R}}^{N \times M}\) represents the convolutional weighting matrix, and \(b_{i}\) denotes bias. The convolution is linear; hence, it is easy to obtain:

$${\text{Cov}}({\mathbf{y}}_{{{\text{conv}}}} ) = \sum\limits_{j = 1}^{C} {\sum\limits_{i = 1}^{C} {{\mathbf{W}}_{i} {\text{Cov}}({\mathbf{x}}_{i} ,{\mathbf{x}}_{j} )} } {\mathbf{W}}_{j}^\mathrm{T} .$$
(4)

2.2.2 Activation layer

In an activation layer, an input is normally fed into a nonlinear function, \(f( \cdot )\):

$${\mathbf{y}}_{{{\text{af}}}} = f({\mathbf{x}}).$$
(5)

Because it is nonlinear, we perform the 1st order Taylor expansion to obtain its linear approximation:

$${\mathbf{y}}_{{{\text{af}}}} = f({\mathbf{x}}) \simeq f({\overline{\mathbf{x}}}) + f^{\prime}({\overline{\mathbf{x}}}) \cdot ({\mathbf{x}} - {\overline{\mathbf{x}}}) = {\mathbf{F}}({\overline{\mathbf{x}}}){\mathbf{x}} + (f({\overline{\mathbf{x}}}) - {\mathbf{F}}({\overline{\mathbf{x}}}){\overline{\mathbf{x}}})$$
(6)

where the Taylor-based coefficient matrix \({\mathbf{F}} \in {\mathbb{R}}^{M \times M}\) is diagonal to \([{\mathbf{F}}]_{m,m} = f^{\prime}([{\overline{\mathbf{x}}}]_{m} )\). Thus, the covariance of the nonlinear transformation layer can be estimated by

$${\text{Cov}}({\mathbf{y}}_{{{\text{af}}}} ) \simeq {\mathbf{F}}\text{Cov}({\mathbf{x}}){\mathbf{F}}^\mathrm{T} .$$
(7)

2.2.3 Batch normalization layer

For a batch normalization layer, the input is normalized as

$${\mathbf{y}}_{{{\text{bn}}}} = \frac{\gamma }{{\sqrt {\sigma_\text{B}^{2} + \varepsilon } }}{\mathbf{x}} + \left( {\beta - \frac{{\gamma u_\text{B} }}{{\sqrt {\sigma_\text{B}^{2} + \varepsilon } }}} \right).$$
(8)

Here, \(\gamma\) and \(\beta\) are hyperparameters that are learned during training and frozen when inferencing. \(u_\text{B}\) and \(\sigma_\text{B}^{2}\) represent the batch mean and variance, respectively, which are also frozen during inference. Hence, the covariance propagating through the batch normalization layer is

$${\text{Cov}}({\mathbf{y}}_{{{\text{bn}}}} ) = \frac{{\gamma^{2} }}{{\sigma_\text{B}^{2} + \varepsilon }}{\text{Cov}}({\mathbf{x}}) \equiv \eta {\text{Cov}}({\mathbf{x}}).$$
(9)

2.2.4 Pooling layer

Average pooling is a widely used method in this field. This can be interpreted as a convolution operation of kernel size \(k\) with stride \(s\), where the convolution kernel is a constant matrix with value \(1/k^{2}\). Similar to the operation of a convolution layer, its output \({\mathbf{y}}_{{{\text{ap}}}} \in {\mathbb{R}}^{N \times 1}\) can be formulated as a linear transformation of the input as follows:

$${\mathbf{y}}_{{{\text{ap}}}} = {\mathbf{Ax}}.$$
(10)

Here, the average pooling matrix \({\mathbf{A}} \in {\mathbb{R}}^{N \times M}\) is sparse with \(N = (M - k)/s + 1\). Thus,

$${\text{Cov(}}{\mathbf{y}}_{{{\text{ap}}}} ) = {\mathbf{A}}{\text{Cov}}({\mathbf{x}}){\mathbf{A}}^\text{T}.$$
(11)

2.2.5 Up-sampling layer

For an up-sampling layer, each element of the input is duplicated and can be expressed as a linear combination of the input:

$${\mathbf{y}}_{{{\text{up}}}} = {\mathbf{Ux}},$$
(12)

where the upsampling matrix \({\mathbf{U}} \in {\mathbb{R}}^{N \times M}\) is a sparse matrix with only one element in each row and \(N = 2M\). The covariance estimated from an up-sampling layer is

$${\text{Cov}}({\mathbf{y}}_{{{\text{up}}}} ) = {\mathbf{U}}{\text{Cov}}({\mathbf{x}}){\mathbf{U}}^\text{T}.$$
(13)

2.3 Example: U-net

We adopt a commonly used U-net structure to denoise the projection, followed by reconstruction with the filter back projection (FBP) method:

$${{\varvec{\upmu}}} = \Phi ({\mathbf{p}}) = {\mathcal{O}}\left( {\varphi_{L} \left( {\varphi_{L - 1} \left( { \cdots \varphi_{l} \cdots \left( {\varphi_{2} \left( {\varphi_{1} ({\mathbf{p}})} \right)} \right)} \right)} \right)} \right).$$
(14)

Here, \({\mathcal{O}}\) represents the linear FBP operator. \({\mathbf{p}} \in {\mathbb{R}}^{M \times 1}\) denotes an input projection, and \({{\varvec{\upmu}}} \in {\mathbb{R}}^{N \times 1}\) represents the corresponding reconstruction. The reconstruction flowchart is illustrated in Fig. 1. A concatenate layer and residual layer are included in U-net. The concatenated layer merges the feature map in the 1st layer to the 6th layer, and the residual layer adds the input projection to the 9th layer to obtain the output projection.

Fig. 1
figure 1

(Color online) U-net structure for CT reconstruction. The projection is first filtered by U-net and then reconstructed by FBP. Two activation functions: ① LeakyReLU and ② Swish, are applied to U-net

We iteratively estimate the covariance of reconstruction predicted from the trained U-net:

$${\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{z}}_{j}^{l} ) \simeq \left\{ {\begin{array}{*{20}l} {\eta_{i}^{l} {\mathbf{F}}_{i}^{l} \left[ {\sum\limits_{{j^{\prime} = 1}}^{{C^{l - 1} }} {\sum\limits_{{i^{\prime} = 1}}^{{C^{l - 1} }} {{\mathbf{W}}_{{i^{\prime},i}}^{l} {\text{Cov}}({\mathbf{z}}_{{i^{\prime}}}^{l - 1} ,{\mathbf{z}}_{{j^{\prime}}}^{l - 1} )({\mathbf{W}}_{{j^{\prime},j}}^{l} )^\text{T} } } } \right](\eta_{j}^{l} {\mathbf{F}}_{j}^{l} )^\text{T} }, \hfill & {l = 1,3,4,5,8,9} \hfill \\ {{\mathbf{A}}_{i} {\text{Cov}}({\mathbf{z}}_{i}^{l - 1} ,{\mathbf{z}}_{j}^{l - 1} ){\mathbf{A}}_{j}^\text{T} }, \hfill & {l = 2} \hfill \\ {{\mathbf{U}}_{i} {\text{Cov}}({\mathbf{z}}_{i}^{l - 1} ,{\mathbf{z}}_{j}^{l - 1} ){\mathbf{U}}_{j}^\text{T} }, \hfill & {l = 6} \hfill \\ \end{array} } \right.$$
(15)

where \({\mathbf{z}}\) represents the latent variable in the hidden layer of U-net. \({\mathbf{z}}^{0} = {\mathbf{p}}\) when \(l = 1\) and \({\text{[Cov}}({\mathbf{p}})]_{n,n} = exp({\overline{\mathbf{p}}})/I_{0}\). The subscript \({\mathbf{W}}_{{i^{\prime},i}}^{l}\) denotes a convolutional weighting matrix from the \(i^{\prime}{\text{th}}\) channel input of the \(l{\text{th}}\) layer to its \(i{\text{th}}\) channel output, where \(C^{l}\) represents the total number of channels in the \(l{\text{th}}\) layer.

With a concatenation operation, the 7th layer contains feature maps from the 1st and 6th layers. Thus, three types of covariances for the 7th layer must be present: (1) covariance between channels in the 6th layer, (2) covariance between channels in the 1st layer, and (3) covariance between channels in the 1st layer and 6th layer. The covariance of cases (1) and (2) has already been estimated using Eq. (15), and the covariance of case (3) can be estimated as

$$\begin{aligned} & {\text{Cov}}({\mathbf{z}}_{i}^{6} ,{\mathbf{z}}_{j}^{1} ) = {\text{Cov}}({\mathbf{z}}_{i}^{1} ,{\mathbf{z}}_{j}^{6} )^{T} = {\mathbf{U}}_{i} {\text{Cov}}({\mathbf{z}}_{i}^{5} ,{\mathbf{z}}_{j}^{1} ){\kern 1pt} \\ & {\text{where}}\;{\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{z}}_{j}^{1} ) = {\mathbf{A}}_{i} {\text{Cov}}({\mathbf{z}}_{i}^{l - 1} ,{\mathbf{z}}_{j}^{1} ),\;l = 2 \\ & {\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{z}}_{j}^{1} ) \simeq \eta_{i}^{l} {\mathbf{F}}_{i}^{l} \left[ {\sum\limits_{{i^{\prime} = 1}}^{{C^{l - 1} }} {{\mathbf{W}}_{{i^{\prime},i}}^{l} Cov({\mathbf{z}}_{{i^{\prime}}}^{l - 1} ,{\mathbf{z}}_{j}^{1} )} } \right]{\kern 1pt},\;l = 3,4,5 \\ \end{aligned}$$
(16)

Therefore, the covariance estimation of the 7th layer is

$${\text{Cov}}({\mathbf{z}}_{i}^{7} ,{\mathbf{z}}_{j}^{7} ) = \left\{ {\begin{array}{*{20}l} {{\text{Cov}}({\mathbf{z}}_{i}^{6} ,{\mathbf{z}}_{j}^{6} ),} \hfill & {i,j = 1,2,3} \hfill \\ {{\text{Cov}}({\mathbf{z}}_{i}^{6} ,{\mathbf{z}}_{j}^{1} ),} \hfill & {i = 1,2,3,j = 4,5,6} \hfill \\ {{\text{Cov}}({\mathbf{z}}_{i}^{1} ,{\mathbf{z}}_{j}^{6} ),} \hfill & {i = 4,5,6,j = 1,2,3} \hfill \\ {{\text{Cov}}({\mathbf{z}}_{i}^{1} ,{\mathbf{z}}_{j}^{1} ),} \hfill & {i,j = 4,5,6} \hfill \\ \end{array} } \right.$$
(17)

With a residual operation, the output projection represents the sum of the input projection and output residue of the 9th layer. Thus, the covariance of the output projection also consists of three parts: (1) covariance of the input projection, (2) covariance of the output residue of the 9th layer, and (3) covariance between the input projection and output residue. Only the covariance estimation of case (3) should be calculated because cases (1) and (2) are estimated using Eq. (15):

$$\begin{aligned} & {\text{Cov(}}{\mathbf{z}}^{9} ,{\mathbf{p}}) = {\text{Cov(}}{\mathbf{p}},{\mathbf{z}}^{9} )^{T} = \sum\limits_{i = 1}^{{C^{8} }} {{\mathbf{W}}_{i}^{9} {\text{Cov}}({\mathbf{z}}_{i}^{8} ,{\mathbf{p}})} \\ & {\text{where}}\;\;{\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{p}}) \simeq \eta_{i}^{l} {\mathbf{F}}_{i}^{l} \left[ {\sum\limits_{{i^{\prime} = 1}}^{{C^{l - 1} }} {{\mathbf{W}}_{{i^{\prime},i}}^{l} {\text{Cov}}({\mathbf{z}}_{{i^{\prime}}}^{l - 1} ,{\mathbf{p}})} } \right],\;\;l = 1,3,4,5,8 \\ & {\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{p}}) = {\mathbf{A}}_{i} Cov({\mathbf{z}}_{i}^{l - 1} ,{\mathbf{p}}),\;\;l = 2 \\ & {\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{p}}) = {\mathbf{U}}_{i} Cov({\mathbf{z}}_{i}^{l - 1} ,{\mathbf{p}}),\;\;l = 6 \\ {\text{Cov}}({\mathbf{z}}_{i}^{l} ,{\mathbf{p}}) = \left\{ \begin{gathered} {\text{Cov}}({\mathbf{z}}_{i}^{6} ,{\mathbf{p}}),\;\;i = 1,2,3 \hfill \\ {\text{Cov}}({\mathbf{z}}_{i}^{1} ,{\mathbf{p}}),\;\;i = 4,5,6 \hfill \\ \end{gathered} \right.,\;\;l = 7 \\ \end{aligned}$$
(18)

The covariance estimation of the 10th layer is then calculated as

$${\text{Cov(}}{\mathbf{z}}^{10} ) = {\text{Cov}}({\mathbf{p}}) + {\text{Cov}}({\mathbf{z}}^{9} ) + {\text{Cov}}({\mathbf{z}}^{9} ,{\mathbf{p}}) + {\text{Cov}}({\mathbf{p}},{\mathbf{z}}^{9} ).$$
(19)

Combining Eqs. (15)–(19), we obtain the final covariance estimation of the reconstruction:

$${\text{Cov(}}{{\varvec{\upmu}}}) = {\mathcal{O}}{\text{Cov(}}{\mathbf{z}}^{10} ){\mathcal{O}}^\text{T}.$$
(20)

A gradient discontinuous function, LeakyReLU, and a gradient continuous function, Swish, are chosen as activation functions to detect the influence of activation functions on the covariance estimation of the CNN.

2.3.1 LeakyReLU activation function

The LeakyReLU function and its corresponding gradient are expressed as

$$\begin{gathered} f_{{{\text{LR}}}} (x) = \left\{ {\begin{array}{*{20}c} {x,} & {{\text{if}}\,x > 0} \\ {\alpha x,} & {{\text{if}}\;x \le 0} \\ \end{array} } \right. \hfill \\ f^{\prime}_{{{\text{LR}}}} (x) = \left\{ {\begin{array}{*{20}c} {1,} & {{\text{if}}\,x > 0} \\ {\alpha ,} & {{\text{if}}\;x \le 0} \\ \end{array} } \right. \hfill \\ \end{gathered}$$
(21)

The Taylor-based coefficient matrix in Eq. (6) is \([{\mathbf{F}}_{{{\text{LR}}}} ]_{m,m} = f^{\prime}_{{{\text{LR}}}} ([{\overline{\mathbf{x}}}]_{m} )\). Plugging \({\mathbf{F}}_{{{\text{LR}}}}\) into Eqs. (15)–(20), we obtain the covariance estimation from U-net with LeakyReLU.

2.3.2 Swish activation function

The Swish function is another commonly used activation function:

$$f_{{{\text{Sw}}}} (x) = \frac{x}{1 + \exp ( - \lambda x)}.$$
(22)

Its gradient is

$$f^{\prime}_{{{\text{Sw}}}} (\overline{x}) = \lambda f_{{{\text{Sw}}}} (\overline{x}) + \frac{{1 - \lambda f_{{{\text{Sw}}}} (\overline{x})}}{{1 + \exp ( - \lambda \overline{x})}}.$$
(23)

Thus, the Taylor-based coefficient matrix is \([{\mathbf{F}}_{{{\text{Sw}}}} ]_{m,m} = f^{\prime}_{{{\text{Sw}}}} ([{\overline{\mathbf{x}}}]_{m} )\). Replacing \({\mathbf{F}}\) in Eqs. (15)–(18) with \({\mathbf{F}}_{{{\text{Sw}}}}\), we obtain the covariance estimation of the projection by U-net with Swish using Eq. (19) and the covariance estimation of its reconstruction by FBP using Eq. (20).

3 Experiments

The projection data used for training are generated from the Grand Challenge dataset of the Mayo Clinic. We randomly choose a reconstruction dataset of one patient and select various ROIs with sizes of 128 × 128 pixels as phantoms. Geometrical parameters of the simulated system are listed in Table 1. By setting the number of incident photons to \(I_{0} = 10^{4}\), we add Poisson noise to the noise-free projections simulated from phantoms to obtain noisy projections.

Table 1 System geometry of simulation experiments

Using noise-free projections as labels and noisy projections as inputs, we train U-net by minimizing an L2 norm loss function between labels and outputs. In addition, we fix the hyper-parameter \(\alpha { = }0.1\) for LeakyReLU and \(\lambda { = }0.5\) for Swish. We simulate 1792 noisy projections for the study. The dataset is randomly split into a training set and a validation set, where 80% of the dataset represents the training set and 20% represents the validation set. We train the network with Keras on a GPU RTX8000 of 48G. The loss function of U-net with LeakyReLU and Swish decreases to approximately \(10^{ - 4}\) during converging. Furthermore, we randomly split the dataset into 5 folds to run a fivefold cross validation on the trained network. The average loss in the fivefold cross validation is also approximately \(10^{ - 4}\), which is similar to the loss of the trained network. Thus, the dataset is sufficient for training the small-size U-net in Fig. 1, and the trained network is stable.

Noisy projections used for inference are generated from another patient dataset in the same manner. We generate noisy projections of different phantoms and noise levels to validate the proposed analytical noise propagation method and analyze the performance of U-net using the analytically estimated covariance. Information on the noisy projections generated for prediction is presented in Table 2. Note that the number of incident photons increases linearly from 103 to 5.05 × 104. The reconstruction of both phantoms using \(I_{0} = 10^{4}\) is illustrated in Fig. 2.

Table 2 Information on the noisy projections generated for inference
Fig. 2
figure 2

Denoised projections of a test phantom A and b test phantom B for \(I_{0} = 10^{4}\) obtained from U-net and their reconstructions obtained from FBP

In addition, we conduct a practical experiment to validate our proposed method. The experimental platform and phantom are presented in Fig. 3. The scanning parameters are presented in Table 3. We repeat the scan 450 times at each angle and acquire projection data of 360 views using \(2\pi\). Considering the computational cost, every four pixels of the detector are binned into one to obtain a projection with a smaller size.

Fig. 3
figure 3

(Color online) Experimental platform and phantom

Table 3 Scanning parameters of the practical experiment

Covariance estimation from a statistical method is used as a reference in this study.

$$\begin{aligned} & {\text{Cov}}^{*} ({\kern 1pt} {\hat{\mathbf{p}}}) = \frac{1}{K - 1}\sum\limits_{k = 1}^{K} {({\hat{\mathbf{p}}}_{k} - {\mathbf{\overline{\hat{p}}}})({\hat{\mathbf{p}}}_{k} - {\mathbf{\overline{\hat{p}}}})^\text{T} } \\ & {\text{Cov}}^{*} ({\kern 1pt} {\hat{\mathbf{\mu }}}) = \frac{1}{K - 1}\sum\limits_{k = 1}^{K} {({\hat{\mathbf{\mu }}}_{k} - {\mathbf{\overline{\hat{\mu }}}})({\hat{\mathbf{\mu }}}_{k} - {\mathbf{\overline{\hat{\mu }}}})^\text{T} } \\ \end{aligned}$$
(24)

where the total number of noise realizations is \(K = 1000\).

The generalization error (GE) [33] represents the sum of the bias and variance and measures the generalization ability of a neural network:

$${\text{[GE]}}_{n} = ([{\hat{\mathbf{\mu }}}]_{n} - [{{\varvec{\upmu}}}^{*} ]_{n} )^{2} + [{\text{Cov}}({\hat{\mathbf{\mu }}})]_{n,n},$$
(25)

where \({{\varvec{\upmu}}}^{*}\) represents noise-free reconstruction.

A pixel-wise noise reduction percentage (NPR) is calculated to analyze the denoising performance of U-net.

$${\text{[NRP]}}_{n} = \left| {\frac{{\sqrt {[{\text{Cov}}({\hat{\mathbf{\mu }}})]_{n,n} } - \sqrt {[{\text{Cov}}({{\varvec{\upmu}}})]_{n,n} } }}{{\sqrt {[{\text{Cov}}({{\varvec{\upmu}}})]_{n,n} } }}} \right| \times 100\%$$
(26)

Here, \({{\varvec{\upmu}}} = {\mathcal{O}}({\mathbf{p}})\) and \({\text{Cov(}}{{\varvec{\upmu}}}) = {\mathcal{O}}{\text{Cov}}({\mathbf{p}}){\mathcal{O}}^\text{T}\), according to Eq. (20). We choose one low-attenuation point and one high-attenuation point in test phantom B to present the trend of GE with the noise level; the two points are marked by red dots in Fig. 2b.

We also calculate the NPS to analyze the noise spatial correlation of U-net in the Fourier domain:

$${\text{NPS}} = \mathcal{F}[{\text{Cov}}({\hat{\mathbf{\mu }}})],$$
(27)

where \(\mathcal{F}\) denotes the Fourier transform operator.

4 Experimental results

Because the linear approximation of nonlinear activation functions requires the mean of the input in Eq. (6), we only use one noise realization as the mean of the input to analytically estimate the covariance of the projection acquired by U-net and its corresponding reconstruction by FBP.

4.1 Validation of the proposed analytical covariance estimation method for U-net

For test phantom A, the variance of the projections obtained by U-net is illustrated in Fig. 4a, and its covariance at the center of the projections is presented in Fig. 4b. The results reveal that the variance estimation is in agreement with the reference for both the activation functions. The error between the variance estimation and reference is not significant compared with the variance itself. We observe that the variance obtained from LeakyReLU varies sharply when approaching the boundary, whereas that from Swish changes smoothly. Meanwhile, the covariance estimation also agrees with the reference, where the error is primarily statistical. The shape of the covariance from the two activation functions is quite different; it is circular for LeakyReLU and elliptical for Swish. For test phantom B, good agreement can still be observed between the variance estimation and reference (as shown in Fig. 4c). Sharp changes also occur near the boundary in the projection variance from LeakyReLU. As shown in Fig. 4d, the covariance estimation for both activation functions is in agreement with the reference.

Fig. 4
figure 4

Variance of the projections obtained from U-net and its covariance at the center of projections when \(I_{0} = 10^{4}\). a and c projection variance of phantom A and B used for testing, respectively. b and d corresponding covariance at the center of the projection

The variance and covariance of the reconstructions obtained by FBP from projections denoised by U-net are presented in Fig. 5. For both phantoms and activation functions, the variance estimation of the reconstructions agrees with the reference because FBP is linear. Meanwhile, the error that propagates through the FBP is not a concern. We discover that the central areas of variance from the two activation functions have different appearances. The central area appears dark for LeakyReLU and bright for Swish in the same display window. The covariance estimation is yet again in agreement with the reference, leaving only statistical noise in the error map.

Fig. 5
figure 5

Variance of the reconstructions by FBP from projections obtained from U-net and its corresponding covariance at the center of the reconstructions for \(I_{0} = 10^{4}\). a the reconstruction variance of test phantom A and b its covariance at the center of reconstructions. c the reconstruction variance of test phantom B, and d its covariance at the center of reconstructions

The profiles of the variance and covariance for the projections and reconstructions are plotted in Fig. 6. For both phantoms and activation functions, the profiles of the variance and covariance estimations match those of the references. As shown in Fig. 6a1, c1, the profile of the projection variance from Swish appears smooth, whereas that from LeakyReLU appears sharp. The profile of covariance (as shown in Fig. 6b1 and d1) from Swish demonstrates a larger spread, whereas that from LeakyReLU exhibits sharper changes. For the profiles of the reconstruction variance, displayed in Fig. 6a2 and c2, we discover that the noise from LeakyReLU is lower than that from Swish. The variance gradually decreases from the edge of the field of view (FOV) to its center for LeakyReLU, whereas it demonstrates an opposite behavior for Swish. Although the values of the projection covariance are close, the absolute value of the reconstruction covariance from LeakyReLU is much smaller than that from Swish; this demonstrates that the covariance from Swish is more structurally related and difficult to deal with.

Fig. 6
figure 6

(Color online) Profiles of the projection variance and covariance obtained from U-net, and profiles of its reconstruction variance and covariance obtained from FBP for \(I_{0} = 10^{4}\). a1d1 profiles of the projection variance and covariance: a profiles of the variance marked by a red solid line in Fig. 3a for test phantom A, and b1 profiles of its covariance marked by a red solid line in Fig. 3b, c profiles of the variance marked by a red dashed line in Fig. 3c for test phantom B, and d1 profiles of its covariance marked by a red dashed line in Fig. 3d. a2d2 profiles of the reconstruction variance and covariance: a2 profiles of the variance marked by a red solid line in Fig. 4a for test phantom A, and b2 profiles of its covariance marked by a red solid line in Fig. 4b, c2 profiles of the variance marked by a red dashed line in Fig. 4c, and d2 profiles of its covariance marked by a red dashed line in Fig. 4d

In addition, we estimate the variance of the projections obtained by U-net with LeakyReLU under different noise levels for test phantom B. As shown in Fig. 7, the variance estimation agrees with the reference for different noise levels. Although the error between the variance estimation and reference increases with the noise level, it is still insignificant compared with its corresponding variance.

Fig. 7
figure 7

Variance of projections obtained from U-net with LeakyReLU under different noise levels for test phantom B

The noise estimation of U-net with LeakyReLU in the practical experiment is illustrated in Fig. 8. It is apparent that both the variance and covariance estimations from the analytical method agree well with the references, which strongly demonstrates the feasibility of the proposed analytical noise propagation method in practical usage.

Fig. 8
figure 8

(Color online) Noise properties of reconstruction from the projection produced by U-net with LeakyReLU for the practical experiment. a The variance and profiles are marked by the red solid line, and b the covariance at the center of the reconstruction and the profiles are marked by the red solid line

4.2 Performance analysis with analytical covariance

Pixel-wise GE maps for test phantom B are illustrated in Fig. 9. For both activation functions, GE increases for each pixel with increasing noise levels, which indicates that U-net is inapplicable for highly noisy projections. When \(I_{0}\) increases to a certain number, the decrease in GE is not significant. The GE in the smooth region is smaller than that at the edge when the number of incident photons increases to \(3.25 \times 10^{4}\), indicating that U-net is more effective in smooth regions. Compared with the GE for LeakyReLU, the GE for Swish is relatively large in smooth regions, whereas it is almost the same at the edge.

Fig. 9
figure 9

GE maps of U-net with varying noise levels for test phantom B

Further, noise reduction percentage (NRPs) are listed in Table 4. For both activation functions, the increase in NRP from \(I_{0} = 10^{3}\) to \(I_{0} = 5.5 \times 10^{3}\) is approximately 20%, and this increase quickly slows to approximately 5% or less. For both low- and high-attenuation points, the noise reduction effect of LeakyReLU is stronger than that of Swish at various noise levels. The NRP of LeakyReLU is approximately 10% higher than that of Swish, particularly for the low-attenuation point at \(I_{0} = 10^{3}\), and the difference decreases to approximately 1% for a low noise level with \(I_{0} = 5.05 \times 10^{4}\). For the high-attenuation point, the difference in the NRP for both activation functions is smaller than 3% and becomes even smaller when the number of incident photons increases. The NRPs at both points for LeakyReLU are comparable, which suggests that LeakyReLU treats low- and high-attenuation areas equally during noise suppression. The NRP at the low-attenuation point for Swish is slightly smaller than that at the high-attenuation point; however, the difference between the NRPs at the low- and high-attenuation points gradually reduces to approximately 1% with decreasing noise levels.

Table 4 NRPs of U-net with varying noise levels. Point ① represents the low-attenuating point, and point ② indicates the high-attenuating point. Both points are marked by red dots in Fig. 2(b)

The NPS at the center of the reconstruction is illustrated in Fig. 10. For each noise level, the NPS at the center of the reconstruction from both activation functions decreases as \(10^{3}\) increases to \(5.05 \times 10^{4}\) and drops by approximately an order of magnitude from \(10^{3}\) to \(5.5 \times 10^{3}\). The NPS from LeakyReLU first increases to a maximum and then decreases as the frequency increases, whereas that from Swish continues to increase with increasing frequency. Both LeakyReLU and Swish exhibit similar NPS shapes at low frequencies, indicating that their performance in dealing with low-frequency noise is comparable. The high-frequency noise in the NPS from LeakyReLU gradually reduces; however, it increases considerably for Swish, suggesting that more structures are present in the reconstruction noise propagated through U-net with Swish.

Fig. 10
figure 10

(Color online) NPS at the center of reconstructions varying with the number of incident photons. a NPS of reconstructions with varying noise levels. bd NPS profiles of different noise levels: b NPS profiles at \(I_{0} = 10^{3}\), c NPS profiles at \(I_{0} = 5.5 \times 10^{3}\) and \(10^{4}\), and d NPS profiles at \(I_{0} = 3.25 \times 10^{4}\) and \(5.05 \times 10^{4}\)

5 Discussion and conclusion

In this study, an analytical noise propagation method for CNNs in CT imaging is proposed. The five basic layers that comprise a typical CNN include the convolution, nonlinear activation, batch normalization, average pooling, and up-sampling layers. Except for the nonlinear activation layer, the other four layers are all linear, which simplifies the estimation of the covariance of their output by linear propagation. The 1st order Taylor expansion is used to obtain the linear approximation of the nonlinear activation layer for linear propagation of noise. By integrating the noise propagation of both linear and nonlinear layers in the CNN, we can estimate the covariance of reconstruction from the projection in a step-by-step manner.

The results indicate that the covariance estimated by the proposed analytical method agrees well with that estimated by the statistical method, regardless of phantoms, noise levels, and activation functions. We demonstrate that it is feasible to propagate noise from only a single projection to image reconstructed from CNN. The covariance of the projection obtained from U-net with the gradient continuous activation function Swish is smooth, whereas that with the discontinuous gradient activation function LeakyReLU exhibits sharp changes near the boundary. The noise in the reconstruction from LeakyReLU is smaller than that in Swish. They demonstrate opposite performances, where the variance from Swish gradually decreases from the FOV edge to its center. Therefore, LeakyReLU and Swish are completely different in terms of noise suppression. The covariance for Swish spreads wider than that for LeakyReLU, which indicates that Swish uses the information of more neighborhood pixels in denoising.

We further qualitatively and quantitatively evaluate network performance from three aspects. The GE, which contains bias and variance, is a tradeoff between the accuracy and noise of the network output and measures the generalization of the neural network. Trained with data under the condition of \(I_{0} = 10^{4}\), the network fails to reduce the GE for projections with lower incident photons, which renders it unacceptable for application in projection denoising with high noise levels. This also limits its application to projections with lower noise when \(I_{0}\) increases to a certain number because the improvement in GE is trivial. A pixel-wise NRP is defined to measure the denoising ability of the network. The effect of noise suppression is strong only when the noise level is sufficiently high; otherwise, it quickly weakens as the noise level decreases. An evident drop in GE can be observed in smooth regions but not at the edge when the number of incident photons increases, although the NPRs for smooth regions and the edge are the same. Therefore, the accuracy of the smooth regions is higher than that of the edges. In addition, the spatial correlation of noise is analyzed using the NPS. Consequently, it is discovered that there is no significant difference in NPS between LeakyReLU and Swish at low frequencies. However, the NPS at high frequencies is completely opposite for these two activation functions, where it weakens with increasing frequencies for LeakyReLU. The variance in projection denoised by the two activation functions is comparable; however, the NRP of the reconstruction from LeakyReLU is larger than that of Swish, and the NPS of the reconstruction from Swish increases as the frequency increases. Thus, both activation functions demonstrate comparable performance in projection denoising; however, their different noise distributions lead to different effects of noise suppression in reconstruction. Swish utilizes information from more adjacent pixels in noise reduction; therefore, its noise is too structural to be handled by FBP. The noise correlation of LeakyReLU is lower and easier to process. Therefore, the image quality of the reconstructions from LeakyReLU is better than that of Swish in terms of noise suppression.

In summary, the proposed analytical noise propagation method is capable of providing a reasonable pixel-wise noise property estimation from only a single sample, whereas other noise estimation methods cannot present comparable performance under the same conditions. Our proposed method can be applied to any inference-ready CNN with a fixed structure and weight for noise estimation. Because the convolution, batch normalization, average pooling, and up-sampling operations are all linear, and the nonlinear activation function is linearly approximated, the noise of the network input propagates linearly in the network. Evidently, the error in the noise estimation of the network output results from the linear approximation of the nonlinear activation function. Two activation functions, LeakyReLU and Swish, are validated in this study; hence, the proposed method is applicable to any network with these two activation functions, regardless of the network structure. Moreover, the noise property estimation of the network output can be used to evaluate the performance of the reconstruction methods. We can characterize noise features based on pixel-by-pixel noise estimation, which also enables us to analyze the spatial correlation and structural properties of noise. The experimental results reveal the significant value of this method in evaluating the output from CNN methods. In future studies, we aim to study the application of covariance estimation to a model observer for subjective image quality assessment. However, the computational cost is expected to increase with increasing network complexity and dimensions. Hence, efficient noise propagation methods for complex and high-dimensional networks must be studied.