Keywords

1 Introduction

The reversible variant of the discrete wavelet transform (DWT) is used in the lossless JPEG 2000 compression to decompose an image into subbands of different characteristics, that are then independently entropy coded [16, 33]. In [27] we noticed, that the lifting steps (LS) [7, 32] employed by DWT may propagate noise between subbands and worsen the compression effects. To limit the noise propagation we proposed the reversible denoising and lifting step (RDLS), which is built based on LS. It prevents the noise propagation by exploiting denoising filters while retaining other desirable effects and properties of LS (like the perfect reversibility). We found that RDLS improves the compression effects of algorithms exploiting DWT [27] and color space transforms [25]. In [28] we showed, for a color space transform consisting only of LSs, that by applying RDLS with a special denoising filter we may practically skip selected LSs of the transform or the entire transform. Results obtained in [27] suggested, that a partial skipping of DWT may be worthwhile. Since DWT consists not only of LSs but also of the non-lifting reorder step, in [29] we proposed the skipped steps DWT (SS-DWT) obtained from DWT by skipping selected steps of its computation. To construct SS-DWT in an image-adaptive way, we employed a heuristic and, based on its outcomes, we defined two simple fixed SS-DWT variants FIX1 and FIX2. SS-DWT resulted in compression ratio (bitrate) improvements similar to the RDLS-modified DWT (RDLS-DWT), but obtained at a smaller cost. The fixed variants are especially interesting from a practical standpoint because they allow obtaining high bitrate improvements at a very small cost and are compliant to the JPEG 2000 part 2 standard [13, 30].

So far, the step skipping was applied in compression of two-dimensional (2D) images only. In this paper, we report the preliminary results of applying it to the 3-dimensional DWT (3D-DWT) in lossless compression of volumetric medical images. In particular we generalize the FIX1 and FIX2 variants to a 3D case and apply them in JP3D, i.e., volumetric extension of JPEG 2000 standard [4, 14].

The task of improving the lossless JP3D bitrates of volumetric medical images is not simple and not many attempts are reported in the literature. Two interesting approaches were evaluated in [4]. In that study, the direction-adaptive DWT (DA-DWT), earlier used for 2D data [5, 8, 9], was applied to volumetric medical images. Also the other approach, the block-based intra-band prediction of DWT transformed subbands (JP3D+BP), was an adaptation of the method presented in [24] for 2D images. These approaches improve average bitrates of JP3D for medical volumes by less than 1% at a cost of a high increase in the compression process complexity. We will compare our results with findings reported in [4] and, for this purpose, we will use the same set of test data. We note, that video coding algorithms, like the state-of-the-art High Efficiency Video Coding (HEVC) standard [15, 35], may be applied to medical volumes treated as sequences of volume slices, however, the average lossless HEVC bitrate for the aforementioned set is inferior to the JP3D bitrate [4].

We also note, that RDLS and step skipping may technically be applied to lossy compression, however, to our best knowledge, no such attempt has been undertaken. An application of lossy compression to medical images is still disputable, guidelines issued by various professional bodies recommend different lossy compression ratios, and in some cases lossy compression of such images is forbidden by law [6, 19]. For some modalities the compression algorithm and lossy compression ratio utilized may have a negative impact on the results of automatic analyses of the decompressed images (e.g., in biomedical applications exploiting texture operators [18, 22]).

The remainder of this paper is organized as follows. In subsections of the next Section, we first briefly describe 2D- and 3D-DWT (Sect. 2.1) and the modifications of DWT investigated earlier: RDLS-DWT, SS-DWT, and fixed variants FIX1 and FIX2 of 2D-SS-DWT (Sect. 2.2). Then, in Sect. 2.3, we propose 3D variants of FIX1 and FIX2. Section 2.4 contains the experimental procedure including the description of the test images and the used implementations. The experimental results are presented and discussed in Sect. 3. Section 4 summarizes the findings.

2 Materials and Methods

2.1 Lifting-Based Discrete Wavelet Transform

DWT employed in image compression algorithms decomposes an image into subbands of different characteristics (they represent image details of different orientations and sizes). A subband is easier to encode efficiently than the original image because its well-defined characteristics allow for better statistical modeling. The decomposition into subbands has many additional advantages for the lossless and lossy coding, e.g., it allows for various kinds of progressive coding. For brevity, as in the previous works [29, 30], we describe here only the lifting-based reversible DWT with Cohen Daubechies Faveau (5,3) wavelet filter, reduced to essentials. Among others, it is exploited in lossless JPEG 2000 compression of 2D and volumetric images. For further details, as well as for more general characteristics of various variants of DWT, their application in JPEG 2000, and the JPEG 2000 standard, the reader is referred to [1, 4, 13, 14, 16, 33].

The one-dimensional DWT (1D-DWT) transforms in-place a discrete signal \(S = s_0 s_1 s_2 \dots s_{l-1}\) of finite length l into two subbands:

  • a low-pass filtered signal L, that represents the low-frequency features of S;

  • a high-pass filtered signal H containing high-frequency features that, along with the L signal, allows the perfect reconstruction of the original signal.

The transformation of S is done in 3 steps. First, in the prediction step, we perform the high-pass filtering of odd samples (hereafter, the parity of the sample is determined by its location and not its value) by applying to each of them the LS presented in below Eq. 1:

$$\begin{aligned} s_x \leftarrow s_x - \lfloor (s_{x-1}+s_{x+1})/2 \rfloor . \end{aligned}$$
(1)

In each LS a single signal sample is modified by adding to it a linear combination of other samples (in a general case the sum may be negated). A transform performed as a sequence of LSs has advantageous properties: it may be computed in-place and it is easily and perfectly invertible.

Another LS (Eq. 2) is then, during the update step, applied to each even sample:

$$\begin{aligned} s_x \leftarrow s_x + \lfloor (s_{x-1}+s_{x+1}+2)/4 \rfloor . \end{aligned}$$
(2)

Finally, in the reorder step, we reposition even samples to the lower half of the original signal, preserving their ordering (sample \(s_x\) is moved to \(s_{x/2}\)), and odd samples are moved to the upper half. We obtain separate subbands L and H, respectively. The reorder step is not an LS.

The 2D-DWT of an image is obtained by first applying 1D-DWT to each image column, which results in L and H subbands of the image (Fig. 1A–B). Then, by applying 1D-DWT to each row, we obtain the 1-level DWT consisting of LL and HL subbands (transformed from L subband) and LH and HH subbands (from H subband)—see Fig. 1C.

Fig. 1.
figure 1

1-level 2D-DWT (A–C) and 3-level 2D-DWT (D)

Full size image

The 3D-DWT of a volumetric image is obtained by first applying 1D-DWT in the axial direction, which produces two (volumetric) subbands L and H (Fig. 2A–B). Next, we proceed like in the 2D-DWT. That is, we apply 1D-DWT to the volume in the vertical direction (Fig. 2C), which results in the LL and HL subbands (transformed from the L subband) and LH and HH subbands (from the H subband). Finally we apply 1D-DWT horizontally obtaining the 1-level 3D-DWT of the volume (Fig. 2D), that consists of 8 subbands: LLL and HLL (transformed from the LL subband), LHL and HHL (from HL), LLH and HLH (from LH), and LHH and HHH (from HH). In Fig. 2B–D each subband is labeled with a number (in round brackets) of the group of steps (GS) that has the most recently been used to filter all the samples contained in the subband. For instance, GS 4 in Fig. 2C corresponds to all the prediction LSs employed when transforming the H subband into LH and HH. We will use GSs in defining 3D fixed variants of SS-DWT.

Fig. 2.
figure 2

1-level 3D-DWT (A–D) and 3-level 3D-DWT (E); coordinate system presented in panel (A), in round brackets presented are GS numbers

Full size image

The higher-level DWT, that provides multiresolution image representation, is obtained by Mallat decomposition [20]. The \(t+1\)-level transform is obtained by applying the 1-level transform to the low-pass subband of the t-level transform, i.e., to the LL in the cases of 2D-DWT (Fig. 1D) and LLL for 3D-DWT (Fig. 2E).

2.2 Reversible Denoising and Lifting Steps and Step Skipping

The idea of skipping selected steps of a transform originates from RDLS. An unwanted side effect of LS is that the sample being modified by LS (filtered by LS in the case of DWT) gets contaminated by noise from other samples. In the case of DWT, the noise gets propagated between subbands. JPEG 2000 encodes the DWT subbands independently; the noise propagation increases the amount of information that has to be encoded and thus worsens the compression effects. RDLS is a modification of LS (and consequently of the lifting-based transform) that integrates LS with denoising filters in order to avoid noise propagation while preserving other properties of LS (and of the lifting-based transform). For instance, in RDLS-DWT the prediction (Eq. 1) and update (Eq. 2) steps are replaced by RDLSs constructed based on them, i.e., by:

$$\begin{aligned} s_x \leftarrow s_x - \lfloor (s_{x-1}^d+s_{x+1}^d)/2 \rfloor \,\,\, \hbox {and} \end{aligned}$$
(3)
$$\begin{aligned} s_x \leftarrow s_x + \lfloor (s_{x-1}^d+s_{x+1}^d+2)/4 \rfloor , \end{aligned}$$
(4)

respectively, where \(s_{i}^d\) denotes the denoised sample \(s_{i}\).

RDLS was successfully applied to DWT [27] and to reversible color space transforms—first [25] to a simple RDgDb [26] and then [28] to more complex color space transforms: LDgEb [26], RCT [16], and YCoCg-R [21]. Both in the cases of reversible color space transforms (for JPEG-LS [12, 34], JPEG 2000, and JPEG XR [10, 17]) and the DWT exploited in lossless JPEG 2000, the application of RDLS resulted in practically useful improvements of image compression ratios of certain types of images. For example, in the latter case the lossless compression ratios of non-photographic images were improved by about 14% [27].

RDLS has very interesting properties. Despite exploiting the inherently irreversible denoising, it is perfectly reversible. The RDLS-modified transform is more general than the original one. Among others, by employing special denoising filters we may obtain, as a special case of the RDLS-modified transform, the unmodified transform. Such a special filter case denoted None, for which \(s_i^d=s_i\), was proposed and investigated in [27]. Another special filter, the Null filter proposed in [28] (for which \(s_i^d=0\)), allows to practically skip the step. For further details and properties of the RDLS approach we refer the reader to [27, 28].

In [27] we found that the noise filtering in RDLS-DWT was the most effective in improving lossless JPEG 2000 bitrates when applied during computing of some RDLS-DWT subbands only and since in some cases the best bitrates were obtained when the DWT stage of JPEG 2000 was skipped, we suspected that similarly to denoising, the optimum might be in-between skipping and applying DWT. As opposed to RDLS-modified color space transforms, employing the Null filter in RDLS-DWT does not allow to skip the entire transform. Color space transforms consist of nothing but the lifting steps, whereas DWT and RDLS-DWT, besides the lifting prediction and update steps (that may be turned into \(s_x \leftarrow s_x\) by employing the Null filter in RDLS-DWT), perform the sample reordering steps.

Therefore in [29] we proposed SS-DWT obtained from DWT by skipping selected steps of its computation. We employed a heuristic for selecting steps to be skipped in an image-adaptive way and, based on outcomes of the heuristic, defined two simple fixed SS-DWT variants FIX1 and FIX2. For brevity, we refer the reader to [29] for a detailed description and properties of the general SS-DWT case and describe below only the fixed variants.

In the FIX1 variant of SS-DWT we skip all the update steps. In FIX2 we skip all the update steps (as in FIX1) and additionally skip the prediction step for the HH subband as well as the reorder step for HH and LH. Noteworthy, FIX2 results in a decomposition of an image into fewer subbands than the regular DWT or FIX1 because the HH and LH subbands are not created from the H subband that remains unchanged after the transform—see Fig. 3. As we noticed in [29] and verified experimentally in [30], the FIX1 and FIX2 variants of SS-DWT are compliant with the JPEG 2000 part 2 by exploiting standard extensions defined in annexes H (FIX1) or F and H (FIX2) [13].

Fig. 3.
figure 3

1-level FIX2 variant of SS-DWT (A–C) and 3-level FIX2 (D)

Full size image

The most interesting results, from a practical standpoint, were obtained by applying entropy estimation of JPEG 2000 coding effects for selecting among the fixed SS-DWT variants, the unmodified DWT, and the skipping of the DWT stage of JPEG 2000. The average bitrate improvement due to selecting among the above-mentioned fixed variants was similar to that of RDLS-DWT (roughly 14% for non-photographic images), but it was obtained at a significantly smaller cost; the overall compression time was only 3% greater than that of the unmodified JPEG 2000. We remark, that so far the research on RDLS has been carried out using very simple denoising filters only (we used simple RCRS filters [11] like the median filter). We suspect, that RDLS effects could be improved by the use of more sophisticated filters, e.g., based on detector precision characteristics or operating in the transform domain [2, 3]. In [29] we also found, that by combining SS-DWT and RDLS-DWT even greater bitrate improvements (of up to about 17.5% for non-photographic images) might be obtained at a significantly increased cost of heuristic-based selecting, based on the actual bitrate instead of an estimated one, of the steps to be skipped and the denoising filters. However, we do not expect so high bitrate improvements in the case of volumetric medical images, because the characteristics of these images resemble characteristics of photographic images, for which the JPEG 2000 bitrate improvement due to applying RDLS-DWT and SS-DWT was below 1%.

2.3 Fixed Variants of Three-Dimensional Skipped Steps Discrete Wavelet Transform

2D-DWT may be obtained as a special case of 3D-DWT when the volume size in the axial direction is 1. Thus, 3D variants of fixed SS-DWT, should be defined in such a way, that for the above volume they become their 2D counterparts (FIX1 and FIX2) and in general follow the same principles (when to skip the step) as their 2D counterparts. A three-dimensional variant of FIX1 (3D-FIX1), naturally, is obtained from 3D-DWT by skipping all the update steps. In FIX2 we additionally skipped prediction of the HH subband (i.e., of samples that had already been subjected to prediction), and reordering of samples from HH and LH (i.e., we skipped reordering of samples if for each of the signal samples to be reordered during 1D-DWT the prediction or update has been skipped). Fortunately, this may be extended to the 3D case. In 3D-FIX2 variant of 3D-SS-DWT (Fig. 4) we skip

  • all the update steps,

  • prediction for HH and reorder for HH and LH (the H subband of the volume does not get decomposed into LH and HH, once created, the H subband remains unchanged after the transform), and

  • prediction for HHL and reorder for HHL and LHL (the HL subband of the volume does not get decomposed into LHL and HHL and remains unchanged after the transform).

Fig. 4.
figure 4

1-level 3D-FIX2 (A) and 3-level 3D-FIX2 (B)

Full size image

Some of the volumes we used for evaluation in this paper use the same resolution in all directions, whereas for others a different resolution is used in the axial direction. Another case, when the axial direction should be treated differently than others, happens when a volume data represents a time series of 2D images. Above suggests, that the pixel correlation in the axial direction may be different than in other directions. Therefore we define also fixed variants of 3D-SS-DWT that either always perform the 1D-DWT in the axial direction (variants denoted 3D-FIX1p and 3D-FIX2p) or always skip such transform (denoted 3D-FIX1s and 3D-FIX2s) and to each volume slice apply 2D FIX1 or FIX2, respectively. In Table 1, for each 3D fixed SS-DWT variant, we list which GSs of 3D-DWT (recall Fig. 2B–D) are skipped by this variant; when applying 1D-DWT, if both prediction and update steps are skipped, then the reorder step is skipped as well.

Table 1. The investigated variants of 3D fixed SS-DWT
Full size table

Variants 3D-FIX2, 3D-FIX2p, 3D-FIX1s, and 3D-FIX2s result in decomposition of a volumetric image into fever subbands, than 3D-DWT. Only some of the 3D fixed SS-DWT variants are compliant to the JPEG 2000 standard and may be obtained without altering the standard implementation. 3D-FIX1 may be obtained by exploiting JPEG 2000 part 2 Annex H (i.e., the arbitrary wavelet transform kernels) supported by JP3D (although not supported by the implementation we used in the research reported herein). Unfortunately, the JPEG 2000 part 2 Annex F that would allow obtaining 3D-FIX2 is not supported by JP3D. Out of the remaining 3D fixed SS-DWT variants, only the 3D-FIX1s may be obtained in compliance with the standard by using JPEG 2000 part 2 extension defined in Annex H and 0-level decomposition in the axial direction (allowed by JP3D).

2.4 Procedure

We investigated the effects of 3D fixed SS-DWT variants on compression ratios of the JP3D compressor. For SS-DWT variants we employed the RDLS-SS-DWT version 0.9Footnote 1—our implementation of SS-DWT (and of RDLS-DWT). RDLS-SS-DWT is available as a patch to the IRIS-JP3D version 1.1.1Footnote 2—a JPEG 2000 part 10 [4, 14] reference software, developed by Tim Bruylants from Vrije Universiteit Brussel (VUB) and the Interdisciplinary Institute for BroadBand Technology (IBBT). In our implementation, the SS-DWT variants were obtained by modifying the DWT stage of JP3D; the transformed subbands were compressed by further JP3D stages as if the regular 3D-DWT was applied. For instance, 3D-FIX2 skips 1D-DWT in vertical direction for the HL subband not decomposing it into LHL and HHL (see Fig. 4A and Fig. 2D). In our implementation, the JP3D statistical modeling for arithmetic coding of HL symbols was divided into two stages between which the model was reinitialized. This could slightly worsen the obtained bitrates—only slightly because, as compared to the 2D image compression [30], the subbands of volumetric images are much larger and the impact on bitrate, of the unnecessary re-adaptation of the model, is correspondingly smaller.

In experiments, we used the set of medical volumetric images, that was earlier used in [4] for evaluation of other modifications of 3D-DWT and was made available to us thanks to the courtesy of Tim Bruylants. The set contains 11 images of various modalities:

  • 6 computed tomography scans (CT1...CT6) of 12-bit depth and sizes (width \(\times \) height \(\times \) depth, in pixels) from 512 \(\times \) 512 \(\times \) 44 to 512 \(\times \) 512 \(\times \) 672,

  • 3 magnetic resonance imaging scans (MRI1...MRI3) of 12-bit depth and sizes from 256 \(\times \) 256 \(\times \) 100 to 432 \(\times \) 432 \(\times \) 250, and

  • 2 ultrasound volumes (US1 and US2) of 8-bit depth and sizes 500 \(\times \) 244 \(\times \) 201 and 352 \(\times \) 242 \(\times \) 136, respectively.

Images are described in detail in [4].

The compression ratio or bitrate r, expressed in bits per pixel (bpp), is calculated using the total size in Bytes of the compressed volumetric image including the compressed file format header. The bitrate is directly proportional to the compressed file size, hence smaller bitrate means better compression result. The effects of 3D fixed SS-DWT variants on JP3D bitrate were analyzed based on bitrate changes with respect to the reference bitrate obtained using unmodified 3D-DWT. The bitrate change \({\varDelta }r\) was expressed in percentage of the reference bitrate.

Besides comparing bitrates obtained using the 3D fixed SS-DWT variants presented in Table 1 to the bitrate of unmodified 3D-DWT, we also tested effects of skipping the transform stage (NO-DWT). For DWT and SS-DWT, we used the 3-level decomposition (0-level for NO-DWT). Except for setting the transform variant and the decomposition level, we invoked the compressor with default parameters.

3 Results and Discussion

In Table 2 we present JP3D bitrates (r) for volumetric images from our test-set and bitrate changes (\({\varDelta }r\)) obtained by replacing 3D-DWT with 3D fixed SS-DWT variants listed in Table 1 or by skipping the DWT stage. For the latter case, we see that skipping the DWT stage results in a significant bitrate worsening for all images, the average bitrate increase is about 57%. These results are consistent with the earlier research on 2D-SS-DWT [29, 30], where although omitting DWT allowed for improving the bitrates of some of the images, but almost exclusively they were non-photographic images (e.g., computer-generated, composed from others including natural ones, or screen content images [23]). In the above research, for evaluating the SS-DWT effects on single-component 2D image compression, the green components of images from a large CT2 set introduced in [31] were used. For photographic images (typical natural continuous-tone images) that have characteristics resembling medical volumes, by skipping the DWT stage an improvement was obtained for 1 out of 499 such images. Also the SS-DWT variants, in which 1D-DWT in the axial direction is skipped for the entire volumetric image (3D-FIX1s and 3D-FIX2s), worsen bitrates for all images, although to a lesser extent (by 11.06% and 12.47%, respectively). We generally see that DWT is an appropriate transform for the preparation of volumetric medical image data for further stages of compression.

However, skipping some transform steps only or even skipping 1D-DWT in a certain direction, but only for a part of the volumetric image being transformed, may result in bitrate improvements. Figure 5 shows the cases for which a bitrate improvement was obtained by using one of the proposed variants of 3D fixed SS-DWT. The best results were obtained for 3D-FIX1p—this variant improves bitrates of 4 images (CT4...CT6 and MR1) on average by 1.23%. Note, that this variant applied to all images worsens the bitrate by 0.09% on average. Therefore, in order to obtain an improvement of the JP3D bitrate with the use of 3D fixed SS-DWT variants, the effects of applying the transform variant to a given volume should be checked (or estimated) and SS-DWT should be applied only if it results in a bitrate improvement. By using 3D-FIX1p in such image-adaptive way, we get an average bitrate improvement for the whole set of images by 0.45%.

Table 2. JP3D bitrate changes due to applying various 3D fixed SS-DWT variants
Full size table
Fig. 5.
figure 5

Bitrate improvements obtained using 3D fixed SS-DWT variants

Full size image

The remaining variants (3D-FIX1, 3D-FIX2, and 3D-FIX2p) allow improving bitrates of fewer images (each time being an image, whose bitrates are improved by 3D-FIX1p). For some images, choosing instead of 3D-FIX1p a different 3D fixed SS-DWT variant gives greater bitrate savings. By adaptively choosing the best variant for the image (for images from the exploited test set, the choice can be limited to 3D-FIX1, 3D-FIX2, and 3D-FIX1p) the average improvement for images, for which SS-DWT is effective, increases from 1.21% to 1.61%. The average improvement for the whole set of images increases to 0.59%.

Looking at the modalities of images we see, that the fixed SS-DWT variants are not effective for ultrasound volumes and in the case of other modalities they are effective for some images. However, the numbers of images in individual modalities are too small to draw significant conclusions with respect to the medical volumetric image modality.

The obtained bitrate improvement appears small from a practical point of view, but the result is not bad compared to other methods of improving the DWT effects for JPEG 2000 compression of continuous-tone images. In [30], using adaptively selected fixed SS-DWT variants for the 499 aforementioned 2D photographic images, an average improvement of 0.62% was obtained for the baseline JPEG 2000; exploiting the extensions of part 2 of JPEG 2000 standard resulted in an improvement of 0.68%. In [29] we showed that the improvement achieved using the general SS-DWT case is greater than for the fixed variants and can be further increased by combining SS-DWT with RDLS-DWT (for the aforementioned 2D images, the improvement was 0.92%). Then, by using only the general SS-DWT case, but with an improved heuristic of the adaptive selection of steps to be skipped, an improvement of 0.98% was obtained. In [4], for the same set of volumetric medical images that is used in this paper, by using much more complex modifications of 3D variants of DWT and JPEG 2000 (i.e., the DA-DWT and JP3D+BP mentioned in Sect. 1), the bitrates were improved only little more, than by using the adaptively selected 3D fixed SS-DWT variants. JP3D+BP was more effective than DA-DWT and resulted in a bitrate improvement of 0.87%.

4 Conclusion

In this paper, we report preliminary results of applying the step skipping to 3D-DWT in lossless compression of volumetric medical images. In particular, we generalized the FIX1 and FIX2 variants of SS-DWT to the 3D case (obtaining 6 fixed variants 3D-FIX1, 3D-FIX2, 3D-FIX1p, 3D-FIX2p, 3D-FIX1s, and 3D-FIX2s) and employed them in the JP3D compressor. We evaluated these variants on a set of 11 medical volumetric images (modalities CT, MRI, and US) and found, that 3D-FIX1, 3D-FIX2, 3D-FIX1p, and 3D-FIX2p allowed for bitrate improvements. By adaptively selecting a fixed 3D variant of SS-DWT for each image, we improved the JP3D bitrate on average by 0.59%. This result is not impressive form a practical standpoint, but it is competitive to much more complex modifications of DWT and JPEG 2000. The results we obtained for 3D data along with earlier results for 2D images suggest, that further JP3D bitrate improvements are possible by using an adaptively constructed general case SS-DWT, probably also combined with RDLS, which is currently being investigated.