1 Introduction

Video and image compression standards have been widely used on the Internet and other digital media. They are created to maintain the image and video fidelity at high compression rates. Three-dimensional (3D) media are rapidly growing with the expansion of digital equipment, and it has the potential to become mainstream soon. Accordingly, much effort has been devoted to reducing the amount of storage and bandwidth needed in 3-D technology [15]. Currently, there are many standards for holography image compression, including Jpeg, Jpeg2000, Vp9, and HEVC / H.265. All image compression methods use similarities between different parts of an image in various domains, such as frequency and time. These methods keep some pieces of similar data to reduce the size of final file with minimum quality compromise. Advanced compression systems employ more sophisticated domains to find weak similarities and remove redundant information. Also, modern compression methods benefit from different mathematical operations and feedbacks to compensate for errors. In brief, receive and transmit sophistication is compromised to reduce the amount of transmitted data without negligible quality loss. Most image compression methods use time, frequency, or time-frequency domains because many structural similarities can be found in these spaces. For hologram images, the depth dimension is also added to the image. Therefore, exploring similarities in 3D matrices is crucial for them. Most of modern image/video compression methods have embedded wavelet and compensator blocks. One of the highly efficient and compact standards is known as advanced video coding (AVC). There are critical challenges in extending such approaches to 3D images. Based on the different natures of recording and storing 3D images, the effectiveness of the conventional AVC is debatable [30]. Here, we consider digital holography (DH), one of the leading technologies in 3D images and videos with widespread adoption and importance in the field. The HEVC / H.265 is among the best standards to efficiently store and transmit the DHs. In a typical HEVC codec, some steps predict and compensate compression error, which results in a reasonable compression rate and weak correlation between the compression error and the original image [24]. This low correlation makes it difficult to estimate the compression error based on the original image and enhance compression quality.

Holography is a 3D image or video recording technology that uses interference between a reference wave and a wavefield. It requires an object reflection on the hologram plane with minimum noise to have a high-quality hologram image. There are three standard methods for holography recording [18]:

  • Phase-shifting holography: It requires sampling the interference intensity field of the object at three different phases. These three-phase shift waves are called interferograms. One of the most valuable advantages of using this technique is that it can quickly make a hologram, but it needs at least three 2D matrices (images) to store a 3D image, which is not optimal in terms of memory. A three-phase shift hologram can be given as [18]:

    $$ \begin{array}{@{}rcl@{}} {E_{0}}(x,y) &=& \frac{{1 - i}}{{4E_{R}^{*}}}\left\{ {{I_{H}}(x,y;0) - {I_{H}}\left( x,y;\frac{\pi }{2}\right)} \right.\\ &&\left. { + i\left[ {{I_{H}}\left( x,y,\frac{\pi }{2}\right) - {I_{H}}(x,y;\pi )} \right]} \right\}, \end{array} $$
    (1)

    where E0 and ER are the complex object and the complex reference wavefield, respectively. The IH represents the various interferograms for three different phases; i is the imaginary number and ∗ is the complex conjugate operator.

  • Phase-Shifted Distances holography: (1) can be rewritten to improve computation speed and memory usage as:

    $$ \begin{array}{@{}rcl@{}} {D_{1}} &=& {I_{H}}(x,y;0) - {I_{H}}\left( x,y;\frac{\pi }{2}\right),\\ {D_{2}} &=& {I_{H}}\left( x,y;\frac{\pi }{2}\right) - {I_{H}}(x,y;\pi ),\\ {E_{0}}(x,y) &=& \frac{{1 - i}}{{4E_{R}^{*}}}\left( {{D_{1}} + i{D_{2}}} \right), \end{array} $$
    (2)

    where D1 and D2 are equal to the difference of phase-shifted interferograms and \(i = \sqrt { - 1} \). Here, only two matrices are required (D1 and D2) instead of three 2D matrices. Memory usage and the necessary computations are substantially lower than the phase-shifting holography.

  • Complex object wavefield: In this technique, holographic data are modeled as a complex function. Two forms of complex representation are prevalent: Cartesian (real-imaginary value) and polar (absolute-phase). In this light, the object wavefield can be expressed as [18]:

    $$ \begin{array}{@{}rcl@{}} {E_{o}}(x,y) &=& {\text{Re}} \left[ {{E_{o}}(x,y)} \right] + ilm\left[ {{E_{o}}(x,y)} \right],\\ {E_{o}}(x,y) &=& \left| {{E_{o}}(x,y)} \right|{e^{i {\measuredangle}{E_{0}}(x,y)}}. \end{array} $$
    (3)

    where E0 is the complex object wavefield; Re[.] and Im[.] represent the real and imaginary parts of the complex number; |.| is the absolute value, and \(\measuredangle \) is the argument of a complex number. In this case, the hologram needs to store two matrices, similar to (B). Hence, the memory usage and the required computation are lower than (A). Meanwhile, it also needs less calculation than (B), because it does not need to multiply \((1-i)/(4E_{R}^{*} )\) by (D1 + iD2), and directly stores the final complex answer.

Based on our research, complex object wavefield is superior to other holography forms. To the best of our knowledge, one of the main challenges in holography is to propose a codec in terms of lower quality loss, higher compression rate, better compatibility with the holographic displays, and noise resilience than state of the art codecs. Although many research has been done in 3D hologram codecs, this topic is still an important field in holography. The HEVC is one of the best, state of the art codecs that works effectively on 3D holograms. In this research, a hybrid HEVC-Wavelet routine is suggested to overcome the HEVC shortcomings. In summary, the main innovations in this research can be grouped into the following categories:

  • Using wavelet to reduce the size of real and imaginary parts of complex object wavefield.

  • Using dissimilarity between nearest-neighbor interpolation and wavelet in image resizing to keep details of holograms.

  • Using the HEVC error to half-size hologram that it can be coded effectively by HEVC and used for enhancing quality.

It is worth mentioning that proposing a new codec better than HEVC is a significant achievement because HEVC is believed to be near optimal codec. Indeed, this codec is emerged as a result of sustained research several years. Empirical results demonstrate that the proposed method can result in significant bitrate savings while maintaining the quality of service in holograms (object plane). In the meantime, the suggested codec can also be applied to traditional 2D images effectively.

This article is organized as follows. In the next section, an overview of the previous works is given. This section is designed to provide a perspective of the proposed approach among the state of the art literature. Then, the theory of the proposed method is detailed in Section 3. In the fourth section, database details and the simulation results are presented. The results analysis is presented in the fourth section. Finally, the conclusion of the work is given in Section 5.

2 Related works

There are many different hologram coding methods, which can be classified into three categories:

2.1 Wavelet-based methods

The wavelet-based methods use the wavelet transform. It has proven excellent compression performance for two-dimensional images because of its adaptation to the time-frequency domain. Darakis et al. [10] compress phase shift holograms using the wavelet transform and compared their method to the Set partitioning in hierarchical trees (SPIHT) algorithm. Wavelet-Bandelets transform is employed for 3D holographic image compression in [2], resulting in a higher compression ratio than conventional wavelet methods. Blinder et al. [4] apply wavelet for Off-axis Holographic image compression. They show Off-axis holographic recordings could be compressed lossy or lossless more effectively by Direction-Adaptive Discrete Wavelet Transforms (DA-DWT) such as Mallat. In a review article, Viswanathan et al. [27] evaluate various wavelet schemes for compressing holographic images. They try to find some selective values for each codec parameter to enhance compression performance. The authors show Gabor wavelets have better time-frequency localization than Fresnelet based wavelets and hence are recommended for hologram compression. Viswanathan et al. used Morlet wavelet in another work [28] for the adaptive reconstruction of holograms. They showed that Morlet wavelets can be used effectively for 3DTV display systems better than conventional wavelet methods. Viswanathan et al. suggested another framework for view-dependent hologram representation and adaptive reconstruction [29]. There, some wavelet coefficients are eliminated according to the user’s position. Also, the best quality is produced by the Shannon wavelet. Blinder et al. used modulo wavelet transform as a multiscale phase unwrapping method in 2D image compression [6]. This method can be generalized to holograms based on its properties on multiscale phases. They speed-up their approach by inserting the reversible modulo operator, resulting in the right candidate for real-time hologram compression systems. Cheremkhin and Kurbatova [8] proposed a method using Meyer and reverse biorthogonal wavelets to compress off-axis digital holograms. The suggested method uses frequency filtering, wavelet decomposition, threshold processing, and quantization.

2.2 Hybrid methods

One of the most prevailed approaches in the field of hybrid methods is the High-Efficiency Video Coding (HEVC) method, which is a hugely optimized method while evaluated mainly against other codecs. Sharabayko compared VP9 and HEVC codecs in 3D images [23]. Based on the comparisons made, HEVC has shown better performance, better compression, and lower quality in 3D images compared to other methods. Bernardo et al. compared JPEG codecs such as JPEG 2000, JPEG XT, and JPEG, quality in holograms against HEVC in different compression ratio/quantization rations and real-imaginary versus amplitude-phase form. They demonstrated that HEVC has better performance in comparison to JPEG codecs in 3D holograms [19]. Peixeiro et al. evaluated various encoding methods on different holographic image databases. Based on the results obtained, the authors concluded that HEVC is also a very efficient method and mostly has better performance than other methods in different conditions [18]. Grange et al. compared AV1 codec as a free codec against VP9 and HEVC. The results proved that in most holograms, the AV1 could equate HEVC and VP9 in quality. The AV1 is a free codec and is more accessible than HEVC, indicating that it can be used instead of HEVC in software and applications of 3D media [31]. Bernardo et al. analyzed some details in HEVC compression, including quantization number and real-imaginary versus amplitude-phase effect on output quality in object plane versus hologram plane. They study the impact of HEVC parameters in different holograms on these two planes [3]. Finally, Corda et al. analyzed the quality of different codecs in holograms using objective tests [9]. Its results confirmed the superiority of HEVC codec over other conventional coding methods.

2.3 Other methods

In some researches, such as the ones presented by Seo et al. [22] and Senoh et al. [21], mathematical operations or geometric patterns were used to compress particular types of holograms. Symeonidou et al. [26] proposed a motion compensation scheme to generate compressed holographic video more efficiently. The efficiency of these methods is often lower than HEVC and is not addressed in this article. As a summary of related works, the HEVC method has shown better compression quality than conventional methods in all hologram types. Most articles and research have compared HEVC with other codecs, while the authors found no study trying to enhance the HEVC itself. Considering that the newest, well-known reference (based on our best knowledge) that considered HEVC quality in holographic images has been done in 2018, and after that, no further work has been done. However, in order to make the Related Works section more complete, several recent research on the HEVC codec has been added that consider the HEVC codec in 3D images, but these research aimed to reduce the complexity of the HEVC codec. In the study by Zhang et al. [32], an improved HEVC method was proposed to increase the speed and reduce the complexity. This work used some dependencies in the components of the holographic images to decrease codec computations and speed up the coding process. However, the output quality has been slightly reduced in this method. Amish and Bourennane in [1] suggested a hardware solution that reduced the complexity of HEVC codec. In the method proposed by Chen et al. [7], some hologram components are predicted that can accelerate the HEVC codec with minimal output quality loss. Hamout and Elyousfi in [13] used an edge-finding algorithm in the 3D images to identify regions of interest in the hologram and increase the speed of the standard HEVC algorithm. The method has also led to a decrease in hologram quality in comparison to the standard codec. Saldanha et al. [20] used tile adaptation to balance the computational load of the HEVC encoder on the holographic image. This method, which speeded up the HEVC codec via parallel processing, has been tested on a multi-core system. Nevertheless, none of the new research discussed quality enhancement, and their goal has been to speed up HEVC process in 3D images and reduce the computational complexity. As a summary, the research done in 2018, such as [9, 18, 19] are of the latest research conducted on the quality of the HEVC codec. In this article, a combination of HEVC codec, wavelet transform, and nearest-neighbor interpolation is applied to create an efficient scheme to enhance DHs quality in the object plane. The proposed method compresses the approximate wavelet channel of a 3D image; it also computes compression error and encodes this error using HEVC. In addition to their reverse procedure in the decoder, there is an extra step based on nearest-neighbor interpolation to reconstruct the original DH using coded image and compression error. As confirmed by extensive simulations, this asymmetric algorithm enhances HEVC efficiency considerably.

3 Proposed approach

3.1 High-efficiency video coding

The HEVC method is one of the best coding standards and is the result of the H.264 upgrade. Due to the increasing demand for high-quality videos, a video coding expert group and a moving picture expert group jointly proposed a video codec called Advanced Video Coding (AVC), later known as H.264. This method, which was then used in many hardware and software implementations, is a very efficient way of transmitting video and images, especially for smartphones. After some time, the same team enhanced AVC by adding some new blocks to optimize the algorithm and compensate compression loss in output and provided a new optimized codec, known as high-efficiency video coding (HEVC). The HEVC algorithm has about a 50% lower bit rate than H.264 while maintaining quality. The main disadvantage of HEVC is that it was not released for commercial use for free [14, 17]. It should be noted that this technology is now almost the best available standard in terms of compression and quality. The proposed method is implemented using HEVC codec version 16.18

3.2 Wavelet-based methods

Discrete Wavelet Transform (DWT) has been widely applied to various fields of image and video processing, such as in image denoising, segmentation, compression, and coding. In this research, the discrete wavelet transform is used to image resizing, i.e., creating a half-scale image of the original image. DWT, due to its nonlinear bank of filters and wavelet mother function, can affect the image pixels contrary to the typical interpolation or geometric operations used in image resizing. The half-size approximated channel of 2D wavelet transform output can produce better visual result than conventional image resizing methods. The use of DWT is not usually applicable in image resizing because the output dimension is limited to some specific sizes, not selectable by the user. According to some literature, such as Dumic et al. [11] and Gajitzki [12], image resizing using wavelet transform is more efficient in its specific dimensions than conventional methods. Here, the wavelet transform is used as a downward image resizer to overcome the inherent weakness of conventional resizing methods. Generally, 2D wavelet transform can only reduce the image dimensions. In contrast, the traditional image resizing methods able to increase or decrease the image size, so in the proposed approach, 2D wavelet is used to reduce hologram size, and classic image resizing methods is used to transform hologram to its original size. Interestingly, the proposed two-way resizing utilizing a combination of wavelet and conventional manner provides lower loss compared to a two-way resizing using just traditional methods in both ways. Thus, one of the novelties of the proposed method is the application of different resizing methods before and after the HEVC encoding step to enhance coding efficiency.

3.3 The proposed hybrid HEVC-Wavelet compression algorithm

The proposed hybrid image compression method uses a half copy of the image and error. First, a prediction step is considered for the conventional encoder and decoder in HEVC to predict the error in the receiver and add it to the original image to enhance codec efficiency. Since HEVC/H.265 is a highly optimized method, its output error is almost independent of input, and it cannot be directly predicted or minimized. Therefore, we kept an approximate channel of 2D wavelet transforms to reduce the image size to half (line 3-4 in encoding step pseudocode). The output of this resized image using HEVC needs lower storage and bandwidth to transmit. The main challenge is that HEVC is a more efficient compression method than any resizing method. For addressing this challenge, a novel compensation scheme is added to the resizing step. Figure 1 depicts the block diagram of the proposed algorithm encoding steps. Based on the wavelet transform properties, the wavelet approximation channel with the HAAR function can produce an image similar to the input image but with half input dimensions. This image is coded using the HEVC codec (codec version 16.18, line 5 in pseudocode). Successively, the image is decoded and returned to its original size using nearest-neighbor interpolation (line 6 in encoding step pseudocode). In this case, the wavelet transform is not efficient enough to return to its original size because only one channel is transmitted. Therefore, we applied nearest-neighbor interpolation to restore the image to its original size. In this light, we can produce an asymmetry between wavelet transform and nearest-neighbor interpolation on both sides of the HEVC. Here, the compression error, which is calculated between expanded and original images, is employed as a compensation reference (pseudocode line 7). The size of this error is reduced to its half by using the nearest-neighbor interpolation. This resized error is then coded using HEVC (line 9). This half-size coded error and coded approximate wavelet channel are merged in a file and sent to the receiver (line 10). Each of these images is half-sized compared to the original image.

Fig. 1
figure 1

Block diagram of the proposed algorithm encoding step

Full size image

In the decoding step, the four images combined in the encoding step are extracted from the input file. The real and imaginary parts belong to main hologram, which had halved dimensions with wavelet transform, are decoded by HEVC. Nearest-neighbor interpolation is used due to the lack of all channels of wavelet transform and only having approximate channel to return these images to their original sizes. Two half-dimensional error images are first decoded with the HEVC and then resized by the nearest neighbor interpolation. In the proposed method, the resizing process of the original images is different in encoding and decoding steps. This dissimilarity has led to the preservation of some details and increased quality. Eventually, the returned original size error images and the main holo images are added to form the real and imaginary holographic channels final output. The details of this method are described in Fig. 2.

Fig. 2
figure 2

Block diagram of the proposed algorithm decoding step

Full size image
figure a

This dedicated routine is more efficient than the standard HEVC codec, which can perform better in terms of compression rates and image quality under the same conditions than the standard HEVC codecs in 3D holographic images.

Some other modes, such as the use of dimensional nearest-neighbor interpolation or other wavelet functions, are examined on both sides of the codec; none could prove more efficient than the proposed method. It should be noted that the proposed approach is more efficient at high compression rates, where the image is highly corrupted.

figure b

Some other modes, such as nearest-neighbor interpolation or other wavelet functions, are examined on both sides of the codec; none could prove more efficient than the proposed method. It should be noted that the proposed approach is more efficient at high compression rates, where the image is highly corrupted.

4 Experimental results

In this research, the holograms have been selected from Interfere I & II datasets built by Peter Schelkens from ETRO-VUB, which can be downloaded from http://www.erc-interfere.eu/. Since each hologram’s parameters are different, Table 1 indicates all the selected holograms and other details required for simulation. Figure 3 shows the selected holograms. All simulations were performed on the object plane for both real and imaginary parts of the holographic image separately with similar conditions for the standard HEVC and the proposed method, and the results were compared. The flowchart of the evaluation system is illustrated in Fig. 4. Here, the input image is first transformed to the range 0 to 255, then converted to an 8-bit integer and submitted to the codec. The codec output after decompression, i.e., the decoded image, is compared with the codec input image based on the quality criteria.

Table 1 Hologram Characteristics
Full size table
Fig. 3
figure 3

Studied holograms

Full size image
Fig. 4
figure 4

Block diagram of the used quality assessment scheme

Full size image

All simulations were performed on the object plane for both real and imaginary parts of the holographic (object plane) input image separately with similar conditions for standard HEVC and the proposed method, and the results were compared. The flowchart of the evaluation system is illustrated in Fig. 4. Here, the input image is first transformed to the range 0 to 255, then converted to an 8-bit integer and submitted to the codec. The codec output after decompression, i.e., the decoded image, is compared with the codec input based on the quality criteria.

For performance evaluations, the proposed method is implemented using a combination of HEVC reference codec version 16.18 [20] and an added function of MATLAB 2018 software. The evaluations was also performed on a computer with a Intel Core i7-9700 CPU (3.00 GHz) and 64GB RAM. The runtime and output of the HEVC standard codec based on [3, 18,19,20] was stored and used for comparison. The simulation parameters for HEVC and the proposed method are precisely the same. The metrics used for efficiency evaluation are bitrate, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Bjontegaard metric (BD-PSNR) under various quantization parameters (QPs), including 51, 48, 46, 44, 42, 40, 38. It should be noted that any compression method in holography images should be checked on two real-imaginary or amplitude-phase images because there are two parts of data in 3D images where the results may be different on these parts. All DH samples belong to the Interfere I and II datasets [14, 17]. Figures 5, 6, 7 and 8 show the PSNR and SSIM obtained for real-imaginary parts of two sample digital holography images (3-DMulti and Dragon8Ks), which are taken from the Interfere I and II datasets, respectively. In these figures, circle markers belong to the proposed method, while the diamond is for the standard HEVC. Both results show that the proposed method was superior to standard HEVC [3, 18,19,20] in all simulated quantization parameters in terms of PSNR. Each QP is written inside the corresponding marker for better comparison.

Fig. 5
figure 5

PSNR for different QPs in imaginary (A) and real part (B) of 3D Multi holography image

Full size image
Fig. 6
figure 6

SSIM for different QPs in imaginary (A) and real part (B) of 3D Multi holography image

Full size image
Fig. 7
figure 7

PSNR for different QPs in imaginary (A) and real part (B) of Dragon8Ks holography image

Full size image
Fig. 8
figure 8

SSIM for different QPs in imaginary (A) and real part (B) of Dragon8Ks holography image

Full size image

To reveal the efficiency of the proposed method, the BD-PSNR and BD-Rate of the proposed method and standard HEVC in some different DH images are calculated, Table 2. There is a natural trade-off between BD-PSNR and BD-rate, meaning the increase of one would bring the decrease of the other. Based on results given in Table 2, one can confirm that the BD-PSNR increased about 0.55 dB, while BD-Rate significantly decreased about -19.5% for imaginary and real parts of the 3DMulti dataset. These two parameters reveal the excellence of the compression efficiency for the proposed method versus normal HEVC. The best enhancement occurred in Chess8KS, which belongs to the Interfere II dataset, with more than 1 dB in both real and imaginary parts, which demonstrates that the proposed method has better compression in high-resolution images. The lowest enhancement belongs to real and imaginary parts of Ball8KS, about 0.35 dB BD-PSNR and -7.5% BD-Rate, which are still better than the standard HEVC algorithm. In addition, Table 3 presents the coding run time for each hologram as an indication of the computational complexity for the normal HEVC and the proposed method. Note that the proposed method encodes four half-size holograms and its computational cost is about 30% higher than a conventional encoder. This is because four halved holograms may require more encoding time than two real and imaginary full-size parts and some additional blocks such as wavelets and resizing steps, which are used in the proposed encoding routine.

Table 2 Average BD-PSNR and BD-Rate of imaginary and real Part for different DHs
Full size table
Table 3 Encoding Run Times for normal HEVC and HEVC-Wavelet
Full size table

5 Conclusions

This study introduced a hybrid HEVC-Wavelet compression algorithm that employs the central core of HEVC and approximate channel of 2D wavelet transform for predicting the compression error and provides a method of compensating it. Unlike other prediction algorithms, the proposed scheme uses an asymmetric methodology to overcome the uncorrelated form of HEVC compression error. Experimental results showed that bit rate saving about 17.5% (based on average BD-Rate values of Table 2) can be achieved at the same or slightly better PSNR compared with the standard HEVC codec. Given the structure of the proposed method that can be applied to other codecs, the effect of the proposed method to increase compression efficiency in other hologram codecs can be considered in future research.