1 Introduction

The 3D multi-view video coding (MVC) [1, 4, 6, 21, 24, 30, 34, 38, 40, 43] achieves efficacious compression performance for 3D video transmission. It has acquired a great concern, lately. The standard of MVC encoding is a supplement of the standard of 2D video encoding, and it is expected to rapidly supersede the 2D compression standard in several directions such as gaming, entertainment, education, medicine, and 3DTV. The 3D sequence is composed of various streams taken for the same object with various cameras. Therefore, it is an imperious task to accomplish adequate encoding to adapt forthcoming bandwidth demands, while achieving the recommended 3DV quality performance. The communication of 3DV data via wireless networks has been dramatically increased [2]. To transport 3D videos through restricted-resource networks, a robustly efficient compression mechanism must be exploited, while obtaining good 3DV reception quality. To achieve sufficient communication of the 3DV information, the 3D compression standard must take the advantage of the inter-view correlations within different 3D streams in addition to the temporal and spatial similarities among contiguous frames in the selfsame video to improve the encoding performance. On the other hand, a highly video encoding is more sentient to communication corruptions.

The 3D video streaming over wireless noisy networks is constantly affected by the transmission packet corruptions of both random and bursty natures [11, 14, 24, 25, 29, 31]. Therefore, it is a critical issue due to the limited available bandwidth and the impossibility of re-transmitting all corrupted MBs. The 3D-MVC compression predictive framework presented in Fig. 1 [8] is utilized to encode the transmitted 3D videos. It comprises the P and B inter-compressed frames and the I intra-compressed frames. Thence, the corruptions might be broadcasted to the adjoining views or to the following frames, and therefore, they may result improper 3D video performance. To decrease the influence of packet faults prior 3D video communication, there are several methods that can be used [32, 39]. Unluckily, they increase the transmission delay and bit rate. It is known that there is a difficulty for re-transmitting all lost or erroneous MBs especially in the case of real-time communication. Thus, there is a need for post-processing concealment schemes at the decoder [9, 12, 19, 42]. The concealment techniques exploit the great advantage of the inter- and intra-view matching within the 3DV streams to reconstruct the erroneous MBs or frames. They can decrease the transmission corruptions without requiring sophisticated encoder amendments or rising the communication delay. The concealment schemes introduced to recover the erroneous 2D streams can be employed and exploited to conceal the corrupted 3D streams [3, 5, 10, 13, 16, 17, 20, 23, 24, 26, 33, 37, 42]. They are predictable to be more efficient in reconstructing the 3D stream corruptions due to the utilization of the inter-view correlation inside the 3DV frames [8].

Fig. 1
figure 1

3D video compression framework

Full size image

The 2D video concealment techniques proposed in the literature can be employed with some modifications to compensate the communication losses of the transmitted 3D streams. These 2D video error control techniques are expected to be further advantageous for recovering the communication losses in the 3D streams due to the great availability of the inter-view correlations inside the different views of the transmitted 3D streams. The DIECA and OBBMA [26] are two of the most efficient ones from several EC techniques that have been proposed to compensate the 2D video degradations resulting from wireless channels. The OBBMA is a temporal EC technique that defines the motion vectors among the external boundaries of the exchanged pixels and the same outer boundaries of the pixels neighboring to the erroneous MB. It only uses the reference macro-block external borders to determine the highest correlated adjoining motion vectors as indicted in Fig. 2a. Otherwise, the DIECA is inter-view and spatial EC technique that recovers the DVs to recover the corrupted pixels of the lossy frame through carefully searching inside the reference macro-blocks to define the best object boundary orientation. The highest defined object boundary value is selected to conceal the corrupted pixels of the erroneous 3D images as presented in Fig. 2b.

Fig. 2
figure 2

Temporal, spatial, and inter-view EC algorithms: a OBBMA and b DIECA

Full size image

Since the previously suggested EC techniques [3, 5, 10, 13, 16, 17, 20, 23, 24, 26, 33, 37, 42] rely on the precision and availability of the contiguous DVs or MVs, the scenarios with dense errors and with all the contiguous MBs lost will not be dealt efficiently with techniques such as the straightforward Frame Temporal Replacement Algorithm (FTRA) [23]. Thence, it is approved that although most of the previous concealment schemes attain reliable concealment results, they cannot perform sufficiently in the state of severe transmission losses, particularly when several surrounding macro-blocks are lost. Furthermore, it is also observed that although some of the traditional algorithms can operate efficiently with random channel corruptions, others fail in lessening the burst slice channel corruptions. Thus, we suggest an efficient concealment technique, which is more reliable in the state of severely erroneous random and bursty channel cases. It outperforms the state-of-the-art EC algorithms, which achieve low reception performance in the case of enormous slice corruptions and heavily lossy MBs. Moreover, the proposed algorithm in this work can predict the optimum candidate motion and disparity vectors for concealment through the employment of the BKF. So, to compensate severe 3D frames losses, in this paper, we suggest an efficient technique to improve the motion and disparity vectors estimation, by exploiting a BKF scheme [37] to the motion and disparity vectors prediction process. The BKF is applied to predict unstable indistinctive processes from a group of noisy experiments [13]. The motion and disparity vectors prediction process is considered as a set of noisy observations. Based on this similarity, the proposed BKF-based EC (BKF-EC) scheme is applied to the received heavily corrupted 3DV to alleviate the communication corruptions through smoothing the determined motion and disparity vectors that are primarily predicted via the utilization of the DIECA and OBBMA. Consequently, the inherent corruptions among the DIECA-determined DVs and the OBBMA-determined MVs could be isolated, and therefore, the heavily erroneous frames can be recovered with high precision. Therefore, the proposed BKF-based EC technique can present more recommended qualities than those that have been attained by the traditional concealment techniques, while presenting minimum computations. The proposed BKF-based EC scheme neither requires complicated encoder mechanisms nor increases the communication latency. It can furthermore perform efficiently with the state of severe frame faults ignored in the traditional error control works.

It is noticed that the KF was used previously in the state-of-the-art works [13, 24, 33, 37]. It was applied to the 2D H.264/AVC videos as a post-processing EC technique. The proposed work is different from the previously presented works with several modifications. The suggested BKF-based EC scheme conceals the 3D H.264/MVC videos with heavily corrupted MBs with random and bursty PLRs beyond 20%. The proposed technique in this paper outperforms those embedded in the related works. The suggested algorithm and the experimental simulation results will be introduced in the next sections. In Sect. 2, the basic BKF model is introduced. Section 3 presents the suggested BKF-based EC scheme. Section 4 gives a discussion of the experimental simulation results and a comparative analysis. In Sect. 5, we summarize the concluding remarks of the whole work.

2 Bayesian Kalman Filter

The BKF is a process that characterizes the discrete-time mechanisms and their mathematical features. It is utilized to determine the events s(k) of the discrete-time process concerned with the linear difference stochastic formula from a group of determined measurements y(k). The BKF mathematical model can be described as in Eqs. (1) and (2).

$$ {\mathbf{s}}(k) = {\mathbf{A}}(k - 1){\mathbf{s}}(k - 1) + {\mathbf{w}}(k) $$
(1)
$$ {\mathbf{y}}\left( k \right) = {\mathbf{H}}\left( k \right){\mathbf{s}}\left( k \right) + {\mathbf{r}}\left( k \right) $$
(2)

In Eqs. (1) and (2), s(k) = [s1(k), s2(k), …, sN(k)]T, r(k) is an M × 1 measurement noise vector with N(0,\( \sigma_{r}^{2} \)), y(k) is an M × 1 measurement vector, H(k) is an M × M measurement matrix, A(k-1) is an N × N transition matrix, and w(k) is an N × 1 white noise vector with N(0,\( \sigma_{w}^{2} \)). In this work, we assumed that the measurement r(k) noise and process white noise w(k) follow independent and normal distributions. In this work, we employed the BKF through the motion and disparity vectors estimation process. We assumed that the motion and disparity vectors prediction model of the lossy macro-blocks from the reference neighboring MBs is a Markov model. Therefore, the BKF may work efficiently for the disparity and motion vectors prediction process for refining the corrupted MBs. The BKF prediction framework consists of two different steps, which are the measurement and time updates as observed in Fig. 3 [28]. The \( \mathop {\mathbf{p}}\nolimits_{k} \), \( {\hat{\mathbf{s}}}_{k} \), \( {\mathbf{p}}_{k}^{ - } \), \( {\hat{\mathbf{s}}}_{{\bar{k}}} \), and \( \mathop K\nolimits_{k} \) refer to posteriori estimated error covariance, posteriori estimated state, priori estimated error covariance, priori estimated state, and KF gain. The BKF is utilized to define the filtered states s(k) from a group of noisy events y(k) [28]. It exploits the determined state through the previous measurement and time updates to define the current state.

Fig. 3
figure 3

The BKF prediction mechanism

Full size image

3 The Proposed BKF-Based Concealment Scheme

The suggested BKF-based EC scheme that will be utilized for smoothing the formerly determined disparity and motion vectors via the DIECA and OBBMA is introduced in this section. Through the BKF-based EC process, to recover the erroneous disparity and motion vectors of the transmitted MBs, we exploit the feature of the 3D stream inter-view, temporal, and spatial correlations among frames and views. In this work, we employ the compression prediction framework introduced in Fig. 1 because of its sufficient compression and decoding performance. It presents the inter-view correlation among different views and also the spatial–temporal correlation among different frames within the same view stream. The motion compensation prediction (MCP) and disparity compensation prediction (DCP) are exploited at the encoder to obtain sufficient 3D video compression and moreover at the decoder for implementing sufficient error control mechanism [10]. In the MVC prediction structure (MVC-PS) depicted in Fig. 1, the MCP and DCP are utilized to estimate the motion vectors and disparity vectors within different adjoining 3D frames in the selfsame view and in the different views of the transmitted 3D streams, respectively. Thence, each 3D frame inside the 3D MVC-PS is determined via the temporally contiguous 3D frames and/or via different view frames. The even (S2, S4, and S6) views are encoded relying on the motion prediction, while their elementary images are encoded through the utilization of the disparity prediction. The motion prediction is also used to compress the frames in the S0 view. For efficiently compression of the frames inside the odd (S1, S3, and S5) views, both of the motion and disparity predictions are jointly employed.

In this work, we define the 3D stream view relying on its elementary 3D video frame. Thus, as depicted in Fig. 1, the (S1, S3, and S5) views are defined as the B views, the (S2, S4, and S6) views are defined as the P views, and the S0 view is defined as the I view. The last view could be odd or even relying on the utilized 3D stream prediction framework. The 3D compression structure presents various sorts of compressed 3D frames, which are the P and B inter-frames and the I intra-frames. The intra-frames in the I view and as well the inter-frames in the P views are jointly exploited to determine the inter-frames in the B views. Thence, if any corruption happens at the P or I 3D frames, it is broadcasted to the relative inter-frames and furthermore to the adjoining temporal 3D images in the present 3D view. Therefore, in this work, we introduce several cases of error recovery assumption for all 3D frames within the 3D views of the suggested 3D video prediction framework. Thence, the proposed error recovery scheme efficiently chooses the convenient concealment algorithm (DIECA or OBBMA or both) relying on the lossy 3D frame sort inside different view types to recover the severely erroneous blocks. Subsequently, to remove the inherent corruptions among the DIECA-determined DVs and the OBBMA-determined MVs, the proposed BKF is utilized to reinforce the 3DV reception performance. Figure 4 shows the detailed structure of the proposed BKF-based concealment scheme, which is explained with more details in the pseudo-code steps of Algorithm (1).

Fig. 4
figure 4

The suggested framework of BKF-based EC algorithm

Full size image

In Fig. 4, the BKF algorithm is employed as a rectification prediction process for the specified measurement corruptions of the DIECA-determined DVs and OBBMA-determined MVs. We supposed that any determined DV or MV is specified through the state vector s = [sn, sm]. Also, we supposed that the formerly reconstructed measurement DV or MV is characterized through y = [yn, ym], where m and n represent the MB height and width. The eligible result s(k) of the kth frame is supposed to be chosen from that state at frame k − 1 utilizing (3). The measurement y(k) of the vector s(k) state is determined using (4). Thence, the state and measurement model for DV or MV estimation may be formulated as in (3) and (4), where r(k) is an M × 1 measurement noise vector with N(0, \( \sigma_{r}^{2} \)) and w(k) is an N × 1 white noise vector with N(0, \( \sigma_{w}^{2} \)). In the suggested BKF-based EC scheme, we supposed that the observation r(k) and constant process w(k) noises follow zero-mean Gaussian noise distributions defined with the constant covariance of \( \sigma^{2}_{w} \) = \( \sigma_{r}^{2} \) = 0.50I2, where I2 is 2 × 2 unity matrix. We assumed that the measurement r(k) noise and process white noise w(k) follow independent and normal distributions as indicated in the lower part of Eqs. (3) and (4). Moreover, we supposed that the constant measurement H(k) and transition A(k) matrices are two-dimensional unity matrices. It is as well assumed that there is more matching amidst the vertically and horizontally adjoining frames.

figure a
$$ \left[ \begin{aligned} \mathop {\mathbf{s}}\nolimits_{{\mathop n\nolimits_{k} }} \hfill \\ \mathop {\mathbf{s}}\nolimits_{{\mathop m\nolimits_{k} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} 1 \hfill \\ 0 \hfill \\ \end{aligned} \right.\begin{array}{*{20}c} {} \\ {} \\ \end{array} \left. \begin{aligned} 0 \hfill \\ 1 \hfill \\ \end{aligned} \right]\left[ \begin{aligned} \mathop {\mathbf{s}}\nolimits_{{\mathop n\nolimits_{k - 1} }} \hfill \\ \mathop {\mathbf{s}}\nolimits_{{\mathop m\nolimits_{k - 1} }} \hfill \\ \end{aligned} \right] + \left[ \begin{aligned} \mathop {\mathbf{w}}\nolimits_{{\mathop n\nolimits_{k - 1} }} \hfill \\ \mathop {\mathbf{w}}\nolimits_{{\mathop m\nolimits_{k - 1} }} \hfill \\ \end{aligned} \right] ;$$
(3)

\( \;\mathop {\mathbf{w}}\nolimits_{{\mathop n\nolimits_{k - 1} }} ,\mathop {\mathbf{w}}\nolimits_{{\mathop m\nolimits_{k - 1} }} \sim N(0,\mathop \sigma \nolimits_{w}^{2} ) \)

$$ \left[ \begin{aligned} \mathop {\mathbf{y}}\nolimits_{{\mathop n\nolimits_{k} }} \hfill \\ \mathop {\mathbf{y}}\nolimits_{{\mathop m\nolimits_{k} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} 1 \hfill \\ 0 \hfill \\ \end{aligned} \right.\begin{array}{*{20}c} {} \\ {} \\ \end{array} \left. \begin{aligned} 0 \hfill \\ 1 \hfill \\ \end{aligned} \right]\left[ \begin{aligned} \mathop {\mathbf{s}}\nolimits_{{\mathop n\nolimits_{k} }} \hfill \\ \mathop {\mathbf{s}}\nolimits_{{\mathop m\nolimits_{k} }} \hfill \\ \end{aligned} \right] + \left[ \begin{aligned} \mathop {\mathbf{r}}\nolimits_{{\mathop n\nolimits_{k} }} \hfill \\ \mathop {\mathbf{r}}\nolimits_{{\mathop m\nolimits_{k} }} \hfill \\ \end{aligned} \right];$$
(4)

\( \;\mathop {\mathbf{r}}\nolimits_{{\mathop n\nolimits_{k} }} ,\mathop {\mathbf{r}}\nolimits_{{\mathop m\nolimits_{k} }} \sim N(0,\mathop \sigma \nolimits_{r}^{2} ) \)

The suggested BKF-based EC estimation scheme is described by the two different steps of the updating and prediction stages as illustrated in Sect. 2. The detailed suggested BKF improvement scheme is described by the following three stages:

  • Stage (A): initialization stage

    1. 1.

      Define the previously measured noisy DIECA-determined DVs or/and OBBMA-determined MVs.

    2. 2.

      Put the prime state and covariance vectors as in (5).

  • Stage (B): prediction stage

    1. 1.

      Estimate the determined priori estimated state (DV or MV) \( {\hat{\mathbf{s}}}_{{\bar{k}}} \) for frame k utilizing the preceding (DV or MV of the MB) updated state \( {\hat{\mathbf{s}}}_{k - 1} \) of frame k-1 as in (6).

    2. 2.

      Determine the calculated predicted priori error covariance \( {\mathbf{p}}_{k}^{ - } \) employing the noise covariance process \( \sigma^{2}_{w} \) and the updated estimated covariance \( {\mathbf{p}}_{k - 1} \) as in (7).

  • Stage (C): updating stage

    1. 1.

      Determine the estimated updated state (finally estimated DV or MV) vector \( {\hat{\mathbf{s}}}_{k} \) employing the measurement state \( \mathop {\mathbf{y}}\nolimits_{{_{k} }} \)(the recovered DIECA-determined DVs or/and OBBMA-determined MVs) and the observation covariance noise \( \sigma_{r}^{2} \) as in (8).

    2. 2.

      Determine the filtered updated covariance error \( \mathop {\mathbf{p}}\nolimits_{k} \) via the estimated covariance \( {\mathbf{p}}_{k}^{ - } \) and the observation covariance noise \( \sigma_{r}^{2} \) as in (9). The BKF gain \( \mathop K\nolimits_{k} \) is noticed by (10).

$$ \left[ \begin{aligned} \mathop {\mathbf{s}}\nolimits_{{\mathop n\nolimits_{0} }} \hfill \\ \mathop {\mathbf{s}}\nolimits_{{\mathop m\nolimits_{0} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} 0 \hfill \\ 0 \hfill \\ \end{aligned} \right]\begin{array}{*{20}c} {} \\ \end{array} ,\begin{array}{*{20}c} {} \\ \end{array} \left[ \begin{aligned} \mathop {\mathbf{p}}\nolimits_{{\mathop n\nolimits_{0} }} \hfill \\ \mathop {\mathbf{p}}\nolimits_{{\mathop m\nolimits_{0} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} 0 \hfill \\ 0 \hfill \\ \end{aligned} \right] $$
(5)
$$\left[ \begin{aligned} {\hat{\mathbf{s}}}_{{n_{k}^{ - } }} \hfill \\ {\hat{\mathbf{s}}}_{{m_{k}^{ - } }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} 1 \hfill \\ 0 \hfill \\ \end{aligned} \right.\begin{array}{*{20}c} {} \\ {} \\ \end{array} \left. \begin{aligned} 0 \hfill \\ 1 \hfill \\ \end{aligned} \right]\left[ \begin{aligned} \mathop {{\hat{\mathbf{s}}}}\nolimits_{{\mathop n\nolimits_{k - 1} }} \hfill \\ \mathop {{\hat{\mathbf{s}}}}\nolimits_{{\mathop m\nolimits_{k - 1} }} \hfill \\ \end{aligned} \right] $$
(6)
$$ \left[ \begin{aligned} {\mathbf{p}}_{{n_{k}^{ - } }} \hfill \\ {\mathbf{p}}_{{m_{k}^{ - } }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} 1 \hfill \\ 0 \hfill \\ \end{aligned} \right.\begin{array}{*{20}c} {} \\ {} \\ \end{array} \left. \begin{aligned} 0 \hfill \\ 1 \hfill \\ \end{aligned} \right]\left[ \begin{aligned} \mathop {\mathbf{p}}\nolimits_{{\mathop n\nolimits_{k - 1} }} \hfill \\ \mathop {\mathbf{p}}\nolimits_{{\mathop m\nolimits_{k - 1} }} \hfill \\ \end{aligned} \right]\left[ \begin{aligned} 1 \hfill \\ 0 \hfill \\ \end{aligned} \right.\begin{array}{*{20}c} {} \\ {} \\ \end{array} \left. \begin{aligned} 0 \hfill \\ 1 \hfill \\ \end{aligned} \right]^{T} + \mathop \sigma \nolimits_{w}^{2} $$
(7)
$$ \left[ \begin{aligned} {\hat{\mathbf{s}}}_{{\mathop n\nolimits_{{\mathop k\limits^{{}} }} }} \hfill \\ {\hat{\mathbf{s}}}_{{\mathop m\nolimits_{{\mathop k\limits^{{}} }} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} {\hat{\mathbf{s}}}_{{n_{k}^{ - } }}\hfill \\ {\hat{\mathbf{s}}}_{{m_{k}^{ - } }} \hfill \\ \end{aligned} \right] + \left[ \begin{aligned} \mathop k\nolimits_{{\mathop n\nolimits_{k} }} \begin{array}{*{20}c} {} \\ \end{array} .\begin{array}{*{20}c} {} \\ \end{array} (\mathop {\mathbf{y}}\nolimits_{{\mathop n\nolimits_{k} }} - {\hat{\mathbf{s}}}_{{n_{k}^{ - } }} ) \hfill \\ \mathop k\nolimits_{{\mathop m\nolimits_{k} }} \begin{array}{*{20}c} {} \\ \end{array} .\begin{array}{*{20}c} {} \\ \end{array} (\mathop {\mathbf{y}}\nolimits_{{\mathop m\nolimits_{k} }} - {\hat{\mathbf{s}}}_{{m_{k}^{ - } }} ) \hfill \\ \end{aligned} \right] $$
(8)
$$ \left[ \begin{aligned} \mathop {\mathbf{p}}\nolimits_{{\mathop n\nolimits_{k} }} \hfill \\ \mathop {\mathbf{p}}\nolimits_{{\mathop m\nolimits_{k} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} \mathop {({\mathbf{I}} - k}\nolimits_{{\mathop n\nolimits_{k} }} )\begin{array}{*{20}c} {} \\ \end{array} .\begin{array}{*{20}c} {} \\ \end{array} {\mathbf{p}}_{{n_{k}^{ - } }} \hfill \\ \mathop {({\mathbf{I}} - k}\nolimits_{{\mathop m\nolimits_{k} }} )\begin{array}{*{20}c} {} \\ \end{array} .\begin{array}{*{20}c} {} \\ \end{array} {\mathbf{p}}_{{m_{k}^{ - } }} \hfill \\ \end{aligned} \right] $$
(9)
$$ \left[ \begin{aligned} \mathop k\nolimits_{{\mathop n\nolimits_{k} }} \hfill \\ \mathop k\nolimits_{{\mathop m\nolimits_{k} }} \hfill \\ \end{aligned} \right] = \left[ \begin{aligned} {\mathbf{p}}_{n_{k}^{-}}/({\mathbf{p}}_{n_{k}^{-}} + \mathop \sigma \nolimits_{r}^{2} ) \hfill \\ {\mathbf{p}}_{m_{k}^{-}}/({\mathbf{p}}_{m_{k}^{-}} + \mathop \sigma \nolimits_{r}^{2} ) \hfill \\ \end{aligned} \right] $$
(10)

So, the above updating and prediction stages are employed as a refinement process for each erroneous MB regardless of its position in the lossy view and 3D frame. We repeat the BKF-based EC stages for the entire erroneous frames till we obtain the superior ultimately determined DVs or/and MVs as declared in (8). Further discussion details, equations, mathematical proofs, conditions, and assumptions for the BKF stages are found in [13, 24, 33, 37]. Therefore, the stages of the proposed scheme are iterated until the whole erroneous frames are recovered. So, if the firstly determined DVs and MVs are not efficient for an optimum EC process due to low availability of surrounding corrected pixel information especially in the case of presence severe errors. Thus, they can be enhanced by the predicted DVs and MVs, which are estimated through the utilization of the suggested BKF-based EC scheme to achieve better 3D video streaming performance.

4 Simulation Results and Comparative Discussions

In this section, we validate the performance of the proposed BKF-based EC scheme. Several simulations on standard 3D (Exit, Ballroom, and Vassar) streams have been carried out [18]. The utilized 3D sequences have different characteristics. The compressed bit streams of the tested 3D sequences are firstly obtained, and after that they are transported over a severe wireless channel with different random PLR values, which are simulated using Gilbert channel model [10]. The delivered 3D bit streams at the decoder are then reconstructed and refined utilizing the proposed BKF-based EC scheme. The JMVC reference codec software [36] is employed in the simulation tests, which is relying on the 2D reference codec software [15]. The JVT standard conditions [18] are used to define the chosen simulation parameters which are used in this work. To validate the quality performance of the proposed concealment algorithm, the peak signal-to-noise ratio (PSNR), the Feature Similarity (FSIM) index, and the Structural Similarity (SSIM) index [41] are used as the measured objective quantities.

To investigate the performance of the proposed concealment scheme, we have carried out different tests. We contrast the performance of the suggested scheme to the case of utilizing the OBBMA, DIECA, a hybrid scheme of them [8], and the case of no concealment. In the declared results, the BKF-EC indicates the suggested enhanced algorithm, which gives the best received qualities. In the simulation tests, we used various quantization parameter (QP) values of 37, 32, and 27 and different random PLRs of 40, 50, and 60% as noticed in Figs. 5, 6, 7, 8, 9, 10, 11, 12, 13. To appraise the efficacious performance of the proposed algorithm for recovering different lossy 3D frame types, we test more simulation results of several erroneous (P- or B- or I-) frame sorts and positions inside different views.

Fig. 5
figure 5

Visual results of the chosen 215th B-frame in I-view (S0) of the “Ballroom” stream: a error-free B215 frame, b lossy B215 with random PLR = 40%, QP = 37, c recovered B215 by OBBMA only without using BKF [8], d recovered B215 by full suggested BKF-based EC scheme

Full size image
Fig. 6
figure 6

Visual results of the chosen 215th B-frame in B-view (S1) of the “Ballroom” stream: a error-free B215 frame, b lossy B215 with random PLR = 40%, QP = 37, c recovered B215 by OBBMA + DIECA only without using BKF [8], d recovered B215 by full suggested BKF-based EC scheme

Full size image
Fig. 7
figure 7

Visual results of the chosen 215th B-frame in P-view (S2) of the “Ballroom” stream: a error-free B215 frame, b lossy B215 with random PLR = 40%, QP = 37, c recovered B215 by OBBMA only without using BKF [8], d recovered B215 by full suggested BKF-based EC scheme

Full size image
Fig. 8
figure 8

Visual results of the chosen 169th I-frame in I-view (S0) of the “Exit” stream: a error-free I169 frame, b lossy I169 with random PLR = 50%, QP = 32, c recovered I169 by DIECA only without using BKF [8], d recovered I169 by full suggested BKF-based EC scheme

Full size image
Fig. 9
figure 9

Visual results of the chosen 169th B-frame in B-view (S1) of the “Exit” stream: a error-free B169 frame, b lossy B169 with random PLR = 50%, QP = 32, c recovered B169 by DIECA only without using BKF [8], d recovered B169 by full suggested BKF-based EC scheme

Full size image
Fig. 10
figure 10

Visual results of the chosen 169th P-frame in P-view (S2) of the “Exit” stream: a error-free P169 frame, b lossy P169 with random PLR = 50%, QP = 32, c recovered P169 by DIECA only without using BKF [8], d recovered P169 by full suggested BKF-based EC scheme

Full size image
Fig. 11
figure 11

Visual results of the chosen 136th B-frame in I-view (S0) of the “Vassar” stream: a error-free B136 frame, b lossy B136 with random PLR = 60%, QP = 27, c recovered B136 by OBBMA only without using BKF [8], d recovered B136 by full suggested BKF-based EC scheme

Full size image
Fig. 12
figure 12

Visual results of the chosen 136th B-frame in B-view (S1) of the “Vassar” stream: a error-free B136 frame. b lossy B136 with random PLR = 60%, QP = 27, c recovered B136 by OBBMA + DIECA only without using BKF [8], d recovered B136 by full suggested BKF-based EC scheme

Full size image
Fig. 13
figure 13

Visual results of the chosen 136th B-frame in P-view (S2) of the “Vassar” stream: a error-free B136 frame. b lossy B136 with random PLR = 60%, QP = 27. c recovered B136 by OBBMA only without using BKF [8], d recovered B136 by full suggested BKF-based EC scheme

Full size image

Figures 5, 6, and 7 show the results of the 3D Ballroom video at the case of QP = 37 and random PLR = 40% for the tested 215th inter-compressed B-frame inside the S0, S1, and S2, respectively. We recover the 215th B inter-frame utilizing the appropriate OBBMA, DIECA, or OBBMA + DIECA algorithms without utilizing the BKF scheme [8] and with exploiting the suggested BKF-based EC algorithm. In Figs. 5d, 6d, and 7d, we observe that the visual results of the proposed BKF-EC scheme are better than the visual results of the state of using only the OBBMA or OBBMA + DIECA algorithms which are declared in Figs. 5c, 6c, and 7c.

Figures 8, 9, and 10 present the results of the Exit 3DV sequence at the case of QP = 32 and random PLR = 50% of the 169th I intra-compressed frame inside the I-view S0, the B inter-frame within the S1 B-view, and the P inter-frame inside the S2 P-view, respectively. It is noticed that in Figs. 8c, 9c, and 10c, the selected frames are concealed by using the appropriate DIECA algorithm only [8] based on their erroneous frame locations inside the 3DV prediction structure as it is declared in Fig. 1. In Figs. 8d, 9d, and 10d, the same selected frames are also recovered but employing the proposed BKF-based EC scheme. It is approved that there is a significant effect of exploiting the BKF scheme incorporated with the conventional error control schemes for reconstructing of the heavy erroneous 3DV sequences. The same comparison results are presented in Figs. 11, 12, and 13 for the Vassar 3DV stream at the case of QP = 27 and random PLR = 60% for the 136th test inter-compressed B-frame inside the I-view S0, B-view S1, and P-view S2, respectively. We can conclude from the introduced results in Figs. 11d, 12d, and 13d compared to which are declared in Figs. 11c, 12c, and 13c [8] that the prominence of utilizing the suggested BKF-based concealment algorithm for reconstructing the corrupted macro-blocks is large in the introduced state of severe random PLR = 60%.

The PSNR values of the utilized 3D streams at different QPs and PLRs are declared in Figs. 14, 15, and Table 1. We contrast the PSNR values of the suggested BKF-based EC scheme with those of the case of not hiring concealment and also with those of the case of employing EC but without deploying the BKF scheme [8]. It is noticed from the PSNR values of the tested 3D streams that the suggested BKF-based EC algorithm always achieves the best objective qualities. It is declared that the BKF-based EC scheme has superior PSNR gain for the whole tested 3D streams of about 1.6 ~ 4.8 dB at various random PLRs and QPs contrast to that of the state of employing EC only without employing the BKF scheme [8]. The proposed BKF-based EC scheme is an appreciated error control algorithm because it achieves about 16–22 dB average gain at various random PLRs and QPs compared to that case of not employing EC algorithms.

Fig. 14
figure 14

PSNR (dB) results for the: a Ballroom, b Exit, and c Vassar 3D streams at QP = 27 and different random PLRs

Full size image
Fig. 15
figure 15

PSNR (dB) results for the: a Ballroom, b Exit, and c Vassar 3D streams at QP = 37 and different random PLRs

Full size image
Table 1 PSNR (dB) results for the Ballroom, Exit, and Vassar 3D streams at QP = 32 and different random PLRs
Full size table

Figures 16, 17, and 18 display the PSNR objective results for the different frames of the three tested 3D Ballroom, Exit, and Vassar streams at various QPs and random PLRs. The PSNR results for the chosen first 50 frames at various QPs and random PLRs of all tested 3DV sequences show that the suggested BKF-based EC algorithm significantly outperforms the state of employing conventional EC only without deploying BKF [8]. The simulation outcomes also reveal that there is a better PSNR performance achieved through utilizing the suggested BKF-based concealment algorithm compared to employing concealment techniques without exploiting the BKF scheme, and it becomes significant while increasing the PLRs or QPs values.

Fig. 16
figure 16

PSNR comparison for the frames of the Ballroom sequence at various QPs and random PLRs

Full size image
Fig. 17
figure 17

PSNR comparison for the frames of the 3D Exit sequence at different QPs and random PLRs

Full size image
Fig. 18
figure 18

PSNR comparison for the frames of the 3D Vassar sequence at various QPs and random PLRs

Full size image

Table 2 shows the average frame execution time values of the suggested BKF-based EC algorithm contrast to the state of employing the state-of-the-art concealment techniques without using BKF [7, 8] for the utilized 3D videos at random PLR = 50% and QPs = 27, 32, and 37. All simulation comparison tests are carried out using an Intel® Core™i7-4500U CPU @1.80 and 2.40 GHz with 8 GB RAM and running with Windows 1064-bit operating system. The average frame execution time numerical results are presented to validate the performance of the proposed scheme for approving its utilization for real-time applications. Therefore, it is noticed that the suggested BKF-based EC algorithm needs a little additional increase in the computational cost of the execution time contrast to the literature techniques. Thence, the proposed scheme presents a recommended execution processing time that can be acceptable for online and real-time 3DV communication applications.

Table 2 The average execution time of the suggested concealment scheme contrast to the literature techniques for the tested Ballroom, Exit, and Vassar streams at random PLR = 50% and different QPs
Full size table

Table 3 shows the FSIM and SSIM index comparison for the utilized 3D streams at different random PLRs. We compare the FSIM and SSIM results of the proposed BKF-based EC scheme with the state of employing the state-of-the-art concealment techniques without using BKF [7, 8] for the utilized 3D videos at QP = 32 and different random PLRs. It is declared from the results that deploying the proposed BKF-based concealment algorithm is highly recommended for the 3D videos streaming over severely erroneous networks. The presented results illustrate that there are PSNR, FSIM, and SSIM values enhancement achieved by the suggested technique contrast to those achieved by state-of-the-art algorithms. Furthermore, it is declared that the suggested BKF-based EC algorithm introduces preferable results for 3D streams with different characteristics.

Table 3 FSIM/SSIM comparison results between the proposed EC scheme and the state-of-the-art techniques for the tested Ballroom, Exit, and Vassar streams at QP = 32 and various random PLRs
Full size table

To emphasize the effectiveness of the proposed scheme, further simulation tests have been carried out to compare the results of the proposed technique with the literature techniques. The achievement of the proposed BKF-based concealment technique is contrast to the hybrid of DIECA and Decoder Motion Vector Estimation Algorithm (DMVEA) EC algorithms introduced in [8], the Temporal Bilateral Error Concealment (TBEC), Inter-view Bilateral Error Concealment (IBEC), and Multi-Hypothesis Error Concealment (MHEC) algorithms suggested in [35], the error resilient Flexible Macro-block Ordering (FMO) scheme and the improved suggested EC methods presented in [27], the dynamic suggested concealment selection scheme proposed in [23], the suggested error control technique introduced in [22], and the joint of DMVEA and DIECA algorithms utilizing the error resilience Dispersed FMO (DFMO) scheme suggested in [7].

The comparisons have been tested with the Exit and Ballroom 3D streams with the same simulation parameters which are utilized in all above tests at QP = 32 and random PLRs = 5, 10, and 20%. Table 4 declares the average PSNR values of the proposed error control technique contrast to the literature error control techniques in [7, 8, 22, 23, 27, 35]. We observe that the proposed technique outperforms the state-of-the-art techniques. It is as well observed that with employing the proposed error recovery algorithm, the 3D reception quality can be enhanced by deploying of the BKF scheme besides the EC algorithms.

Table 4 PSNR (dB) comparison results of the proposed BKF-based EC scheme and the state-of-the-art techniques for the 3D Exit and Ballroom streams at QP = 32 and different random PLRs
Full size table

To further approve the efficacy of the suggested BKF-based EC scheme, we have run more simulation tests on more 3DV sequences of the Flamenco2 and Objects2 3D video test sequences [18], which have various spatial resolutions and temporal characteristics. Firstly, the compressed 3D bit streams are obtained, and then, the bit streams are transported through severe channels with different slice PLR values, which are simulated using Markov channel model [33]. The delivered 3D bit streams at the decoder are then reconstructed and refined utilizing the proposed BKF-based EC scheme. The JMVC reference codec software [36] is employed in the simulation tests, which is relying on the 2D reference codec software [15]. The JVT standard conditions [18] are used to define the chosen simulation parameters which are used in this work. To investigate the performance of the proposed concealment scheme, we have carried out different tests. We contrast the performance of the suggested scheme to the case of utilizing the OBBMA, DIECA, a hybrid scheme of them [8], and the case of no concealment. The comparisons at QP values of 32 and 27 and at different lengths of severe burst slice PLR of 50% of the chosen 3D frames of the utilized 3D (Flamenco2 and Objects2) streams are declared in Figs. 19, 20, 21, 22, 23, and 24. To appraise the efficacious performance of the proposed algorithm for recovering different lossy 3D frame types, we test more simulation results of several erroneous (P- or B- or I-) frame sorts and positions inside different views to approve the better performance of the suggested BKF-based EC algorithm in recovering the erroneous 3D frames at the case of different lengths of burst slice PLRs.

Fig. 19
figure 19

Visual results of the chosen 25th I-frame in I-view (S0) of the “Flamenco2” stream: a error-free I25 frame, b lossy I25 with burst slice PLR = 50%, QP = 32, c recovered I25 by DIECA only without using BKF [8], d recovered I25 by full suggested BKF-based EC scheme

Full size image
Fig. 20
figure 20

Visual results of the chosen 25th B-frame in B-view (S1) of the “Flamenco2” stream: a error-free B25 frame. b lossy B25 with burst slice PLR = 50%, QP = 32, c recovered B25 by DIECA only without using BKF [8], d recovered B25 by full suggested BKF-based EC scheme

Full size image
Fig. 21
figure 21

Visual results of the chosen 25th P-frame in P-view (S2) of the “Flamenco2” stream: a error-free P25 frame. b lossy P25 with burst slice PLR = 50%, QP = 32, c recovered P25 by DIECA only without using BKF [8], d recovered P25 by full suggested BKF-based EC scheme

Full size image
Fig. 22
figure 22

Visual results of the chosen 48th B-frame in I-view (S0) of the “Objects2” stream: a error-free B48 frame, b lossy B48 with burst slice PLR = 50%, QP = 27, c recovered B48 by OBBMA only without using BKF [8], d recovered B48 by full suggested BKF-based EC scheme

Full size image
Fig. 23
figure 23

Visual results of the chosen 48th B-frame in B-view (S1) of the “Objects2” stream: a error-free B48 frame. b lossy B48 with burst slice PLR = 50%, QP = 27, c recovered B48 by OBBMA + DIECA only without using BKF [8], d recovered B48 by full suggested BKF-based EC scheme

Full size image
Fig. 24
figure 24

Visual results of the chosen 48th B-frame in P-view (S2) of the “Objects2” stream: a error-free B48 frame. b lossy B48 with burst slice PLR = 50%, QP = 27, c recovered B48 by OBBMA only without using BKF [8], d recovered B48 by full suggested BKF-based EC scheme

Full size image

Figures 19, 20, and 21 present the visual results of the Flamenco2 3DV sequence at the case of QP = 32 and burst slice PLR = 50% of the 25th I intra-frame inside the I-view S0, the B inter-frame within the S1 B-view, and the P inter-frame inside the S2 P-view, respectively. It is noticed that in Figs. 19c, 20c, and 21c, the selected frames are concealed by using the appropriate DIECA algorithm only [8] based on their erroneous frame locations within the 3DV prediction structure as it is noticed in Fig. 1. In Figs. 19d, 20d, and 21d, the same selected frames are also recovered but employing the suggested BKF-based EC scheme. The significant effect of exploiting the BKF scheme incorporated with the conventional concealment techniques for refining the heavy burst slice erroneous 3DV sequences is clear. Figures 22, 23, and 24 show the results of the 3D Objects2 video at the case of QP = 27 and burst slice PLR = 50% for the 48th inter-compressed B-frame inside the S0, S1, and S2, respectively. We recovered the 48th B inter-frame utilizing the OBBMA, DIECA, or OBBMA + DIECA algorithms without deploying the BKF scheme [8] and with employing the suggested BKF-based EC algorithm. From Figs. 22d, 23d, and 24d, it is noticed that the visual results of the suggested BKF-based EC scheme are better than the visual results of the state of using only the OBBMA or OBBMA + DIECA algorithms, which are declared in Figs. 22c, 23c, and 24c.

The PSNR comparisons of the 3D Flamenco2 and Objects2 streams at different QPs and burst slice PLRs are declared in Figs. 25, 26, and 27. We contrast the PSNR values of the suggested BKF-based EC scheme with those of the case of not hiring concealment and with those of the state of employing EC but without deploying the BKF scheme [8]. It is noticed from the PSNR values of the tested 3D streams that the suggested BKF-based EC algorithm always achieves the best objective qualities. It is declared that the BKF-based EC scheme has superior PSNR gain for the whole tested 3D streams of about 2.15–4.65 dB at various burst slice PLRs and QPs contrast to that of the state of employing EC only without employing the BKF scheme [8]. The proposed BKF-based EC scheme is an appreciated error control algorithm because it achieves about 15–27 dB average gain at various random PLRs and QPs compared to that case of not employing EC algorithms.

Fig. 25
figure 25

PSNR (dB) results for the: a Flamenco2 and b Objects2 3D streams at QP = 27 and different burst slice PLRs

Full size image
Fig. 26
figure 26

PSNR (dB) results for the: a Flamenco2 and b Objects2 3D streams at QP = 32 and different burst slice PLRs

Full size image
Fig. 27
figure 27

PSNR (dB) results for the: a Flamenco2 and b Objects2 3D streams at QP = 37 and different burst slice PLRs

Full size image

Table 5 shows the average frame execution time values of the suggested BKF-based EC algorithm contrast to the state of employing the state-of-the-art error control techniques without using BKF [7, 8] for the utilized 3D Flamenco2 and Objects2 videos at burst slice PLR = 50% and QP = 27, 32, and 37. All simulation comparison tests have been implemented using an Intel® Core™i7-4500U CPU @1.80 and 2.40 GHz with 8 GB RAM and running with Windows 10 64-bit operating system. The average frame execution time numerical results are presented to validate the efficacy of the proposed error control technique for approving its utilization in the state of real-time applications. Therefore, it is noticed that the suggested BKF-based EC algorithm has a little additional increase in the computational cost of execution time contrast to the literature error control algorithms. Thence, the proposed scheme presents a recommended execution processing time that can be acceptable for online and real-time 3DV communication applications.

Table 5 The average execution time comparison between the proposed BKF-based EC scheme and the state-of-the-art methods on the utilized Flamenco2 and Objects2 streams at burst slice PLR = 50% and different QPs
Full size table

To emphasize the effectiveness of the proposed scheme for robust 3D streams delivery through severe burst slice wireless channels, we implement different comparisons to compare the results of the proposed technique with the literature techniques. The achievement of the proposed BKF-based concealment technique is contrast to the hybrid of DIECA and DMVEA EC algorithms introduced in [8], the introduced KF-based concealment algorithms introduced in [13, 24, 33], and the joint of DMVEA and DIECA algorithms utilizing the DFMO scheme suggested in [7]. The comparisons have been tested with the Flamenco2 and Objects2 streams with the same simulation parameters which are utilized in all above simulation results at QP = 32 and burst slice PLRs = 30, 40, and 50%. Table 6 declares the average PSNR values of the proposed error control technique contrast to the literature error control techniques in [7, 8, 13, 24, 33]. We observe that the proposed technique outperforms the state-of-the-art techniques. It is as well observed that with employing the proposed error recovery algorithm, the 3D reception quality can be enhanced by deploying of the BKF scheme besides the EC algorithms.

Table 6 PSNR (dB) comparison results of the proposed BKF-based EC scheme and the state-of-the-art methods on the 3D Flamenco2 and Objects2 streams at QP = 32 and different burst slice PLRs
Full size table

5 Conclusions

This paper introduced an improved EC algorithm for reliable transmission of 3D streams over noisy wireless networks based on using a recursive Bayesian Kalman filtering scheme. The considerable advantage of this paper is that the most convenient error control technique is adaptively selected relying on the lossy 3D frames and views. Furthermore, the suggested error control technique confirmed that it is more appreciated and recommended in the case of randomly and bursty noisy transmission channels. Experimental results proved the performance capability of the suggested BKF-based EC algorithm in compensating the communication corruptions of the transported 3D streams at the case of large random and burst PLRs. The proposed scheme attained better objective and subjective reception qualities contrast to the related error control works.