Keywords

1 Introduction

The algorithm in this paper is targeted at YUV video streaming. Y component is brightness, so it is necessary to process the Y component and convert it with U, V components to RGB format to be displayed together [1]. Research has shown that the result from processing the three channels of RGB, respectively, will encounter distortion in color. Therefore, to avoid such a distortion, it is necessary to directly process the Y.

1.1 Pre-filtering

The gray value of most salt and pepper noise points is concentrated at 255 or 0, while most signal points are not concentrated at this area [2]. Hence, the author proposes this algorithm: (1) assume the pixel value is judged as the salt and pepper noise point when the gray range is between [0, 16] or [235, 255] and other areas are regarded as signal points; (2) the noise point (Z ij ) can be judged from the above and the peripheral pixels of the 3*3 window centered at Z ij are marked as (Z 1, Z 2, …, Z 8), from which the points whose gray is not in the salt and pepper noise gray range can be selected and also marked as {X 1, X 2, …, X l }, 0 ≤ l ≤ 8; Z ij is the gray value of the noise point ij, and Z 1, Z 2, …, Z 8 and X 1, X 2, …, X l are the gray value of the point around the pixel ij; (3) when there is l = 0, there are signal points around the pixel ij, and S ij will not be processed and its value is still equal to itself; (4) when there is l ≠ 0, there are some signal points around the pixel ij, and now the gray mid-value of the surrounding signal points can be used as the gray value of this point; order Z ij  = median {X 1, X 2, …, X l }, in which median {X 1, X 2, …, X l } is the mid-value operator to solve the set {X 1, X 2, …, X l }, and this is called as the noise mid-value algorithm as {X 1, X 2, …, X l } is possible signal points. For the Forman sequence, when the Gaussian noise (variance: 36; PSNR: 32.6 dB) or the salt and pepper noise (PSNR: 32.6 dB) is added into it, the result in Table 69.1 can be gained after the pre-filtering [3].

Table 69.1 The noise reduction effect of pre-filtering on the forman sequence added with salt and pepper or Gaussian noise
Full size table

From the above, it can be known that pre-filtering has a large elimination effect on the salt and pepper noise, but it is not obvious for the sequence disturbed by Gaussian noises. After an objective evaluation by PSNR is made, the subjective evaluation can be seen as shown in Fig. 69.1.

Fig. 69.1
figure 1

The seventh frame image of forman: a image after salt and pepper noise experienced pre-filtering; b image with 24 dB salt and pepper noise; c image after Gaussian noise experienced pre-filtering; d image with 32 dB Gaussian noise; e original image

Full size image

1.2 Time Domain Filtering

The time domain filtering method in this paper is based on the weighted recursive filtering algorithm of front and back frames. In this algorithm, when the front frame and the frame before the front frame are the images after experiencing pre-filtering, their salt and pepper noise have been removed, but have do not proceed by the spatial domain smoothing method. In selecting the threshold, the video sequence Football within movement can be used for detection for the sake of solving the critical threshold. When it is larger than this threshold (T1), it is unnecessary to be processed. Gaussian noise (variance: 36; PSNR: 32.6 dB) can be used for detection to test the sequence input result when the threshold is between 5 and 100. The details can be seen in Table 69.2.

Table 69.2 The time domain filtering effect under different threshold (T1)
Full size table

From the above, PSNR value decreases when threshold is too small; the difference between two frames is larger than the set thresholds most until the result of the front frame is output; PSNR value also decreases when threshold is too big, but results of the front frame and the frame before the front frame are still output by the recursive filtering within the threshold range; when the difference between two frames is larger than 18, W is 1 and is weighed with the front frame. The movement intensity is large when the difference between two frames is larger than threshold, but is small when the difference is smaller than threshold. Test the second threshold from 1 to 17, and assume W is 0.8 (larger than T2 but smaller than T1) or 0.5 (smaller than T2). The domain filtering has a good effect on the stationary sequence, so the stationary sequence News is selected to be added with Gaussian noise (variance 36, PSNR: 32.6 dB) for testing T2 so as to solve the best T2 value, as shown in Table 69.3.

Table 69.3 The filtering effect of diffident threshold (T2) on gaussian noise
Full size table

From the above, the threshold (T2) is 5. Determine the value of W1 first, namely, the weighted value when the difference in the pixel brightness values related to two frames is between [5, 15]. To test W1, the value range [0.6, 0.9] can be considered. The details are shown in Table 69.4.

Table 69.4 The filtering effect of diffident weighted value (W1) on gaussian noise
Full size table

The value of W2 can be determined from Table 69.5.

Table 69.5 The filtering effect of diffident weighted value (W1) on Gaussian noise
Full size table

Thus, the best value for W1 is 0.4. Based on Tables 69.2, 69.3, 69.4, 69.5, the thresholds T1, T2, W1 and W2 are determined, and the equation of the first movement detection of the domain weighed recursive filtering can be gained as follows:

$$ \begin{gathered} W = \begin{array}{*{20}c} {1,} & {|F_{1} (i,j) - F_{2} (i,j)| > 18} \\ \end{array} \hfill \\ W = \begin{array}{*{20}c} {0.8,} & {5 < |F_{1} (i,j) - F_{2} (i,j)| < 18} \\ \end{array} \hfill \\ W = \begin{array}{*{20}c} {0.4,} & {|F_{1} (i,j) - F_{2} (i,j)| > 5} \\ \end{array} \hfill \\ \end{gathered} $$
(69.1)

In Eq. (69.1), F 1(i, j) is the pixel value of the last frame, and F 2(i, j) is the pixel value of the current frame. Next, the second weighed recursive filtering is on, and the equation is as follows:

$$ F(i,j) = (1 - W)*F_{1} (i,j) + W*F_{2} (i,j) $$
(69.2)

In Eq. (69.2), F (i, j) is the result of the domain filtering, namely, the gray value of this point.

1.3 Spatial Domain Filtering

1.3.1 Edge Detection

First, edge detection can be made. Gauss-Laplacian is also called LOG operator. This operator will carry out a smoothing processing on the original image with Gaussian algorithm before solving edge, to realize the maximum suppression on noise. Two-dimension Gaussian function is used to smooth the image as follows:

$$ G(x,y) = {{e^{{ - \frac{{x^{2} + y^{2} }}{{2\delta^{2} }}}} } \mathord{\left/ {\vphantom {{e^{{ - \frac{{x^{2} + y^{2} }}{{2\delta^{2} }}}} } {\left( {2\pi \delta^{2} } \right)}}} \right. \kern-0pt} {\left( {2\pi \delta^{2} } \right)}} $$
(69.3)

Its shape is like the clock inverted in the two-dimension space. For this function, Laplace transform can be used, namely, the second-order directional derivative can be taken for Gaussian function, so the function equation of the 2-D LOG operator can be gained as follows:

$$ \nabla^{2} G(x,y) = \frac{{\partial^{2} G}}{{\partial x^{2} }} + \frac{{\partial^{2} G}}{{\partial y^{2} }} = \frac{1}{{2\pi \delta^{4} }}\left( {\frac{{x^{2} + y^{2} }}{{\delta^{2} }} - 2} \right)e^{{ - (x^{2} + y^{2} )/(2\delta^{2} )}} $$
(69.4)

Use this LOG operator to do convolution operation on input image I(x, y) and the output image is as follows:

$$ \begin{gathered} F(x,y) = \int\limits_{\alpha \equiv - \infty }^{ + \infty } {\int\limits_{\beta \equiv - \infty }^{ + \infty } {\nabla^{2} G(\alpha ,\beta )I(x - \alpha ,y - \beta )d\alpha d\beta } } \hfill \\ = \frac{1}{{2\pi \delta^{4} }}\int\limits_{\alpha \equiv - \infty }^{ + \infty } {\int\limits_{\beta \equiv - \infty }^{ + \infty } {\left( {\frac{{x^{2} + y^{2} }}{{\delta^{2} }}e^{{ - \frac{{x^{2} + y^{2} }}{{2\delta^{2} }}}} } \right)} } * \hfill \\ I(x - \alpha ,y - \beta ) - 2e^{{ - \frac{{x^{2} + y^{2} }}{{2\delta^{2} }}}} I(x - \alpha ,y - \beta )d\alpha d\beta \hfill \\ \end{gathered} $$
(69.5)

LOG operator is a low pass filtering process, which is used to eliminate the image intensity change when the spatial size is much less than Gaussian space coefficient to reduce noise. Then, use Laplace (\( \nabla^{2} \)) to gain the second-order directional derivative image F(x, y) of smooth image I(x, y). From the shape of LOG operator in Fig. 69.2, LOG operator is about symmetry of round, and is the same in sizes of edge detection in all directions.

Fig. 69.2
figure 2

LOG operator

Full size image

The 5*5 template of the LOG edge detection operator in this algorithm is as follows:

$$ \begin{array}{*{20}c} { - 2} & { - 4} & { - 4} & { - 4} & { - 2} \\ { - 4} & 0 & 8 & 0 & { - 4} \\ { - 4} & 8 & {24} & 9 & { - 4} \\ { - 4} & 0 & 8 & 0 & { - 4} \\ { - 2} & { - 4} & { - 4} & { - 4} & { - 2} \\ \end{array} $$
(69.6)

1.3.2 Noise Level Detection and Self-Adaptive Filtering

The noise level detection method is based on statistical method. The image without edges should be smooth under no noises. Under no noises, Forman, Football video sequences are detected, and noise detection is increased gradually, gaining cumulative probabilities of gray differences between [20, 256]. PSNR values are in Table 69.6 after sequences experience straight passing, 3*3 center-weighted filtering, 3*3 mean filtering, 5*5 mean filtering, and 7*7 mean filtering.

Table 69.6 PSNR values after forman and football sequences cumulative probabilities reduce noise when entering different filters or straight under different Gaussian noise
Full size table

From the above, the divided four thresholds are T 1 = 0.7, T 2 = 0.5, T 3 = 0.2 and T 4 = 0.1. Order P is the cumulative probability of adjacent point brightness difference between [20, 256]. If there is P > T 1, 7*7 mean filtering is used; if T 1 > P > T 2, 5*5 mean filtering is used; if T 2 > P > T 3, 3*3 mean filtering is used; if T 3 > P > T 4, 3*3 center-weighted filtering is used; If P < T 4, straight output is used, which is based on Weber’s Law, and now the noise is in the acceptable range. The algorithms of mean filtering, mid-value filtering and center-weighted filtering have been introduced above. Template of 3*3 center-weighted is as follows:

$$ \begin{array}{*{20}c} {0.08} & {0.08} & {0.08} \\ {0.08} & {0.36} & {0.08} \\ {0.08} & {0.08} & {0.08} \\ \end{array} $$
(69.7)

1.4 Weighted Output

The space-time combined filter is used in this algorithm, and its results are shown in Table 69.7; the Wz is the weighted value of the spatial domain.

Table 69.7 Filtering effect of different space–time weighted value Wz on different level Gaussian noise
Full size table

From the above, it can be known: Wz = 0.9, if P > T1; Wz = 0.8, if T1 > P > T2; Wz = 0.55, if T2 > P > T3; Wz = 0, if P < T3 > P > T4; Wz = 0, if P < T4. The weighted equation is as follows:

$$ Y(x,y) = Y_{1} (x,y)*(1 - Wz) + Y_{2} (x,y)*Wz $$
(69.8)

In Eq. (69.8), Y(x, y) is the brightness value of space-time outputs; Y1(x, y) is the brightness value of domain filtering; the Y2(x, y) is the brightness value of spatial domain filtering. The final result is outputted with the form of RGB converted by Y component after processed and the U and V component. The conversion method is introduced above.

2 Conclusion

The time domain in this algorithm can keep the edges and details of each frame, and the spatial domain has a good effect on smooth noise reduction. For different speed and sequences with different characteristics, the noise reduction effect is excellent. Subjective evaluation on it is good; the PSNR objective evaluation also suggests it has a good noise reduction effect.