Abstract
Predicting a given pixel from surrounding neighbouring pixels is of great interest for several image processing tasks. Previous works focused on developing different Gaussian based models. Simultaneously, in real-world applications, the image texture and clutter are usually known to be non-Gaussian. In this paper, we develop a pixel prediction framework based on a finite generalized inverted Dirichlet (GID) mixture model that has proven its efficiency in several machine learning applications. We propose a GID optimal predictor, and we learn its parameters using a likelihood-based approach combined with the Newton-Raphson method. We demonstrate the efficiency of our proposed approach through a challenging application, namely image inpainting, and we compare the experimental results with related-work methods.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Pixel prediction has shown to be one of the most needed tool to perform several applications in image processing such as anomaly detection [18, 31], object detection [8, 17], edge detection [28], video compression [7, 24], semantic segmentation [27, 36], image restoration [11, 32] and keypoint prediction [37]. Meanwhile, pixel prediction is represented by approximating the predicted pixel using its neighbors. For that, it is usually represented by a linear or a non-linear combination of the neighboring pixels plus an error value [29, 38]. In this paper, inspired by the work proposed in [20], we take the optimal predictor of \({x}_n\) as the conditional expectation \(E\big (x_{ij}\mid \forall x_{i'j'} \in {N_{i,j}}\big )\), where \(x_{i'j'}\) are the neighbors of \(x_{ij}\) within the set of pixels \(N_{i,j}\).
Exploiting the ease of analytical derivations, the authors in [40] derived optimal predictors for Gaussian distribution and mixture of Gaussians. However, the field of non-Gaussian distributions exhibited an exciting expansion in terms of mathematical theorems [23, 35]. Yet, a large number of researchers proved that Gaussian assumption is generally inappropriate and other alternative distributions are more effective in modeling data than Gaussian distribution by unveiling more appropriate patterns and correlations among data features [4,5,6, 10]. In our recent work [19], we proved that finite inverted Dirichlet mixture model effectively represents positive vectors [3, 9, 39]. However, it suffers from significant drawbacks such as its minimal, strictly positive covariance structure. Therefore, to overcome this limitation, we consider applying the generalized inverted Dirichlet which belongs to Liouville family of distributions [22]. This distribution provides a more decent representation of the variability of the data [12]. Indeed, considering the fact that generalized inverted Dirichlet could be factorized into a set of inverted Beta distributions [13], gives more flexibility for modeling data in real-world applications.
In this work, we derive a novel optimal predictor based on generalized inverted Dirichlet distribution which results in a linear combination of the neighboring pixels. Meanwhile, we evaluate the proposed approach on image inpainting application. We choose a publicly available dataset namely Paris StreetView to validate our approach [16]. For the purpose of proving the efficiency of our proposed optimal predictor, we consider two types of pixel discarding. The first pixel removal is random, whereas, in the second experiment, we discard lines from the image. We use a 3\(^\mathrm{rd}\) order non-symmetrical half-plane casual (NSHP) neighborhood system to compute the missing pixel [11]. Finally, we perform two image comparison metrics to evaluate our proposed model and compare it to other similar optimal based predictors.
The rest of the paper is organized as follows: in Sect. 2, we describe our prediction model, and we derive the analytical expression of the GID optimal predictor. In Sect. 3, we consider the image inpainting application on Paris StreetView dataset with two different data masks to demonstrate the effectiveness of the proposed predictor, and we discuss the experimental results. Finally, a summary is provided in Sect. 4.
2 GID Prediction Model
The generalized inverted Dirichlet mixture model has shown high flexibility for modeling and clustering positive vectors. In this section, we start by reviewing the finite GID mixture model. Then, we introduce the parameters learning through the EM algorithm and, later, we extend this model to the prediction.
2.1 Mixture of Generalized Inverted Dirichlet Distributions
Let \(\mathcal {X}=(\mathbf {X}_1,\dots ,\mathbf {X}_N)\) be a set of N d-dimensional positive vectors where each vector \(\mathbf {X}_i\) follows a mixture of K generalized inverted Dirichlet (GID) distributions characterized by parameters \(\mathbf {\theta }_{j}=(\mathbf {\alpha }_j,\mathbf {\beta }_j)\) and mixing weight \(\pi _j\) of the jth component.
where \(\Theta =(\mathbf {\theta }_{1},\dots , \mathbf {\theta }_{K}, \pi _1,\dots ,\pi _K)\) represents the GID mixture model parameters and \(P(\mathbf {X}_i|\mathbf {\theta }_{j})\) is the generalized inverted Dirichlet distribution, which has the following form [26]:
where \(\gamma _{jl}=\beta _{jl}+\alpha _{jl}-\beta _{jl+1}\), for \(l=1,\dots ,d\) (\(\beta _{jd+1}=1\)). It is to be noted that the GID is reduced to the inverted Dirichlet distribution when the parameter \(\gamma _{jl}\) is set to zero (\(\gamma _{j1}=\dots =\gamma _{jd}=0\)).
The flexibility of the generalized inverted Dirichlet is by dint of the concept of “Force of mortality” of the population where we introduce, here, a doubly non-central Y independent-variables defined as
where \(T_{il} = 1+ X_{i1} + X_{i2} + \dots + X_{il-1},~~ l=1,\dots ,d\)
The characteristic function underlying the \(\mathcal {Y}=(\mathbf {Y}_1,\dots ,\mathbf {Y}_N)\) independent variables follows a product of 2-parameters inverted Beta distribution, where \(\theta _{l}=(\alpha _{l},\beta _{l})\)
In which the probability of inverted Beta is given by:
Many characteristics of the distribution are defined in [30]. We mention some interesting statistics for this distribution.
First, the mixed moments such as the \(n^{th}\) moment is given by:
where \(\beta - n\) is positive.
Then, the covariance between two variable \(Y_1\) and \(Y_2\) is defined as:
The covariance between two features for inverted Beta is always positive, which means that both they tend to increase or decrease together.
Finally, the variance of a variable Y is conveyed by:
2.2 Likelihood-Based Learning
Theoretically, a plethora of literature agrees on the effectiveness of the likelihood-based approach for estimating the mixture parameters. One of the well-known methodologies is the Expectation-Maximization technique [15], beginning with a tuned initialization for the set of parameters to the expectation step where the posterior is inferred (named often as “responsibilities”), then the iterations are proceeded to update the required variables until convergence. The heart of the matter comes with estimating the parameters based on the second derivative of the log-likelihood function with regards to each parameter. First, we introduce the log-likelihood as follows:
Initializing Process. As a first step, an unsupervised-method, namely “K-means,” is applied to obtain the initial K clusters. Consequently, for each cluster, the method-of-moments is implemented to get the initial \(\mathbf {\alpha }_j\) and \(\mathbf {\beta }_j\) parameters of each component j. The mixing weight is set in the initial phase as the number of elements in each cluster divided by the total number of vectors. As mentioned earlier, with conditionally independent features, the GID is converted by the inverted Beta distribution factorization. Thus, given the moments of inverted Beta distribution [2], the initial \(\alpha _{jl_0}\) and \(\beta _{jl_0}\) are deduced by
where \(\mu _{jl}\) is the mean and \(\sigma _{jl}\) is the standard-deviation, \(j=1,\dots ,K,~ l=1,\dots ,D\).
Expecting the Responsibilities. The responsibilities or posterior probabilities play an essential role in the likelihood-based estimation technique. It affects the update of the parameters in the next following step using the current parameter value.
Maximizing and Upgrading the GID Parameters. At the beginning, we set the gradient of log-likelihood function with respect to the mixing weight parameter equals to zero:
Then, we obtain the update formula for \(\pi _j\), for \(j=1,\dots ,K\) as
where \(P(j|\mathbf {Y}_i)\) is the posterior computed in the E-step.
To learn the parameters \(\mathbf {\alpha }_j\) and \(\mathbf {\beta }_j\), the Fisher scoring algorithm [25] is used. Thus, we need to calculate the first and the second derivatives of the log-likelihood function based on the following update [1]:
The first derivatives of \(\log P (\mathcal {Y},\Theta )\) are given by:
The second derivative with respect to \(\alpha _{jl}\) is given by:
The second derivative w.r.t \(\beta _{jl}\) is obtained through the same development.
2.3 GID Optimal Predictor
In this section, we present our novel non-linear optimal predictor method based on generalized inverted Dirichlet distribution. We consider the conditional expectation property to predict one random variable from the other neighboring variables.
We consider p data points \( ({X}_i,{X}_{i+1},\dots ,{X}_{i+p-1})\), knowing their values, we predict the neighboring data point \(\hat{{X}}_{i+p}\) on the base of minimizing the mean squared error (MSE). Therefore, we model the joint density of \({X}_{i+p}\) and its neighbors using the generalized inverted Dirichlet. We take \(i=0\) and we derive the equations.
Considering generalized inverted Dirichlet properties [22], the conditional random variable \(Y_{p}\) follows an inverted Beta distribution:
Consequently, the conditional probability density function of \(X_{p}\) is
where \(T_{p-1} = 1+\sum _{k=1}^{p-1}X_k\).
Hence, the conditional expectation expression of \(X_{p}\) is expressed as follows.
In the case of mixture models, the optimal predictor expression can be derived directly by following steps defined in [19] (more details are in [40]):
where
Finally, the GID optimal predictor is resumed in the following linear combination of \( X_{p}\) neighbors:
3 Experimental Results
Image inpainting is the process of restoring deteriorating, damaged or missing parts of an image to produce a complete picture. It is an active area of image processing research [32,33,34] where machine learning has exciting results comparable to artists’ results. Mainly, in this process, we will be completing a missing pixel by an approximated value that depends on its neighborhood. In our work, we use the 3\(^{rd}\) order non-symmetrical half-plane casual neighborhood system [19, 21]. We apply the model on a publicly available dataset; Paris StreetView [16]. Then, we compare it with the widely used mixture of Gaussian predictor, generalized Dirichlet mixture predictor and inverted Dirichlet mixture predictor. We are not trying to restore the ground-truth image, our goal is to get an output image that is close enough or similar to the ground-truth. Therefore, we use the structural similarity index (SSIM) [11] to gauge the differences between the predicted images and the original ones. We also perform signal to noise ratio (PSNR) [14] to evaluate the performance of the models.
We reduce the size of the original images to \(256\times 256\) to minimize the complexity of computing the model’s parameters. We train the model on \(70\%\) of the database and we test on the rest. We apply two types of masks. The first one is randomly distributed as shown in Fig. 1a. And, for the second one, we discard lines of the images, as in Fig. 1b. Finally, we compute the SSIM and PSNR of each test image with its corresponding ground-truth, and we average all over the test set.
As we mentioned earlier, we discard around \(15\%\) of the pixels randomly. Figure 2 reveals that the difference between models’ prediction is undetectable visually. Moreover, Table 1 shows that the difference between the models is not significant. There is a slight advantage for the use of GIDM model compared to the others. Thus, we conclude that this approach of models’ evaluation is not appropriate. For that, we decide to remove slightly thick lines of pixels and re-evaluate the models.
To evaluate the models’ performance, we used TensorFlow to calculate the PSNR and Skimage python package for the SSIM metric.
After discarding lines from the images, we are able to generate back again the missing pixels, and Fig. 3 demonstrates that GIDM is the most efficient model among all the others. This is also clear in Table 2, where we can notice in the chosen images that GIDM is the most accurate re-generator of discarded pixels. Therefore, our work has shown that image data is better represented by generalized inverted Dirichlet. It is noteworthy to mention that these models’ performance is hugely dependent on the size of the masks, the hyper-parameters, the type and order of the neighbouring system.
4 Conclusion
In this paper, we have developed a new optimal predictor based on finite generalized inverted Dirichlet mixtures. The GID demonstrates its efficiency in representing positive vectors due to its statistical characteristics through its covariance structure.
We learnt the model parameters using MLE approach with Newton Raphson method, and we considered the NSHP neighbouring system to compute the predicted pixel. We evaluated the GID optimal predictor on image inpainting and we compared the proposed model to other similar related works. The experimental results demonstrate its capability that offers reliable prediction and modeling potential.
References
Al Mashrgy, M., Bdiri, T., Bouguila, N.: Robust simultaneous positive data clustering and unsupervised feature selection using generalized inverted dirichlet mixture models. Knowl. Based Syst. 59, 182–195 (2014)
Mashrgy, M.A., Bouguila, N.: A fully Bayesian framework for positive data clustering. In: Laalaoui, Y., Bouguila, N. (eds.) Artificial Intelligence Applications in Information and Communication Technologies. SCI, vol. 607, pp. 147–164. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19833-0_7
Bdiri, T., Bouguila, N.: An infinite mixture of inverted Dirichlet distributions. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011. LNCS, vol. 7063, pp. 71–78. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24958-7_9
Bdiri, T., Bouguila, N.: Learning inverted dirichlet mixtures for positive data clustering. In: Kuznetsov, S.O., Ślezak, D., Hepting, D.H., Mirkin, B.G. (eds.) RSFDGrC 2011. LNCS (LNAI), vol. 6743, pp. 265–272. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21881-1_42
Bdiri, T., Bouguila, N.: Positive vectors clustering using inverted dirichlet finite mixture models. Expert Syst. Appl. 39(2), 1869–1882 (2012)
Bdiri, T., Bouguila, N.: Bayesian learning of inverted dirichlet mixtures for SVM kernels generation. Neural Comput. Appl. 23(5), 1443–1458 (2013)
Bdiri, T., Bouguila, N., Ziou, D.: Visual scenes categorization using a flexible hierarchical mixture model supporting users ontology. In: 25th IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2013, Herndon, VA, USA, 4–6 November 2013, pp. 262–267. IEEE Computer Society (2013)
Bdiri, T., Bouguila, N., Ziou, D.: Object clustering and recognition using multi-finite mixtures for semantic classes and hierarchy modeling. Expert Syst. Appl. 41(4), 1218–1235 (2014)
Bdiri, T., Bouguila, N., Ziou, D.: A statistical framework for online learning using adjustable model selection criteria. Eng. Appl. Artif. Intell. 49, 19–42 (2016)
Bdiri, T., Bouguila, N., Ziou, D.: Variational Bayesian inference for infinite generalized inverted dirichlet mixtures with feature selection and its application to clustering. Appl. Intell. 44(3), 507–525 (2016). https://doi.org/10.1007/s10489-015-0714-6
Bouguila, N.: Non-gaussian mixture image models prediction. In: 2008 15th IEEE International Conference on Image Processing, pp. 2580–2583. IEEE (2008)
Bourouis, S., Al Mashrgy, M., Bouguila, N.: Bayesian learning of finite generalized inverted dirichlet mixtures: application to object classification and forgery detection. Expert Syst. Appl. 41(5), 2329–2336 (2014)
Boutemedjet, S., Bouguila, N., Ziou, D.: A hybrid feature extraction selection approach for high-dimensional non-Gaussian data clustering. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1429–1443 (2008)
Channoufi, I., Bourouis, S., Bouguila, N., Hamrouni, K.: Image and video denoising by combining unsupervised bounded generalized gaussian mixture modeling and spatial information. Multimedia Tools Appl. 77(19), 25591–25606 (2018). https://doi.org/10.1007/s11042-018-5808-9
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B (Methodol.) 39(1), 1–22 (1977)
Doersch, C., Singh, S., Gupta, A., Sivic, J., Efros, A.A.: What makes Paris look like Paris? ACM Trans. Graph. (SIGGRAPH) 31(4), 101:1–101:9 (2012)
Elguebaly, T., Bouguila, N.: Finite asymmetric generalized Gaussian mixture models learning for infrared object detection. Comput. Vis. Image Underst. 117(12), 1659–1671 (2013)
Fan, W., Bouguila, N.: Topic novelty detection using infinite variational inverted dirichlet mixture models. In: Li, T., et al. (eds.) 14th IEEE International Conference on Machine Learning and Applications, ICMLA 2015, Miami, FL, USA, 9–11 December 2015, pp. 70–75. IEEE (2015)
Graja, O., Bouguila, N.: Finite inverted dirichlet mixture optimal pixel predictor. In: 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 1–5. IEEE (2019)
Grimmett, G., Grimmett, G.R., Stirzaker, D., et al.: Probability and Random Processes. Oxford University Press, New York (2001)
Guidara, R., Hosseini, S., Deville, Y.: Maximum likelihood blind image separation using nonsymmetrical half-plane Markov random fields. IEEE Trans. Image Process. 18(11), 2435–2450 (2009)
Gupta, A., Song, D.: Generalized liouville distribution. Comput. Math. Appl. 32(2), 103–109 (1996)
Kotz, S., Balakrishnan, N., Johnson, N.L.: Continuous multivariate distributions, Volume 1: Models and applications. Wiley, Hoboken (2004)
Li, Y., Sayood, K.: Lossless video sequence compression using adaptive prediction. IEEE Trans. Image Process. 16(4), 997–1007 (2007)
Lindley, D., Rao, C.R.: Advanced statistical methods in biometric research. (1953)
Lingappaiah, G.: On the generalised inverted dirichlet distribution. Demostratio Mathematica 9(3), 423–433 (1976)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Maini, R., Aggarwal, H.: Study and comparison of various image edge detection techniques. Int. J. Image Process. (IJIP) 3(1), 1–11 (2009)
Makhoul, J.: Linear prediction: a tutorial review. Proc. IEEE 63(4), 561–580 (1975)
Mathai, A., Moschopoulos, P.: A multivariate inverted beta model. Statistica 57, 189–198 (1997)
Matteoli, S., Diani, M., Corsini, G.: A tutorial overview of anomaly detection in hyperspectral images. IEEE Aerosp. Electron. Syst. Mag. 25(7), 5–28 (2010)
Mosleh, A., Bouguila, N., Hamza, A.B.: Bandlet-based sparsity regularization in video inpainting. J. Vis. Commun. Image Represent. 25(5), 855–863 (2014)
Mosleh, A., Bouguila, N., Hamza, A.B.: A video completion method based on bandlet transform. In: Proceedings of the 2011 IEEE International Conference on Multimedia and Expo, ICME 2011, Barcelona, Catalonia, Spain, 11–15 July 2011, pp. 1–6. IEEE Computer Society (2011)
Mosleh, A., Bouguila, N., Hamza, A.B.: Automatic inpainting scheme for video text detection and removal. IEEE Trans. Image Process. 22(11), 4460–4472 (2013)
Ng, K.W., Tian, G.L., Tang, M.L.: Dirichlet and Related Distributions: Theory, Methods and Applications, vol. 888. Wiley, Hoboken (2011)
Sefidpour, A., Bouguila, N.: Spatial color image segmentation based on finite non-Gaussian mixture models. Expert Syst. Appl. 39(10), 8993–9001 (2012)
Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1145–1153 (2017)
Therrien, C.W., Quatieri, T.F., Dudgeon, D.E.: Statistical model-based algorithms for image analysis. Proc. IEEE 74(4), 532–551 (1986)
Tirdad, P., Bouguila, N., Ziou, D.: Variational learning of finite inverted dirichlet mixture models and applications. In: Laalaoui, Y., Bouguila, N. (eds.) Artificial Intelligence Applications in Information and Communication Technologies. SCI, vol. 607, pp. 119–145. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19833-0_6
Zhang, J., Ma, D.: Nonlinear prediction for gaussian mixture image models. IEEE Trans. Image Process. 13(6), 836–847 (2004)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Graja, O., Najar, F., Bouguila, N. (2020). Generalized Inverted Dirichlet Optimal Predictor for Image Inpainting. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2020. Lecture Notes in Computer Science(), vol 12509. Springer, Cham. https://doi.org/10.1007/978-3-030-64556-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-64556-4_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64555-7
Online ISBN: 978-3-030-64556-4
eBook Packages: Computer ScienceComputer Science (R0)