Abstract
In anomalous statistical physics, deformed algebraic structures are important objects. Heavily tailed probability distributions, such as Student’s t-distributions, are characterized by deformed algebras. In addition, deformed algebras cause deformations of expectations and independences of random variables. Hence, a generalization of independence for multivariate Student’s t-distribution is studied in this paper. Even if two random variables which follow to univariate Student’s t-distributions are independent, the joint probability distribution of these two distributions is not a bivariate Student’s t-distribution. It is shown that a bivariate Student’s t-distribution is obtained from two univariate Student’s t-distributions under q-deformed independence.
H. Matsuzoe—This work was partially supported by MEXT KAKENHI Grant Numbers 15K04842 and 26108003.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
- Deformed exponential family
- Deformed independence
- Statistical manifold
- Tsallis statistics
- Information geometry
1 Introduction
In the theory of complex systems, heavily tailed probability distributions are important objects. Power law tailed probability distributions and their related probability distributions have been studied in anomalous statistical physics ([6, 12, 15]). One of an important probability distribution in anomalous statistical physics is a q-Gaussian distribution. It is a noteworthy fact that a q-Gaussian distribution coincides with a Student’s t-distribution in statistics. Hence we can discuss Student’s t-distributions from the viewpoint of anomalous statistical physics. Though Student’s t-distributions have been studied by many authors (cf. [3, 7]), our motivation is quite different from the others.
Heavily tailed probability distributions including Student’s t-distributions are represented using deformed exponential functions (cf. [11, 12]). However, these functions do not satisfy the law of exponents. Hence deformed algebraic structures are naturally introduced (cf. [4, 6]). Once such a deformed algebra is introduced, the sample space can be regarded as a some deformed algebraic space, not the standard Euclidean space (cf. [11]). Hence it is natural to introduce suitable deformed expectations and independences of random variables. In fact, we find that the duality of exponential and logarithm can express the notion of independence of random variables. Hence we can generalize the independence using deformed exponential and deformed logarithm functions [9].
In this paper, we summarize such deformed algebraic structures, then we apply these deformed algebras to multivariate Student’s t-distributions. Even if two independent random variables follow to univariate Student’s t-distributions, the joint probability distribution is not a bivariate Student’s t-distribution. Hence we show that a bivariate Student’s t-distribution can be obtained from two univariate Student’s t-distributions under q-deformed independence with a suitable normalization.
We remark that deformed algebraic structures for statistical models and generalization of independence are discussed in information geometry. (cf. [5, 9, 11]. See also [1].) Though normalizations of positive densities are necessary in the arguments of generalized independence, statistical manifold structures are changed by normalizations of positive densities. In particular, generalized conformal equivalence relations for statistical manifolds are needed (cf. [9, 10]). Hence a statistical manifold of the set of bivariate Student’s t-distributions with q-independent random variables is not equivalent to a product statistical manifold of two sets of univariate Student’s t-distributions.
2 Deformed Exponential Families
In this paper, we assume that all objects are smooth for simplicity. Let us begin by reviewing the foundations of deformed exponential functions and deformed exponential families (cf. [9, 12]).
Let \(\chi \) be a strictly increasing function from \((0,\infty )\) to \((0,\infty )\). We define a \(\chi \) -logarithm function or a deformed logarithm function by
The inverse of \(\ln _{\chi }s\) is called a \(\chi \) -exponential function or a deformed exponential function, which is defined by
where the function u(s) is given by \(u(\ln _{\chi }s) = \chi (s)\).
From now on, we suppose that \(\chi \) is a power function, that is, \(\chi (t) = t^q\). Then the deformed logarithm and the deformed exponential are defined by
We say that \(\ln _q s\) is a q-logarithm function and \(\exp _q t\) is a q -exponential function. By taking a limit \(q \rightarrow 1\), these functions coincide with the standard logarithm \(\ln s\) and the standard exponential \(\exp t\), respectively. In this paper, we focus on q-exponential case. However, many of arguments for q-exponential family can be generalized for \(\chi \)-exponential family ([9, 11]).
A statistical model \(S_{q}\) is called a q-exponential family if
where \(F_1(x), \dots , F_n(x)\) are functions on the sample space \(\varOmega \), \(\theta = {}^t(\theta ^1, \dots , \theta ^n)\) is a parameter, and \(\psi (\theta )\) is the normalization with respect to the parameter \(\theta \).
Example 1
(Student’s t-distribution). Fix a number \(q \ (1 < q < 1+2/d, \ d \in \mathbf {N})\), and set \(\nu = -d - 2/(1-q)\). We define an n -dimensional Student’s t -distribution with degree of freedom \(\nu \) or a q -Gaussian distribution by
where \(X = {}^t(X_1, \dots , X_d)\) is a random vector on \(\mathbf {R}^d\), \(\mu = {}^t(\mu ^1, \dots , \mu ^d)\) is a location vector on \(\mathbf {R}^d\) and \(\varSigma \) is a scale matrix on \(\mathrm {Sym}^+(d)\). For simplicity, we assume that \(\varSigma \) is invertible. Otherwise, we should choose a suitable basis \(\{v^{\alpha }\}\) on \(\mathrm {Sym}^+(d)\) such that \(\varSigma = \sum _{\alpha }w_{\alpha }v^{\alpha }\). Then, the set of all Student’s t-distributions is a q-exponential family. In fact, set
Then we have
Since \(\theta \in \mathbf {R}^d\) and \(\tilde{R} \in \mathrm {Sym}^+(d)\), the set of all Student’s t-distributions is a \(d(d+3)/2\)-dimensional q-exponential family. The normalization \(\psi (\theta )\) is given by
3 Statistical Manifold Structures Based on q-Fisher Metric
In this section we give a brief review of statistical manifold structures on a q-exponential family. We consider a q-Fisher metric in this paper. However, it is known that a q-exponential family naturally has three kinds of statistical manifold structures. See [2, 9, 11] for more details.
Let \(S_q\) be a q-exponential family. The normalization \(\psi (\theta )\) on \(S_q\) is convex, but may not be strictly convex. Hence we assume that \(\psi \) is strictly convex throughout this paper. In fact, we obtain the following proposition.
Proposition 1
Let \(S_q = \{p(x;\theta )\}\) be a q-exponential family. Then the normalization function \(\psi (\theta )\) is convex.
Proof
Set \(u(x) = \exp _q x\) and \(\partial _i = \partial /\partial \theta ^i\). Then we have
Since \(\partial _i\int _{\varOmega }p(x;\theta )dx = \int _{\varOmega }\partial _ip(x;\theta )dx = 0\) and \( \int _{\varOmega }\partial _i\partial _jp(x;\theta )dx =0\), we have
For an arbitrary vector \(c= {}^t(c^1, c^2, \dots . c^n) \in \mathbf{R}^n\), since \(Z_{q}(p) > 0\) and \(u''(x) >0\), we have
This implies that the Hessian matrix \((\partial _i\partial _j\psi (\theta ))\) is semi-positive definite. \(\square \)
From the assumption for \(\psi (\theta )\), we can define the q -Fisher metric and the q -cubic form by
respectively. For a fixed real number \(\alpha \), set
where \(\nabla ^{q(0)}\) is the Levi-Civita connection with respect to g. Since g is a Hessian metric, from standard arguments in Hessian geometry [13], \(\nabla ^{q(e)} := \nabla ^{q(1)}\) and \(\nabla ^{q(m)} := \nabla ^{q(-1)}\) are flat affine connections and mutually dual with respect to g. Hence the quadruplet \((S_q, g, \nabla ^{q(e)}, \nabla ^{q(m)})\) is a dually flat space.
Next, we consider deformed expectations for q-exponential families. We define the escort distribution \(P_q(x;\theta )\) of \(p(x;\theta ) \in S_q\) and the normalized escort distribution \(P_q^{esc}(x;\theta )\) by
respectively. Let f(x) be a function on \(\varOmega \). The q-expectation \(E_{q,p}[f(x)]\) and the normalized q-expectation \(E_{q,p}^{esc}[f(x)]\) are defined by
respectively. Under q-expectations, we have the following proposition. (cf. [8])
Proposition 2
For \(S_q\) a q-exponential family, (1) set \(\phi (\eta ) = E_{q,p}^{esc}[\log _qp(x;\theta )]\), then \(\phi (\eta )\) is the potential of g with respect to \(\{\eta _i\}\). (2) Set \(\eta _i = E_{q,p}^{esc}[F_i(x)]\). Then \(\{\eta _i\}\) is a \(\nabla ^{q(m)}\)-affine coordinate system such that
\(\square \)
We define an \(\alpha \) -divergence \(D^{(\alpha )}\) with \(\alpha = 1-2q\) and a q -relative entropy (or a normalized Tsallis relative entropy) \(D_q^T\) by
respectively. It is known that the \(\alpha \)-divergence \(D^{(1-2q)}(r,p)\) induces a statistical manifold structure \((S_q, g^F, \nabla ^{(2q-1)})\), where \(g^F\) is the Fisher metric on \(S_q\) and \(\nabla ^{(2q-1)}\) is the \(\alpha \)-connection with \(\alpha = 2q-1\), and the q-relative entropy \(D_q^T(r,p)\) induces \((S_q, g, \nabla ^{q(e)})\).
Proposition 3
(cf. [10]). For a q-exponential family \(S_q\), the two statistical manifolds \((S_q, g^F, \nabla ^{(2q-1)})\) and \((S_q, g, \nabla ^{q(e)})\) are 1-conformally equivalent. \(\square \)
We remark that the difference of a \(\alpha \)-divergence and a q-relative entropy is only the normalization \(q/Z_q(p)\). Hence a normalization for probability density imposes a generalized conformal change for a statistical model.
4 Generalization of Independence
In this section, we review the notions of q-deformed product and generalization of independence. For more details, see [9, 11].
Let us introduce the q-deformed algebras since q-exponential functions and q-logarithm functions do not satisfy the law of exponent. Let \(\exp _q x\) be a q-exponential function and \(\ln _q y\) be a q-logarithm function. For a fixed number q, we suppose that
We define the q-sum \(\tilde{\oplus }^q\) and the q-product \(\otimes _q\) by the following formulas [4]:
Since the base of an exponential function and the argument of a logarithm functions must be positive, conditions (2) and (3) are necessary. We then obtain q-deformed law of exponents as follows.
We remark that the q-sum works on the domain of a q-exponential function and a q-product works on the target space. This implies that the domain of q-exponential function (i.e. the total sample space \(\varOmega \)) may not be a standard Euclidean space.
Let us recall the notion of independence of random variables. Suppose that X and Y are random variables which follow to probabilities \(p_1(x)\) and \(p_2(y)\), respectively. We say that two random variables are independent if the joint probability p(x, y) is given by the product of \(p_1(x)\) and \(p_2(y)\):
Hence \(p_1(x)\) and \(p_2(y)\) are marginal distributions of p(x, y). When \(p_1(x)>0\) and \(p_2(y)>0\), the independence is equivalent to the additivity of information:
Let us generalize the notion of independence based on q-products. Suppose that X and Y are random variables which follow to probabilities \(p_1(x)\) and \(p_2(y)\), respectively. We say that X and Y are q -independent with e-normalization (or exponential normalization) if a probability density p(x, y) is decomposed by
where c is the normalization defined by
We say that X and Y are q -independent with m-normalization (or mixture normalization) if a probability density p(x, y) is decomposed by
where \(Z(p_1, p_2)\) is the normalization defined by
In the case of q-exponential families, including the standard exponential families, we can change normalizations from exponential type to mixture type and vice versa. (See the calculation in Example 1.) Hence we can carry out e- and m-normalization simultaneously. However, e- and m-normalizations are different in general [14].
In some problems, the normalization of probability density is not necessary. In this case, we say that X and Y are q -independent if a positive function f(x, y) is decomposed by a q-product of two probability densities \(p_1(x)\) and \(p_2(y)\):
The function f(x, y) is not necessary to be a probability density. In addition, the total integral of f(x, y) may diverge.
5 q-independence and Student’s t-distributions
In this section, we consider relations between univariate and bivariate Student’s t-distributions
Suppose that \(X_1\) and \(X_2\) are random variables which follow to univariate Student’s t-distributions \(p_1(x_1)\) and \(p_2(x_2)\), respectively. Even if \(X_1\) and \(X_2\) are independent, the joint probability \(p_1(x_1)p_2(x_2)\) is not a bivariate Student’s t-distribution [7]. We show that q-deformed algebras work for bivariate Student’s t-distributions.
Theorem 1
Suppose that \(X_1\) and \(X_2\) are random variables which follow to univariate Student’s t-distributions \(p_1(x_1)\) and \(p_2(x_2)\), respectively, with same parameter \(q \ (1<q<2)\). Then there exist a bivariate Student’s t-distribution \(p(x_1,x_2)\) such that \(X_1\) and \(X_2\) are q-independent with e-normalization.
Proof
Suppose that \(X_1\) follows to a univariate Student’s t-distribution (or a q-Gaussian distribution) given by
where \(\mu _1 \ (-\infty < \mu < \infty )\) is a location parameter, and \(\sigma _1 \ (0 < \sigma < \infty )\) is a scale parameter. Similarly, suppose that \(X_2\) follows to \(p(x_2;\mu _2,\sigma _2)\). By setting
we obtain a q-exponential representation as follows:
where \(\theta ^{1}\) and \(\theta ^{11}\) are natural parameters defined by
We remark that the normalization \(z_q(\sigma _1)\) can be determined by the parameter \(\theta ^{11}\). Therefore, \(p(x_1;\mu _1,\sigma _1)\) is uniquely determined from natural parameters \(\theta ^{1}\) and \(\theta ^{11}\). Set \(\theta ^{2}\) and \(\theta ^{22}\) by changing parameters to \(\mu _2\) and \(\sigma _2\). Then we obtain a positive density by
where \(A(\theta )\) is given by
Recall that \(p(x_1;\mu _1,\sigma _1) \otimes _qp(x_2;\mu _2,\sigma _2)\) is not a probability distribution. Set the e-normalization function c by
where \(z_q\) is the m-normalization function of bivariate Student’s t-distribution.
As a consequence, we have
This implies that \(X_1\) and \(X_2\) are q-independent with e-normalization, and the joint positive measure \(p(x_1, x_2)\) is a bivariate Student’s t-distribution. \(\square \)
Let us give the normalization function \(z_q\) in \(\theta \)-coordinate, explicitly. Using a property of gamma function, we have
Hence the m-normalization function of bivariate Student’s t-distribution is simply given by
From Equation (1) and (4), the constant \(z_q\) should be given by
6 Concluding Remarks
In this paper, we showed that a bivariate Student’s t-distribution can be obtained from two univariate Student’s t-distributions using e- and m-normalizations. Recall that statistical manifold structures of statistical models are changed by their normalizations. Hence a statistical manifold structure of a bivariate Student’s t-distribution does not coincide with the product manifold structure of two univariate Student’s t-distributions.
References
Amari, S., Nagaoka, H.: Method of Information Geometry. American Mathematical Society, Providence. Oxford University Press, Oxford (2000)
Amari, S., Ohara, A., Matsuzoe, H.: Geometry of deformed exponential families: invariant, dually-flat and conformal geometry. Phys. A 391, 4308–4319 (2012)
Berg, C., Vignat, C.: On the density of the sum of two independent Student \(t\)-random vectors. Statist. Probab. Lett. 80, 1043–1055 (2010)
Borgesa, E.P.: A possible deformed algebra and calculus inspired in nonextensive thermostatistics. Phys. A 340, 95–101 (2004)
Fujimoto, Y., Murata, N.: A generalization of independence in naive bayes model. In: Fyfe, C., Tino, P., Charles, D., Garcia-Osorio, C., Yin, H. (eds.) IDEAL 2010. LNCS, vol. 6283, pp. 153–161. Springer, Heidelberg (2010)
Kaniadakis, G.: Theoretical foundations and mathematical formalism of the power-law tailed statistical distributions. Entropy 15, 3983–4010 (2013)
Kotz, S., Nadarajah, S.: Multivariate \(t\) Distributions and Their Applications. Cambridge University Press, New York (2004)
Matsuzoe, H.: Statistical manifolds and geometry of estimating functions. In: Prospects of Differential Geometry and its Related Fields, World Scientific Publishing, pp. 187–202 (2013)
Matsuzoe, H., Henmi, M.: Hessian structures and divergence functions on deformed exponential families. In: Nielsen, F. (ed.) Geometric Theory of Information. Signals and Communication Technology. Springer, Switzerland (2014)
Matsuzoe, H.., Ohara, A..: Geometry for \(q\)-exponential families. In: Recent Progress in Differential Geometry and its Related Fields, World Scientific Publishing, pp. 55–71 (2011)
Matsuzoe, H., Wada, T.: Deformed algebras and generalizations of independence on deformed exponential families. Entropy 17, 5729–5751 (2015)
Naudts, J.: Generalised Thermostatistics. Springer, London (2011)
Shima, H.: The Geometry of Hessian Structures. World Scientific Publishing, Singapore (2007)
Takatsu, A.: Behaviors of \(\varphi \)-exponential distributions in Wasserstein geometry and an evolution equation. SIAM J. Math. Anal. 45, 2546–2556 (2013)
Tsallis, C.: Introduction to Nonextensive Statistical Mechanics: Approaching a Complex World. Springer, New York (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Sakamoto, M., Matsuzoe, H. (2015). A Generalization of Independence and Multivariate Student’s t-distributions. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2015. Lecture Notes in Computer Science(), vol 9389. Springer, Cham. https://doi.org/10.1007/978-3-319-25040-3_79
Download citation
DOI: https://doi.org/10.1007/978-3-319-25040-3_79
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25039-7
Online ISBN: 978-3-319-25040-3
eBook Packages: Computer ScienceComputer Science (R0)