Abstract
Let (P i , Q i ), i = 0, 1, be two pairs of probability measures defined on measurable spaces (Ω i ,F i ) respectively. Assume that the pair (P1, Q1) is more informative than (P0,Q0) for testing problems. This amounts to say that If (P1,Q1) ≥ If (P0,Q0), where If (·, ·) is an arbitrary f-divergence. We find a precise lower bound for the increment of f-divergences If(P1,Q1) − If(P0,Q0) provided that the total variation distances ||Q1 − P1|| and ||Q0 − P0|| are given. This optimization problem can be reduced to the case where P1 and Q1 are defined on the space consisting of four points, and P0 and Q0 are obtained from P1 and Q1 respectively by merging two of these four points. The result includes the well-known lower and upper bounds for If(P,Q) given ||Q − P||.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
S. M. Ali and S. D. Silvey, “A General Class of Coefficients of Divergence of One Distribution from Another”, J. Roy. Statist. Soc., Ser. B (Methodological) 28, 131–142 (1966).
D. Blackwell, “Comparison of Experiments”, in Proc. Second Berkeley Symp. on Math. Statist. and Probab. (Univ. of Calif. Press, 1951), pp. 93–102.
I. Csiszár, “Eine InformationstheoretischeUngleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten”, Magyar Tud. Akad.Mat. KutatóInt. Közl. 8, 85–108 (1963).
I. Csiszár, “Information-TypeMeasures ofDifference of ProbabilityDistributions and IndirectObservations”, Stud. Sci.Math. Hung. 2, 299–318 (1967).
D. Feldman and F. Österreicher, “ANote on f-Divergences”, Stud. Sci. Math. Hung. 24, 191–200 (1989).
G. L. Gilardoni, “On the Minimum f-Divergence for Given Total Variation”, C. R. Math. Acad. Sci. Paris 343, 763–766 (2006).
G. L. Gilardoni, “An Improvement on Vajda’s Inequality”, in Progress in Probability (Birkhäuser, Basel, 2008), Vol. 60, pp. 299–304.
G. L. Gilardoni, “On Pinsker’s and Vajda’s Type Inequalities for Csiszár’s f-Divergences”, IEEE Trans. Inform. Theory 56, 5377–5386 (2010).
A. Guntuboyina, S. Saha, and G. Schiebinger, “Sharp Inequalities for f-divergences”, IEEE Trans. Inform. Theory 60, 104–121 (2014).
A. A. Gushchin, “On an Extension of the Notion of f-Divergence”, Theory Probab. Appl. 52, 439–455 (2008).
L. Le Cam, “Sufficiency and Approximate Sufficiency”, Ann. Math. Statist. 35, 1419–1455 (1964).
L. Le Cam, “On the Information Contained in Additional Observations”, Ann. Statist. 2, 630–649 (1974).
E. L. Lehmann and J. P. Romano, Testing Statistical Hypotheses (Springer Science & Business Media, New York, 2006).
F. Liese, “f-Divergences, Sufficiency, Bayes Sufficiency, and Deficiency”, Kybernetika 48, 690–713 (2012).
F. Liese and I. Vajda, Convex Statistical Distances (Teubner, Leipzig, 1987).
F. Liese and I. Vajda, “On Divergences and Information in Statistics and Information Theory”, IEEE Trans. Inform. Theory 52, 4394–4412 (2006).
F. Liese and I. Vajda, “f-Divergences: Sufficiency, Deficiency and Testing of Hypotheses”, in Advances in Inequalities from Probability Theory and Statistics, Ed. by N. S. Barnett and S. S. Dragomir (Nova Science, New York, 2008), pp. 113–158.
F. Österreicher and D. Feldman, “Divergenzen von Wahrscheinlichkeitsverteilungen Integralgeometrisch Betrachtet”, ActaMath. Sci. Hungar. 37, 329–337 (1981).
M. D. Reid and R. C.Williamson, “Generalized Pinsker Inequalities”, in The 22nd Ann. Conf. on Learning Theory (COLT 2009) (2009), pp. 1–10.
H. Strasser, Mathematical Theory of Statistics (de Gruyter, Berlin, New York, 1985).
E. Torgersen, Comparison of Statistical Experiments (Cambridge Univ. Press, Cambridge, 1991).
I. Vajda, “On the f-Divergence and Singularity of Probability Measures”, Period. Math. Hung. 2, 223–234 (1972).
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Gushchin, A.A. The minimum increment of f-divergences given total variation distances. Math. Meth. Stat. 25, 304–312 (2016). https://doi.org/10.3103/S1066530716040049
Received:
Published:
Issue Date:
DOI: https://doi.org/10.3103/S1066530716040049