Abstract
The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter ε. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion coefficients up to second order in ε. Using a conjecture we extend the calculation to 11th order and discuss the convergence of the resulting series
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Y. Ephraim N. Merhav (2002) ArticleTitleHidden Markov processes IEEE Trans. Inform. Theory 48 1518–1569 Occurrence Handle10.1109/TIT.2002.1003838 Occurrence Handle2003f:94024
A. Schliep A. Schönhuth C. Steinhoff (2003) ArticleTitleUsing hidden Markov models to analyze gene expression time course data Bioinformatics 19 IssueIDSuppl. 1 i255–i263
L.R. Rabiner (1989) ArticleTitleA tutorial on hidden Markov models and selected applications in speech recognition Proc. IEEE 77 257–286 Occurrence Handle10.1109/5.18626
I. Kanter A. Frydman A. Ater (2005) ArticleTitleIs a multiple excitation of a single atom equivalent to a single ensemble of atoms? Europhys. Lett. 69 IssueID6 874–878 Occurrence Handle10.1209/epl/i2004-10441-9
I. Kanter A. Frydman A. Ater (2005) ArticleTitleUtilizing hidden Markov processes as a new tool for experimental physics Europhys. Lett. 69 IssueID5 798–804 Occurrence Handle10.1209/epl/i2004-10407-y
Shannon C.E. A mathematical theory of communication. Bell Sys. Tech. J. 27:379–423 and 623–656 (1948).
Nattermann T., Theory of the random field Ising model, in Spin Glasses and Random Fields, Young A.P., ed. (World Scientific, 1997).
Jacquet P., Seroussi G., and Szpankowski W., On the Entropy of a Hidden Markov Process, Data Compression Conference, Snowbird (2004).
Ordentlich E., and Weissman T., New Bounds on the Entropy Rate of Hidden Markov Processes, San Antonio Information Theory Workshop (Oct. 2004).
A preliminary presentation of our results is given in Zuk O., Kanter I., and Domany E., Asymptotics of the Entropy Rate for a Hidden Markov Process, Data Compression Conference, Snowbird (2005).
T.M. Cover J.A. Thomas (1991) Elements of Information Theory Wiley New York
Saul L.K., and Jordan M.I., Boltzmann Chains and Hidden Markov Models, Advances in Neural Information Processing Systems, Volume 7 (MIT Press, 1994).
D.J.C. MacKay (1996) ArticleTitleEquivalence of Boltzmann chains and Hidden Markov Models Neural Comput. 8 IssueID1 178–181 Occurrence Handle97i:81051
B. Derrida M.M. France J. Peyriere (1986) ArticleTitleExactly solvable one-dimensional inhomogeneous models J. of Stat. Phys 45 IssueID3–4 439–449
D.S. Fisher P. Le Doussal P. Monthus (2001) ArticleTitleNonequilibrium dynamics of random field Ising spin chains: Exact results via real space renormalization group Phys. Rev. E 64 IssueID6 066–107 Occurrence Handle10.1103/PhysRevE.64.066107 Occurrence Handle2002k:82071
G. Grinstein D. Mukamel (1983) ArticleTitleExact solution of a one-dimensional ising-model in a random magnetic field Phys. Rev. B 27 4503–4506 Occurrence Handle10.1103/PhysRevB.27.4503 Occurrence Handle1983PhRvB..27.4503G
Nieuwenhuizen T.M., and Luck J.M., Exactly soluble random field Ising models in one-dimension, J. Phys. A: Math. Gen. 19:1207–1227 (May 1986).
Derrida B., and Hilhorst H.J. (1983). Singular behavior of certain infinite products of random 2× 2 matrices. J. Phys. A 16:2641–2654
http://www.maplesoft.com/
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zuk, O., Kanter, I. & Domany, E. The Entropy of a Binary Hidden Markov Process. J Stat Phys 121, 343–360 (2005). https://doi.org/10.1007/s10955-005-7576-y
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10955-005-7576-y