Abstract
We consider infinite-dimensional optimization problems involving entropy-type functionals in the objective function as well as as in the constraints. A duality theory is developed for such problems and applied to the reliability rate function problem in information theory.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Kullback, S., andLeibler, R. A.,On Information and Sufficiency, Annals of Mathematical Statistics, Vol. 22, pp. 79–86, 1951.
Kullback, S.,Information Theory and Statistics, John Wiley and Sons, New York, New York, 1959.
Akaike, H.,Information Theory and an Extension of the Maximum Likelihood Principle, Proceedings of the 2nd International Symposium of Information Theory, Berkeley, California, 1972.
Charnes, A., Raike, W. M., andBettinger, C. O.,An External and Information Theoretic Characterization of Some Interzonal Transfers, Socio-Economic Planning Sciences, Vol. 6, pp. 531–537, 1972.
Charnes, A., Cooper, W. W., andLearner, D. B.,Constrained Information Theoretic Characterizations in Consumer Purchase Behavior, Journal of the Operational Research Society, Vol. 29, pp. 833–840, 1978.
Charnes, A., andCooper, W. W.,An Extremal Principle for Accounting Balance of a Resource-Value Transfer Economy: Existence, Uniqueness, and Computations, Accademia Nazionale dei Lincei, Series 8, Vol. 56, pp. 556–561, 1974.
Charnes, A., andCooper, W. W.,Constrained Kullback-Leibler Estimation, Generalized Cobble-Douglas Balance, and Unconstrained Convex Programming, Accademia Nazionale dei Lincei, Series 8, Vol. 58, pp. 568–576, 1975.
Shannon, C. E.,A Mathematical Theory of Communication, Bell Systems Technical Journal, Vol. 27, pp. 379–423, 1948 and Vol. 27, pp. 623–656, 1948.
Ben-Tal, A., andCharnes, A.,A Dual Optimization Framework for Some Problems of Information Theory and Statistics, Problems of Control and Information Theory, Vol. 8, pp. 387–401, 1979.
Gallager, R. G.,Information Theory and Reliable Communication, John Wiley and Sons, New York, New York, 1968.
Jelinek, F.,Probabilistic Information Theory, McGraw-Hill, New York, New York, 1968.
Gallager, R. G.,A Simple Derivation of the Coding Theorem and Some Applications, IEEE Transactions on Information Theory, Vol. IT-11, pp. 3–18, 1965.
Csiszar, I., andKorner, J.,Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press, New York, New York, 1981.
Blahut, R. E.,On Hypothesis Testing and Information Theory, IEEE Transactions on Information Theory, Vol. IT-20, pp. 405–417, 1976.
Rockafeller, R. T.,Duality and Stability in Extremum Problems Involving Convex Functions, Pacific Journal of Mathematics, Vol. 21, pp. 167–186, 1976.
Rockafeller, R. T.,Conjugate Duality and Optimization, SIAM Regional Conference Series in Applied Mathematics, Vol. 16, 1974.
Laurent, P. J.,Optimization et Approximation, Hermann, Paris, Paris, France, 1972.
Ponstein, J.,Approaches to the Theory of Optimization, Cambridge University Press, Cambridge, England, 1980.
Rockafeller, R. T.,Minimax Theorems and Conjugate Saddle Functions, Mathematica Scandinava, Vol. 14, pp. 151–173, 1964.
Arimoto, S.,Computation of Random Coding Exponent Functions, IEEE Transactions on Information Theory, Vol. IT-22, pp. 665–671, 1976.
Author information
Authors and Affiliations
Additional information
Communicated by M. Avriel
This research was supported by ONR Contracts N00014-81-C-0236 and N00014-82-K-0295 with the Center for Cybernetics Studies, University of Texas, Austin, Texas. The first author was partly supported by NSF.
Rights and permissions
About this article
Cite this article
Ben-Tal, A., Teboulle, M. & Charnes, A. The role of duality in optimization problems involving entropy functionals with applications to information theory. J Optim Theory Appl 58, 209–223 (1988). https://doi.org/10.1007/BF00939682
Issue Date:
DOI: https://doi.org/10.1007/BF00939682