Abstract
Example of dual control of linear uncertain system have been presented. The control task with short horizon (N=2) were solved using dynamic programming. It was shown that the optimal solution is ambiguous, the cost function is non-convex and has many local minima. Optimal control depends in a discontinuous manner on the initial conditions. It was also observed that active learning occurs only when the uncertainty of the initial state exceeds a certain threshold. In this case, the amount of information transmitted from sensor to the controller is much greater than in the case of passive learning.
Access provided by CONRICYT-eBooks. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
[1] Alpcan T., Shames I., (2015): An Information-Based Learning Approach to Dual Control . IEEE Trans. on Neural Networks and Learning Systems Vol.26, Issue 11 Pages: 2736–2748.
[2] Alpcan T.; Shames I; Cantoni M.; Nair G. (2013): Learning and information for dual control 2013 9th Asian Control Conference (ASCC), Pages: 1–6
[3] Åström K, Wittenmark B. (1995): Adaptive Control. Addison–Wesley, 2nd edition.
[4] Åström K., Helmersson A. (1986): Dual control of an integrator with unknown gain. Comp. & Maths. With Appls.,12A:6, pp. 653–662.
[5] Åström K., Wittenmark B. (1971): Problems of identification and control. Journal of Mathematical Analysis and Applications, 34, pp. 90–113.
[6] Banek T. (2010): Incremental value of information for discrete–time partially observed stochastic systems Control and Cybernetics vol. 39, No. 3.
[7] Banek T., Kozłowski E. (2005): Active and passive learning in control processes application of the entropy concept, Systems Sciences, vol. 31, N.2, pp. 29–44.
[8] Banek T., Kozłowski E. (2006): Adaptive control of system entropy Contr. and Cybernetics 35, 2.
[9] Banek T., Kozłowski E. (2010): Active learning in discrete–time stochastic systems In: Jerzy Jozefczyk and Donat Orski (ed.), Knowledge-based Intelligent System Advancements: Systemic and Cybernetic Approaches, pp.350–371.
[10] Bania P., Baranowski J. (2016): Field Kalman Filter and its approximation. In proc of 55th IEEE Conference on Decision and Control December 12–14, Las Vegas, USA.
[11] Bar-Shalom Y., Tse E. (1976): Caution, probing, and the value of information in the control of uncertain systems. Ann. Econ. Social. Measurement. Vol 5. pp. 323–337.
[12] Bar-Shalom Y. (1981): Stochastic Dynamic Programming: Caution and Probing. IEEE trans Aut. Contr. Vol AC-26, No 5 pp. 1184–1195.
[13] Bensoussan, A. (1983), Maximum Principle and Dynamic Programming Approach of the Optimal control of Partially Observed Diffusions, Stochastics, vol.9, issue 3, 169–222.
[14] Bernhardsson B. (1989): Dual control of a first-order system with two possible gains. Int. J. of Adaptive Control and Signal Processing, 3, pp. 15–22.
[15] Bohlin T. (1969): Optimal dual control of a simple process with unknown gain. Report PT 18.196, IBM Nordic Laboratory, Lidingö, Sweden.
[16] Cao S., Qian F., Wang X. (2016): Exact optimal solution for a class of dual control problems. International Journal of Systems Science, 47:9, 2078–2087.
[17] Casiello F., Loparo K. A. (1989): Optimal control of unknown parameter systems. IEEE Trans. Aut. Contr. AC-34, pp.1092–1094.
[18] Chen R., Loparo K. A. (1991): Dual control of linear stochastic systems with unknown parameters. IEEE International Conference on Systems Engineering pp. 65–68.
[19] Chen, R., (1990): Dual Control of Linear Stochastic Systems with Unknown Parameters. Ph.D. Dissertation, Systems Engineering, Case Western Reserve University.
[20] Fang S., Chen J., Hideaki I., (2017): Towards integrating control and information theories. Lecture notes in control and information sciences 465. Springer.
[21] Feldbaum A.A. (1965): Optimal control systems. Elsevier Science.
[22] Feldbaum, A.A. (1960): Dual Control Theory I–II, J. Aut. Remote Cont., 21, 874–880, 1033–1039.
[23] Feldbaum, A.A. (1961), Dual Control Theory III–IV, J. Aut. Remote Cont., 22, 1–12, 109–121.
[24] Feng X., Loparo K., (1997): Optimal State Estimation for Stochastic Systems: An Information Theoretic Approach IEEE Transactions on Automatic Control, Vol. 42, No. 6.
[25] Filatov N. M, Unbehauen H. (2000): Survey of adaptive dual control methods. IEE Proceedings - Control Theory and Applications 147(1):118 – 128.
[26] Filatov, N.M., Unbehauen, H. (2004): Adaptive Dual Control: Theory and Applications, Lecture Notes in Control and Information Sciences No. 302.
[27] Fleming, W.H., Pardoux, H. (1982): Optimal Control of Partially Observed Diffusion, SIAM Journal on Control and Optimization, 20, 261–285.
[28] Heirung T. A., Ydstie B., Foss B. (2017): Dual Adaptive Model Predictive Control. Automatica, Feb. 2017, to appear.
[29] Hijab O. (1984): Entropy and dual control Proc. of 23rd C DC, Las Vegas NV.
[30] Hordjewicz T., Kozłowski E. (2006): Comparison of stochastic optimal controls with different level of self-learning Annales UMCS Informatica AI 5 (2006) 343–356.
[31] Hordjewicz T., Kozłowski E. (2007): The self-learning active problem in dynamic systems Annales UMCS Informatica AI 7 (2007) 171–180
[32] Kumar K, Heirung T. A,. N., Patwardhan S. C., Foss B. (2015): Experimental Evaluation of a MIMO Adaptive Dual MPC. 9th IFAC Symposium on Advanced Control of Chemical Processes ADCHEM 2015 Whistler, Canada, 7–10 June 7 – 10, 2015, IFAC-PapersOnLine Volume 48, Issue 8, 545–550 .
[33] Li D.; Qian F.; Fu P. (2008): Optimal nominal dual control for discrete-time linear-quadratic Gaussian problems with unknown parameters Automatica 44, 119–127
[34] Li D; Fu P.; Qian F. (2003): Optimal nominal dual control for discrete-time LQG problem with unknown parameters, SICE Annual Conference, http://ieeexplore.ieee.org/document/1323396/
[35] Lindoff B., Holst J., Wittenmark B. (1999): Analysis of approximations of dual control. Int. J. Adapt. Control Signal Process. 13, 593–620.
[36] Potschka H. C, Schloder J. P., Bock H. G. (2016): Dual Control and Information Gain in Controlling Uncertain Processes. 11th IFAC Symposium on Dynamics and Control of Process Systems, including Biosystems June 6-8, 2016. NTNU, Trondheim, Norway..
[37] Rishel, R. (1986): An Exact Formula for a Linear Quadratic Adaptive Stochastic Optimal Control Law, SIAM Journal on Control and Optimization, 24, 667–674.
[38] Rishel, R. (1990), A Comment on a Dual Control Problem in Proc. of 19th IEEE Conference on Decision and Control Including the Symposium on Adaptive Processes, pp. 337–340.
[39] Sagawa T., Ueda M. (2013): Role of mutual information in entropy production under information exchanges. New J. Phys. 15 125012,
[40] Saridis G. N. (1988): Entropy formulation of optimal and adaptive control. IEEE Trans. Aut. Cont. Vol: 33, Issue: 8.
[41] Saridis G. N. (2001): Entropy in control engineering. Series in Inteligent Control and Intelligent Automation. World Scientific Publishing.
[42] Scardovi L. (2005): Information based control for state and parameter estimation. PhD thesis. University of Genoa Dep. of Communication, Faculty of Engineering Computer and System Sciences.
[43] Sternby J. (1976): A simple dual control problem with an analytical solution. IEEE Trans.Aut. Cont., 21(6):840–844.
[44] Tenno R. (2010), Dual adaptive controls for linear system with unknown constant parameters, International Journal of Control, 83:11, 2232–2240
[45] Touchette H., Lloyd S. (2000): Information-theoretic limits of control. Phys Rev Lett. 2000 Feb 7;84(6):1156–9.
[46] Touchette H., Lloyd S. (2004): Information-theoretic approach to the study of control systems. Phys. A 331, 140–172.
[47] Tsai Y. A., Casiello F. A., Loparo K. A. (1992): Discrete-time entropy formulation of optimal and adaptive control problems. IEEE Trans. Aut. Cont. vol 37, No. 7.
[48] Tse E. (1974): Adaptive Dual Control Methods. Annals of Economic and Social Measurement, Vol. 3. No. 1. (1974)
[49] Tse, E., and Bar-Shalom, Y. (1973), An Actively Adaptive Control for Linear Systems with Random Parameters via the Dual Control Approach, IEEE Trans. Aut. Control, 18, 109–117.
[50] Tse, E., and Bar-Shalom, Y. (1976), Actively Adaptive Control for Nonlinear Stochastic Systems, Proceedings of the IEEE, 64, 1172–1181.
[51] Tse, E., Bar-Shalom, Y., Meier L. (1973a): Wide sense adaptive dual control for nonlinear stochastic systems. IEEE Trans. Aut. Control, 18, 98–108.
[52] Uciński D., Patan M. (2016): D-optimal spatio-temporal sampling design for identification of distributed parameter systems In proc of 55th IEEE Conference on Decision and Control December 12–14, Las Vegas, USA, 3985–3990.
[53] Wittenmark B. (1995): Adaptive dual control methods; an overview. Proc. Of 5th IFAC symposium of adaptive systems in control and signal processing. Budapest, pp. 67–72.
[54] Zabczyk J. (1996): Chance and decision. Stochastic control in discrete time. Quaderni Scuola Normale di Pisa
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bania, P. (2017). Simple example of dual control problem with almost analytical solution. In: Mitkowski, W., Kacprzyk, J., Oprzędkiewicz, K., Skruch, P. (eds) Trends in Advanced Intelligent Control, Optimization and Automation. KKA 2017. Advances in Intelligent Systems and Computing, vol 577. Springer, Cham. https://doi.org/10.1007/978-3-319-60699-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-60699-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60698-9
Online ISBN: 978-3-319-60699-6
eBook Packages: EngineeringEngineering (R0)