Abstract
We extend the line of research initiated by Fortnow and Klivans [6] that studies the relationship between efficient learning algorithms and circuit lower bounds. In [6], it was shown that if a Boolean circuit class \(\mathcal{C}\) has an efficient deterministic exact learning algorithm, (i.e. an algorithm that uses membership and equivalence queries) then \(\mathsf{EXP}^{\mathsf{NP}} \not \subseteq \mathsf{P/poly}[\mathcal{C}]\). Recently, in [14] EXP NP was replaced by DTIME(n ω(1)). Yet for the models of randomized exact learning or Valiant’s PAC learning, the best result so far is a lower bound against BPEXP (the exponential-time analogue of BPP). In this paper, we derive stronger lower bounds as well as some other consequences from randomized exact learning and PAC learning algorithms, answering an open question posed in [6] and [14]. In particular, we show that
-
1
If a Boolean circuit class \(\mathcal{C}\) has an efficient randomized exact learning algorithm or an efficient PAC learning algorithm then \(\mathsf{BPTIME}(n^{\omega(1)})/1 \not \subseteq \mathsf{P/poly}[\mathcal{C}]\).
-
2
If a Boolean circuit class \(\mathcal{C}\) has an efficient randomized exact learning algorithm then no strong pseudo-random generators exist in \(\mathsf{P/poly}[\mathcal{C}]\).
We note that in both cases the learning algorithms need not be proper. The extra bit of advice comes to accommodate the need to keep the promise of bounded away probabilities of acceptance and rejection. The exact same problem arises when trying to prove lower bounds for BPTIME or MA [3,7,16,20]. It has been an open problem to remove this bit. We suggest an approach to settle this problem in our case. Finally, we slightly improve the result of [5] by showing a subclass of MAEXP that requires super-polynomial circuits. Our results combine and extend some of the techniques previously used in [6,14] and [20].
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)
Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)
Barak, B.: A probabilistic-time hierarchy theorem for slightly non-uniform algorithms. In: Rolim, J.D.P., Vadhan, S.P. (eds.) RANDOM 2002. LNCS, vol. 2483, pp. 194–208. Springer, Heidelberg (2002)
Buhrman, H., Fortnow, L.: One-sided versus two-sided error in probabilistic computation. In: Meinel, C., Tison, S. (eds.) STACS 1999. LNCS, vol. 1563, pp. 100–109. Springer, Heidelberg (1999)
Buhrman, H., Fortnow, L., Thierauf, T.: Nonrelativizing separations. In: Proceedings of the 13th Annual IEEE Conference on Computational Complexity (CCC), pp. 8–12 (1998)
Fortnow, L., Klivans, A.R.: Efficient learning algorithms yield circuit lower bounds. J. Comput. Syst. Sci. 75(1), 27–36 (2009)
Fortnow, L., Santhanam, R.: Hierarchy theorems for probabilistic polynomial time. In: FOCS, pp. 316–324 (2004)
Gentry, C., Halevi, S.: Fully homomorphic encryption without squashing using depth-3 arithmetic circuits. In: Proceedings of the 52nd Annual FOCS, pp. 107–109 (2011)
Goldreich, O., Zuckerman, D.: Another proof that bpp ⊆ ph (and more). In: Studies in Complexity and Cryptography, pp. 40–53 (2011)
Harkins, R.C., Hitchcock, J.M.: Exact learning algorithms, betting games, and circuit lower bounds. In: Aceto, L., Henzinger, M., Sgall, J. (eds.) ICALP 2011, Part I. LNCS, vol. 6755, pp. 416–423. Springer, Heidelberg (2011)
Impagliazzo, R., Wigderson, A.: Randomness vs. time: De-randomization under a uniform assumption. In: FOCS, pp. 734–743 (1998)
Kabanets, V., Impagliazzo, R.: Derandomizing polynomial identity tests means proving circuit lower bounds. Computational Complexity 13(1-2), 1–46 (2004)
Kearns, M.J., Valiant, L.G.: Cryptographic limitations on learning boolean formulae and finite automata. J. ACM 41(1), 67–95 (1994)
Klivans, A., Kothari, P., Oliveira, I.: Constructing hard functions from learning algorithms. In: Proceedings of the 28th Annual IEEE Conference on Computational Complexity (CCC), pp. 86–97 (2013)
Klivans, A.R., Sherstov, A.A.: Cryptographic hardness for learning intersections of halfspaces. J. Comput. Syst. Sci. 75(1), 2–12 (2009)
van Melkebeek, D., Pervyshev, K.: A generic time hierarchy with one bit of advice. Computational Complexity 16(2), 139–179 (2007)
Miltersen, P.B., Vinodchandran, N.V., Watanabe, O.: Super-polynomial versus half-exponential circuit size in the exponential hierarchy. In: Asano, T., Imai, H., Lee, D.T., Nakano, S.-i., Tokuyama, T. (eds.) COCOON 1999. LNCS, vol. 1627, pp. 210–220. Springer, Heidelberg (1999)
Naor, M., Reingold, O.: Number-theoretic constructions of efficient pseudo-random functions. J. ACM 51(2), 231–262 (2004)
Razboeov, A.A., Rudich, S.: Natural proofs. J. of Computer and System Sciences 55(1), 24–35 (1997)
Santhanam, R.: Circuit lower bounds for merlin–arthur classes. SIAM J. Comput. 39(3), 1038–1061 (2009)
Shamir, A.: IP=PSPACE. In: Proceedings of the Thirty First Annual Symposium on Foundations of Computer Science, pp. 11–15 (1990)
Trevisan, L., Vadhan, S.P.: Pseudorandomness and average-case complexity via uniform reductions. Computational Complexity 16(4), 331–364 (2007)
Valiant, L.G.: A theory of the learnable. Communications of the ACM 27(11), 1134–1142 (1984)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Volkovich, I. (2014). On Learning, Lower Bounds and (un)Keeping Promises. In: Esparza, J., Fraigniaud, P., Husfeldt, T., Koutsoupias, E. (eds) Automata, Languages, and Programming. ICALP 2014. Lecture Notes in Computer Science, vol 8572. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43948-7_85
Download citation
DOI: https://doi.org/10.1007/978-3-662-43948-7_85
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43947-0
Online ISBN: 978-3-662-43948-7
eBook Packages: Computer ScienceComputer Science (R0)