Abstract
In this chapter, I show that priors over network parameters can be defined in such a way that the corresponding priors over functions computed by the network reach reasonable limits as the number of hidden units goes to infinity. When using such priors,there is thus no need to limit the size of the network in order to avoid “overfitting”. The infinite network limit also provides insight into the properties of different priors. A Gaussian prior for hidden-to-output weights results in a Gaussian process prior for functions,which may be smooth, Brownian, or fractional Brownian. Quite different effects can be obtained using priors based on non-Gaussian stable distributions. In networks with more than one hidden layer, a combination of Gaussian and non-Gaussian priors appears most interesting.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1996 Springer Science+Business Media New York
About this chapter
Cite this chapter
Neal, R.M. (1996). Priors for Infinite Networks. In: Bayesian Learning for Neural Networks. Lecture Notes in Statistics, vol 118. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-0745-0_2
Download citation
DOI: https://doi.org/10.1007/978-1-4612-0745-0_2
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-94724-2
Online ISBN: 978-1-4612-0745-0
eBook Packages: Springer Book Archive