Abstract
Probabilistic neural networks (PNN), introduced by Specht [1, 2] have their predecessors in the theory of statistical pattern classification. In the fifties and sixties, problems of statistical pattern classification in the stationary case were accomplished by means of parametric methods, using the available apparatus of statistical mathematics (e.g. [3,4,5,6,7]). The knowledge of the probability density to an accuracy of unknown parameters was assumed and the parameters were estimated based on the learning sequence. Having observed tendencies present in the literature within the next decades, we should say that these methods have been almost completely replaced by the non-parametric approach (see e.g. [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24]). In the non-parametric approach it is assumed that a functional form of probability densities is unknown.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Specht, D.: Probabilistic neural networks and the polynomial Adaline as complementary techniques for classification. IEEE Trans. Neural Netw. 1, 111–121 (1990)
Specht, D.F.: A general regression neural network. IEEE Trans. Neural Netw. 2(6), 568–576 (1991)
Bishop, C.: Neural Networks for Pattern Recognition. Clarendon Press, Oxford (1995)
Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley, London (2001)
Fu, K.: Sequential Methods in Pattern Recognition and Machine Learning. Academic, New York (1968)
Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic, New York (1990)
Webb, A.: Statistical Pattern Recognition. Wiley, Chichester (2002)
Devroye, L., Györfi, L.: Nonparametric Density Estimation: The \(L_1\) View. Wiley, New York (1985)
Devroye, L., Györfi, L., Lugosi, G.: Probabilistic Theory of Pattern Recognition. Springer, New York (1996)
Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer, New York (2001)
Efromovich, S.: Nonparametric Curve Estimation. Methods, Theory and Applications. Springer, New York (1999)
Eubank, R.L.: Spline Smoothing and Nonparametric Regression. Marcel Dekker, INC., New York (1988)
Eubank, R.: Nonparametric Regression and Spline Smoothing. Marcel Dekker, New York (1999)
Györfi, L., Hżrdle, W., Sarda, P., Vieu, P.: Nonparametric Curve Estimation from Time Series. Springer, New York (1989)
Györfi, L., Kohler, M., Krzyżak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer, New York (2002)
Härdle, W.: Applied Nonparametric Regression. Cambridge University Press, Cambridge (1990)
Härdle, W., Kerkyacharian, G., Picard, D., Tsybakov, A.: Wavelets, Approximation, and Statistical Applications. Springer, New York (1998)
Ibragimov, I., Khasminskii, R.: Statistical Estimation: Asymptotic Theory. Springer, New York (1981)
Pagan, A., Ullah, A.: Nonparametric Econometrics. Cambridge University Press, London (1999)
Rafajłowicz, E.: Consistency of orthogonal series density estimators based on grouped observations. IEEE Trans. Inf. Theory 43(1), 283–285 (1997)
Rao, B.L.S.P.: Nonparametric Functional Estimatio. Academic, New York (1983)
Thompson, J., Tapia, R.: Nonparametric Function Estimation and Simulation. SIAM, Philadelphia (1990)
Wertz, W.: Statistical Density Estimation: a Survey. Vandenhoeck & Ruprecht, Göttingen (1978)
Wertz, W., Schneider, B.: Statistical density estimation: a bibliography. Int. Stat. Rev. 47, 155–175 (1979)
Rosenblatt, M.: Remarks on some estimates of a density function. Ann. Math. Stat. 27, 155–175 (1956)
Parzen, E.: On estimation of probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)
Cacoullos, T.: Estimation of a multivariate density. Ann. Inst. Stat. Math. 18, 179–189 (1965)
Čencov, N.: Evaluation of an unknown distribution density from observations. Sov. Math. 3, 1559–1562 (1962)
Schwartz, S.: Estimation of probability density by an orthogonal series. Ann. Math. Stat. 1261–1265 (1967)
Kronmal, R., Tarter, M.: The estimation of probability densities and cumulatives by Fourier series methods. J. Am. Stat. Assoc. (1968)
Walter, G.: Properties of Hermite series estimation of probability density. Ann. Stat. 5, 1258–1264 (1977)
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13, 21–27 (1967)
Loftsgaarden, D., Quesenberry, C.: A nonparametric estimate of a multivariate density function. Ann. Math. Stat. 36, 1049–1051 (1965)
Devroye, L.: Universal consistency in nonparametric regression and nonparametric discrimination. Technical report. School of Computer Science, Mc Gill University (1978)
Greblicki, W.: Asymptotically optimal pattern recognition procedures with density estimate. IEEE Trans. Inf. Theory 24, 250–251 (1978)
Greblicki, W., Rutkowski, L.: Density-free Bayes risk consistency of nonparametric pattern recognition procedures. Proc. IEEE 69(4), 482–483 (1981)
Rutkowski, L.: Sequential estimates of probability densities by orthogonal series and their application in pattern classification. IEEE Trans. Syst. Man Cybern. SMC-10(12), 918–920 (1980)
Rutkowski, L.: Sequential estimates of a regression function by orthogonal series with applications in discrimination. Lectures Notes in Statistics, vol. 8, pp. 236–244. Springer, New York (1981)
Rutkowski, L.: Sequential pattern recognition procedures derived from multiple Fourier series. Pattern Recognit. Lett. 8, 213–216 (1988)
Ryzin, J.: Bayes risk consistency of classification procedures using density estimation. Sankhya Ser. A (1966)
Wolverton, C., Wagner, T.: Asymptotically optimal discriminant functions for pattern classification. IEEE Trans. Inf. Theory 15, 258–265 (1969)
Kramer, C., Mckay, B., Belina, J.: Probabilistic neural network array architecture for ECG classification. In: Proceedings of the Annual International Conference on IEEE Engineering in Medicine and Biology Society, vol. 17, pp. 807–808 (1995)
Musavi, M., Chan, K., Hummels, D., Kalantri, K.: On the generalization ability of neural-network classifier. IEEE Trans. Pattern Anal. Mach. Intell. 16, 659–663 (1994)
Raghu, P., Yegnanarayana, B.: Supervised texture classification using a probabilistic neural network and constraint satisfaction model. IEEE Trans. Neural Netw. 9, 516–522 (1998)
Romero, R., Touretzky, D., Thibadeau, G.: Optical Chinese character recognition using probabilistic neural networks. Pattern Recognit. 3, 1279–1292 (1997)
Streit, R.L., Luginbuhl, T.: Maximum likelihood training of probabilistic neural networks. IEEE Trans. Neural Netw. 5, 764–783 (1994)
Burrascano, P.: Learning vector quantization for the probabilistic neural network. IEEE Trans. Neural Netw. 2, 458–461 (1991)
Zaknich, A.: A vector quantization reduction method for the probabilistic neural network. In: Proceedings of the IEEE International Conference on Neural Networks: Piscataway, NJ, pp. 1117–1120 (1997)
Specht, D.: Enhancements to the probabilistic neural networks. In: Proceedings of the IEEE International Joint Conference on Neural Networks: Baltimore, MD, pp. 761–768 (1992)
Mao, K., Tan, K.C., Ser, W.: Probabilistic neural-network structure determination for pattern classification. IEEE Trans. Neural Netw. 11(4), 501–507 (2000)
Jones, M., Marron, J., Sheather, S.: A brief survey of bandwidth selection for density estimation. J. Am. Stat. Assoc. 91, 401–407 (1996)
Földes, A., Révész, P.: A general method for density estimation. Stud. Sci. Math. Hung. 9, 81–92 (1974)
Nikolsky, S.: A Course of Mathematical Analysis. Mir Publishers, Moscow (1977)
Szegö, G.: Orthogonal Polynomials, vol. 23. American Mathematical Society, Colloquium Publications (1959)
Sansone, G.: Orthogonal Functions. Interscience Publishers Inc., New York (1959)
Alexits, G.: Convergence Problems of Orthogonal Series. Akademiai Kiado, Hungary, Budapest (1961)
Zygmund, A.: Trigonometric Series. Cambridge University Press, Cambridge (1959)
Sjölin, P.: Convergence almost everywhere of certain singular integrals and multiple Fourier series. Ark. Math. 9, 65–90 (1971)
Yamato, H.: Sequential estimation of a continuous probability density function and the mode. Bull. Math. Stat. 14, 1–12 (1971)
Nadaraya, E.A.: On estimating regression. Theory Probab. Appl. 9(1), 141–142 (1964)
Watson, G.S.: Smooth regression analysis. Sankhyā: Indian J. Stat. Ser. A 359–372 (1964)
Devroye, L., Wagner, T.: On the convergence of kernel estimators of regression functions with applications in discrimination. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 51, 15–21 (1980)
Devroye, L.: On the almost everywhere convergence of nonparametric regression function estimates. Ann. Stat. 9, 1301–1309 (1981)
Devroye, L., Krzyżak, A.: An equivalence theorem for \(l_1\) convergence of the kernel regression estimate. J. Stat. Plan. Inference 23, 71–82 (1989)
Rutkowski, L.: On system identification by nonparametric function fitting. IEEE Trans. Autom. Control AC-27, 225–227 (1982)
Rutkowski, L., Rafajłowicz, E.: On global rate of convergence of some nonparametric identification procedures. IEEE Trans. Autom. Control AC-34(10), 1089–1091 (1989)
Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbance. IEEE Trans. Inf. Theory IT-37, 214–216 (1991)
Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Trans. Signal Process. (1993)
Gałkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proc. IEEE 73, 942–943 (1985)
Gałkowski, T., Rutkowski, L.: Nonparametric fitting of multivariable functions. IEEE Trans. Autom. Control AC-31, 785–787 (1986)
Devroye, L., Wagner, T.: Nonparametric discrimination and density estimation. Technical report 183, Electronic Research Center, University of Texas (1976)
Wahba, G.: Interpolating spline methods for density estimation, variable knots. Technical report 337, Department of Statistics, University of Wisconsin, Madison (1973)
Wahba, G.: Optimal convergence properties of variable knot, kernel, and orthogonal series methods for density estimatio. Ann. Stat. (1975)
Wahba, G.: Smoothing noisy data with spline function. Numer. Math. (1975)
Wahba, G.: Interpolating spline methods for density estimation, equi-spaced knot. Ann. Stat. (1975)
Wahba, G.: A survey of some smoothing problems and the method of generalized cross-validation for solving the, TR-457, Department of Statistics, University of Wisconsin, p. brak (1976)
Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)
Devroye, L.: Necessary and sufficient conditions for the almost everywhere convergence of nearest neighbor regression function estimates. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 61, 467–481 (1982)
Devroye, L., Györfi, L., Krzyżak, A., Lugosi, G.: On the strong universal consistency of nearest neighbor regression function estimates. Ann. Stat. 22, 1371–1385 (1994)
Aizerman, M., Braverman, E., Rozonoer, L.: Theoretical foundations of the potential function method in pattern recognition learning. Autom. Remote Control 25, 821–837 (1964)
Chen, S., Cowan, C., Grant, P.: Orthogonal least squares learning algorithm for radial basis network. IEEE Trans. Neural Netw. 2, 302–309 (1991)
Kecman, V.: Learning and Soft Computing. MIT, Cambridge (2001)
Chui, C.: Wavelets: a Tutorial in Theory and Applications. Academic, Boston (1992)
Meyer, Y.: Wavelets: Algorithms and Applications. SIAM, Philadelphia (1993)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Rutkowski, L., Jaworski, M., Duda, P. (2020). Basic Concepts of Probabilistic Neural Networks. In: Stream Data Mining: Algorithms and Their Probabilistic Properties. Studies in Big Data, vol 56. Springer, Cham. https://doi.org/10.1007/978-3-030-13962-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-13962-9_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-13961-2
Online ISBN: 978-3-030-13962-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)