Abstract
Signals occurring in applications like medical imaging and telecommunications are inherently complex-valued, and processing them in their natural form preserves the physical characteristics of these signals. Therefore, there is a widespread research interest in developing efficient complex-valued neural networks along with their learning algorithms. However, operating in the Complex domain presents new challenges; foremost among them being the choice of an appropriate complex-valued activation function. Basically, an activation function for a neural network is required to be nonlinear, bounded and differentiable in every point on the considered plane [1]. This implies that in the Complex domain, the function has to be nonlinear, bounded and entire. However, Liouville’s theorem states that an entire and bounded function in the Complex domain is a constant (function) [2]. As neither the analyticity and boundedness can be compromised, nor is a constant function acceptable as an activation function as it cannot project the input space to a non-linear higher dimensional space, choices for activation functions for complex-valued neural network are limited. In this chapter, the different complex-valued neural networks existing in the literature are discussed in detail, along with their limitations.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Keywords
- Independent Component Analysis
- Extreme Learn Machine
- Recurrent Neural Network
- Independent Component Analysis
- Radial Basis Function Network
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall, New Jersey (1998)
Remmert, R.: Theory of Complex Functions. Springer, New York (1991)
Leung, H., Haykin, S.: The complex backpropagation algorithm. IEEE Transactions on Signal Processing 39(9), 2101–2104 (1991)
Kim, T., Adali, T.: Fully complex multi-layer perceptron network for nonlinear signal processing. Journal of VLSI Signal Processing 32(1/2), 29–43 (2002)
Yang, S.-S., Ho, C.-L., Siu, S.: Sensitivity analysis of the split-complex valued multilayer perceptron due to the errors of the i.i.d. inputs and weights. IEEE Transactions on Neural Networks 18(5), 1280–1293 (2007)
Zhang, H., Zhang, C., Wu, W.: Convergence of batch split-complex backpropagation algorithm for complex-valued neural networks. Discrete Dynamics in Nature and Society 16, Article ID 329173 (2009), Online Journal, http://www.hindawi.com/journals/ddns/2009/329173.html
Savitha, R., Suresh, S., Sundararajan, N., Saratchandran, P.: A new learning algorithm with logarithmic performance index for complex-valued neural networks. Neurocomputing 72(16-18), 3771–3781 (2009)
Jianping, D., Sundararajan, N., Saratchandran, P.: Complex-valued minimal resource allocation network for nonlinear signal processing. International Journal of Neural Systems 10(2), 95–106 (2000)
Deng, J.P., Sundararajan, N., Saratchandran, P.: Communication channel equalization using complex-valued minimal radial basis function neural networks. IEEE Transactions on Neural Networks 13(3), 687–696 (2002)
Benvenuto, N., Piazza, F.: On the complex backpropagation algorithm. IEEE Transactions on Signal Processing 40(4), 967–969 (1992)
Brandwood, D.H.: A complex gradient operator and its application in adaptive array theory. IEE Proceedings 130, 11–16 (1983)
Georgiou, G.M., Koutsougeras, C.: Complex domain backpropagation. IEEE Transactions on Circuits and Systems-II: Analog and Digital Signal Processing 39(5), 330–334 (1992)
Kim, T., Adali, T.: Approximation by fully complex multi-layer perceptrons. Neural Computation 15(7), 1641–1666 (2003)
You, C., Hong, D.: Nonlinear blind equalization schemes using complex-valued multilayer feedforward neural networks. IEEE Transactions on Neural Networks 9(6), 1442–1455 (1998)
Savitha, R., Suresh, S., Sundararajan, N., Saratchandran, P.: Complex-valued function approximation using an improved BP learning algorithm for feed-forward networks. In: IEEE International Joint Conference on Neural Networks (IJCNN 2008), June 1-8, pp. 2251–2258 (2008)
Li, M.B., Huang, G.-B., Saratchandran, P., Sundararajan, N.: Fully complex extreme learning machine. Neurocomputing 68(1-4), 306–314 (2005)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE International Joint Conference on Neural Networks (IJCNN 2004), 25-29, vol. 2, pp. 985–990 (2004)
Huang, G.B., Siew, C.K.: Extreme learning machine with randomly assigned RBF kernels. Int. J. Inf. Technol. 11(1) (2005)
Kantsila, A., Lehtokangas, M., Saarinen, J.: Complex RPROP-algorithm for neural network equalization of GSM data bursts. Neurocomputing 61, 339–360 (2004)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 1-3, pp. 586–591 (1993)
Chen, S., McLaughlin, S., Mulgrew, B.: Complex valued radial basis function network,part I: Network architecture and learning algorithms. EURASIP Signal Processing Journal 35(1), 19–31 (1994)
Chen, S., McLaughlin, S., Mulgrew, B.: Complex valued radial basis function network, part II: Application to digital communications channel equalization. Signal Processing 36(2), 175–188 (1994)
Li, M.B., Huang, G.B., Saratchandran, P., Sundararajan, N.: Complex-valued growing and pruning RBF neural networks for communication channel equalisation. IEE Proceedings- Vision, Image and Signal Processing 153(4), 411–418 (2006)
Yingwei, L., Sundararajan, N., Saratchandran, P.: A sequential learning scheme for function approximation using minimal radial basis function neural networks. Neural Computation 9(2), 461–478 (1997)
Huang, G.B., Saratchandran, P., Sundararajan, N.: A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Transactions on Neural Networks 16(1), 57–67 (2005)
Wang, J.: Recurrent neural networks for solving systems of complex-valued linear equations. Electronics Letters 28(18), 1751–1753 (1992)
Mandic, D., Chambers, J.: Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. John Wiley and Sons, West Sussex (2001)
Li, C., Liao, X., Yu, J.: Complex-valued recurrent neural network with IIR neuron model: training and applications. Circuits Systems Signal Processing 21(5), 461–471 (2002)
Goh, S.L., Mandic, D.P.: An augmented extended kalman filter algorithm for complex-valued recurrent neural networks. Neural Computation 19(4), 1039–1055 (2007)
Mandic, D.P.: Complex valued recurrent neural networks for noncircular complex signals. In: International Joint Conference on Neural Networks (IJCNN 2009), June 14-19, pp. 1987–1992 (2009)
Zhou, W., Zurada, J.M.: Discrete-time recurrent neural networks with complex-valued linear threshold neurons. IEEE Transactions on Circuits and Systems 56(8), 669–673 (2009)
Mandic, D.P., Javidi, S., Goh, S.L., Kuh, A., Aihara, K.: Complex-valued prediction of wind profile using augmented complex statistics. Renewable Energy 34(1), 196–201 (2009)
Gangal, A.S., Kalra, P.K., Chauhan, D.S.: Performance evaluation of complex valued neural networks using various error functions. Proceedings of the World Academy of Science, Engineering and Technology 23, 27–32 (2007)
Chen, X.M., Tang, Z., Variappan, C., Li, S.S., Okada, T.: A modified error backpropagation algorithm for complex-valued neural networks. International Journal of Neural Systems 15(6), 435–443 (2005)
Rattan, S.S.P., Hsieh, W.W.: Complex-valued neural networks for nonlinear complex principal component analysis. Neural Networks 18(1), 61–69 (2005)
Fiori, S.: Nonlinear complex-valued extensions of Hebbian learning: an essay. Neural Computation 17(4), 779–838 (2005)
Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley and Sons, New York (2001)
Hyvarinen, A., Oja, E.: Independent component analysis: Algorithms and applications. Neural Networks 13(4-5), 411–430 (2000)
Lv, Q., Zhang, X., Jia, Y.: Blind Separation Combined Frequency Invariant Beamforming and ICA for Far-field Broadband Acoustic Signals. In: Wang, J., Liao, X.-F., Yi, Z. (eds.) ISNN 2005. LNCS, vol. 3497, pp. 538–543. Springer, Heidelberg (2005)
Chang, A.-C., Jen, C.-W.: Complex-valued ICA utilizing signal-subspace demixing for robust DOA estimation and blind signal separation. Wireless Personal Communications 43(4), 1435–1450 (2007)
Lee, I., Kim, T., Lee, T.-W.: Fast fixed-point independent vector analysis algorithms for convolutive blind source separation. Signal Processing 87(8), 1859–1871 (2007)
He, Z., Xie, S., Ding, S., Cichocki, A.: Convolutive blind source separation in the frequency domain based on sparse representation. IEEE Transactions on Audio, Speech, and Language Processing 15(5), 1551–1563 (2007)
Jen, C.-W., Chen, S.-W., Chang, A.-C.: High-resolution DOA estimation based on independent noise component for correlated signal sources. Neural Computing and Applications 18(4), 381–385 (2008)
Calhoun, V.D., Adali, T., Pearlson, G.D., van Zijl, P.C., Pekar, J.J.: Independent component analysis of fMRI data in the complex domain. Magnetic Resonance in Medicine 48(1), 180–192 (2002)
Calhoun, V., Adali, T.: Complex infomax: Convergence and approximation of infomax with complex nonlinearities. The Journal of VLSI Signal Processing 44(1-2), 173–190 (2006)
Adali, T., Calhoun, V.D.: Complex ICA of brain imaging data. IEEE Signal Processing Magazine 24(5), 136–139 (2007)
Bingham, E., Hyvarinen, A.: Ica of complex valued signals: A fast and robust deflationary algorithm. In: International Joint Conference on Neural Networks (IJCNN 2000), vol. 3 (2000)
Bingham, E., Hyvarinen, A.: A fast fixed-point algorithm for independent component analysis of complex valued signals. International Journal of Neural Systems 10(1), 1–8 (2000)
Fiori, S.: Neural independent component analysis by maximum-mismatch learning principle. Neural Networks 16(8), 1201–1221 (2003)
Yang, T., Mikhael, W.B.: A general approach for image and co-channel interference suppression in diversity wireless receivers employing ICA. Circuits, Systems, and Signal Processing 23(4), 317–327 (2004)
Erikkson, J., Koivunen, V.: Complex random vectors and ICA models: Identifiability, uniqueness, and separability. IEEE Transactions on Information Theory 52(3), 1017–1029 (2006)
Sallberg, B., Grbic, N., Claesson, I.: Complex-valued independent component analysis for online blind speech extraction. IEEE Transactions on Audio, Speech, and Language Processing 16(8), 1624–1632 (2008)
Li, H., Adali, T.: A class of complex ICA algorithms based on the kurtosis cost function. IEEE Transactions on Neural Networks 19(3), 408–420 (2008)
Novey, M., Adali, T.: Complex ICA by negentropy maximization. IEEE Transactions on Neural Networks 19(4), 596–609 (2008)
Li, X.-L., Adali, T.: Complex independent component analysis by entropy bound minimization. IEEE Transactions on Circuits and Systems I 57(7), 1417–1430 (2010)
Ollilaa, E., Koivunen, V.: Complex ICA using generalized uncorrelating transform. Signal Processing 89(4), 365–377 (2009)
Novey, M., Adali, T.: On extending the complex fast ICA algorithm to noncircular sources. IEEE Transactions on Signal Processing 56(5), 2148–2154 (2008)
Brown, J., Churchill, R.: Complex Variables and Applications. McGrawHill, New York (1996)
Flanigan, F.: Complex Variables: Harmonic and Analytic Functions. Dover Publications, New York (1983)
Le Page, W.: Complex Variables and the Laplace Transforms for Engineers. Dover Publications, New York (1980)
Fisher, S.: Complex Variables, 2nd edn. Dover Publications, New York (1999)
Wirtinger, W.: Zur formalen theorie der funktionen von mehr komplexen vernderlichen. Annals of Mathematics 97 (1927)
Hjorungnes, A., Gesbert, D.: Complex-valued matrix differentiation: Techniques and key results. IEEE Transactions on Signal Processing 55(6), 2740–2746 (2007)
Adali, T., Li, H., Novey, M., Cardoso, J.-F.: Complex ICA using nonlinear functions. IEEE Transactions on Signal Processing 56(9), 4536–4544 (2008)
Loss, D.V., de Castro, M.C.F., Franco, P.R.G., de Castro, E.C.C.: Phase transmittance RBF neural networks. Electronics Letters 43(16), 882–884 (2007)
Uncini, A., Vecci, L., Campolucci, P., Piazza, F.: Complex-valued neural networks with adaptive spline activation function for digital radio links nonlinear equalization. IEEE Transactions on Signal Processing 47(2), 505–514 (1999)
Yang, S.S., Siu, S., Ho, C.L.: Analysis of the initial values in split-complex backpropagation algorithm. IEEE Transactions on Neural Networks 19(9), 1564–1573 (2008)
Nitta, T.: An extension of the back-propagation algorithm to complex numbers. Neural Networks 10(8), 1391–1415 (1997)
Hirose, A.: Continuous complex-valued back-propagation learning. Electronic Letters 28(20), 1854–1855 (1992)
Kim, M.S., Guest, C.C.: Modification of back propagation networks for complex-valued signal processing in frequency domain. In: International Joint Conference on Neural Networks (IJCNN 1990), vol. 3, pp. 27–31 (1990)
Karim, A., Adeli, H.: Comparison of the fuzzy-wavelet RBFNN freeway incident detection model with the california algorithm. Journal of Transportation Engineering 128(1), 21–30 (2002)
Jogensen, T.D., Haynes, B.P., Norlund, C.C.F.: Pruning artificial neural networks using neural complexity measures. International Journal of Neural Systems 18(5), 389–403 (2008)
Mayorga, R.V., Carrera, J.: A radial basis function network approach for the computational of inverse continuous time variant functions. International Journal of Neural Systems 17(3), 149–160 (2007)
Pedrycz, W., Rai, P., Zurada, J.: Experience-consistent modeling for radial basis function neural networks. International Journal of Neural Systems 18(4), 279–292 (2008)
Chen, S., Hong, X., Harris, C.J., Hanzo, L.: Fully complex-valued radial basis function networks: Orthogonal least squares regression and classification. Neurocomputing 71(16-18), 3421–3433 (2008)
Chen, S.: Information Science Reference. Complex-valued Neural Networks: Utilizing High-dimensional Parameters, ch. VII. IGI Global snippet, PA (2009)
Savitha, R., Suresh, S., Sundararajan, N.: A fully complex-valued radial basis function network and its learning algorithm. International Journal of Neural Systems 19(4), 253–267 (2009)
Huang, G.B., Li, M.B., Chen, L., Siew, C.K.: Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71(4-6), 576–583 (2008)
Suresh, S., Savitha, R., Sundararajan, N.: A sequential learning algorithm for a complex-valued self-regulatory resouce allocation network-csran. IEEE Transactions on Neural Networks 22(7), 1061–1072 (2011)
Hirose, A.: Complex-valued neural networks for more fertile electronics. Journal of the Institute of Electronics, Information and Communication Engineers (IEICE) 87(6), 447–449 (2004)
Hirose, A.: Complex-valued Neural Networks: Theories and Applications. Series on Innovative Intelligence, vol. 5. World Scientific Publishing Company, Singapore (2004)
Cha, I., Kassam, S.A.: Channel equalization using adaptive complex radial basis function networks. IEEE Journal on Selected Areas in Communications 13(1), 122–131 (1995)
Pandey, R.: Feedforward neural network for blind equalization with PSK signals. Neural Computing and Applications 14(4), 290–298 (2005)
Patra, J.C., Pal, R.N., Baliarsingh, R., Panda, G.: Nonlinear channel equalization for QAM constellation using artificial neural networks. IEEE Transactions on System, Man and Cybernetics, Part B: Cybernetics 29(2), 262–271 (1999)
Du, K.L., Lai, A.K.Y., Cheng, K.K.M., Swamy, M.N.S.: Neural methods for antenna array signal processing: A review. Signal Processing 82(4), 547–561 (2002)
Bregains, J.C., Ares, F.: Analysis, synthesis and diagnosis of antenna arrays through complex-valued neural networks. Microwave and Optical Technology Letters 48(8), 1512–1515 (2006)
Yang, W.H., Chan, K.K., Chang, P.R.: Complex-valued neural network for direction of arrival estimation. Electronics Letters 30(7), 574–575 (1994)
Shen, C., Lajos, H., Tan, S.: Symmetric complex-calued RBF receiver for multiple-antenna-aided wireless systems. IEEE Transactions on Neural Networks 19(9), 1659–1665 (2008)
Suksmono, A.B., Hirose, A.: Intelligent beamforming by using a complex-valued neural network. Journal of Intelligent and Fuzzy Systems 15(3-4), 139–147 (2004)
Amin, M.F., Murase, K.: Single-layered complex-valued neural network for real-valued classification problems. Neurocomputing 72(4-6), 945–955 (2009)
Buchholz, S., Bihan, N.L.: Polarized signal classification by complex and quaternionic multi-layer perceptron. International Journal of Neural Systems 18(2), 75–85 (2008)
Ozbay, Y., Kara, S., Latifoglu, F., Ceylan, R., Ceylan, M.: Complex-valued wavelet artificial neural network for doppler signals classifying. Artificial Intelligence in Medicine 40(2), 143–156 (2007)
Ceylan, M., Ceylan, R., Ozbay, Y., Kara, S.: Application of complex discrete wavelet transform in classification of doppler signals using complex-valued artificial neural network. Artificial Intelligence in Medicine 44(1), 65–76 (2008)
Sinha, N., Saranathan, M., Ramakrishna, K.R., Suresh, S.: Parallel magnetic resonance imaging using neural networks. In: IEEE International Conference on Image Processing (ICIP 2007), vol. 3, pp. 149–152 (2007)
Aizenberg, I., Moraga, C.: Multilayer feedforward neural network based on multi-valued neurons (MLMVN) and a backpropagation learning algorithm. Soft Computing 11(2), 169–183 (2007)
Aizenberg, I., Moraga, C., Paliy, D.: A feedforward neural network based on multi-valued neurons. In: Computational Intelligence, Theory and Applications. Advances in Soft Computing, XIV, pp. 599–612. Springer, Berlin (2005)
Aizenberg, I., Aizenberg, N.: Pattern Recognition Using Neural Network Based on Multi-Valued Neurons. In: Mira, J. (ed.) IWANN 1999. LNCS, vol. 1607, pp. 383–392. Springer, Heidelberg (1999)
Aizenberg, I., Paliy, D.V., Zurada, J.M., Astola, J.T.: Blur identification by multilayer neural network based on multivalued neurons. IEEE Transactions on Neural Networks 19(5), 883–898 (2008)
Amin, M.F., Islam, M.M., Murase, K.: Ensemble of single-layered complex-valued neural networks for classification tasks. Neurocomputing 72(10-12), 2227–2234 (2009)
Noest, A.J.: Phasor neural networks. Neural Information Processing Systems 2, 584–591 (1989), Online Journal, http://books.nips.cc/nips02.html
Kobayashi, M.: Pseudo-relaxation learning algorithm for complex-valued associative memory. International Journal of Neural Systems 18(2), 147–156 (2008)
Muezzinoglu, M.K., Guzelis, C., Zurada, J.M.: A new design method for the complex-valued multistate Hopfield associative memory. IEEE Transactions on Neural Networks 14(4), 891–899 (2003)
Kawata, S., Hirose, A.: Frequency-multiplexing ability of complex-valued Hebbian learning in logic gates. International Journal of Neural Systems 18(2), 173–184 (2008)
Isokawa, T., Nishimura, H., Kamiura, N., Matsui, N.: Associative memory in quaternionic Hopfield neural network. International Journal of Neural Systems 18(2), 135–145 (2008)
Tanaka, G., Aihara, K.: Complex-valued multistate associative memory with nonlinear multilevel functions for gray-level image reconstruction. IEEE Transactions on Neural Networks 20(9), 1463–1473 (2009)
Pande, A., Goel, V.: Complex-valued neural network in image recognition: A study on the effectiveness of radial basis function. Proceedings of World Academy of Science, Engineering and Technology 20, 220–225 (2007)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Springer-Verlag GmbH Berlin Heidelberg
About this chapter
Cite this chapter
Suresh, S., Sundararajan, N., Savitha, R. (2013). Introduction. In: Supervised Learning with Complex-valued Neural Networks. Studies in Computational Intelligence, vol 421. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29491-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-29491-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29490-7
Online ISBN: 978-3-642-29491-4
eBook Packages: EngineeringEngineering (R0)