Biological Neural Networks

Abstract

A biological neural network is composed of a group of connected neurons. A single neuron may be connected to many other neurons and the total number of neurons and connections in a network may be significantly high. One of the amazing aspects of biological neural networks is that when the neurons are connected to each other, higher-level intelligence, which cannot be observed from a single neuron, emerges.

1 Introduction

A biological neural network is composed of a group of connected neurons. A single neuron may be connected to many other neurons and the total number of neurons and connections in a network may be significantly high. One of the amazing aspects of biological neural networks is that when the neurons are connected to each other, higher-level intelligence, which cannot be observed from a single neuron, emerges. The exact mechanism of the emergence of intelligence from the neuronal network has been an intense research topic for neuroscientists, biologists, and engineers, and is not yet fully understood. In fact, computational modeling and mathematical analysis of biological neural networks are integral parts of the neuroscience discipline called computational neuroscience, which is also closely related to the artificial neural network community. The main assumption in this discipline is that through the computational modeling the probable working mechanism of the biological network can be unveiled. Moreover, understanding the working principles of biological neuronal networks has been believed to open the horizon to designing high-performance artificial neuronal networks.

Therefore, in this chapter, we will review the basic neurobiology regarding individual neurons and their networks, and introduce some interesting neuro-scientific discoveries that have inspired artificial neural networks. However, these introductory materials are by no means extensive, so interested readers are advised to read standard textbooks in neuroscience [17,18,19].

2 Neurons

2.1 Anatomy of Neurons

A typical neuron consists of a cell body (soma), dendrites, and a single axon (see Fig. 5.1). The axon and dendrites are filaments that extrude from the cell body. Dendrites typically branch heavily and extend a few hundred micrometers from the soma. The axon leaves the soma at the axon hillock, and moves up to 1 m in humans or more in other species. The end branches of an axon are called telodendria. At the extreme tip of the axon’s branches are synaptic terminals, where the neuron can transmit a signal to another cell via the synapse.

Fig. 5.1

Anatomy of neurons

The endoplasmic reticulum (ER) in the soma performs many general functions, including folding protein molecules and transporting synthesized proteins in vesicles to the Golgi apparatus. Proteins synthesized in the ER are packaged into vesicles, which then fuse with the Golgi apparatus. These cargo proteins are modified in the Golgi apparatus and destined for secretion via exocytosis or for use in the cell as shown in Fig. 5.2.

Fig. 5.2

ER and Golgi apparatus for protein synthesis and transport

2.2 Signal Transmission Mechanism

Neurons specialize in forwarding signals to individual target cells via synapses. At a synapse, the membrane of the presynaptic neuron comes into close proximity to the membrane of the postsynaptic cell (see Fig. 5.3). Although there are electric synapses where the presynaptic and postsynaptic neurons are directly fused together for fast electric signal transmission [18, 19], chemical synapses, which transmit the action potential via neurotransmitters, are the most common and are of great interest for artificial neural networks.

Fig. 5.3

Chemical synapse between presynaptic terminal and postsynaptic dendrite

As shown in Fig. 5.3, in a chemical synapse, electrical activity in the presynaptic neuron is converted into the release of neurotransmitters that bind to receptors located in the membrane of the postsynaptic cell. The neurotransmitters are usually packaged in a synaptic vesicle, as shown in Fig. 5.3. Therefore, the amount of the actual neurotransmitter at the postsynaptic terminal is an integer multiple of the number of neurotransmitters in each vesicle, so this phenomenon is often referred to as quantal release. The release is regulated by a voltage-dependent calcium channel. The released neurotransmitter then binds to the receptors on the postsynaptic dendrites, which can trigger an electrical response that can produce excitatory postsynaptic potentials (EPSPs) or inhibitory postsynaptic potentials (IPSPs).

The axon hillock (see Fig. 5.1) is a specialized part of the cell body that is connected to the axon. Both IPSPs and EPSPs are summed in the axon hillock and once a trigger threshold is exceeded, an action potential propagates through the rest of the axon. This switching behavior of the axon hillock plays a very important role in the information processing of neural networks, as will be discussed in detail later in Chap. 6.

2.3 Synaptic Plasticity

Synaptic plasticity is the ability of synapses to strengthen or weaken over time as their activity increases or decreases. In fact, synaptic plasticity is one of the important neurochemical foundations of learning and memory that is often mimicked by artificial neural networks.

Two of the best studied forms of the synaptic plasticity in the neuronal cell are long-term potentiation (LTP) and long-term depression (LTD). Specifically, LTP is a sustained strengthening of the synapses based on recent patterns of activity. These are patterns of synaptic activity that cause a long-lasting increase in signal transmission between two neurons. The opposite of LTP is long-term depression (LTD), which leads to a long-lasting decrease in synaptic strength.

In contrast to the artificial neural network, in which the synaptic plasticity changes are usually modeled by simple weight changes, the synaptic plastic change in biological neurons often results from the change in the number of neurotransmitter receptors located on a synapse. For example, as shown in Fig. 5.4, during the LTP additional receptors are fused to the membrane by exocytosis, which are then moved to the postsynaptic dendrite by lateral diffusion within the membrane. On the other hand, in the case of LTD, some of the redundant receptors are moved into the endocytosis region by lateral diffusion within the membrane, and then absorbed by the cell via endocytosis.

Fig. 5.4

Biological mechanism of LTP and LTD

Because of the dynamics of learning and synaptic plasticity, it becomes clear that the trafficking of these receptors is an important mechanism to meet the demand and supply of the receptors at various synaptic locations in the neurons. There are various mechanisms that are being intensively researched by neurobiologists. For example, assembled receptors leave the endoplasmic reticulum (ER) and reach the neural surface via the Golgi network. Packets of nascent receptors are transported along microtubule tracks from the cell body to synaptic sites through microtubule networks. Figure 5.5 shows critical steps in receptor assembly, transport, intracellular trafficking, slow release and insertion at synapses.

Fig. 5.5

Receptor trafficking for synaptic plasticity

3 Biological Neural Network

One of the most mysterious features of the brain is the emergence of higher-level information processing from the connections of neurons. To understand this emergent property, one of the most extensively studied biological neural networks is the visual system. Therefore, in this section we review the information processing in the visual system.

3.1 Visual System

The visual system is a part of the central nervous system that enables organisms to process visual detail as eyesight. It detects and interprets information from visible light to create a representation of the environment. The visual system performs a number of complex tasks, from capturing light to identifying and categorizing visual objects.

As shown in Fig. 5.6, the reflected light from objects shines on the retina. The retina uses photoreceptors to convert this image into electrical impulses. The optic nerve then carries these impulses through the optic canal. Upon reaching the optic chiasm, the nerve fibers decussate (left becomes right). Most of the optic nerve fibers terminate in the lateral geniculate nucleus (LGN). The LGN forwards the impulses to V1 of the visual cortex. The LGN also sends some fibers to V2 and V3. V1 performs edge detection to understand spatial organization. V1 also creates a bottom-up saliency map to guide attention.

Fig. 5.6

Anatomy of visual system and information processing

3.2 Hubel and Wiesel Model

One of the most important discoveries of Hubel and Wiesel [20] is the hierarchical visual information flow in the primary visual cortex. Specifically, by examining the primary visual cortex of cats, Hubel and Wiesel found two classes of functional cells in the primary visual cortex: simple cells and complex cells. More specifically, simple cells at V1 L4 respond best to edge-like stimuli with a certain orientation, position and phase within their relatively small receptive fields (Fig. 5.7). They realized that such a response of the simple cells could be obtained by pooling the activity of a small set of input cells with the same receptive field that is observed in LGN cells. They also observed that complex cells at V1 L2/L3, although selective for oriented bars and edges too, tend to have larger receptive fields and have some tolerance with regard to the exact position within their receptive fields. Hubel and Wiesel found that position tolerance at the complex cell level could be obtained by grouping simple cells at the level below with the same preferred orientation but slightly different positions. As will be discussed later, the operation of pooling LGN cells with the same receptive field is similar to the convolution operation, which inspired Yann LeCun to invent the convolutional neural network for handwritten zip code identification [21].

Fig. 5.7

Hubel and Wiesel model for primary visual cortex

The extension of these ideas from the primary visual cortex to higher areas of the visual cortex led to a class of object recognition models, the feedforward hierarchical models [22]. Specifically, as shown in Fig. 5.8, as we go from V1 to TE, the size of the receptive field increases and the latency for the response increases. This implies that there is a neuronal connection along this path, which forms a neuronal hierarchy. A more surprising finding is that as we go along this pathway, neurons become sensitive to more complex inputs that are not sensitive to transforms.

Fig. 5.8

Hierarchical models of visual information processing

3.3 Jennifer Aniston Cell

An extreme form or surprising example of this information processing hierarchy can be found in the discovery of the so-called “Jennifer Aniston Cell” [23], which represents a complex but specific concept or object. For those who do not know Jennifer Aniston, she was one of the most popular American actresses of the 1990s, having starred in America’s favorite sitcom, Friends.

The study involved eight epilepsy patients who were temporarily implanted with a single cell recording device to monitor the activity of brain cells in the medial temporal lobe (MTL). The medial temporal lobe contains a system of anatomically related structures that are essential for declarative memory (conscious memory for facts and events). The system consists of the hippocampal region (Cornu Ammonis (CA) fields, dentate gyrus, and subicular complex) and the adjacent perirhinal, entorhinal, and parahippocampal cortices (see Fig. 5.9).

Fig. 5.9

The anatomical location of the medial temporal lobe

During the single cell recording, the authors in [23] noticed a strange pattern on the medial temporal lobe (MTL) of the brain in one of their participants. Every time the patient saw a picture of Jennifer Aniston, a specific neuron in the brain fired. They tried to show the words “Jennifer Aniston,” and again it would fire. They tried other ways to summon Jennifer Aniston in other ways, and each time it fired. The conclusion was inevitable: for this particular person, there was a single neuron that embodied the concept of Jennifer Aniston.

The experiment showed that individual neurons in the MTL respond to the faces of certain people. The researchers say that these types of cell are involved in sophisticated aspects of visual processing, such as identifying a person, rather than just a simple shape. This observation leads to a fundamental question: can a single neuron embody a single concept? Although this issue will be investigated thoroughly throughout the book, the short answer is “no” because it is not the single neuron in isolation, but a neuron from a densely connected neural network that can extract the high-level concept.

4 Exercises

• 1. Explain the role of the following structure in a neuron:

  • a. Soma

  • b. Dendrite

  • c. ER

  • d. Golgi apparatus

  • e. Axon hillock

  • f. Synapse

• 2. It is important to have a sense for the relative orders of magnitude of cellular components. Please specify each physical parameter for a synapse.

  • a. Vesicle diameter

  • b. Synapse width

  • c. Vesicles released per active zone per action potential

  • d. Synaptic cleft width

• 3. Explain the differences between electrical and chemical synapses.

• 4. Explain the different types of neurotransmitters and their roles.

• 5. Explain the differences between ionotropic receptors and metabotropic receptors.

• 6. Explain the mechanism of LTD and LTP.

• 7. What is the role of the neurotransmitter trafficking?

• 8. Explain the visual information processing step by step.

• 9. Explain why the Hubel and Wiesel model implies the convolutional processing in the visual cortex.

• 10. What is the main observation from the Jennifer Aniston cell?