1 Introduction

The visual system is one of the basic information processing systems responsible for memory, awareness, attention, and other complex cognitive function formation. Light stimulus is trapped and processed to extract information in terms of orientation and direction selectivity, object identification, motion detection [41], etc. Apart from object segregation and identification, direction selectivity, multi-scale feature extraction, and salient maps are basic functionality of the V1 layer of the primary visual cortex [43,44,45]. Rather than the overall functionality, the role of individual neurons in such complex functionality is of utmost importance. Specifically, their morphological structures and their primary connectomic organization that results in their contribution toward such robust nature. Literature [3, 54] suggests the existence of two-dimensional uniform functional maps in V1 and V2 regions whereas other literature [9, 24, 34, 35] found the existence of extraordinarily precise micro-architectures in a three-dimensional plane. Literature also suggests cortical maps build with single-cell precision [24, 34, 35] that significantly affects the computational capability of ganglion cells. Along with connectome specificity and degree of arborization, distinct morphologies of dendrites seem to perform much complex functional computations [25, 56]. It is also observed that the neurons display a dynamic range of spiking patterns corresponding to its spatial extent and spatio-temporal dynamics [10, 25, 50]. Such complex activity can be closely associated in terms of single-neuron dynamics [11], cell biophysics [26, 33, 58] and connectivity to the retinal network [7, 12]. Looking closer into layer connecting photoreceptors and direction-selective ganglion cells via ON/OFF bipolar cells, literature [14, 37, 53] also suggests basic pattern detection, edge and shape perception, object depth estimation, etc. as its probable inherent characteristics. Such dynamic behavior brings the need of understanding the mechanisms underlying significance of structural organization, dendritic morphology, and cell biophysics in shaping neuronal response in the primary visual cortex due to visual stimuli.

In this work, attempts are made to develop models to interpret visual data as probably perceived through a single layer of the primary visual cortex. Two distinct neuron morphologies organized in precise repetitive micro-architecture [9, 24, 34, 35], along with distinct cell physiology and dynamics of each neuron with localized ion channels [26, 33, 58] has been incorporated. The input to the model is similar to the response of bipolar cells (ON and OFF) toward the intensity of light. This response, in turn, is feedforward to a single layer of direction-selective RGC, and the output of the network is interpreted in terms of firing rate of neurons to reconstruct a vague perception of visual representation in a single layer of the neuronal network in primary visual cortex. The proposed model is also incorporated in the famous hmax model discussed in [42, 46,47,48] which is inspired by the functional architecture of the striate cortex discussed in Hubel and Wiesel et al [16] to understand the modeled structure’s role in learning and recognition.

2 Methods

An attempt is made to simulate a biological primary visual cortex single-layered neuron model targeted to perceive probable functional aspects. The model network of the primary visual cortex is modeled in three primary sections as shown in Fig. 1, here the input section (First section in Fig. 1) receives (transduced) signals from the ON and OFF bipolar cells connected to photoreceptors, the second section is the detailed RGCs organized in repetitive precise micro-architectures responsible for encoding the inputs into probable functional space, and the third section is the rate encoded response where the probable perceived image is reconstructed in terms of spikes per second.

Fig. 1
figure 1

Organization of different cells from photoreceptors to the RGC network and the respective sections considered for modeling of the proposed single layered network of visual cortex as shown in [28]

Full size image

2.1 ON and OFF bipolar cells

Two types of bipolar cell responses are selected, considering their sensitivity to light or darkness which are in turn connected to the photoreceptor cells. ON bipolar cell, directly connected to photoreceptor cells, responds with sustained depolarization corresponding to light intensity and sustained hyperpolarization to darkness and vice-versa for OFF bipolar cells, which are connected to photoreceptor cells via horizontal cells [1, 23, 30, 36]. Considering computational complexity, the response of bipolar cells is considered as a rectangular current pulse, either positive or negative, with a temporal spread of 250 mS. The amplitude of the square pulses corresponding to respective ON and OFF bipolar cells is proportional to the intensity of light. These current pulses are fed as input to the RGCs, and its effects on the post-synaptic neuron are assumed to be linear in terms of spiking frequency as shown in Figs. 2 and 3 [13, 22]. The input to the photoreceptor cells is “tif” or “png” images. Each pixel is considered a stimulus that is connected to the bipolar cells. The bipolar cells, in turn, convert these stimuli into temporal data. The amplitude of sustained depolarization and hyperpolarization corresponding to light or dark is considered that can deliver between \(-128\) to 127 nA current to the connected RGC’s with a multiplication factor of 5 and \(-5\) for ON type and OFF type bipolar cells, respectively, to map the stimulus dynamics within the sensitivity range of the connected neurons [18, 20, 21].

Fig. 2
figure 2

ON Bipolar cell response is encoded as square pulses (sustained depolarization or hyperpolarization) due to light intensity transduction, fed to the post synapse of the RGCs, and the corresponding spike rates

Full size image
Fig. 3
figure 3

OFF Bipolar cell response is encoded as square pulses (sustained depolarization or hyperpolarization) due to light intensity transduction, fed to the post synapse of the RGCs, and the corresponding spike rates

Full size image

2.2 Retinal Ganglion cell

2.2.1 Morphology

In the proposed model, two distinct morphologies for RGCs have been considered to compare the difference in computation functionality. The neuron morphology and respective connectivity matrices are inspired from the well-known Sobel edge detector kernel and directional gradient kernels. One cell of comparatively less arborized morphology with 4 dendritic terminal connecting with ON and OFF bipolar cell whereas the other having 6 dendritic terminals contributing to neuronal computation as shown in Figs. 4, 5.

Fig. 4
figure 4

RGC with 4 dendritic terminals connecting to the ON and OFF bipolar cells

Full size image
Fig. 5
figure 5

RGC with 6 dendritic terminals connecting to ON and OFF bipolar cells

Full size image

Shown in Figs. 4 and 5, the red synaptic connections are connected to ON-bipolar cells, and blue terminals are connected to OFF bipolar cells of the corresponding photoreceptor cells. The dendritic junctions and post-synaptic connectivity are rich in ionic channel concentration at the localized regions [5, 29, 32, 49] and for simplification of the model these regions are considered to generate “bursting” or “chattering” type of action potentials [18, 20, 21]. The rest of the section of the dendritic fibre is assumed to be passive and divided into isopotential compartments to facilitate decremental conduction [2, 31]. The equivalent neuron function of the RGC in Fig. 4 is shown in Fig. 6. Parameters for spiking activity are chosen as discussed in [18, 20, 21]. Propagation delay due to propagation through the passive fibres is neglected and assumed to reach concurrently at the junctions and soma sequentially from daughter branches to the parent branch. Diameters of the daughter dendritic branches are taken as equal because emphasis has been given to structure-function relationship rather than morphological plasticity. Connectivity of dendrites with bipolar cells is primarily through distal dendritic end connections. Probable connectivity of dendritic spines with bipolar cells has not been considered to avoid connectome plasticity and to investigate the structure-function relationship. Morphological structures used in the proposed model are symmetric with equal branching layers. Combined with Rall’s 3/2 branching rule and electronic length [15, 39, 40], daughter dendrites lengths are equal. The propagation velocity in the two fibres of equal length and diameter is equal [17]. Equal conduction velocity along with equal lengths of fibre with equal daughter dendrites diameters results in the signals reaching the branching junctions coherently for two simultaneous incoming signals.

Fig. 6
figure 6

Functional aspect of the neuron in Fig. 4

Full size image

2.2.2 Connectivity of RGC’s with bipolar cells

Connectivity of non-sister dendrites of RGC’s with the ON or OFF bipolar cells is convergent as discussed in [4]. Each RGC’s are arranged in precise modular repetitive structures as shown in Figs. 7 and 8, respectively. Each neuron is spread over a \(3\times 3\) grid of bipolar cells (both ON and OFF) where the connectivity of 1 represents connectivity with ON bipolar cell whereas \(-1\) describes connectivity with the OFF bipolar cell that is in turn connected to the corresponding pixel representation of photoreceptor cells. Two \(-1's\) in the central pixel in Fig. 7 infers connectivity of two dendritic branches in the central OFF bipolar cell, and 0 denotes no connectivity with the corresponding bipolar cells.

Fig. 7
figure 7

Connectivity matrices for detection of four directional edges namely vertical, horizontal, and two diagonal components, respectively, from left to right for neuron model in Fig. 4

Full size image
Fig. 8
figure 8

Connectivity matrices for detection of four directional edges namely vertical, horizontal, and two diagonal components, respectively, from left to right for neuron model in Fig. 5

Full size image

2.2.3 Functions of dendritic fibres, junctions, and soma

The excitation and hyperpolarization in localized active regions of the dendrites near the post-synaptic terminal are dependent on the strength of the ON/OFF bipolar cell response which in turn transduces the strength of light stimuli into a change in membrane potential and spike encoding in those localized regions. The membrane dynamics in these regions are modeled using the ‘Izhikevich’ spiking neuron model [18,19,20,21] due to its ability to simulate a wide range of spiking dynamics and computational simplicity. The equations governing the spiking activity can be summarized as

$$\begin{aligned} C\frac{{\rm d}v}{{\rm d}t}& = k\left( \left( v-v_r\right) \left( v-v_t\right) -u+I\right) ,\quad if\; v \ge v_t \end{aligned}$$
(1)
$$\begin{aligned} \frac{{\rm d}u}{{\rm d}t} & = a\left[ b\left( v-v_r\right) -u\right] , v \leftarrow c, u \leftarrow u+d \end{aligned}$$
(2)

where v is the membrane potential with I as the stimuli to the neurons, u as the recovery current, \(v_r\) as the resting membrane potential and \(v_t\) as the threshold potential. The activity of regular spiking, chattering, intrinsic bursting is controlled by parameters a, b, c, d, k, C as discussed in [19, 20]. In the proposed model, values considered are \(a=0.01, b=5, c=-56, d=130, C=150; k=1.2; v_r=-65, v_t=-35, v_{\rm peak}=40\) for bursting activity and \(a=0.03, b=-2, c=-50, d=100, C=100, k=0.7, v_r=-60, v_t=-30, v_{\rm peak}=40\) for chattering type activity. On the other hand, propagation of signal is modeled using the passive fibre model as discussed in [2] which basically behaves as a low pass filter resulting in decremental conduction along the channel and can be mathematically described as

$$\begin{aligned}&I_{inTotal}=I_t+I_{\rm out} \end{aligned}$$
(3)
$$\begin{aligned}&\quad I_{\rm inTotal} = \frac{\left( V_{\rm out}-V_{in}\right) }{R_{\rm lon}} \end{aligned}$$
(4)
$$\begin{aligned}&\quad I_t+C_m\frac{{\rm d}V_{\rm out}}{{\rm d}t}+G_L\left( V_{\rm out}-E_L\right) =0 \end{aligned}$$
(5)

where \(I_{\rm inTotal}\) is the total propagating current toward the junction/ point of measurement, \(I_t\) is the transmembrane current due to membrane dynamics, \(I_{\rm out}\) is the total delivered current, \(V_{in}\) is the action potential generated by the localized active region, \(V_{out}\) is the membrane potential at the junction or point of measurement with initial membrane potential equals to resting membrane potential, \(C_m\) is the equivalent capacitance of the isopotential compartment of fibre, \(R_{\rm lon}\) is the axial resistance against the flow of propagating current, \(G_L\) is the membrane leakage conductance, and \(E_L\) is the equilibrium potential due to the leakage ion channels. The junction of dendritic bifurcation and the ‘soma’ is assumed to be a summing node where the current from the two branches progressively propagates and accumulates. This cumulative current will trigger a combined effect due to the presence of localized active ion channels near the junctions in terms of action potential, and the process repeats itself till the current reaches ‘soma’ to trigger the overall response due to the cumulative input stimulus connected to the distal dendritic ends. The distal ends of neurons are connected to the inputs with precise arrangements as shown Figs. 7 and 8 representing the connectivity matrix. Shown in Figs. 10, 11, 12 and 13 are some of the responses of the simpler neuron morphology shown in Fig. 4 at different locations of localized ion channels, namely near the synapses, near the junctions and at the ‘soma’, respectively, due to different stimulus matrices shown in Fig. 9.

Fig. 9
figure 9

Different orientation \(3\times 3\) photoreceptor stimulus matrix connected to the neuron in Fig. 4 via bipolar cells in configuration of vertical edge selective connectivity matrix shown in Fig. 7 to simulate spiking activity at different ion concentrated localized locations of the neuron shown in Figs. 10, 11, 12, and 13

Full size image
Fig. 10
figure 10

Spiking activity due to input matrix 1 near the synapse, at the junctions with localized ion clannels and the soma, respectively

Full size image
Fig. 11
figure 11

Spiking activity due to input Matrix 2 near the synapse, at the junctions with localized ion clannels and the soma, respectively

Full size image
Fig. 12
figure 12

Spiking activity due to input Matrix 3 near the synapse, at the junctions with localized ion clannels and the soma, respectively

Full size image
Fig. 13
figure 13

Spiking activity due to input Matrix 4 near the synapse, at the junctions with localized ion clannels and the soma, respectively

Full size image

Figures 10, 11, 12, and 13 show a number of interesting results starting with post inhibitory rebound, post inhibitory bursting, intrinsic bursting type activity at different locations of the neuron. The extreme right figures in Figs. 10, 11, 12, and 13 show the overall response of the neuron which is finally converted to spike rates to quantitatively interpret the output response of the layer of neurons. Extreme left Matrix 1 in Fig. 9 represents a horizontal gradient that is fed to a horizontal edge-sensitive configuration that results in high regular spiking activity whereas the response of the same neuron to a segment of diagonal gradient results in post inhibitory rebound spiking activity with an overall decrease in the number of spikes. On the other hand, the second last test matrix Matrix4 corresponding to a vertical gradient in Fig. 9 also results in post inhibitory spiking activity due to unmatched edge orientation which can be seen in actual direction-selective ganglion cells [51, 55]. For encoding and visualization of the response of the modeled neuron network, the spiking rate is considered irrespective of regular spiking activity and inhibitory rebound activity.

3 Simulation and results

All simulations are done in “Python3.6” interpreter and for solving the differential equations, odeint method from “scipy.integrate” package is used. For plotting the membrane potentials, other related plots and responses to input image, “matplotlib” package has been used. The inputs to the neurons are “tif” or “png”images where each pixel is considered as stimuli to spatially localized photoreceptor cells, and the range of values for each pixel in the image is converted within the range of \(-128\) to 127 representing the extremums of light and dark. These photoreceptor cells, namely rods are connected to two types of bipolar cells, one ON type and the other OFF type cells. ON type cell response is considered to be a positive square pulse (sustained depolarized pulse) of a temporal spread of 250 ms for corresponding light intensity. Step width of 250 ms is considered to support the minimum response time of the eye, and the amplitude of the bipolar cell response is amplified 5 times that of the input stimuli to encode the stimuli within the sensitivity range of the bursting and chattering type membranes as discussed in [4, 20]. On the other hand, OFF type cell response is considered to be a negative square pulse (sustained hyperpolarized pulse) with the same gain factor of 5 and a temporal spread of 250 ms. Square pulse is assumed in the proposed model to decrease the computational complexity in generating similar responses of the bipolar cells [13]. The spiking activity of the neurons is governed by “Izhikevich” neuron model” [19, 20], specifically bursting and chattering type of membrane. Connectivity of RGC with the ON and OFF bipolar cells is convergent and precisely organized in repetitive modular patterns [3, 4, 9]. The RGC’s are rich in ionic concentration at some precise locations, assumed near the synapse, at the junctions and the soma as discussed in [5, 29, 32, 49, 50] for signal encoding, signal renewal, and overall response encoding. Shown in Fig. 14 are the test images taken as inputs to simulate the single layer of RGC networks consisting of distinct morphologies shown in Figs. 4 and 5, respectively.

Fig. 14
figure 14

Test images used as input to simulate the model

Full size image
Fig. 15
figure 15

Output response of the network of RGC cell to inputs in Fig. 14 due to modular network of simple neuron Fig. 4 is found to be similar to an overall image segmentation and fine spatial detail extractor

Full size image

Figure 15 is the response of the network of ganglion cells, each neuron of morphological structure shown in Fig. 4 where the response of the network is quantified in terms of spikes per second. The model shows a response similar to a segmented image used for clustering and classification. The response also shows sensitivity toward fine detailed textures similar to midget cells [6, 38] and capable of capturing minute gradient change from one photoreceptor to the successive photoreceptor cell.

Fig. 16
figure 16

Output response of the network of RGC cell to inputs in Fig. 14 due to modular network of arborize neuron Fig. 5 is found to be similar to edge orientation specific resonator

Full size image

Figure 16 shows response of single layer RGC network constructed using neuron shown in Fig. 5 with connectivity matrix shown in Fig. 8. The morphology of the neuron referred to in Fig. 5 is comparatively arborized as compared to neuron shown in Fig. 4 but with similar biophysical properties. The response shows the better direction-selective response for coarser textures as compared to responses shown in Fig. 15 and less sensitive toward fine textures which suggest the existence of the role of complex structures in function formation [25, 56, 57]. The responses in Fig. 16 show higher spiking rate corresponding to preferred directional coarser edge orientation as compared to simple arborized neuron model Fig. 15. Neuronal response corresponding to first columns in Fig. 16 shows horizontal edge selectivity whereas the second column shows vertical edge selectivity as compared to the two diagonal counterparts shown in third and fourth columns of Fig. 16. On the other hand, a significant reduction in spiking frequency is also seen as the edge orientation changes from the preferred directional edges to its opposite counterparts. A similar response can be seen in Fig. 15 but in terms of fine texture orientation detection, suggesting the functional significance of neuronal morphologies in multi-scale feature extraction. Responses from the model suggest extraction of multi-scale features depending on the spatial spread of the dendritic arbours [43,44,45] that can be used in progressive layers for pattern detection and classification.

In order to study the viability of the proposed model in learning and object recognition, the model has been integrated into the hmax model discussed in [42, 46,47,48] with minor changes in the number of filters. The hmax model is inspired by the receptive field architecture discussed in Hubel and Wieser et al. [16]. The simple cell layers namely S1 and S2 in the hmax model have been replaced with the proposed architecture to extract directional features which in turn are fed to populations of complex cell layers C1 and C2’s responsible for pooling maximum activation information. The S1 unit in hmax model discussed in [42, 46,47,48] comprises of 2 phases \(\times 4\) orientations \(\times 17\) sizes of receptive fields. But due to the computational complexity of the proposed model, the S1 unit of the proposed model is accommodated with 2 phases \(\times 4\) orientations \(\times 3\) sizes of receptive fields. The 2 phases discussed in the model are the morphological connectivity matrices discussed in Figs. 7, 8 with the polarity of the connections reversed. The S1 layer discussed in hmax model implements a Gabor filter responsible for extraction of local directional feature information and parameters such as \(\theta , \gamma , \sigma , \phi \), and \(\lambda \) controles the receptive field properties such as orientation, aspect ratio, effective width, phase, and wavelength. Very similar to the hmax model, proposed morphologically defined neuron model also works as a local directional texture information extractor from the input image. In the proposed model, the connectivity matrix controls the orientation, spread of neurons control the aspect ratio, wavelength tuning is controlled by the type of localized ion channels at specific site, and phase is controlled by the polarity of connectivity matrices. On the other hand, sensitivity of the neuron morphology is also controlable by controling the amount of input current coming from the bipolar cells. In the proposed model, the bandwith of the neuron is controled by the combinations of spiking activity at the junctions as well as the cell body of the neuron. For optimal behavior of the model and considering types of spiking activity seen in visual cortex, bursting and chattering type of neuron spiking activity has been considered. The C1 neuron layer in the hmax model is responsible for the pooling of maximum activation of neurons within a specific receptive field. The sizes of the receptive field of C1 cells are normally higher than simple cells S1 and S2 since complex cell units are insensitive to the location of the stimulus within their respective receptive fields. Therefore, 6 receptive fields 5, 7, 9, 11, 13, 15 are considered for pooling operations of C1 cells. In the next simple cell layer S2, the pooled activity of C1 cells is reprojected in 4 directional orientations capable of capturing more complex patterns such as contours and combinations of different orientation information, which will be later fed to the C2 complex cells. C2 cells incorporated in the model are functionally similar to C1 complex cells and responsible for max-pooling operations. Initial learning and recognition model incorporating simple detailed morphological and connectomic structure of neuron to hmax model [42, 46,47,48] has been implemented on the face database provided with the official computational hmax model (found in https://maxlab.neuro.georgetown.edu/hmax.html #updated). Facial features from the face database have been learned in the C2 layer from the facial database in terms of population histogram, and testing and validation have been carried out on 6 shifted or tilted face images, position-independent face images for each subject, and the overall recognition and retrieval accuracy is calculated from the best 4 retrieved images. Face detection and retrieval accuracy of the model is averaged at \(71.166\%\) which seems very interesting as only 3 receptive field sizes are incorporated in S1 layer, 6 in the C1 layer and is expected to show better performance by increasing the numbers of receptive field dependent filters in S1 and S2 region. On the other hand, hmax model shows better performance with a face identification benchmark of \(90.4\%\) which might be due to the number of direction information extracted. Face detection and retrieval accuracy for each subject for the proposed model are shown in Table 1.

Table 1 Retrival performance of the hmax model after incorporating the proposed morphologically detailed RGC neuron model in the S1 and S2 layer
Full size table

4 Conclusion

The visual cortex is a complex information processor, capable of processing visual stimuli and extracting relevant information parallelly to generate meaningful responses. No doubt such a robust system is equipped with multiple types of RGN morphologies with their distinct connectome specificity and dendritic spread organized in modular patterns, segregated in multiple layers. As discussed in the literature, at least 50 different types of RGC’s [7, 8, 27] including midgets and parasol cells with distinct morphologies can be found. The proposed model suggests the probable role of different RGC structures linked to individual functional aspects that plays an important role in feature extraction, object detection, and recognition. Preliminary test to understand its role in face recognition has been conducted by incorporating the morphologically detailed RGC neurons in the S1 and S2 layers of hmax model inspired by Hubel and Wieser. The model shows good results in terms of face recognition and retrieval considering the number of filters incorporated to the model. The efficacy of the proposed model in object recognition is anticipated to improve by increasing the range of receptive field widths that can focus on a wide range of spatial frequencies for multi-scale feature extraction. Such robust networks would not only be capable of extracting multi-scale features, shapes, and edge information but also global information such as depth estimation, motion detection, gaze stabilization, etc. Coupling of such local features with global features might result in noise-free detection of shapes, edges, objects, etc. On the other hand, multi-scale local feature detectors in conjuncture with multi-scale global feature extractors might play a significant role in learning feature optimization for identification of objects possibly in terms of self-organization neural nets via extension and retraction of dendritic spines [52, 59]. Thus, a detailed study of the importance of morphology and their relevant interlayer and intralayer connectivity might open a new understanding of the dynamic change in dendritic morphology and arbors’ growth during learning.