Keywords

1.1 The Systems View of Neuroscience

1.1.1 Goal-Directed Behavior and the Result

In contemporary neuroscience there is a problem of isolation of meaningful segments of behavior and related to them brain activity. In many cases brain activity is averaged over the period right after the presented stimulus, despite the fact that in this case all predictive activity is completely lost. Considering all behavior as goal-directed entails the necessity of viewing brain activity as related to the future events. Such perspective was suggested in the framework of the functional systems theory by P.K. Anokhin (1974) and has been developed further within the system-evolutionary theory (Shvyrkov 1986).

An important point of the Theory of Functional Systems (TFS) is the definition of a factor that unites sparse brain and body elements into a system (“systems-creating factor”). The factor is a result of the system—an adaptive effect in the organism-environment interaction achieved upon realization of that system. The result is isomorphic in relation to any system. Therefore, the systems approach can be applied to various research objects and behavioral situations. Notably, in relation to performed behavior the result is a future event, not a past one, like stimulus. The system is understood as a dynamic organization of activity of components with different anatomical localization, where interaction becomes a mutual facilitation in the process of ensuring an adaptive for an organism result (see also Alexandrov et al. 2000).

How can the result, a future event, be a reason of current activity? In the TFS this “time paradox” is explained via the concept of goal—a model of the result which contains its predicted parameters and is provided by the “acceptor of the result of an action”. Thus P. Anokhin (1974) had resolved the disconnection between causality and teleology in the description of behavior in the form acceptable for those dedicated to causality as a necessary principle of scientific analysis (Bunge 1963).

The TFS enables a holistic view of behavior through studying the result-driven organization of entire organism-environment interactions (Anokhin 1974), unlike the traditional view of functions as direct effects of a certain substrate, including the nerve tissue, e.g. motor functions, sensory, emotional, motivational etc. A similar view expressed by other authors also suggests considering the activity of any brain area with respect to behavioral performance as explained by implementation and selection of systems (Cisek and Kalaska 2010). The “task space” representation, proposed by Weible et al. (2009) for the anterior cingulate cortex, is an idea to a certain extent alike, if expanded to the whole organism. In the TFS the function is defined as achievement of a result. Such systemic function can not be localized, it is applied to the whole organism that interacts with the environment.

Multiple research efforts within the TFS framework have led to creation of the system-evolutionary approach (Shvyrkov 1986; see also Alexandrov 2015). One of the most important steps of this approach was the solution of the psycho-physiological (mind-body) problem. The psychic processes describe the organism and its behavior as a whole. Physiological processes are considered on the level of elements. The organization of physiological processes into a system is based on neither psychic, nor physiological, but specific (informational) systemic processes. Their substrate is physiological activity, whereas their informational content is psychic. Thus, behavioral acts are not only based on localized physiological processes, but also on the processes of their organization. In other words, psychical and physiological processes are different aspects of the same systemic processes.

It is important to note that the described solution of the psychophysiological problem excludes “theoretical reductionism” (see Dudai 2002 for types of reductionism), i.e. degrading the psychic processes to physiological ones. This aspect seems especially important within the problem of consciousness (Alexandrov and Sams 2005): since consciousness cannot be simplified by analytical means, it is often discarded from the scope of scientific investigation (Kandel 2006).

The specific issues under investigation of systemic psychophysiology are formation and realization of systems, their taxonomy, and the dynamics of intersystem relations in behavior. The systemic psychophysiology rejects the reactivity paradigm and employs goal-directed principles in the analysis of activity of individuals and, more importantly, neurons. Therefore, it is free from the eclectic explanations of goal-directed behavior by reflex-based mechanisms (see Alexandrov 2015 for more details).

We acknowledge that the presented view shares certain aspects of cognitive structures. For example, U. Neisser’s cognitive schema concept also includes prediction of incoming information, guidance of the exploration, and modification during execution (Neisser 1976; see also Moscovitch et al. 2016). Moreover, it presumes simultaneous activation of schemas on different levels of hierarchy (see related assertions in paragraph 4). Other resemblant views are presented in the brain activity interpretations by Engel et al. (2001) and von Stein et al. (2000). The cognitive maps (Tolman 1948), reconstructed via place-cell activity analysis etc. (Burgess and O’Keefe 2011; Hartley et al. 2013; O’Keefe 1976), and cognitive schemas (Bartlett 1995), modeled in consolidation research (Hennies et al. 2016; Tse et al. 2007, 2011) also reveal some of these properties (see also Dudai et al. 2015). However, a clear formulation of what makes a system (the result) is of critical importance for considering cognitive units in terms of the individual and its interaction with the environment, rather than in terms of environment proper. This leads to interpretation of neuronal firing and other physiological measures from living organisms as a manifestation of their activity.

1.1.2 Activity Paradigm

The view of behavior as aimed at future results assumes that the principal feature of the living matter is its activity. The concrete form of this activity is defined by the level of the matter organization (Anokhin 1974). The activity principle presumes that behavior is driven by a model of its result.

The classical TFS includes the concept of the “starting stimulus”. All the processes of system organization are goal-directed, whereas the starting stimulus solely triggers the execution of integrated elements. And even this role of the stimulus, that seems necessary, disappears when the behavioral act is considered not as an isolated entity, but as a component of a behavioral continuum, that is, the succession of behavioral acts performed by an individual during lifetime. The given behavioral act within a continuum is deployed after the result of the previous act has been achieved and evaluated. The evaluation of the results of the given act is a necessary part of the next act initiation. These processes serve as a transition from the execution of one act to a subsequent one. There is no room for a stimulus in the continuum. The environmental changes described as stimuli are contained in the model of a preceding result. They are conditions, but not causes of behavior. Unexpected changes will either have no effect on the continuum (i.e. “ignored”), or serve a condition for behavior that interrupts the succession: either repetition of the interrupted behavioral act, or building a new one via systemogenesis (see Sect. 1.2.1). In any case, both are aimed at the future, and their cause is a mismatch (see Sect. 1.3.1).

Provided that the whole organism is active on the level of behavior, the neuronal firing would also be considered as manifestation of activity.

1.1.3 Active Neuron

Within the reactivity paradigm behavior is a reaction based on the transmission of excitation in a circuit (or a net). The function of a neuron is therefore forwarding of excitation. Events recorded in the neuron are considered as a response to a stimulus, that had affected some part of it and may travel further along the cell to be a stimulus to other nerve cells. Thus, a neuron, just like an organism, responds to stimuli.

The activity paradigm also dictates coherent understanding of both the whole organism functioning, and that of a single cell in a multicellular organism. This correspondence has been achieved by treating events in a neuron or any other living cell as execution of a genetic program. The execution requires receiving metabolites from other cells (Shvyrkov 1986; Alexandrov 2015). Consequently, neuronal activity, alike behavior of an organism, is not a response, but a way of changing its relation to environment. Events in the neuron are “actions” that change its microenvironment with respect to its “needs”, causing modifications in blood flow, metabolic inflow from glial cells, and activity of other neurons. Therefore, a neuron is not a conductor or a calculator—it’s an organism inside organism.

A neuron can satisfy its metabolic “needs” only by co-action with other elements of an organism to form a functional system. Their cooperative, joint activity leads to a new relation between the whole organism and its environment, as well as (at the cellular level) to satisfying metabolic “needs” of the cells. As soon as the result is achieved by the organism (and metabolites are received by a neuron), the firing of the neuron ceases.

This view of neuronal activity corresponds to the evolutionary perspectives that show similarities between survival principles of single-cell organisms and neurons within nervous system. It has been shown that the colonies of single-cell organisms and cells in a multicellular organism provide for breath, nutrition, and other group functions via cooperation—they synchronize their metabolic processes (e.g. Weber et al. 2012). The satisfaction of all various metabolic requirements of an organism is achieved by diverse behavioral acts. It can also be argued that besides regular functioning a neuron is active during apoptosis, or “altruistic suicide” (see Sect. 1.3.3).

The systemic view of the neuronal activity requires a corresponding approach to investigation of learning and memory (see also Alexandrov 2008).

1.2 The Formation of Memory during Learning and Systemic Structure of Behavior

1.2.1 Systemogenesis

The key notion in the TFS besides the system is that of development, revealed in the concept of systemogenesis. Systemogenesis refers to the idea that maturation of organs is not homogeneous—the first elements to mature during early ontogenesis are those parts of organs and tissues that are essential for achieving the results of the systems to ensure the survival of the organism at these stages of individual development (Anokhin 1974).

It has been argued that the systemogenesis also occurs during learning in adults—the emergence of a new system provides a new behavioral act. It was proposed that due to successive emergence of new systems during lifetime the role of different neurons in the organization of behavior should be considered on the basis of individual history (Alexandrov and Alexandrov 1982). This hypothesis has led to the systems-evolutionary theory and the system-selection concept of learning (Shvyrkov 1986). The latter construct is in accordance with G. Edelman’s (1987) view of neuronal ensemble formation during learning as a selection process (excerpt of cells on the basis of their features), as opposed to learning as instruction (modification of neuronal features caused by stimulation). The principle of selection underlies the immune and evolutionary processes in a similar way, although at different time scales.

According to G. Edelman, the selection process starts in the early ontogeny, when lots of neurons die during brain maturation. The cells that survive this selection were termed a “primary assortment”. The “secondary assortment” is formed upon learning via behavioral interaction with the environment at the second stage of the selection.

Changeux had also defined two stages of the selection: the “neural Darwinism” in the early (including prenatal) ontogeny, proposed to be a selection of effective synapses, and the “mental Darwinism” in adults considered as modification of the existing synaptic connections. However, the units assumed to be subject for the selection are not the connections, but groups of neurons that were selected at the first stage and had cooperative activity (Changeux and Connes 1999).

The essence of the selection process in both individual development and evolution is the resulting outcome. As Wright (1995) puts it, it’s not the “truth” that the brain needs, but a success. Similarly, the evolution “supports” those who have survived, not those who were right (Cacioppo and Gardner 1999). The selection during evolution applies not to independent features, but to the holistic organisms, the phenotypes. The basis for the selection is the achievement of results beneficial for the given phenotypic variation. These phenotypes are the only objects of the selection (Shvyrkov 1986; Fodor 2007). Success in the selection process, defined by the quality of the achieved results, includes formation of “pre-specialized” (see below in this section) and “specialized” neurons (see example in Fig. 1.1).

Fig. 1.1
figure 1

Activity of a representative specialized neuron #1938 recorded in rabbit’s anterior cingulate cortex. (a) The raster plot shows spikes during successive turns of the animal towards a pedal aligned to the pedal pressing (PP). The inset shows spike waveforms, selected after sorting. Below is the histogram of these spikes (50 ms in a bin, the ticks of the ordinate show tens of spikes). The bottom panel shows superimposed behavioral markers (up-deflection for pedal-pressing and down-deflection for lowering the head into the feeder). In this experiment (Sozinov et al. 2012) the activity of neurons was recorded with glass electrodes (3–6 MOhms @ 1 KHz) after reaching the learning criterion—see text and panel (b). For definition of specialization, see text. (b) The schematic view of the experimental chamber from the top. The behavioral markers (beam crossing) were used to identify the following behavioral acts (that corresponded to the stages of learning): lowering head and taking food from the feeder (LH); lifting head from feeder and turning it toward pedal (TP); moving to pedal corner (RP); pedal pressing (PP); running from pedal to feeder (PF). This sequence of acts was looped (arrows) during neuronal recordings (10+ cycles on each side)

Full size image

In the frames of the systemic approach, the individual development is a sequence of systemogeneses that provide new interactions with environment. Formation of a new system during systemogenesis is considered as emergence of a new element of individual (subjective) experience during learning. The new system consists of the neurons that were selected from the “reserve” cells—presumably, those are low active or silent cells. These cells correspond to the primary assortment (according to Edelman 1987) and termed here “pre-specialized”. These are the neurons subjected to selection during learning, when some of them become specialized in relation to a system of a new behavioral act. This selection process is defined by specific metabolic features of these cells. Accordingly, the group of selected units can be termed the secondary assortment. Therefore, the new system is an addition to the existing ones, it is ‘superimposed’ on them.

These evolutionary considerations presume that the specialization of neurons remains for their whole lifetime. The learning process would then be provided by recruiting new cells, rather than by retraining the ones previously trained. This is in accordance with experimental data (Schmidt et al. 1976; Thompson and Best 1990; Wilson and McNaughton 1993; Swadlow and Hicks 1997; Williams et al. 1999; Greenberg and Wilson 2004; Brecht et al. 2005; Chestek et al. 2007; Jackson et al. 2007; Fraser and Schwartz 2012) that show “silent” cells in the brain of different species, increase of the number of active cells during learning, and constancy of new neuronal specializations (during the whole registration period—weeks or even months and years, see also McMahon et al. 2014).

The adult neurogenesis in birds and mammals (Paton and Nottebohm 1984; Carleton et al. 2003) has been shown to be related to learning. The learning process facilitates the survival of newborn neurons (the “use it or loose it” principle—Kempermann et al. 1998), as well as proliferation (Prickaerts et al. 2004), whereas inhibition of neurogenesis disrupts memory formation (Shors et al. 2001). On the basis of these and other related data (Anacker and Hen 2017; Frankland et al. 2013) we propose that the adult neurogenesis may support the formation of new systems (see general scheme on Fig. 1.7). Therefore, both the “reserve” cells and newborn cells can be specialized in relation to new systems during learning. The adult neurogenesis may also contribute to the reinstatement of the primary and secondary assortments of neurons in pathology (Xue 1998). This compensation of loss of neurons, including the pre-specialized neurons can possibly also occur in a healthy organism. Since the latter assumption is less grounded, the corresponding relation is marked with a question on Fig. 1.7.

Fig. 1.2
figure 2

Percentage of neurons with different specialization in motor and retrosplenial cortical areas in rats and rabbits. Black—unidentified, grey—“old”, white—“new” (horizontal—feeder-related, vertical—pedal-related). Asterisk: significant differences between percentages of neurons. See text for details

Full size image
Fig. 1.3
figure 3

Firing frequency of neuron #261204-cl6 in consecutive series of behavioral acts on the first side of the chamber before (solid line) and after (dashed) training on the second side. Ordinate: mean ± SEM spike frequency (spikes per second). Abscissa: acts of food-acquisition behavior on first side of the cage (see Fig. 1.1b for definition of behavioral acts; LP—locating in pedal corner before pedal pressing; EF—visiting empty feeder). Significant difference between the two series: Mann-Whitney *p < 0.05; **p < 0.01

Full size image
Fig. 1.4
figure 4

Activity of a rat RSC single neuron specialized in relation to approaching and pressing the first pedal during stable performance of the previously acquired behavior and learning to press the second pedal on the opposite side of the experimental chamber. Recording time: 27 min. See Fig. 1.1b for designation of behavioral acts. (a) Spike raster plot (top) and histogram (bottom) aligned to the end of pedal-pressing (EP) on the side trained first. Neuronal activity during repeated trials before (22 trials, above arrow, black histogram) and after (below arrow, grey histogram) onset of learning to press the second pedal (20 trials, 10 effective trials in a row). Ordinate: ticks of the histogram—tens of spikes in 50-ms bins Abscissa: hundreds of milliseconds Horizontal bars represent spans of act onsets (PP and LH). * significant differences between numbers of spikes within 100 ms bins. (b) Spike raster plot and histogram aligned to the end of pedal-pressing (EP) on the second side. (67 trials). All markers as in panel (a). (c) Dynamics of activity of the same neuron in different acts during acquisition of the pedal-pressing on the second side: mean ± SD spike frequencies in tens of consecutive trials

Full size image
Fig. 1.5
figure 5

Activity of neuron RAT27904–1 in consecutive series of trials after onset of training on the second side of experimental chamber. (a) Mean frequencies in different acts of behavior on the side trained first (left panel) and second (right panel). LP—locating in pedal corner before pedal pressing; see Fig. 1.1b for designation of behavioral acts. The frequencies are normalized to maximum. Lines 1, 2, and 3 represent series of behavior, separated by switches to alternative side. (b) Frequency excess over background in consecutive trials (along abscissa) for specific act LP2 on the second side. The series 2 starts from trial 41. (c) Duration of correponding trials of act LP2 seconds

Full size image
Fig. 1.6
figure 6

Activity of neurons in the rabbits’ anterior cingulate (ACC) and retrosplenial (RSC) cortices in the course of the first (light grey) and the second (black) weeks of pedal-pressing training. Top: Percentage of specialized neurons recorded during the two weeks. Bottom: Spike frequencies of unidentified neurons in ACC and average number of non-specific acts in RSC. The asterisk shows significant differences

Full size image
Fig. 1.7
figure 7

General framework of the systemic organization of behavior: Theoretical schema of the “Change or Die” principle. EG—“early” genes; LG—“late” genes; DG—“death” genes. See Sects. 1.2.1 and 1.3.3 for explanations

Full size image

1.2.2 The Formation of Neuronal Specializations during Individual Development Continues Phylogenesis

The emergence of the nervous system is a “revolutionary” event in the evolution, because it had provided radical increase of complexity and variability of behavior. The complexity of organisms and the genome size do not seem to correlate (Gregory 2001). However, the number of cell types does correspond to the phylogenetic complexity (Bonner 1988). Importantly, it is the nervous system that had contributed the most to this increase. The cell types in the nervous systems are of great, evidently innumerable, variety (DeFelipe 2011). Moreover, the combinations of different cell specializations are individual, because the specialization is formed in relation to the elements of individual experience—the functional systems. Therefore, number of unique sets of specializations equals the number of individuals. In other words, every individual has a unique (although culture-specific) composition of systems. The scope of all possible specialization types depends on the species and the subset of neurons pre-specialized during early ontogeny (the primary assortment). Accordingly, the composition of neuronal specializations (the secondary assortment) is individual. Within the presented view of development as a formation of new specializations the ontogeny appears as phylogeny continued through the increase of the number of cell specialization types.

1.2.3 The Patterns of Neuronal Specializations in Different Species

The research in our laboratory includes recording of neuronal spikes from brains of animals during cyclic operant appetitive behavior (see below in this section). The experimental protocols are in accordance with the Council of the European Communities Directive of November 24, 1986 (86/609 EEC) and the National Institutes of Health “Guidelines for the Care and Use of Animals for Experimental Procedures”, and were approved by the ethics committee of the Institute of Psychology, Russian Academy of Sciences. The specialization of a neuron in relation to a system is assessed via probability of activation in behavioral acts. If this probability reaches 100% in one or more acts, then the neuron is considered specialized, and the activations of a given act are called “specific” activations (see Alexandrov et al. 2013 for more details). An example of specific activations of a specialized neuron is presented in Fig. 1.1a.

According to the framework presented above, the individual reflects interaction with the physical world, rather than the world itself. This reflection depends on the individual goals and history and can be described on the basis of the individual structure of memory (see paragraph 4). Any individual is essentially a composition of both the phylogenic and ontogenic memory. Thus, we have proposed that different species and even different individuals, who acquire new behavior in the same “resultative milieu” (operant food-acquisition behavior), would have memory structure that has similarities and differences, revealed by comparing patterns of neuronal specializations in various brain regions. The similarities would be explained by the identity of achieved results, whereas the differences would reveal the peculiarities of species and the history of learning. The patterns of neuronal specializations are relative numbers of neurons specialized in relation to different systems. Therefore, the pattern reveals a particular” “set” of systems.

We have compared the patterns of neuronal specializations in the homological areas of cingulate and motor cortices of rats and rabbits (retrosplenial cortex, RSC, according to Paxinos and Watson 1997 in rats, and according to Vogt et al. 1986 in rabbits). The animals acquired operant appetitive behavior in a chamber with two pedals that activated corresponding feeders. The chamber viewed from the top was axially symmetric with a pedal and a feeder in adjacent corners of each of the two sides of a square—Fig. 1.1b). The rats’ chamber was 1/3 of the size of the rabbits’ chamber.

The correct performance was a looped movement from pressing a pedal through turning to corresponding feeder facing a wall to eating in the feeder, and turning back to the same pedal (10–15 loops until switch to the opposite side). The behavioral cycle was divided into several acts (Fig. 1.1b): lowering head and taking food from the feeder; lifting head from feeder and turning it toward pedal; moving to pedal corner; pedal pressing; running from pedal to feeder. This division was based on the stages of learning that had been introduced daily during training. The significant increase of the firing rate above background frequency was termed activation. Details of the experimental setup and procedures have been described in more detail elsewhere (e.g. Alexandrov et al. 1990, 2013).

The symmetric arrangement of the chamber allows for classification of neurons according to how discriminatory its firing is in relation to the behavioral acts. For example, the neuron on Fig. 1.1a has activations in each of the consecutive runs to the pedal on the right side of the chamber. Then its classification would depend on specific activations on the left side. If there’s none, the neuron is specialized in relation to a system that subserves approaching the right pedal, which is peculiar for the new behavior (the case for this neuron). However, if there’s activation during approaching the right feeder, the firing might be explained by a left turn of the animal’s head and/or body (verified in additional tests). Then the neuron might belong to a system that subserves (and presumably have subserved) acts of behavior beyond the experimental chamber. Consequently, as opposed to the neurons of the “new” systems, the latter were termed neurons of the “old” systems.

In one of the experiments within this paradigm, activity of single neurons was recorded from RSC and motor (M) cortices in rats and rabbits (see Gavrilov et al. 2002 for more details). In the RSC most of the specialized units were classified as belonging to the systems of “new” behavioral acts, whereas most of the M neurons were of “old” systems (e.g. context-independent activations during any particular movement, or during taking any food or non-eatable objects from the feeder or anywhere in the chamber). Thus, the percentage of units with “new” specializations in RSC was (at least seven times) higher than in the M (Fig. 1.2). The difference between the numbers of neurons with old and new specializations between the two cortical areas was significant and equi-directional in rats and rabbits. The relative old/new systems content in the two cortices shows general similarity of the specialization patterns in the two species.

Meanwhile, the details of specialization patterns differed—presumably, according to the ethological peculiarities of the species. The rats had relatively more neurons specialized in relation to the new acts of taking food from the feeder. We contrasted the numbers of neurons with feeder-related specializations (acts PF and LH on Fig. 1.1b) and those with pedal-related specializations (RP and PP). While these numbers were about the same in rats’ RSC, the rabbits had significantly (some four times) more pedal-related neurons than the feeder-related ones. This difference may be explained by the peculiarities of food taking and manipulation, which is of great variety in the rats (Whishaw et al. 1998). Apparently, the greater number of neurons with new specializations in rats is also due to their more differentiated and complex behavior.

Consequently, the data presented reveal both the task-related similarities and species-derived differences of the neuronal subserving of similar behavior between rats and rabbits in the homological cortical areas. In this experiment the recording of neurons was performed after acquisition of the pedal-pressing behavior, and the specializations were revealed during asymptotic performance. Meanwhile, the investigation of cognitive components necessitates consideration of their emergence. Within our framework, the systems that underlie behaviors appear via systemogenesis, whereas in the conventional terms, new memory undergoes a process of consolidation.

1.2.4 The Traditional View of Memory Consolidation

The processes of the acquisition and consolidation of memory attract the best modern expertise in both the methods and conceptual schemes (e.g. Feld and Born 2017; Kitamura et al. 2017; Moscovitch et al. 2016). However, most of the schemes and investigations are based on the old Descartes’ concept of memory traces: the traces are made of the pores that become more permeable as the spirit repeatedly passes through them during the behavior execution.

The issues that follow this idea are those of the mechanisms and limitations of pore enlargement, the brain structures with different amount of pores, of pore permeability duration, etc. These issues, translated from the seventeenth century to the modern terms (from pores to synapses, from spirits to neuronal firing), maintain their essence under the concept of engram. Unfortunately, the approaches to consolidation, albeit very distinguished (see Dudai 2012 for review), rely mainly on long-term increases of conduction effectiveness in circuits, networks, etc. “The current central dogma of synaptic consolidation is that it involves stimulus (“teacher”)-induced activation of intracellular signaling cascades, resulting in posttranslational modifications, modulation of gene expression, and synthesis of gene products that alter synaptic efficacy” (Dudai 2012, p. 228).

1.2.5 The Systems View of Memory Consolidation

From the systems point of view, the neuron is a result achiever, rather than a conductor of excitation. Therefore, the issue of the conduction efficiency increase is out of the scope of the systems approach. The learning process is considered as formation of a new system of co-active cells (including neurons) of different localization, not necessarily directly connected. This view excludes the concept of “trace” left solely by the instructive input due to plasticity of the nervous system.

The systems view of consolidation was formed on the basis of the systems approach (above). However, the experimental evidence leads other authors to similar conclusions. For example, G. Horn claims that the cross-correlational analysis of neuronal activity in IMHV of domestic chicks does not confirm that the connectivity of “imprint-responsive” neurons is increased during learning, as predicted by the Hebbian rule. “Rather, – the author concludes, – the neurons might form a set of parallel, largely uncoupled elements that are likely to provide a larger storage capacity than a system with tightly coupled elements” (Horn 2004, p. 121). Although functional connectivity may indeed increase after learning (Abdou et al., 2018), we believe that the connectivity affords synchronization of metabolic activity between structurally connected neurons (see Sect. 1.1.3), and G. Horn’s conclusion remains accurate for the rest of neurons of the same specialization with no direct connections, and even more so for the somatic cells. Different kinds of network approach in the analysis of synaptic (Hoshiba et al. 2017), cellular (Adams et al. 2017) and whole-brain (Lohmann et al. 2016) processes share some aspects of the systems approach, albeit they largely retain instruction-based view on learning, and hence the issue of a unit-of-analysis (Korhonen et al. 2017).

The systems description of the consolidation process necessarily includes two groups of interdependent processes: the systemic specialization, and accommodative reconsolidation. The former applies to the morphological and functional modifications of a neuron that provide its involvement into a new system (described above). The definition of the latter process requires several preliminary considerations.

Of importance is that a new memory is dynamic and adaptive, rather than a stable entity (Bartlett 1995). The recent progress of memory reconsolidation research shows the modification of memory after post-consolidation retrieval at the molecular level (Nader 2015; Sara 2000). Memory formation and reactivation require protein synthesis, although the consolidation and reconsolidation processes are not identical (Anokhin et al. 2002; Dudai and Eisenberg 2004). Therefore, the protein synthesis-dependent consolidation reveals a wide range of “active” memory processes (Nader 2003, 2015), rather than just those of “new” memory.

The idea of reconsolidation does not contradict to the above notion of permanent specialization. The reconsolidation does not rule out the changes that had underlied the long term memory formation (Nader et al. 2000). However, it does constitute another, supposively less influential, step of differentiation process for a neuron.

We consider learning as specialization of a group of neurons in relation to a new system. The new system is not a substitution, but an addition to the previously formed systems. It follows that this addition would necessitate the coordination between new and prior elements. Current scope of evidence on reconsolidation shows that reconsolidation may indeed be the general mechanism of prior memory reorganization after new learning (see Dudai et al. 2015; Hupbach et al. 2008; McKenzie and Eichenbaum 2011).

We have suggested earlier that the neurons that are specialized in relation to a system of one behavioral act may modify their activity and be involved in another behavior without changing the specialization (see Alexandrov 2008). Later, the acute (Alexandrov et al. 2001) and chronic tetrode (see Alexandrov 2008) recordings provided more evidence for reorganization of an existing system upon acquisition of a new behavioral act. Namely, the chronic recording of neuronal activity was made during acquisition of the appetitive operant behavior, described in Sect. 1.2.3. When the animals reached an asymptote level of pedal-pressing on the first side of the chamber, the pedal was turned off to start training on the second side. Upon reaching the same criterion there, the animals were returned to the first side. Consequently, the sides were alternated 10–20 cycles of pedal-pressing each. Thus, activity of several neurons was tracked on the first side of the chamber before and after training on the second one.

Three of these neurons with activations specific to acts on the first side changed their activity patterns after initial training series on the second side. Activity of one of these cells is shown in Fig. 1.3 (the firing frequency of this cell changed significantly in several acts, including the specific act LH (see Sect. 1.2.3); see also panel A on Fig. 1.4 for learning-induced activation changes in a neuron with activations specific to preceding behavioral acts). Notably, these changes remained significant in all subsequent series unlike temporary changes of activity of specialized neurons in the first trials of specific acts after alternation or rest periods. The modifications of this kind were termed by us “accommodative” reconsolidation (Alexandrov et al. 2001).

Results that point to reorganization of previously formed system after acquisition of a new one were also received by us via immediate early gene (IEG) expression analysis (Svarnik et al. 2013). This study was designed to control for learning prior to operant food-acquisition by pedal-pressing to reveal activation of the first-skill-specific neurons during acquisition of the second one. In the experimental group of rats the first skill was a “whisking task”—that of using left or right whiskers to receive a water drop. These animals were overtrained for 5 days before the second skill of pedal pressing for food had been introduced. The control group acquired the same food-acquisition behavior, but the first task was a non-instrumental drinking instead of the “whisking”. Albeit the second skill did not involve the whiskers, we have found c-Fos expression in significantly greater number of barrel-field neurons in animals of the experimental group compared to the control. These data may suggest that c-Fos induction during the second training took place in neurons that were specialized in relation to the first, “whisking” task, which is a sign of accommodative reconsolidation. Therefore, besides the specialization of neurons in relation to new systems, we consider morphological and functional modifications of previously specialized neurons. These modifications do not change the specialization and provide inclusion of a new system into the existing structure of individual experience.

It had been previously proposed within the cognitive theories that the memory reorganization may be either routine (reordering the interactions of existing schemas), or heuristic (emergence of new components along with the modification of the prior ones) (Piaget 1951). The specialization and accommodative reconsolidation processes refer to the second type of the reorganization. As far as the first type, the modifications of neurons that belong to existing systems without emergence of a new system may be referred to as “reorganizational” reconsolidation. Presumably, a gradual increment of efficacy may be one of the manifestations of the latter, whereas the former would be signified by curt transition to good performance—like the one we see during our food-acquisition training.

We consider the difference between the specialization and accommodative reconsolidation processes as essential for investigation of underpinnings of learning. Ignored in most of the studies, these processes may be indistinguishable in the data on molecular and cellular learning-related processes. The differentiation of the processes that manifest emergence of new experience from those of prior experience modification is necessary in the contemporary research of memory principles.

In the systems perspective that we develop, learning is the key process under investigation, as it covers the most essential changes of individual experience—the emergence of new systems and modifications within existing ones—and presumes that memory is active and dynamic. Therefore, we next present our view of fundamental processes that underlie or accompany learning: the mismatch between “needs” and recent environmental “input” in the whole organism as well as in neurons, “altruistic suicide”, and long-term potentiation.

1.3 Fundamentals of Learning within the Systems Perspective

1.3.1 Memory Formation Starts with Mismatch

As we mentioned earlier the initial step of the cascade of subcellular molecular events that determine the morphological modifications of neurons, both in the process of morphogenesis (early ontogenesis), and in memory consolidation in adults, is the expression of immediate early genes, followed by the expression of “late “genes that might be directly related to the structural modifications of a neuron. These days the relations between IEG expression and learning, noted a while ago (Maleeva et al. 1989; Tischmeyer et al. 1990; Anokhin and Rose 1991), have become widely accepted (e.g., Horn 2004; Kubik et al. 2007; Barry and Commins 2011; Minatohara et al. 2016).

In the framework of systems neuroscience it seems a logical assumption that the expression of IEGs and the formation of neuronal specializations are related. Indeed, we showed earlier that those structures that contained a lot of neurons specialized in relation to operant behavior also demonstrated a higher number of Fos-positive neurons after learning (Svarnik et al. 2005).

Induction of IEG expression in the adult takes place not only during learning, but also during stress, intoxication, lesions of the nervous system, brain ischemia and other conditions (Herrera and Robertson 1996; Meyer 2015). It was also shown that an artificial change in the microenvironment of neurons causes the appearance of activity in previously silent cells and the expression of IEGs (Stone et al. 1993). Given that activity at neuron is determined by mismatch between neuron’s needs and the current influx of metabolites (as discussed above), IEG expression—a specific manifestation of cellular activity (Clayton 2000) arising in a situation of novelty (Anokhin and Sudakov 2003; Aggleton et al. 2012)—is suggested to be evident in a general bottom line of all these situations i.e.—during the mismatch.

The mismatch arises due to the fact that the previous possibilities of meeting the metabolic neuronal “needs” within the existing memory structure turned out to be ineffective in the condition of a stable change in microenvironment of neurons. The latter occurs upon change in the circumstances of corresponding behavior. Neuron, as noted above, may provide “needs” of its metabolism by combining with other elements of the organism and forming the functional system. Achieving the result of the system simultaneously eliminates the mismatch between “needs” and the state of the microenvironment of neurons, and provides the desired result for the organism on the behavioral level. This may happen only when the corresponding behavior has already been formed. However, learning in normal conditions and recovery in pathology (for example, after a stroke, traumatic brain injury) occur when the “needs” cannot be conformed with existing matching methods of the individual (i.e., within the available individual experience). The mismatch in this situation is different from that in the definitive behavior: it is eliminated not by reactivating existing memory, but by systemogenesis, i.e. selection and fixation of new elements and variants of combining them (see Sect. 1.2.1).

1.3.2 From Mismatch through Match to Consolidation

The emergence of a new system (systemogenesis) may lead both to achievement of the desired result for the organism, and to satisfaction of metabolic “needs” of neurons. However, the new integration is not constant. It was shown that the activity of the human brain changes not only in the process of learning, but also during hours (and days) after learning criteria achievement (e.g. Karni et al. 1995). It was also shown in animal experiments that the parameters of neuronal activations, as well as number of activated cells change within hours and days from the first successful behavioral trial (Erickson and Desimone 1999; Kuzina et al. 2016; Horn 2004; McKenzie and Eichenbaum 2011; Smith et al. 2012; etc).

Our results (Svarnik et al. 2005) show that the number of neurons in which the IEG expression is detected exceeds in many times the number of neurons in this area specialized in relation to the system of the formed behavior. We believe that part of these genetically activated cells are neurons specialized in the relation to the earlier formed systems, and the IEG expression in those cells reflects the beginning of a process of accommodative reconsolidation (see above). Others are pre-specialized neurons, and their gene expression induction is a prerequisite for the transition of cells into a state of readiness for selection during the trials.

As we hypothesized, it is in the trial-and-error process that certain neurons are selected from activated ones (activated both genetically and, presumably, electrophysiologically) and become specialized in relation to the formed system. Decrease in the number of activations as well as in heterogeneity of activity of specialized neurons in the course of memory consolidation demonstrated by us earlier (Kuzina et al. 2016) reflects this selection process and changes in neuronal subserving of new behavior. We compared neuronal activity in rat RSC recorded during the pedal-pressing (see Sect. 1.2.3) within either first five days (group 1), or from days 7 to 15 after its acquisition (group 2), which corresponds to the “early” and “later” stages of consolidation in rodents (e.g. Buitrago et al. 2004). The second group of animals was kept in the homecage during the first week after acquisition. There were significantly more neurons specialized in relation to new behavioral acts in group 1 that had specific activations during both approaching and pressing the pedal, i.e. acts that were acquired just before the start of recordings. In contrast, most neurons of the “pedal” category of group 2 were specifically active only in one of these acts: either approaching, or pressing the pedal. Within the first 5 days (in group 1) there were significantly more pedal-specific neurons with highly differentiated activity in other acts than in group 2. On the other hand, the enhanced selectivity of individual neurons in group 1 was accompanied by more variable activity in acts associated with a “feeder” part of the behavioral cycle. Activity of “pedal” neurons in group 2 was consistently reduced in “feeder” acts. Apparently, the reduction in the variability of activity may be associated with the completion of the selection process and stabilization of neuronal population involved in the newly formed behavior. It is possible that such stabilization requires not only time, but also a certain number of repetitions of experience reactivation (Weible et al. 2009, 2012; McKenzie et al. 2013).

It has been shown that some cells are activated only during the initial stages of learning, and when behavior is stabilized, their activations decrease and disappear (Shima et al. 1996; Wirth et al. 2003). In our view, some of these cells are likely to be pre-specialized neurons activated during trials. In the case of training for behavioral acts similar to previously formed ones (e.g., pedal-pressing on the second side in our setup) activity of neurons during pressing the first pedal may look like variable nonspecific activity, presumably reflecting the process of specialization (ref. to Fig. 13 in Alexandrov 2008). In addition, as our data show, neurons specialized in relation to previously formed behavior may be active during formation of a new one. For example, activity of a neuron on Fig. 1.4 was recorded during the following periods of the experiment: pressing pedal 1 before pedal 2 training (Fig. 1.4a, above arrow, black histogram), acquisition of pedal 2 pressing (Fig. 1.4b), and pedal 1 pressing after acquisition of both (Fig. 1.4a, below arrow, grey histogram). Spike frequencies in certain behavioral acts (TP, PF, and LH, see Fig. 1.1b) had significantly decreased after acquisition of the pedal-pressing on the second side. The activity of this neuron during the specific acts (RP and PP) on the first side had also changed: activations started and ended earlier after acquisition of the pedal-pressing on the second side. Additionally, activity of neurons during pedal 2 training and the following pedal 1 pressing might be considered as a neuronal basis of learning transfer: it accompanies speeded up learning after previous similar experience. Accordingly, mean spike frequencies were significantly higher along the whole period of learning in PP, than in any other acts including RP (Fig. 1.4b, c). Also, there was a significant decrease of activity in all acts, except PP, from the beginning to the end of learning to press pedal on the second side (Fig. 1.4c).

It might be also suggested that the first trials during learning in organisms with highly developed nervous system are subserved by co-activation of not only changing sets of specialized and pre-specialized neurons but also so called “novelty” neurons possibly specialized in relation to orienting behavior (for further details see Ranganath and Rainer 2003; Aleksandrov 2006). This co-activation may provide trial performance as well as achievements of the first positive results during learning. After stabilization of the behavior “novelty” neurons as well as a number of other previously specialized neurons cease their activity. Therefore, the decline of the activity of previously specialized and “novelty” neurons corresponds to the consolidation process and signifies serious reorganization of neuronal supply of memory reactivation. Meanwhile, the specific activation of specialized neurons may be evident from the very first implementation of the new behavior.

The chronic tetrode recordings described above (Sect. 1.2.5) have also revealed neurons specialized in relation to the behavior on the second side. Activation of these neurons that satisfied the criterion of specific activity (see Sect. 1.2.3) was indeed evident since the first relevant behavioral act acquired during learning. An example of such neuronal activity is shown in Fig. 1.5. Smith et al. (2012) also found the phenomenon of emergence of supposedly specific activity in neurons during the first trials of new behavior. However, in this study, animals were not pre-trained (except for familiarization with new environment), as in our experiments, where they were pre-trained to press the first pedal before they learned to press the second one. Perhaps this difference is one of the reasons why the activation (in the place field of a neuron) during the first implementations of behavior were evident in the hippocampus, but not in the RSC.

Earlier (Alexandrov et al. 1991) we showed that under acute ethanol treatment as compared to control the percentages of neurons of different specializations were not changed in the motor cortex. Meanwhile, the neuronal population is different under ethanol: the upper layer neurons are mostly excluded, and the lower layer neurons become more included into the neuronal population that subserves the behavior. Thus, at the early stages of learning the processes of neuronal specialization may proceed differently in different brain structures.

In studies of brain activity reorganization during learning changing roles of brain structures have been repeatedly demonstrated at various stages of training (Rose 1993; Kelly and Garavan 2005). It is known that learning scores, memory, and “cognitive control” depend on the intact cingulate cortical regions of the human brain (Hayden et al. 2010). Furthermore, outwardly the same behavior is accompanied by activation of different areas or layers of the cingulate cortex as learning progresses. On the one hand, by both methods of functional anatomy in humans (Tracy et al. 2003) and multiunit activity recording in animals (Freeman and Gabriel 1999) it was shown that activation of the anterior regions of the cingulate cortex declines and activation of the posterior regions increases in the process of learning. On the other hand, the posterior cingulate cortex is activated during aversive behavior (and also needed for its implementation) both at early and late stages of learning (Gabriel et al. 1991; Katche et al. 2013). It was also shown that the anterior cingulate cortex is involved in the “context-freezing” task at both early and late stages of learning and necessary for reconsolidation of this memory (Vetere et al. 2011; Einarsson and Nader 2012). If the specialization of the neuron, as we noted above, is constant (i.e. neuronal differentiation is irreversible—Sect. 1.2.1), and evident from the first implementations of the newly formed acts (above), it might be assumed that the most significant contribution into the described reorganization is made not by the dynamics of the activity of the specialized neurons but other (“unidentified”) neurons.

In our experiment we recorded activity of the neurons in anterior and posterior areas of rabbit cingulate cortices at “early” (the first week of learning) and “late” (the second week of learning) stages of training of pedal-pressing. We analyzed average frequency of spikes, percentage of specialized and unidentified neurons, and the number of behavioral acts with non-specific activations (i.e. acts with probability of activation between 40% and 100% in unidentified neurons, see Sozinov et al. 2012). We found that all these variables for specialized neurons did not differ between the first and the second week of training (Fig. 1.6, top), whereas the average frequencies and the number of acts with non-specific activations were different for neurons with unidentified specializations (Fig. 1.6, bottom). Therefore, neuronal activity turned out to differ between the first and the second week of training of food-acquisition skill. These differences were primarily indicators of activity of unidentified neurons, rather than specialized neurons.

On the basis of these results it might be possible to propose that in unidentified neurons of the anterior cingulate cortex the frequency decreases from the first to the second week of training, but in such neurons of the posterior cingulate cortex the number of activations in new behavioral acts increases during the same period. In other words, the dynamics of brain activations is less accounted for newly specialized neurons, but is due to activity of those neurons whose specializations are not identified. As we argued earlier (Alexandrov et al. 1993) unidentified neurons are probably specialized in relation to systems of other behavioral acts than those formed in our training. That is why the established differences in characteristics of neuronal activity at successive learning stages might be connected to the processes of reorganization of that experience which served as a basis for newly formed behavior, rather than to changes in cohort of specialized neurons.

These data support mentioned above reasoning that it is necessary to differentiate characteristics of new experience formation and old experience reorganization (Alexandrov et al. 2001; Grosmark and Buzsáki 2016; McKenzie and Eichenbaum 2011). However, saying about higher manifestation of reconsolidative changes we should take into account possibility of maintenance of the stable percentage of differently specialized neurons and, at the same time, changes in neurons of other specializations.

Treating a neuron as an active living organism has additional consequences for several well-known phenomena. Among them are “altruistic suicide” and long-term potentiation covered in the remainder of this paragraph.

1.3.3 “Altruistic Suicide”

As it was mentioned above IEG expression is induced when the organism does not have experience of satisfaction of metabolic “needs” of its cells in some situation, or when repetitive impulses of co-activated neurons do not lead to the result achievement (goal achievement). The IEG expression might be considered not only as the first step of consolidation process, but also as induction of other transcriptional factors underlying cell’s “decision to live or die” (Lee et al. 1998, p. 2736). If the mismatch between “needs” of neurons and their microenvironment is prolonged, neurons become hyperactive, and waves of IEG expression repeat. In such cases “death” gene expression might be induced, which will result in neuronal death—apoptosis (see Fig. 1.7). Thus when the mismatch between “needs” of the neuron and its microenvironment cannot be eliminated in the conditions of existing experience the neuron has two alternatives: to be changed during systemogenesis (new system formation) or to die (Fig. 1.7). These two alternatives exist both in normal conditions (during early ontogenesis and adulthood) and in pathological conditions. The involvement into systemogenesis might be either system specialization process or the process of accommodative (reorganizational) reconsolidation. Cell death is often observed during early development and under pathology, when existing experience of the organism is inapplicable for agreement among metabolisms of different cells of the organism. But this is true not only for such cases. There are data showing that apoptosis is evident in brains of healthy adults and is necessary for normal functioning of animals (e.g. Leist and Jäättelä 2001). Since systemogenesis is a general principle for early development and learning at any age, adaptation and recovery, the discussed data allow concluding that “change or die” options exist in normal conditions. It was shown (Sherstnev et al. 2013) that elimination of neurons (observed as neuronal selection in early ontogenesis important for behavioral repertoire formation at that stage) also contributes to the process of systemogenesis during adulthood (Fig. 1.7). Thus the formulated position states that there are no two alternatives (“systemogenesis or death”) but two interconnected roads to systemogenesis: modification of a neuron or its death. It might be suggested that death of some cells is a necessary payment for a possibility of successful systemogenesis during individual ontogenesis in all those situations when metabolic needs of some cells are in an unavoidable conflict with new means of agreement among cellular needs. The activity principle is applicable for all periods and aspects of existence of a neuron including the processes connected to the “change or die” alternative. Each stage of cellular elimination is an active process (Raoul et al. 2000), and thus neuronal elimination is a suicide (Leist and Jäättelä 2001). This suicide is altruistic in a sense that the neuron turns on the program of self-elimination in order to abolish metabolic conflict and provide survival of other neurons that belong to the same cellular clone. Earlier other authors have already argued for the existence of “altruistic cellular suicide” in the nervous system (Allsopp and Fazakerley 2000) and in unicellular organisms (Strassmann et al. 2000).

1.3.4 Long-Term Potentiation: Traditional and Systems Approach

Above we described the system approach to learning and memory processes. If someone wanted to argue for alternative traditional framework of learning mechanisms, she (he) would probably refer to the phenomenon of long-term potentiation (LTP), which is considered to be a physiological mechanism of long-term memory and regarded as an experimental model of activity-dependent plasticity. Studies of LTP have for many years been seen as the most important and urgent approach mainly because this phenomenon is well demonstrated in the framework describing the formation of memory as an increase in synaptic efficiency of impulse conductance in neuronal networks. Within the systems approach, LTP can be regarded as an electrophysiological description of the mismatch (see Sect. 1.3.1). If we consider neuronal activity as determined by the mismatch we may conclude that an artificial electrical (or chemical) stimulation used to elicit an influx not accordant with the neuron’s preceding activity and not caused by it serves as a powerful mismatch factor. And the increased cell excitability persistently found during testing is a reflection of this mismatch. Not only theoretical framework but also experimental data argue for the link between LTP and the mismatch process, among them the data that show similarity between LTP and the processes that take place during pathological conditions, when metabolic cellular environment is strongly changed (McEachern and Shaw 1996; Vikman et al. 2003). Thus, although experimenters using tetanization do not intend to induce the mismatch, they do. And the mismatch is, as argued above, the initial stage of learning and the formation of a new memory. Therefore, we do consider LTP as a phenomenon that may be related to mechanisms of learning and memory but for different reasons: because it models the initial stage of learning—the mismatch. However, it is not known whether the mismatch obtained during the experimental induction of LTP has the properties of characteristics of natural mismatch during learning. Note that the discrepancy between the traditional concept of LTP and data accumulated from studies of this phenomenon requires an alternative explanation even for those authors who have no doubt that the increase in synaptic efficiency between neurons provides the basis for the formation of memories. McEachern and Shaw (1996) believe that the mechanisms of receptor regulation allow neurons an attempt to prevent long-term changes in their synaptic excitability, which is harmful for neurons. LTP (like depression), acting against this regulation, is not the basis of learning but is an initial manifestation of the cascade of processes leading to the reorganization of the activity of a neuronal group, which “strives” to maintain homeostasis. Shors and Matzel (1997) also came to the conclusion that there is a non-correspondence between the properties of LTP, particularly its duration, and those required if LTP is to support the retention of long-term memory. They believe that LTP is a mechanism related not to maintenance of long-term memory but to the initial stage of its formation. As we see the presented conclusions are in accordance with the suggestion that the mismatch is the initial stage of systemogenesis, and that LTP is an electrophysiological description of the mismatch process. If we consider this suggestion about LTP, we may conclude that although the duration of LTP is insufficient for it to be regarded as the basis of long-term memory, it may be adequate for it to be regarded as an electrophysiological manifestation of prolonged mismatch leading to cell death. Put more simply, the logic of the ideas proposed here suggests a link between LTP and neuronal death. There are some data showing this connection (McEachern and Shaw 1996; Manahan-Vaughan et al. 1999; Ambrogini et al. 2004). Thus, within the systems approach the phenomenon of LTP might be related to the mechanisms of learning and memory, but not because it models increased effectiveness of impulse transmission in neuronal networks, but because it models the mismatch process, which is a characteristic of initial stages of learning.

1.4 The History of Memory Formation and the Memory Structure Are Interrelated

We showed earlier that any behavior is subserved by activation of not only new systems formed during learning but also older systems activation formed at earlier stages of individual ontogenesis (see Alexandrov 2008; Alexandrov et al. 2000). Thus, behavior is reflection of history of its formation (phylogenetic history as well as ontogenetic), i.e. realization of multiple systems, each of which fixates a stage of behavior acquisition. It might be suggested from this statement that system organization of overt similar actions differs if the history of their formation differs. We showed earlier for complex operant behavior in rabbits that neuronal activity in the cingulate cortex differs significantly when rabbits learned the same behavioral acts but in different order (Gorkin and Shevchenko 1996).

In other experiments we checked the hypothesis about connection between activity of recent-task-related system-specialized neurons and the number of stages of learning. The following logic underlied this hypothesis. We showed earlier that in different brain structures of rabbits there were neurons activated during different acts of the acquired (on a daily basis) instrumental behavior: approach to the feeder, turn from the feeder to the pedal, approach to the pedal, the pedal pressing. Since all these acts constituted the sequence during performance, this behavior is accomplished due to reactivation of all systems of these acts, and, hence, due to activation of the system-specialized neurons. Thus, on the basis of the transformation of learning stages into systems of the learned behavioral acts we could propose that if the number of learning stages differ, then organization of neuronal activity during overtly identical behavior differ also. In order to check this assumption we compared neuronal activity in the RSC after rats learned the instrumental behavior within one stage (only pedal pressing was reinforced) with the neuronal activity during the same behavior but when rats learned it in several stages (each stage of instrumental behavior described above was reinforced). We found that the number of newly specialized neurons did not differ between the two cases. However, aggregated activations of all new system-specialized neurons in RP and PP were higher in the case of multiple stages learning, than in the case of one-stage learning. In the latter group we found more neurons with specific activations during turning to the pedal and higher spike frequency of specialized neurons in all acts. These data suggest that there is a connection between activities of the neurons specialized in relation to newly acquired behavior and the number of stages used to acquire this behavior. Thus, in different species we find consistent general principle—the organization of neuronal activity during behavior depends on the history of its acquisition.

This principle is also manifested at the molecular level. We showed that the number of neurons that changed their gene expression (detected by Fos expression) during new behavior learning depended on the number of acquisition stages of the previous training (Svarnik et al. 2013). In these experiments we trained animals to press a pedal on one side of the experimental cage in one or several stages, and then compared the number of Fos-positive neurons after the acquisition of the same skill on the other side of the cage. Despite the fact that the overt behavior during the second acquisition was similar in these two cases, the number of neurons with changes in gene expression was significantly different. It might be suggested that such changes depended on the processes of accommodative reconsolidation described above. Thus, learning involves not only acquisition of new information but also assimilation of this information into earlier established experience structure or schemas (see also Tse et al. 2011).

1.5 Conclusion

We put forward the following sequence of memory formation and functioning, brought together on Fig. 1.8. Learning starts from the mismatch between individual needs and possibilities to meet them using acquired memory. It is manifested on a cellular level as a mismatch between metabolic cellular “needs” and recent metabolic input. In a familiar situation this mismatch might be cleared up by performance of previously formed behavior: Fig. 1.8a shows memory reactivation during behavior acquired earlier. Memory reactivation might be connected to changes of experience structure due to “reactivational reconsolidation”. This type of reconsolidation may appear as continuous memory formation.

Fig. 1.8
figure 8

General framework of the systemic organization of behavior: Types and stages of modification of the individual experience structure. See Fig. 1.7 for abbreviations and Conclusion for explanations

Full size image

In many experimental models of reconsolidation the acquisition is followed by presentation of a reminder—namely, by additional training (see, e.g. Davis et al. 2010). Therefore, Fig. 1.8b shows modification of individual experience structure in a new situation limited by reorganization of previously formed systems. In this situation new element of experience (a new system) is not formed. This type of modification occurs due to “reorganizational reconsolidation”.

The type of modification on Fig. 1.8c is the one where the mismatch can neither be eliminated by reactivation of existing memory (as in A), nor by reorganization of earlier formed systems and intersystems relations (as in B). Then the mismatch is eliminated by formation of a new system. This process involves several stages: C1—early gene expression (EG), which is manifested at early stages of acquisition in pre-specialized neurons (circles), as well as in neurons that belong to the systems of prior individual experience; C2—selection during trial behavior: among activated pre-specialized neurons (that appeared also during adult neurogenesis) a necessary combination (darker circles) is selected; C3—during the process of selection a neuron has a choice of being changed and involved into a new system consolidated later through late gene expression (LG), or die (crossed circles; “death” gene expression, DG); C4—accommodative modification of neurons, specialized in relation to earlier formed systems (the rectangle), determined by inclusion of a new system into the structure of individual experience.

The formation of new integration, preceded by “internal” testing and hypothesis selection, is also manifested in trial behavior. At the cellular level this trial behavior is based on testing combinations of activated neurons; successful combinations provide result achievement and elimination of mismatch (Fig. 1.8, C2). The success is accomplished by modifications of some cells and elimination of others (Fig. 1.8, C3). As the first results are achieved, the cells presumably pre-specialized in relation to searching behavior, as well as other cells that belonged to earlier formed systems, gradually decrease and eliminate their activity. It is probably manifested in temporal changes in overt behavior (that seems as already formed) and in changes of the set of activated cells. Gradual stabilization of the set of activated cells may be manifested in more stable relations between neuronal activations and behavior. Late gene expression provides reorganization of selected neurons and their transition to being constantly specialized in relation to a newly formed system. This system modifies earlier specialized neurons during the process of accommodative reconsolidation (Fig. 1.8, C4). Thus, stability of neuronal specialization in a sense of belonging to certain system does not mean that formed memory is unchangeable. Some of the processes proposed by the schema on Fig. 1.8 remain hypothetical (including the modification of intersystem relations), but we consider this as a consistent framework that provides testable propositions.