A Modeling Environment for Reified Temporal-Causal Networks: Modeling Plasticity and Metaplasticity in Cognitive Agent Models

Abstract

Plasticity is a crucial adaptive characteristic of the brain. Relatively recently mechanisms have been found showing that plasticity itself is controlled by what is called metaplasticity. In this paper a modeling environment is introduced to develop and simulate reified temporal-causal network models that can be applied for cognitive agent models. It is shown how this environment is a useful tool to model plasticity combined with metaplasticity.

1 Introduction

Real-world cognitive agents are often adaptive, described by adaptation principles. For example, mental or neural networks equipped with a Hebbian learning mechanism [5] are able to adapt connection weights over time and learn in this way. This is usually called plasticity (modeled by the middle layer in the example in Fig. 1). In some circumstances it is better to learn fast, but in other circumstances it is better to stay stable and persist what has been learnt in the past. To control this, a type of (higher-order) adaptation called metaplasticity is used (highest layer in Fig. 1); e.g., [1, 6].

Fig. 1.

Overview of an example reified network architecture addressing plasticity and metaplasticity for a cognitive agent model, with: (1) base level (lower plane, pink), (2) first reification level (middle plane, blue) for plasticity of the weight ω of the base connection from srs_s to ps_a and the excitability thresholds τ of these two base states (by the W state and the two T states), and (3) second reification level (upper plane, purple) for metaplasticity for the first-order adaptation speed η and the persistence μ (by the H state and M state). The upward causal connections (blue) and downward causal connections (red) define the interlevel relations. For more explanation of this example network, see [13]. (Color figure online)

In [8, 11] any form of adaptation had to be added by specific procedural program code like usually is done for adaptive networks; there was no standard or principled way to explicitly specify adaptive causal relations. To offer a more principled way to specify adaptive networks, recently the notion of network reification was proposed as an addition to the temporal-causal network modeling approach, and illustrated by some case studies that were implemented in a more or less ad hoc - proof of concept – manner [9, 10]. These initial explorations suggest that this notion of network reification could be useful to model in a systematic and transparent manner from a network-oriented perspective, cognitive and social agent processes that are adaptive of any order, and in particular those involving plasticity and metaplasticity (e.g., see Fig. 1).

Following this, the current paper introduces a specification format based on declarative mathematical relations and a modeling environment for reified temporal-causal networks, implemented in Matlab in a principled and structure-preserving manner. Due to this dedicated overall Network-Oriented Modelling approach for adaptive networks, no procedural, algorithmic or programming skills are needed to design cognitive agents or social networks which show complex adaptive behaviour of any order.

In the paper, in Sect. 2 the reified temporal-causal network architecture is explained in some detail. After this, more details are described of the specification format (Sect. 3) and the implemented modeling environment and its computational reified network engine (Sect. 4) developed. Finally, Sect. 5 is a discussion.

2 Modeling Adaptive Processes by Reified Networks

A conceptual representation of the network structure of a temporal-causal network model involves three main characteristics of the network structure; see [8], Chapter 2, or [11]. First, for the connectivity characteristics of the network, connection weights ω_X,Y are used as a labels for connections from X to Y. Second, for the aggregation characteristics of a network, for each state Y a combination function c_Y(..) is used to aggregate (and modulate) causal impacts on state Y; they can contain parameters p. Third, for the timing characteristics of a network, for each state Y a speed factor η_Y is used for timing of the causal effects. The difference equations used for simulation and mathematical analysis incorporate these three types of network characteristics ω_X,Y, c_Y(..), η_Y:

$Y(t+{\Delta{\textit{t}}}) = Y{\textit{(t)}} + {\boldsymbol{\upeta}}_{Y} [{\mathbf{c}}_{Y}({\bf{\upomega}}_{{X_{1}},Y}X_{1}{\textit{(t)}}, {\ldots}, {\bf{\upomega}}_{{X_{k}},Y}X_{k}{\textit{(t)}}) - Y{\textit{(t)}}] {\Delta{\textit{t}}}$

Here X₁, …, X_k are the states from which state Y gets its incoming connections. For aggregation a library with a number (currently 35) of standard combination functions are available as options, but also own-defined functions can be added.

Modeling adaptive networks asks for a dedicated network architecture in which different levels of adaptivity or plasticity can be modeled. Such an architecture has been proposed based on the notion of network reification [9, 10]. Reification (e.g., [4]), in general means making an abstract notion concrete. For network models this is done by introducing additional states in the network that explicitly represent characteristics of the network such as connectivity, aggregation, and timing, and makes them adaptive:

• Adaptation of a connection weight ω_X,Y: reified connection weight representations W_X,Y

• Adaptation of a speed factor η_Y: reified speed factor representations Η_Y

• Adaptation of a combination function c_Y(..): reified combination function weight representations C_i,Y (for the i^th combination function used)

• Adaptation of a combination function parameter p_Y: reified combination function parameter representations P_i,j,Y (for the j^th parameter of the i^th combination function for Y)

In a graphical representation in a 3D format these new states are depicted in a second plane above the plane for the base network; see the blue plane in the example model depicted in Fig. 1, also indicated as the first reification level. This step can be repeated so that a third plane is added for second-order reification (see the purple third plane in Fig. 1). Three types of causal connections are distinguished: upward causal connections, downward causal connections and leveled (horizontal) causal connections. The downward causal connections have their own fixed role and meaning in the sense that they are causally effectuating one of the four types of adaptations listed above.

Combination functions are built as a weighted average from a number of basic combination functions bcf_i(..) available in a library; these weights can be prespecified as constant values or can be adaptive based on reification states. Examples of basic combination functions often used are the euclidean combination function ${\mathbf{eucl}}_{{\varvec{n},{\varvec{\uplambda}}}} \left( \ldots \right)$ with order n > 0 and scaling factor λ > 0 (generalising the linear scaled sum function for n = 1) and the advanced logistic sum combination function ${\mathbf{alogistic}}_{{{{\upsigma,\uptau }}}} \left( \ldots \right)$ with steepness parameter σ > 0 and excitability threshold parameter τ:

$$ {\mathbf{eucl}}_{{\varvec{n},{\varvec{\uplambda}}}} \left( {V_{1} , \ldots ,V_{k} } \right) = \sqrt[\varvec{n}]{{\frac{{V_{1}^{\varvec{n}} + \ldots + V_{k}^{\varvec{n}} }}{{\varvec{\uplambda}}}}}\varvec{ } $$

$$ {\mathbf{alogistic}}_{\upsigma,\uptau } \left( {V_{1} , \ldots ,V_{k} } \right) = \left[ {\frac{1}{{\mathop {1 + {\text{e}}}\nolimits^{ - \upsigma } \left( {V_{1} + \ldots + V_{k} -\uptau} \right)}} - \frac{1}{{1 + \mathop {\text{e}}\nolimits^{\upsigma \uptau } )}}} \right]( 1+ {\text{e}}^{{ -\upsigma \uptau }} ) $$

Here the $V_{i}$ denote the single impacts ${\varvec{\upomega}}_{{X_{i} ,Y}} X_{i} \left( t \right)$ on state Y for each of the incoming connections from states X₁, …, X_k. Moreover, for Hebbian learning (‘neurons that fire together, wire together’), among others the following combination function is available (used for the reification state W_X,Y in the middle layer in Fig. 1):

$$ {\mathbf{hebb}}_{{\varvec{\upmu}}} \left( {V_{1} , V_{2} , W} \right) = V_{1} V_{2} \left( {1 - W} \right) + {\varvec{\upmu}} W $$

where $V_{1} , V_{2}$ indicate the single impacts from the connected states (base states at the bottom layer in Fig. 1) and $W$ the connection weight (represented by reification state W_X,Y in the middle layer in Fig. 1), and μ is a persistence parameter. In Fig. 1:

• W_X,Y plays the role of connection weight for the connection from X to Y

• Η_Y the role of speed factor for Y

• C_i,Y the role of combination function weight of bcf_i(..) for Y

• P_i,,j,Y the role of combination function parameter value; examples of such reified parameters used in Fig. 1 are the excitability parameters τ (reified by the two T states in the middle plane) and the persistence parameter μ (reified by the M state in the upper plane)

These values are used in the computations for base states Y depending on their role. For any base state Y the following universal combination function c*_Y(..) is used:

c*_Y(H, C₁, …., C_m, P_1,1, P_2,1, _…, P_1,m, P_2,m, W₁, …, W_k, V₁, …, V_k, V) =

H $\frac{{C_{1} {\text{bcf}}_{1} \left( {P_{1,1} , P_{2,1} , W_{1} V_{1} , .., W_{k} V_{k} } \right) + \ldots .. + C_{m} {\text{bcf}}_{m} \left( {P_{1,m} , P_{2,m} , W_{1} V_{1} , .., W_{k} V_{k} } \right) }}{{C_{1} + \ldots .. + C_{m} }}$ + (1−H) V

where

• H is used for the speed factor reification H_Y(t)

• C_j for the combination function weight reification C_i,Y(t)

• P_i,j for the combination function parameter reification P_i,j,Y(t)

• W_i for the connection weight reification $\textbf{W}_{{X_{i}},Y}{\textit{(t)}}$

• V_i for the state value X_i(t) of base state X_i

• V for the state value Y(t) of base state Y

This universal combination function is used in the following universal computational (difference) equation (leaving t out of most of the notation):

Y(t + Δt) = Y(t) +

$[\textbf{c}*_{Y}(\textbf{H}_{Y}, C_{1,Y}, {\ldots}, \textbf{C}_{m,Y}, \textbf{P}_{1,1,Y}, \textbf{P}_{2,1,Y},{\ldots}, \textbf{P}_{1,m,Y}, \textbf{P}_{2,m,Y},\textbf{W}_{{X_{1}},Y}, {\ldots}, \textbf{W}_{{X_{k}},Y}, X_{1}, {\ldots}, X_{k}, Y(t)) - Y(t)] \Delta t$

= Y(t) + H_Y [

$$ \frac{{{\mathbf{C}}_{1,Y} {\text{bcf}}_{1} \left( {{\mathbf{P}}_{1,1,Y} , {\mathbf{P}}_{2,1,Y} , {\mathbf{W}}_{{X_{1}},Y} X_{1} , .., {\mathbf{W}}_{{X_{k}},Y} X_{k} } \right) + \ldots .. + {\mathbf{C}}_{m,Y} {\text{bcf}}_{m} \left( {{\mathbf{P}}_{1,m,Y} , {\mathbf{P}}_{2,m,Y} , , {\mathbf{W}}_{{X_{1}},Y} X_{1} , .., {\mathbf{W}}_{{X_{k}},Y} X_{k} } \right) }}{{{\mathbf{C}}_{1,Y} + \ldots .. + {\mathbf{C}}_{m,Y} }}{-}Y\left( t \right)]\Delta t $$

In these formulas, by its place in the formula, each role indeed contributes a different type of effect according to its intended semantics. In Sect. 3 it is shown how in a network model design, the roles of these reification states are specified by role matrices mb (base connection role), mcw (connection weight role), ms (speed factor role), mcfw (combination function weight role), and mcfp (combination function parameter role).

Note that in a reified network the specific names of the reification states are computationally irrelevant: in this network modeling style the connections and their roles define meaning and processing, not the state names. This may be considered in contrast to reification in logic-based languages like (meta)Prolog [2, 7] where usually syntactical structures of names are processed.

3 Specification Format for a Reified Temporal-Causal Network

In role matrices it is specified which other states have impact on a given state (the incoming arrows in Fig. 1), but distinguished according to their role: base or non-base connections, from which for the latter a distinction is made for the roles connection weight, speed factor, combination function weight and combination function parameter reification (see also Fig. 1). Role matrices enable to apply structure-preserving implementation. The matrices all have rows according to the numbered states X₁, X₂, X₃, …..

For a given application a limited sequence of combination functions is specified by mcf = [….], for the example mcf = [1 2 3], where the numbers 1, 2, 3 refer to the numbering in the function library which currently contains 35 combination functions, the first three being ${\mathbf{eucl}}_{{\varvec{n},{\varvec{\uplambda}}}} \left( \ldots \right)$, ${\mathbf{alogistic}}_{{{\varvec{\upsigma}}\varvec{,}{\varvec{\uptau}}}} \left( \ldots \right)$, ${\mathbf{hebb}}_{{\varvec{\upmu}}} \left( \ldots \right)$. In Box 1 the role matrices mcfw and mcfp (3D matrix) are shown. The first role matrix mb for base connectivity specifies on each row for a given state from which states at the same or a lower level it has incoming connections; see Box 1. For example, in the third row it is indicated that state X₃ (= bs_s) only has one incoming base connection, from state X₂ (= srs_s). As another example, the fifth row indicates that state X₅ (= ${\mathbf{W}}_{{{\text{srs}}_{s} ,{\text{ps}}_{a} }}$) has incoming base connections from X₂ (= srs_s), X₄ (= ps_a) and from X₅ itself, and in that order, which is important as the Hebbian combination function ${\mathbf{hebb}}_{{\varvec{\upmu}}} \left( \ldots \right)$ used here is not symmetric.

Box 1.

Specification in role matrices format for the example reified network for plasticity and metaplasticity

In a similar way the four types of role matrices for non-base connectivity (i.e., connectivity from reification states at a higher level of reification: the downward arrows in Fig. 1), were defined: role matrices mcw for connection weights and ms for speed factors, and role matrices mcfw for combination function weights and mcfp for combination function parameters (see Box 1).

Within each role matrix a difference is made between cell entries indicating (in red) a reference to the name of another state that as a form of reification represents in a dynamic manner an adaptive characteristic, and entries indicating (in green) fixed values for nonadaptive characteristics. Indeed, in Box 1 it can be seen that the red cells of the non-base role matrices are filled with the (reification) states X₅ to X₉ of the first and second reification levels. For example, in Box 1 the name X₅ in the red cell row-column (4, 1) in role matrix mcw indicates that the value of the connection weight from srs_s to ps_a (as indicated in role matrix mb) can be found as value of the fifth state X₅. In contrast, the 1 in green cell (5, 1) of mcw indicates the static value of the connection weight from X₂ (= srs_s) to X₅ (= ${\mathbf{W}}_{{{\text{srs}}_{s} ,{\text{ps}}_{a} }}$). Similarly, role matrix ms indicates (in red) that X₈ represents the adaptive speed factor of X₅, and (in green) that the speed factors of all other states have fixed values. For more explanation about this role matrix specification format and the above example, see [12, 13] or the forthcoming book [14].

4 The Computational Reified Network Engine

The computational reified network engine developed takes a specification in the format as described in Sect. 3 and runs it. First each role matrix (which can be specified easily as table in Word or in Excel) is copied to Matlab in two variants: a values matrix for the static values (adding the letter v to the name) in the green cells, and an adaptivity matrix for the adaptive values represented by reification states (adding the letter a to the name) in the red cells. For example, from mcw two matrices mcwa (adaptivity matrix) and mcwv (values matrix) are derived in this way. The numbers in mcwa indicate the state numbers of the reification states where the values can be found, and in mcwv the numbers indicate the static values directly. States X_j are represented in Matlab by their index number j. Empty cells are filled with NaN (Not a Number) indications. During a simulation, for each step from k to k + 1 (with step size Δt, in Matlab dt) based on the above role matrices first for each state X_j the right values (either the fixed value, or the adaptive value) are assigned to:

Then, as a second part of the computational reified network engine, for the step from k to k + 1 the following is applied; here X(j,k) denotes X_j(t) for t =  t(k)   =   kdt :

Note that functions with multiple groups of arguments here in Matlab get vector arguments where groups of arguments become vectors of variable length. For example, the basic combination function bcf_i(P_1,i, P_2,i, W₁V₁, … , W_kV_k) as expressed in Sect. 3 becomes bcf(i, p, v) in Matlab with vectors p = [P_1,i, P_2,i] for function parameters and v = [W₁V₁, … , W_kV_k] for the values of the function arguments. This format bcf(i, p, v) is used as the basis of the combination function library developed (currently numbered by i = 1 to 35). As can be seen, the structure of the code of this computational reified network engine is quite compact, based on the universal difference equation discussed in Sect. 3: structure-preserving implementation. The combination function library used contains 35 functions at the time of writing. To obtain a general format easily usable within the simulations these functions were numbered and rewritten in the standard basic combination function form bcf(i, p , v ) where i is the number of the function, p is its vector of parameters an v is a vector of values. A more detailed description of the software and a complete specification of the current combination function library can be found at [12].

5 Discussion

In this paper a modeling environment for reified temporal-causal networks was introduced, and applied to model a cognitive agent with plasticity and metaplasticity known from neuroscientific literature; e.g., [1, 6]. The environment includes a new specification format for reified networks and comes with a newly implemented dedicated computational reified network engine, which can simply run such specifications. Moreover, a library of currently 35 combination functions is offered, which can be used; this library can also be extended easily. Using this software environment, the development process of a model can focus in a declarative manner on the reified network specification and therefore is quite efficient, while still all kinds of complex (higher order) adaptive dynamics are covered without being bothered by implementation details. In a forthcoming book [14], more details and many more examples for this modeling approach will be presented.

Application may extend well beyond the neuro-inspired cognitive agents area, as also in Social Science cases are reported where network adaptation is itself adaptive; for example in [3] the second-order adaptation concept called ‘inhibiting adaptation’ for network organisations is described. For further work, it would be interesting to explore the applicability of the introduced modeling environment for such social agent domains as well.