1 Introduction

How can we model the effects of stress and other behavioral moderators on cognition? Adding a physiological representation to cognitive architectures offers an attractive option for modeling these effects. We present the case for integrating a physiological simulation with a cognitive architecture. This approach is demonstrated with ACT-R/Φ, an extended version of the ACT-R cognitive architecture (Anderson 2007) that is coupled with HumMod, an integrative simulation of human physiology (Hester et al. 2011a). This extension allows one to begin modeling how cognition and physiology can influence each other using models built to run in the original ACT-R architecture.

We use ACT-R/Φ to demonstrate lessons one can draw from connecting a physiological substrate to a cognitive architecture and developing a corresponding process model. By comparing the predictions made by the model that uses the physiological substrate in the hybrid architecture to predictions made by the same models that use the ACT-R cognitive architecture, we demonstrate a benefit of the inclusion of a physiological substrate to model some aspects of behavior. Though the extension only begins to model and use a few of the many possible connections between physiology and cognition, it has the potential to be very useful for anyone exploring how the human mind and body can occur in the physical universe, and, consequently, how the human mind and the physical universe may influence each other.

In this paper, we provide a short review of past implementations of cognitive moderators in computational cognitive and agent architectures. Then, we introduce ACT-R/Φ (pronounced act-are-fee), an extended version of the ACT-R architecture that is connected to a model of human physiology. Variables from the physiological model that change with stress are used to modulate ACT-R parameters to simulate a stress response. We demonstrate this physiological modulation of cognition using a modified version of an existing mental arithmetic model and discuss the results from this model.

2 Past implementations of cognitive moderators in architectures

We briefly review several existing moderator implementations represented in cognitive architectures. These existing implementations provide lessons for modeling the effects of moderators on cognition and behavior and integrating a physiological system with a cognitive architecture.

2.1 CoJACK

CoJACK (Ritter et al. 2012) is an extended version of the JACK (Java Agent Construction Kit) agent architecture that is based on the beliefs, desires, and intentions (BDI) model (Rao and Georgeff 1995). CoJACK extends JACK with cognitive limitations and representations of cognitive moderators, including caffeine and fear. Cognitive limitations are represented in CoJACK, with the architecture allowing limited access to plans and belief-sets; in addition, models developed within the architecture can also retrieve incorrect plans or belief-sets.

Under the effect of a moderator, such as caffeine, a set of changes to architectural parameters are overlaid onto the architecture. With caffeine, there is a dose-dependent curve of how processing speed changes with caffeine levels. Stress is represented in the architecture in a similar way.

The work with CoJACK shows that extending an existing architecture with representations of the effects of physiology is possible and potentially useful. However, representing moderators as direct changes to cognitive parameters will lead to intractable conflicts arising from trying to combine multiple moderators (e.g., from PMFServ’s list; Silverman et al. 2004) and not having an appropriate way to represent interactions between moderators (e.g., cognitive changes when fatigued but having recently ingested caffeine). Providing a more explicit representation of the underlying physiology connected to a cognitive architecture provides a way to represent the effects of multiple moderators in a unified, integrated, and tractable system.

2.2 MicroPSI

MicroPSI (Bach 2009) is a hybrid architecture with both symbolic and subsymbolic (neural network) representations based on the Principle of Synthetic Intelligence (PSI) theory (Bartl and Dörner 1998). The architecture has an underlying module that provides representations for emotional components, perceptions, and urges. In particular, the urges are determined by body parameters and urge generators that in turn affect an agent’s motivational state.

There are several urges represented in MicroPSI: intactness and energy (physiological level), competence and reduction of uncertainty (cognitive level), and affiliation (social level). These urges are important as they bring about a certain level of autonomy in MicroPSI agents. Thus, problems like perseveration are less pervasive in MicroPSI models because with time, urges will lead to new motivations that lead to new tasks. These urges also allow for emergent agent behavior over time in a complex environment; this is especially true for MicroPSI as it has learning mechanisms, as well as symbolic and subsymbolic memory representations.

Though Bach (2009) admits that the mechanisms currently in MicroPSI fail to represent several of the complexities of human cognition, MicroPSI’s hybrid memory structure and modulator representation are important architectural distinctions. This architecture provides an important functional middle ground between representing human-like cognitive abilities and the often downplayed modulators of cognition and behavior (e.g., physiology). Perhaps most importantly, MicroPSI illustrates that cognition might be interrupted by physiological urges, and that this process is important, underexplored, and pervasive.

2.3 Fatigue in ACT-R

Gunzelmann and colleagues developed an ACT-R model that simulates the effects of fatigue (arising from sleep-deprivation) and circadian rhythms on human behavior and cognition; this is accomplished by altering ACT-R module parameters for utility calculation of procedural rules (Gunzelmann et al. 2009). Two biomathematical modelsFootnote 1 (CNPA and SAFTE) drive parameter change over time while the ACT-R model is performing the psychomotor vigilance test (PVT).

This fatigue work is an interesting departure from the previously discussed architectures because of the explicit reliance on an external mathematical model. The highest and lowest alert values for the models are found off-line (i.e., before the model is run for the particular task) and linearly tied to corresponding utility parameters that produce the best and worst performance data. Production rule utility in the ACT-R model is then directly moderated by the biomathematical model’s alertness.

Though novel, the ACT-R biomathematical connection presented by Gunzelmann et al. (2009) has a few drawbacks, including how well the derived equations and results may generalize to tasks other than those used by the model (for tasks that have been modeled see, Gunzelmann et al. 2009, 2012, 2011). As mentioned previously, the output from the biomathematical models was also found off-line; this limits flexibility of the model during a task (e.g., performance spikes during acute changes in alertness; Gunzelmann et al. 2011). This separation between the running biomathematical model and ACT-R architecture may make it difficult to generalize the use of this connection in modeling tasks in more dynamic environments. Nonetheless, this work suggests that there are theories that can be used to work with physiology and that there are numerous useful applications of connecting a physiological representation to a cognitive architecture. This work also suggests that ACT-R can be a useful cognitive architecture to base the combined architecture upon.

2.4 Summary: Why should we represent the physiological level?

Work in modeling the effects of stress and fatigue on cognition in ACT-R, as well as the work with the CoJACK architecture, represent applications of overlays (e.g., Ritter et al. 2007). In this case, an overlay is a model of how a single moderator affects cognition realized as a set of changes that are on top of the architecture, possibly including time-based components and reservoirs. In the end, overlays are probably not the best level for representing the effects of physiology on cognition.Footnote 2 Though overlays offer particular insights into potential routes of quantitatively altering behavior of cognitive models, the discussed implementations are often task-specific, virtually impossible to combine, and will be difficult to generalize in the future to further moderators and deep physiology.

Though the projects reviewed all work relatively well for their prescribed functions, a more unified approach should be pursued to represent specific moderators (e.g., fear or stress) and understand how these moderators affect systems that modulate cognition and behavior. Adding an account of the physiological level allows the representation of these modulators and their interactions in a more tractable and appropriate fashion. A physiological representation also allows a more realistic and straightforward quantification of experimental representations (e.g., quantifying the effects of stress on cognition using existing experimental literature on peripheral catecholamines and consequent changes to cognition and behavior). It provides a theoretical way for combining the effects by using intermediate physiological representations including hormones, autonomic nerves, and receptors. Adding an underlying physiological substrate to a cognitive architecture could be used to provide a more unified manner to model these cognitive modulators quantitatively and qualitatively.

Providing a physiological substrate allows for a more unified (e.g., Newell 1990) approach to representing cognitive modulators, potentially providing a cognitive architecture for developing more diverse and interesting computational models. Having the ability to model human behavior on multiple levels within one system is useful as expansions and explorations of cognitive architectures (and subsequent models) continue not only on the physiological level (e.g., Anderson 2007), but also on the affective (e.g., Marsella et al. 2010) and social levels (e.g., Morgan et al. 2010; Zhao et al. this issue). Developing computational models of human behavior with a physiological and cognitive perspective (and with a suitable environment) may potentially provide support for examining theories on the effects of physiology on cognitive and social behavior, and, conversely, theoretical takes on the effects of cognitive and social behavior on physiological systems.

3 A model of mental subtraction in ACT-R/Φ

In the next sections, we review two systems (ACT-R and HumMod) that computationally represent two traditionally separate levels of inquiry (cognitive and physiological). We also discuss how we connected these systems and present this connection as the architecture ACT-R/Φ. This connection allows the simulation of physiological modulation of cognitive function and the cognitive affect on physiology. We demonstrate this capability using a modified version of an existing model of mental serial subtraction.

3.1 ACT-T

ACT-R (Anderson 2007) is a modular, hybrid cognitive architecture that provides both symbolic and subsymbolic representations; the inclusion of both is important because it allows one to alter processes like declarative memory retrieval on both gross and fine-grained levels. Thus, a model’s ability to retrieve a memory (from the declarative module) is affected not only by the availability of the declarative chunk in long-term memory storage, but also the current activation value of the memory. Figure 1 shows that the declarative module is one of several modules that exist within the ACT-R architecture. The production system is made up of several modules.

Fig. 1
figure 1

A high-level representation of ACT-R and its modules

Full size image

Modules in ACT-R have been correlated to structures in the brain (Anderson 2007; Anderson et al. 2008). Table 1 notes areas of the brain that have been related to ACT-R modules. This work allows one to make connections between representations in ACT-R and functional structures in the central nervous system. The most important correlations (for the work presented in this paper) are those between brain regions related to the declarative module (including the retrieval buffer).

Table 1 ACT-R modules (bold) and buffers that are correlated with activity in specific regions of the brain
Full size table

The declarative module and retrieval buffer are associated with activity in the hippocampus and ventral lateral prefrontal cortex (respectively). These correlations allow one to hypothesize changes in the declarative module due to changes in central and peripheral physiology. Thus, one may have specific parameters in ACT-R that are modulated by representations of hormones or catecholamines; stress may elicit changes in the locus coeruleus and hypothalamus, i.e., the sympathetic–adrenal–medullary (SAM) axis and hypothalamic–pituitary–adrenal (HPA) axis respectively), both of which directly affect areas of the prefrontal cortex and hippocampus. While there are several ways one may choose to represent physiological modulation of declarative parameters (as we discussed earlier), we suggest a specific physiological model and simulation system that provides gross anatomical representations of physiology.

3.2 HumMod

HumMod (Hester et al. 2011a) is a simulation system that provides a top-down integrative computational model of human physiology (see Hester et al. 2011b, for a discussion on approaches to computational physiology). It is an extension of the physiological research of Arthur Guyton who originally applied engineering system analysis to the cardiovascular system under normal and pathologically significant physiological states. Guyton’s work continues to serve as the basis of contemporary medical knowledge regarding cardiovascular physiology (Guyton et al. 1972; Montani and Van Vliet 2009). The physiology model in HumMod is a derivative of the original Guyton model (Guyton et al. 1972) that includes over 1,500 linear and non-linear equations and over 6,500 state variables. HumMod provides a user with the opportunity to simulate physiology under normal and abnormal conditions over multiple time scales. The model also provides several points of access to the simulated body through parameters that can change many aspects of the physiological output including output related to both the endocrine and nervous systems. Table 2 illustrates some of the major systems and example variables.

Table 2 Some of HumMod’s major systems
Full size table

There are two ways to change the values attached to variables in HumMod—changing the underlying XML-based model or changing the parameters after the model has been loaded into the simulator system (e.g., change EpiPump.Switch to pump epinephrine into the body). An alteration of the base model allows changing initial variables, derivations, and connections between variables. Changing the parameters has perhaps less systematic power than a change to the actual model, but allows one to work within the given model and quickly view the effect of these changes. The complex connections between state variables can make it difficult to change values of variables themselves—it is easier to modify parameters designated by HumMod (e.g., one may choose to use the epinephrine pump parameter or adrenal nerve parameter instead of directly modifying the epinephrine variables). Using the parameters allows another simulation (e.g., ACT-R) to be modified by and to modify the physiology model in a fairly straightforward fashion.

3.3 ACT-R/Φ: connecting a cognitive architecture to a physiological model

ACT-R/Φ is an extension of the ACT-R cognitive architecture with the addition of a physiological substrate (represented using the HumMod physiological simulation system).Footnote 3 HumMod and ACT-R are connected using a physio module (Fig. 2) that allows two-way communication between the two systems. Thus, one can simulate cognitive effects on physiology (e.g., stress caused by a high cognitive workload and time pressure) and consequent effects of physiology on cognition. The use of this simulation system also allows the exploration of possible emergent external behaviors arising due to non-linear changes in the physiology (e.g., a continuous gradual change in peripheral epinephrine over time) and interactions among physiological systems; this computational exploration will become more important as one begins to develop more robust models of human behavior that run over longer periods of time and in more extreme environments.

Fig. 2
figure 2

A high-level schematic of ACT-R/Φ (top) and the physio module and its buffers (bottom). Thicker solid lines in the top figure represent current direct connections between the physiological module and cognitive modules

Full size image

Within the physio module, a physio-substrate buffer is used to request the module to begin retrieving physiological data from the HumMod simulation. The use of a buffer here is a convenient way to represent (in software) the set of connections and for explanation; we do not propose that the body provides a buffer to the brain—the body and buffer are better viewed as a substrate that the mind is based upon, is influenced by, and attempts to direct. One can start the HumMod simulation (using an ACT-R/Φ model) by putting a chunk of type phys-var into the physio-substrate buffer. To explicitly request the value of any physiological variable one must send a request to the phys-substrate buffer with the specific name of the variable or parameter. Physiological variables in HumMod can be explicitly set to a certain value by adding a chunk to the efferent buffer.

3.4 A mental arithmetic model

Our example application of ACT-R/Φ uses a modified version of the ACT-R 6.0 subtraction model developed by Ritter etal. (2009).Footnote 4 This model completes a mental serial subtraction task (analogous to the subtraction task in the Trier social stress test, or TSST). Figure 3 displays a high-level description of the subtraction model and its relation to ACT-R/Φ. This model includes a representation of the CNS-PNS loop, i.e. a production rule affects the central nervous system (CNS) in HumMod that consequently affects the peripheral nervous system (PNS); the change in the PNS feeds back to affect the CNS over time. In the model, we represent a fluctuation of sympathetic nervous system (SNS) activity due to a scheduled aversive sound (during the task) that causes a form of a startle response.Footnote 5 The startle response in turn causes a change in HumMod (the orange boxes in the right half of Fig. 3) and the affected HumMod variables modify the noise in the declarative knowledge retrieving process. The epinephrine variable in HumMod is tied to declarative memory noise in ACT-R (the :ans parameter in the ACT-R software); epinephrine was chosen due to existing literature indicating its importance in stress responseFootnote 6 and declarative memory encoding/retrieval (e.g., Cahill and Alkire 2003; Miyashita and Williams 2006). Peripheral epinephrine levels are known to affect neural structures (the nucleus of the solitary tract, NTS, and locus coeruleus, LC) that control neural norepinephrine levels (Miyashita and Williams 2006; Ulrich-Lai and Herman 2009).

Fig. 3
figure 3

A schematic of the interactions in the ACT-R/Φ process model built to use the ACT-R/Φ physio module. Peripheral physiology affects memory noise whether or not the model processes the sound cognitively. The boxes in the left half of the figure that have a double border represent the process of the original ACT-R 6.0 model

Full size image

Production rules are added to handle the fast processing of the aversive sound stimulus. After sensing the loud noise in the aural-location buffer, the model clamps (sets to 1) the central nervous system autonomic nerve integration variable (SympsCNS; via the efferent buffer in the physio module) that positively affects the adrenal nerve variable, thus simulating a feature of SNS activation. We developed two separate equations to tie the HumMod epinephrine variable to the ACT-R declarative memory noise (the :ans parameter). We used the model with these two different equations because we were interested in simulating how the different equations may result in different task-related and physiological behavior.

In both Eqs. 1 and  2, the ansMultiplier variable was determined by solving for the equation when declarative memory noise was equal to the value found in the non-caffeine parameter set found by Ritter et al. (2009), and the (current level) epinephrine value is equal to the result of HumMod adrenal nerve activity leading to sympathetic activity (e.g., heart-rate) similar to that found in the original TSST study (Kirschbaum et al. 1993). A (HumMod) heart-rate of roughly 95 was used to calibrate the adrenal nerve activity variable. In this model, ansMultiplier was determined to be 21.4 and ansACT-R-All is 0.7 (from Ritter et al. 2009). These values were chosen to calibrate the model because they were previously used/found experimentally (e.g., Ritter et al. 2007, 2009) and this makes it more straightforward to compare this model to related previous models. Every time physiological variable values are updated in ACT-R/Φ (the interval between updating is determined by the :phys-delay parameter that is 0.25 s by default), the declarative memory noise value is determined using either Eqs. 1 or 2. Equation 2 is Eq. 1 modified to cause an inverted u-shaped curve for performance as epinephrine levels increase from baseline; this partially follows the Yerkes–Dodson (1908) law for complex problems. (This u-shaped performance effect is shown in the next section.)

$$ :ans = \frac{{ansMultiplier*\left( {Epi_{Current \, value} - Epi_{Baseline} } \right)}}{{Epi_{Max} }} $$
(1)
$$ :ans = \left\{ {\begin{array}{*{20}c} {\frac{{ - 2*ansMultiplier*\left( {Epi_{Current \, value} - Epi_{Baseline} } \right)}}{{Epi_{Max} }} + :ans_{ACT - R - All} \quad \forall Epi_{Current \, value} < 50.25 } \\ {\frac{{2*ansMultiplier*(Epi_{Current \, value} - Epi_{Baseline} )}}{{Epi_{Max} }} - :ans_{ACT - R - All} \;\quad \forall Epi_{Current \, value} \ge 50.25 } \\ \end{array} } \right. $$
(2)

Physiological change in ACT-R/Φ is accomplished by using a production rule to send a query to the efferent buffer in the physio module that specifies the name of a HumMod parameter and a new value for it. Perceiving the sound also results in a short processing of the specific sound. The startle response does not last long (cognitively), and the rules that perform the subtraction task continue firing shortly after the encounter with the noise. However, the SympsCNS variable is not changed until a production rule is fired to setup the mental representation of the next subtraction problem,Footnote 7 this happens at the beginning of the block and after an incorrect subtraction answer is given. Thus, epinephrine increases non-linearly until the model resets the internal subtraction representation. (This is not to imply that production rules particularly represent the correct theoretical construct for cognitive change of physiology. It is an artifact of the current implementation and will be altered in future iterations to better reflect theoretical constraints.)

4 Model results and comparison

To illustrate the potential of the ACT-R/Φ architecture, we ran a model that used the components principally affected by the physiological substrate. In this example model, we focus on the effects of memory noise modulation by varying the epinephrine levels (based on a startle response) during the modified subtraction task. We compare model performance, specifically the total number of attempts and proportion of problems answered correctly, when using either Eqs. 1 or 2 to the performance of the ACT-R-All model (a model with static parameters set to match those used in Ritter et al. 2009).

All models completed 4 blocks of 4 min (240 s) of mental serial subtraction. Blocks 1, 2, 3, and 4 had a starting value of 9,095, 6,233, 8,185, and 5,245 and a subtraction constant of 7, 13, 7, and 13 (respectively). More details of the task are available in Ritter et al. (2009). We ran each model 200 times; this number is acceptable based on criteria described by Ritter et al. (2011).Footnote 8

4.1 Model physiology

Figure 4 shows average epinephrine levels during the ACT-R/Φ-Eq. 1 and ACT-R/Φ-Eq. 2 runs; Fig. 5 shows average declarative memory noise over the course of the task (changed according to the epinephrine values). The shaded area above and below each point represents the standard deviation at that point in time based on 200 runs. The model’s average epinephrine levels displayed a higher maximum average during block 1 and 3 as compared to block 2 and 4. Overall, block 2 displayed the lowest average epinephrine levels; other physiological variables modulated by activity in the sympsCNS HumMod variable followed the same general pattern.

Fig. 4
figure 4

The solid line (mean) and dotted line (median) represents epinephrine levels of models (n = 200) while the area around the solid line represents the standard deviation for Eq. 1 (Top) and Eq. 2 (Bottom) models. The line at 2, 6, 10, and 14 minutes represents the point at which the startle was presented in each block. (Color figure online)

Full size image
Fig. 5
figure 5

Mean declarative memory noise values for the ACT-R/Φ-Eq. 1 (top) and ACT-R/Φ-Eq. 2 (bottom) models (n = 200). The line at 2, 6, 10, and 14 minutes represent the points at which the startles were presented. (Color figure online)

Full size image

In block 2 of the task, both ACT-R/Φ models had smaller range of epinephrine values that resulted in a lower declarative memory noise (as seen in Fig. 5). This attenuated epinephrine response occurred because when there is an incorrect answer, the model refocuses on the problem and sets it up as a task; the activation of the SNS is then stopped. So, in block 2 and 4, where the problems are more difficult, the physiology response to the startle has a lower amplitude. Thus, using this hybrid architecture allows one to explore how seemingly small effects of interruption during a task can affect the overall outcome of the task over time and how complexity and attention of the task can affect reaction to non-integral stimuli. This allows one to look at the emergent effects physiology may have on cognition and behavior over time and how problem difficulty can interact with reactions to outside stressors.

4.2 Model performance

General descriptive statistics of the models’ performance in ACT-R/Φ over 200 runs is presented in Table 3 along with the performance of models run in the ACT-R architecture; we also include performance data by human subjects from the original study conducted by Ritter et al. (2009) who appraised the task as threatening and were more reactive to the task. The ACT-R/Φ-Eq. 1 average performance was closest to the human performance during the serial subtraction task.

Table 3 Percent correct and number of attempts (average) by human subjects who fell under the Threat appraisal condition (Ritter et al. 2009), the original ACT-R based models, and the ACT-R/Φ model with Eqs. 1 and  2. B1, B2, B3, and B4 represent Block 1, Block 2, Block 3, and Block 4 (respectively)
Full size table

We compared the models’ task-performance (percentage correct) to see how models that were identical to those used by Ritter et al. (2009) compared to models that had similar cognitive components, but were modulated by certain physiological change. We also compared model output when using the positive linear slope equation (ACT-R/Φ-Eq. 1) and the piecewise equation developed to achieve inverted u-shaped performance during the task. A Mann–Whitney U test was used to compare the models. The ACT-R/Φ-Eq. 1 performance (percent correct) was found to be significantly different (p < .0001; z = 11.74) than performance of the ACT-R/Φ-Eq. 2 model. Performance of the ACT-R-All and ACT-R-Threat models were both found to be significantly different than both ACT-R/Φ-Eq. 1 and ACT-R/Φ-Eq. 2 model performance (p < .0001; z = −7.99; and z = −19.08); the difference between output from the ACT-R-All model and the ACT-R-Threat model was not found to be statistically significant. Histograms, showing the distribution of each model’s performance, are presented in Fig. 6. The distribution shapes may indicate that a higher number of runs would yield a more normal distribution (the relatively high standard deviation of the results found when running the models 200 times also indicate it may be beneficial to run the model even more times, see Byrne 2013, for a related discussion).

Fig. 6
figure 6

Histograms of performance (% correct) for the ACT-R-All (top-left), ACT-R-Threat (top-right), ACT-R/Φ-Eq. 1 (bottom-left), and ACT-R/Φ-Eq. 2 (bottom-right) models; each model was run 200 times

Full size image

To see what range of performance the physiology would impose on the model in the hybrid cognitive architecture, we ran the ACT-R-All model with two sets of declarative memory noise (:ans) values (see Fig. 7 and the related discussion below to get an indication of how declarative memory noise affects the performance, percent correct, of this model).

Fig. 7
figure 7

The effect of declarative memory noise for the ACT-R-All model on performance (% correct) on the serial subtraction task

Full size image

Figure 7 shows that the model’s accuracy (percentage of problems answered correctly) partially depends on its ability to retrieve declarative memories as the declarative memory noise parameter increases, percent correct decreases. Despite the models’ reliance on declarative memory, a declarative memory noise value of roughly 0.55 or higher was still needed to have the models’ percent correct consistently go below 90 %.

Each time the ACT-R/Φ models in Fig. 6 (Eqs. 1 or 2 ) were run, we recorded the :ans values used over each ¼ secondFootnote 9 of the task; this gave us two sets of 3,955 :ans values; Fig. 8 and Fig. 7 display the results from these runs. We then ran the ACT-R-All model (syllable-rateFootnote 10 (:syl) = 0.44, base-level constantFootnote 11 (:blc) = 2.49) with each of these :ans values held constant; this resulted in two model-sets where \( :ans_{{model_{i} }} = :ans_{{time_{i} }} \) and where \( :ans_{times} \) is the average :ans value found at time i after running the ACT-R/Φ-Eq. 1 or ACT-R/Φ-Eq. 2 models. As an example, at 175 s into the task, our ACT-R/Φ-Eq. 2 Footnote 12 model had an average :ans (declarative memory noise) value of 0.53; consequently model 175 in that model-set was run with an :ans value of 0.53. Each model in the two model sets was then run 200 times. This process resulted in over 1,582,000 model runs of the ACT-R-All model. These runs gave us a distribution of mean performance and the noise of those mean values. There was an inverted u-shaped performance distribution, shown in the bottom graph in Fig. 8, due to this physiological modulation in the Eq. 2 set of models. The bottom graph in Fig. 8 shows that as epinephrine values rise, performance improves, but performance begins to decrease back towards the original baseline performance after a continued stress response; this is most evident when observing the effect of memory noise values found during the first block of the task.

Fig. 8
figure 8

The solid line represents mean performance (percent correct) of a single serial subtraction model while the area around the line represents the standard deviation. Each point in the line represents the mean performance of a model run with an :ans parameter value (declarative memory noise) as determined by a list obtained by averaging :ans values used by the ACT-R/Φ-Eq. 1 (Top) and ACT-R/Φ-Eq. 2 (bottom) models, i.e. the point at time 125 s represents the mean percent correct of the ACT-R serial subtraction model run with an :ans value of 0.1 (Top) and an :ans value of 0.53 (Bottom)

Full size image

In Fig. 8 at t = 125 s (in block 1, shown with a dashed line), the y-value is the mean of percent of subtraction problems answered correctly for the ACT-R-All model with an :ans (declarative memory noise) parameter value of 0.1 and 0.53 (top and bottom, respectively). The area above and below each point (red) represents the standard deviation at that particular point; thus, we generally see a higher standard deviation when declarative memory noise (:ans) is higher; a lower mean performance also accompanies higher declarative memory noise.

The higher mean performance for models using block 2 values for the Eq. 1-based models (top) was due to epinephrine levels that failed to reach a value as high as the other models (using values from blocks 1, 3, or 4) before beginning to decrease towards the baseline. This resulted in noise values that did not have a large effect on the mean performance of the models.

4.3 Summary

We were able to reuse an existing subtraction model in this new hybrid architecture and demonstrate a useful way to begin using an extended version of the ACT-R architecture with a physiological substrate. Though the modifications were reasonably simple, quantitatively and qualitatively different behavior was obtained when compared to the original model built to run in ACT-R. We were also able to use output from the extended models run in ACT-R/Φ to explore the range of values output by the original subtraction model with different static declarative memory noise values.

5 Discussion and conclusions

Here we review and discuss the results reported and potential future directions for the work. Models built to use the extensions in the ACT-R/Φ architecture exhibited task performance that was significantly different than the same models that do not use the new physiological substrate and were closer to observed human behavior. This difference is due to the physiological substrate that continuously affects cognitive performance. Though ACT-R/Φ and the subsequent models present a novel method for simulating effects of a stressor on cognition, there remains room for improvement in the architecture and model to better encompass the many dynamics of the interaction between physiology and cognition. We next discuss results and some of the potential ways one could expand this hybrid architecture.

5.1 Performance results

As expected, the results from the ACT-R/Φ-Eq. 1 and ACT-R/Φ-Eq. 2 models differed significantly; consequently the model-sets constructed from those models’ declarative memory noise values (e.g., Fig. 8) also exhibited a different range of performance. The pattern performance found using declarative memory values from the ACT-R/Φ-Eq. 2 model was more realistic than the ACT-R/Φ-Eq. 1 model, with the first models in the model-set beginning the task with a higher noise level (and lower sympathetic activation) that resulted in a lower initial performance. The presented stressor causes a brief increase in performance, but over time the stress causes an adverse effect on performance as physiological variables continue to change due to the stressor; the effect of stress also does not stop immediately, the model-set continues to be affected with the performance range never returning to the initial range of values. The ACT-R/Φ models show it is possible to non-linearly change model performance with a relatively simple modification to an original ACT-R model and the use of physiological equations provided in the ACT-R/Φ system.

The different mean performance among models can be attributed to a varying :ans value that is directly connected to epinephrine (discussed below). Mean performance was expected to be significantly different among the models due to a difference in setting the :ans statically (e.g., once during the model run) versus dynamically. We believe ACT-R/Φ-Eq. 2 provides a good prospect for a higher fidelity model of the effects of stress on serial subtraction (and vice versa). This is due to the known dynamic interaction between stress systems and physiological precedents to the cognitive processes embodied in the ACT-R architecture (e.g., Anderson 2007; Joëls et al. 2006; Sara 2009).

5.2 Physiological variable results

Perhaps most noteworthy, one can see an obvious feedback between epinephrine levels and task performance. The lower epinephrine levels in block 2 and 4 are likely due to the way the ACT-R/Φ models affect the underlying physiology. The models stop the initial activation of the autonomic integration nerve variable with a rule that is applied when the model must restart a subtraction problem; this was meant to represent a refocusing of attention following an opportunity to restart on a subtraction problem session. Though this switch is effectively off shortly after the event, epinephrine levels do not immediately drop, instead they fall over time non-linearly based on the equations in HumMod.

Although the epinephrine level falls during the second half of the blocks, it does not reset to the original baseline; it never actually reaches the original baseline before being raised again the next block. Thus, the model performance in block 4 is affected by the startle response in block 1.

5.3 Summary of results

There are improvements to be made to this model and the ACT-R/Φ extension (some of which are discussed below), however, we believe the current state of the work represents an opportunity to begin to model not only cognitive processes, but also underlying physiological processes affecting and being affected by cognition. The model is affected by both immediate physiological changes and by longer term physiological changes, thus physiology never resets. The data from this model illustrates the potential for developing cognitive models with factors that vary linearly and non-linearly over time and that take account of the effects of physiological factors on cognition and performance, such as noted in the review.

The data also indicate that models that use the stochastic functions available in architectures like ACT-R/Φ need to be reported with an indication of how many times the model has been run. These data show that models with a stochastic component need to be run a larger number of times than non-stochastic models to get a more stable representation of the descriptive statistics often used to judge them against human data (see Ritter et al. 2011 for a discussion on this topic). Larger runs will become even more important as the mechanisms include more variance arising from physiology, and when distributions, rather than just means, are examined.

Though our stress representation provides a more explicit and realistic representation of some interactions between physiology and cognition during a response to a stressor, there are additional representations of stress (both on the physiological and cognitive levels) that would potentially benefit this and related models. More globally for ACT-R/Φ, homeostatic motivations (e.g., Bach 2009) would also likely lead to higher model fidelity and an increase in autonomy.

One could also begin to model stress on a more social-level and describe the interaction between stress and social processes. In previous work, Morgan et al. (2010) looked at processes affecting cognitive and social behavior. The use of ACT-R/Φ provides another way to represent how social and other processes affect cognition and even how this interaction can mediate social network formation (see Zhao et al., this issue, for an example of how a cognitive architecture can be used to explore cognitive mediation of network formation).

5.4 Limitations and potential future work in extending the representation of stress

Though we discussed several different physiological effects of encountering a stressor, currently ACT-R/Φ only represents a single aspect of the sympathetic adrenal medullary (SAM) axis. Only three system parameters/variables are used in ACT-R/Φ but other parameters in HumMod could be connected to the cognitive representations in ACT-R/Φ (e.g., those controlling adrenocorticotropin hormone (ACTH), α-amylase, corticotropin releasing factor (CRF), and cortisol) and countless others likely have secondary or tertiary effects on variables directly implicated in affecting cognition. We plan on continuing to expand the representation within the extended architecture by exploring the combined modulation of architecture (and model) behavior by both the HPA and SAM axes. More recent articles on the physiological antecedents to perception of a stressor (e.g., Joëls and Baram 2009) and physiological changes due to a stressor (e.g., Klein et al. 2010) present potential roadmaps for quantifying physiological modulation in ACT-R/Φ; additional representations may require alternative connections in ACT-R/Φ or even modification of the underlying physiological model. Particular causes for stressors also need to be explored in the continuation of this work.

We have developed a particular event-based stressor (disruptive noise), however it would be useful to develop additional stressors based on factors like cognitive workload or time pressure. Work on the AMBR project (Gluck and Pew 2005) particularly with the work on representing cognitive workload in the ACT-R architecture (e.g. Lebiere et al. 2001), offers useful guidelines for adding cognitive-based stressors to the architecture. Time pressure may also potentially be accomplished by leveraging the goal module/buffer along with the temporal module. Models of cognitive appraisal, which focus on how changes in the agent-environment relationship (e.g., Marsella and Gratch 2009), will also prove particularly useful for exploring known causes of stress like time pressure.

One could also examine stress on a more microgenetic basis. One could then look at the standard deviation of performance and error types with this new hybrid architecture. Intra-individual differences in cognition and several physiological variables (e.g., heart-rate, blood-pressure, etc.) can also be simulated with this type observation.

5.5 Potential work in homeostatic–appetitive motivations

Homeostatic-appetitive (e.g., energy balance, thirst, and skin temperature) motivations are fundamental modulators on human cognition and behavior (e.g., Aarts et al. 2001; Mogg et al. 1998; Wright et al. 2012); ACT-R does not yet provide an architectural representation for this modulation. ACT-R/Φ provides a basic representation for these modifications by using the osmoreceptor, gi-lumen, and heatskin variables for thirst, hunger, and skin temperature (respectively). The physiological variables are tied to variables placed into the goal state by the physio module.

Though this method is admittedly crude, identification of particular physiological variables to tie to cognitive architecture changes is an important step; the ACT-R side of the connection will need to be expanded for more meaningful testing of the representations. It should be mentioned that most models built in ACT-R do not complete tasks over a time-period that would suitably leverage these motivations to display more autonomous behaviorFootnote 13; though one may still begin to simulate other cognitive effects of these motivations (e.g., participating in a psychological experiment while thirsty; Wright et al. 2012). As previously noted, existing architectures like MicroPSI offer potential insights into ways one may add physiological representations to ACT-R by expanding current connections in ACT-R/Φ.

5.6 Potential applications

Extending the architecture to include additional aspects of behavior like circadian rhythms or affect and emotion, may also facilitate the use of the architecture to develop models that can be used as more realistic agents in tutoring systems (e.g., Anderson and Gluck 2001; Fincham et al. 2010), for systems engineering (e.g., Pew and Mavor 1998), or for the design of interfaces (e.g., Hudlicka and Mcneese 2002) as traditional ACT-R models have already been used for some of these purposes. Models that use the new components of ACT-R/Φ would likely benefit from the ability to change physiology over time. One could model how the hunger experienced right before lunch affects cognitive performance and adjust a system to account for these affects. Furthermore, one could provide a caffeine overlay that predicts how a caffeinated beverage changes physiology and that changes cognition. The new hybrid architecture makes development of process models that include a depiction of physiological changes on cognition over time more tractable.

6 Final thoughts

Minds need brains to support them and brains need bodies to support them. As models of cognition continue to develop, we will need to add a physiological representation of the substrates that support and implement cognition on multiple levels. Though it is as of yet unclear the best manner to represent the connections between the levels of physiology and cognition (one may ask, for example, where emotions fit into the picture), it is important to continue to develop systems that can test quantitative predictions made by theoretical models. ACT-R/Φ is a step in this direction, and likely will continue to provide insights as the connections and representations are expanded. It is also important to develop these systems so that migration of older models to the new architecture is tractable and one can build on work already done; ACT-R/Φ allows this because ACT-R 6 models can be run in ACT-R/Φ.

Modeling physiological, cognitive, and social effects on human behavior is a fairly complex task given the large amount of background knowledge needed to produce accurate process models. However, as physiological sensing devices continue to become less invasive and increase in resolution, the amount of physiological data one will be able to collect during a psychological experiment is likely to lighten the load on anyone wishing to develop a model within an architecture like ACT-R/Φ; that is, develop a model within an architecture that provides representations on both the physiological and the cognitive levels.

The most beneficial level of physiological representation for a computational system to provide quantitative predictions remains an open question (Hester et al. 2011b) and depends on the function of the computational system. This question has to be further explored to determine the best levels of representation for a hybrid computational architecture like ACT-R/Φ; reviews that explore behavioral effects of physiological changes on multiple levels (e.g., Joëls and Baram 2009) help make potential answers to this question more clear. Nonetheless, mechanistic models used in hybrid architectures like ACT-R/Φ can be used to represent and resolve ambiguous theoretical interactions between physiological and cognitive levels. The architecture could, for example, be used to explain how noisiness in behavior may be the product of differences in physiological states over time and between people. Lastly, ACT-R/Φ can be used to explore implications of physiological and cognitive interactions on different time-scales, providing new insights into the precedents to emergent effects found over time.