Introduction

In human communication, cognitive binding of information transmitted in more than one sensory mode (e.g., acoustic and visual cues), known as cross-modal integration, is important in perception and/or localization of signals (Bee and Micheyl 2008; Miller and Bee 2012; Ghazanfar 2013). The innate nature of cross-modal binding of auditory and visual signals in human speech is often illustrated by lip-reading in noisy environments (Sumby and Pollack 1954), the “McGurk effect” (McGurk and Macdonald 1976) created by combined visual and auditory input, and the “ventriloquism effect” (Hauser 1996), in which co-occurring signals slightly offset in space or time are perceived to be a single, synchronous multimodal signal originating from a single location. This cognitive process is less well known in animals, despite the fact that the ability to accurately perceive multimodal signals may have high fitness consequences. For example, in courtship and mating interactions, receivers need to be able to perceive multimodal signals and integrate the information they contain in order to localize the sender and respond appropriately (Miller and Bee 2012; Taylor et al. 2011). This is especially important when a signal from one individual occurs simultaneously with signals of others (Bee and Micheyl 2008; McDermott 2009; Taylor et al. 2011). Although well studied in humans, cross-modal integration and cognitive processing have only recently garnered attention in animal communication research (Shettleworth 2001; Narins et al. 2005; Taylor et al. 2011), with a focus on neurophysiology of receiver sensory capacity (Fuster et al. 2000; Narayan et al. 2007; Schmidt and Römer 2011) and signal production (Lombardo et al. 2008; Vélez and Bee 2010; Bee 2012), but almost exclusively in vertebrates (but see VanderSal and Hebets 2009).

Across many species, males convey information on mate quality through a variety of sensory modalities, i.e., acoustic, visual, chemical, and vibratory (Candolin 2003; Michaelidis et al. 2006; Murai and Backwell 2006). In order to choose the best possible mate, females must be able to accurately perceive and assess male signals in different modalities and determine their location (Candolin 2003; Michaelidis et al. 2006; Murai and Backwell 2006; Bee and Micheyl 2008; McDermott 2009; Richardson and Lengagne 2010). However, it is currently unknown how perception of the presence of multiple, disparate male signals plays a role in signal localization and female mate choice decisions (Miller and Bee 2012; Ronald et al. 2012). This is especially true for invertebrate animals, for which cross-modal integration is largely unstudied.

Although cross-modal integration in animals has recently been studied in a few vertebrate models (Martin-Malivel and Fagot 2001; Narins et al. 2005; Hoke et al. 2007; Lombardo et al. 2008; Proops et al. 2009; Lampe and Andre 2012), invertebrates have been considered too neurologically simple to possess more complex cognitive mechanisms other than simple responses to stimuli. There is, however, mounting evidence of flexibility in invertebrate behavior (Bushman 1999; Hopper 2003), as well as the possibility of higher cognitive processes, e.g., risk-balancing behavior (Jackson et al. 2001; Wullschleger and Nentwig 2002; Li et al. 2003). As such, invertebrate models are providing insights to mechanisms of cognitive processes in the so-called simple nervous/neural systems (Barth 2002; Giurfa 2003; Hochner et al. 2003, 2006; Jackson and Li 2004; Nagarah et al. 2011).

Among invertebrate models, the well-studied wolf spider Schizocosa ocreata is an excellent organism for the study of sensory integration (Uetz et al. 2016). They detect environmental stimuli via multiple sensory inputs (e.g., eight eyes and myriad vibration sensors on eight legs), and communicate in multiple sensory modes (Uetz 2000; Taylor et al. 2006; Uetz et al. 2009). Males produce courtship signals in both visual (active tapping, raising, and extending the first pair of legs—see Uetz 2000; Delaney et al. 2007 for details) and vibratory (production of substratum-borne vibration by stridulation and percussion—see Stratton and Uetz 1981, 1983; Scheffer et al. 1996; Gibson and Uetz 2008 for details) modes. These signals may be redundant (sensu Partan and Marler 2005), as female S. ocreata display receptivity to males courting in either isolated signal mode (Scheffer et al. 1996; Gibson and Uetz 2008; Uetz et al. 2009). Males have demonstrated plasticity in signaling based on the substrate and the amount of available light (Taylor et al. 2005, 2006; Gordon and Uetz 2011), indicating they may be compensating for attenuated signal transduction in the complex environment in which they live (Uetz et al. 2013). Additionally, they exhibit eavesdropping and signal matching behavior (Clark et al. 2012, 2015), demonstrating a level of behavioral complexity and cognitive processing similar to that seen in some vertebrate animals (Peake et al. 2005; Phelps et al. 2007).

Female S. ocreata likely encounter several males throughout the breeding season (Cady 1984) and may be courted simultaneously by multiple males (Clark et al. 2012; Uetz pers. obs). Because the complex leaf litter environment may obscure or degrade visual and vibratory signals (Uetz et al. 2013), females may receive signals from multiple males in different sensory modes from different locations. Consequently, we investigated how female S. ocreata integrate spatially and temporally disparate male signals in multiple sensory modes (visual and vibratory), and how that affects mate choice decisions.

Methods

Study species

Immature S. ocreata spiders were collected in the field from the Cincinnati Nature Center Rowe Woods, Clermont County (39°7′31.15″N; 84°15′4.29″W) in the fall of 2012. Spiders were reared in the laboratory in individual cylindrical plastic deli containers (9 cm diam. x 5 cm ht.) with lids. Spiders were fed twice each week with 3–5 small crickets (Acheta domesticus), and water was provided ad libitum via dental wicks attached to a reservoir beneath the container. Laboratory conditions were maintained at 23–25 °C and relative humidity of 65–75 %, and a 13:11 h light/dark cycle. Females (N = 185 in all) were tested approximately 3 weeks after reaching maturity, i.e., during peak receptivity (Uetz and Norton 2007).

Ethical note

To our knowledge, no animal welfare laws or regulations in the USA or the State of Ohio govern the use of invertebrates such as spiders in research. Wherever possible, we adhered to the “Guidelines for the treatment of animals in behavioral research and teaching” (Animal Behavior 85 (2013) 287–295) of the Animal Behavior Society. At the end of this study, spiders were either transferred to another researcher in the laboratory for further study, or ultimately humanely euthanized with CO2 and freezing.

Experimental apparatus

Trials were conducted in a 20-cm-diameter, clear plastic polycarbonate, circular arena placed upon a black granite base (30.48 cm × 30.48 cm × 3.81 cm). Sorbothane® (Isolate it! #0510131-30-4-PSA) 2-cm rubber bumpers underneath the granite served to effectively isolate the base from extraneous environmental vibration. Piezoelectric disk benders (APC International, Ltd. #20-1205) were affixed flush with the granite using adhesive tape, and Reynolds Wrap® parchment paper was placed over the entire area of the arena, on top of the disk benders but under the polycarbonate arena. Vibration signals were delivered to the disk benders from an iPod® touch via a preamp (FiiO #EO6) and amplifier (Pyle model PTA2). Disk bender output was calibrated using a Laser Doppler Vibrometer (LDV, Polytech model PDV-100) and Raven bioacoustics software (Cornell laboratory of Ornithology, version 1.3 Build 23) to closely match the playback amplitude and frequency to original recordings from live male S. ocreata courtship. In addition, disk bender output was measured to assure that directional signal attenuation over distance across the parchment paper surface matched natural levels (Uetz et al. 2013). A single iPod Touch® was placed at one end the arena such that the bottom of the screen was flush with the top of the granite base, in a notch cut into the granite, in order that spiders view the screen at the same level as the exemplar is shown on the screen. Disk benders were placed at different angles in a 360° array around the inside circumference of the arena, creating a range of potential angles (at 0, 45, 90, and 180 degrees relative to the iPod location and measured from the position of females at the center of the arena at the start of a trial) for vibration source separation from the iPod® (Fig. 1).

Fig. 1
figure 1

Experimental arena for both spatial and temporal disparity trials. Small circles disk benders, black rectangle iPod Touch®. a View from top; b view from side

Full size image

Experimental trials

All trials were conducted when females were between 15 and 25 days mature, when females are at peak receptivity (Uetz and Norton 2007). Female hunger was controlled by feeding all females one 10-day old cricket 12–24 h before trials were conducted. Each female was placed in the center of the experimental arena under a translucent plastic vial and allowed to acclimate for 1–2 min; during this time, there was no playback of visual or vibratory signals. Trials commenced with the start of playback and the careful removal of the vial so as not to disturb the female; all trials lasted 10 min and were video recorded from two perspectives: a) directly in front of and b) directly above the arena (facing and aerial shots, respectively) using high definition digital camcorders (Sony #HDR-XR260 V).

Digital video recordings of trials were scored for female signal detection (orientation latency in seconds, number of approaches) and female receptivity toward each stimulus source location. Recordings were scored by a single individual (ECK). Scoring of approach and receptivity behaviors are fairly straightforward, but in questionable cases, consultation with the coauthor and other laboratory members was used to arrive at a consensus. Previous studies of this type in our laboratory have involved scoring by multiple individuals, and interobserver reliability has always been high (>90 %). Female Schizocosa exhibit stereotypical behavioral displays in response to male courtship signals (Table 1) (Montgomery 1903; Uetz and Denterlein 1979; Miller et al. 1998) and typically do not mate until they display at least one (or more) of these behaviors, either singly or in combination (Scheffer et al. 1996; Delaney 1997; Norton and Uetz 2005; Uetz and Norton 2007). Female receptivity toward a stimulus was scored as the sum of the number of individual display behaviors as in previous studies of this species (Uetz and Norton 2007; Uetz et al. 2009).

Table 1 Female receptivity displays recorded for Schizocosa ocreata
Full size table

Spatial disparity: experimental treatments

Females (N = 107) were presented with experimental treatments in a repeated measures design over the course of 4 days (1 trial/treatment/day); only those females that were tested in all four treatments were later included in analysis. Order of presentation of treatments was varied across the four groups, to which females were randomly assigned, in order to control for any effect the order of treatment presentation may have had. Treatments consisted of 4 disk bender positions relative to the iPod Touch®. Degree of separation between the iPod and the disk bender was measured in terms of the angle between them, rather than the linear distance between signals, because of the nearly 360° range of visual and vibration senses of lycosid spiders (DeVoe 1972; Rovner 1993), and conditions female S. ocreata likely experience in the field (Cady 1984; Uetz et al. 2013). All angles were measured from the center of the arena as above. Disk benders were placed at 0°, 45°, 90°, and 180° relative to the iPod Touch®; in the 45° and 90° treatments, disk benders were placed on both sides of the arena, which allowed for presentation from either side of the arena and therefore controlled for any side bias (Fig. 1). In all treatments, vibratory playback was synchronized with spider behavior in video playback.

Temporal disparity: experimental treatments

These experiments were conducted in the same apparatus as spatial disparity experiments (above, Fig. 2). Females (N = 78) were presented with each of three temporal disparity treatments in a repeated measures design over three consecutive days (1 trial/treatment/day); additionally, females were sorted into one of three treatment order presentation groups, in which order of treatments females were presented with was varied, to control for both priming and habituation effects. Temporal disparity treatments consisted of an in synchrony (IS) stimulus, in which both visual and vibratory male signals were completely synchronous; an out-of-synchrony (OS) stimulus, in which male vibratory signals were delayed by 1.2 s (approximately one-half the average bout cycle length); and an interleaved/alternating (IL) stimulus, in which male vibratory and visual signals were alternated such that there was no temporal overlap between signals (i.e., with the vibratory signal commencing only after the visual signal completed, and vice versa as in Fig. 2).

Fig. 2
figure 2

Diagram of temporal disparity treatments. Small blue rectangles visual signals; below them are oscillograms of the vibratory signals, placed according to the time of vibratory signal onset. Dashed line non-overlap of visual and vibratory signals in the asynchrony treatment (color figure online)

Full size image

Statistical analysis

All data were analyzed using JMP PRO version. 10 (SAS Institute, Cary, NC) and JASP version 0.7.5.6.6 (https://jasp-stats.org/) statistical packages. Data were tested for normality, and when significantly different from a normal distribution, transformed with a square root transformation (for count data). An alpha level of p < 0.05 was held as the standard for statistical significance. In the case of the spatial disparity experiments, a series of one-way ANOVA analyses (with repeated measures accounting for variation among individuals) were first performed on the three major response variables (orient latency, approach, comprehensive receptivity score) to test for any priming or habituation effects. As none were found, all data were pooled over time periods and the analysis was collapsed around treatment as the main effect, with the same main response variables. Repeated-measures ANOVA and subsequent matched-pairs analyses were run on the spatial disparity data. These analyses were followed by a series of one-way repeated-measures ANOVA with Tukey’s HSD multiple comparison post hoc testing (α level = 0.05) on responses to individual signal modes (visual- and vibratory-only signals) across treatments. The temporal disparity data set was likewise subjected to repeated-measures ANOVA with Tukey’s HSD multiple comparison post hoc testing. When testing revealed marginal p values (0.05 < p < 0.1), subsequent analyses with Bayesian statistics were used to validate acceptance of the null hypothesis (Jarosz and Wiley 2014).

Results

Spatial disparity experiments

One-way ANOVA analyses (with repeated measures accounting for variation among individuals) showed no evidence of behavioral priming or habituation effects; i.e., neither order of treatment presentation nor day of trial was significant predictors of any response: order of treatment (latency to orient F 3,105 = 0.0404; p = 0.989; number of approaches F 3,102 = 1.067; p = 0.367; comprehensive receptivity score F 3,109 = 0.076; p = 0.973); day of trial (latency to orient: F 3,424 = 1.048; p = 0.371; number of approaches F 3,424 = 0.429; p = 0.732; comprehensive receptivity score F 3,424 = 0.539; p = 0.656). As a consequence, data were pooled over time periods and the analysis was collapsed around treatment as the main effect, with orient, approach, and a comprehensive receptivity score as the main response variables.

One-way ANOVA analyses (with repeated measures as above) showed a significant effect of treatment on all response variables (latency to orient F 3,451 = 39.782; p < 0.0001; number of approaches F 3,451 = 16.141; p < 0.0001; comprehensive receptivity score F 3,451 = 28.574; p < 0.0001) (Table 1). Subsequent matched-pairs analysis ANOVAs of spatial disparity data compared responses to individual signal modes across treatments (multimodal, visual-only or vibratory-only) and revealed significance in all cases (Orient Latency: Within pairs F 107 = 44.6291 < 0.0001; Among pairs F 107 = 40.8102, p < 0.0001; N Approaches: Within pairs F 107 = 2.2697, p < 0.0001; Among pairs F 107 = 16.3633, p < 0.0001; Receptivity Score: Within pairs F 107 = 38.9199, p < 0.0001; Among pairs F 107 = 29.0161, p < 0.0001) (Fig. 3). Comparisons across treatments showed no significant difference in latency to orient to the visual signal, but latency to orient to vibratory signals did vary significantly, with females orienting most slowly to vibratory signals separated from visual signals by 45° (Fig. 3) (Visual F 3,422 = 0.8225, p = 0.482; Vibratory F 3,422 = 48.1664, p < 0.0001). Matched-pairs analysis showed there was no significant difference in the total number of approaches to either signal when separated by 180°, otherwise females approached the visual signal significantly more often. When approach responses to individual signal modes were compared across treatments, approaches to either signal mode varied significantly (Visual F 3,422 = 5.50528, p = 0.0022; Vibratory F 3,422 = 39.9006, p < 0.0001). Females tended to approach multimodal signals most often and least often to vibratory signals separated from visual signals by 45° (Fig. 4). There was a reduction in approaches to the visual signal when separated by ≥90° but an increase in approaches to the vibratory signal, with no significant differences seen between the 90° and 180° treatments for either visual or vibratory signal responses (Fig. 4).

Fig. 3
figure 3

Matched-pairs analysis of mean latency (s) to orient to spatially varied visual and vibratory signals (N = 107). Vertical error bars one SEM. Letters over bars significance across treatments by visual- or vibratory-only Tukey’s HSD post hoc test of one-way repeated-measures ANOVA (α = 0.05). All pairs were significantly different (p < 0.0001)

Full size image
Fig. 4
figure 4

Matched-pairs analysis of mean number of approaches females made to male courtship signals that varied by spatial disparity (N = 107). Vertical error bars one SEM. Letters over bars significance from Tukey’s HSD post hoc test of one-way repeated-measures ANOVA. Brackets over bars indicate outcome of matched-pairs analysis

Full size image

Females were significantly more receptive to the visual signal in all treatments, although this disparity decreased with increasing spatial separation of signal modes (Fig. 5). (Receptivity Score: Visual F 3,422 = 9.3825; p < 0.0001; Vibratory F 3,422 = 75.1745, p < 0.0001). When receptivity to individual signal modes was compared across treatments, females were least receptive to vibratory signals separated by only 45° from visual signals (Fig. 5). Mean comprehensive receptivity score was highest for the multimodal (0° disparity) signal treatment, and not significantly different from the mean score for visual signals in the 45° treatment (t 426 = 0.527; p = 0.599) but was significantly different from all other signals (Fig. 5). A subsequent Bayes analysis strongly supported the null hypothesis of no difference for visual signals between 0° and 45° (BF10 = 0.170). Females tended to exhibit increasing receptivity to vibratory signals as they became more spatially disparate from visual signals (>90°), but there was no significant difference between the 90° and 180° (t 212 = 0.454; p = 0.650) treatments in the mean level of receptivity directed to visual signals (Fig. 5). A Bayes analysis strongly supported the null hypothesis of no difference (BF10 = 0.164).

Fig. 5
figure 5

Matched-pairs analysis of mean comprehensive receptivity scores for spatially disparate male courtship signals (N = 107). Vertical error bars one SEM. Letters over bars outcome of Tukey’s HSD post hoc testing of one-way ANOVA for visual-only and for vibratory-only data. All pairs within treatments were significantly different (p < 0.0001)

Full size image

Temporal disparity experiments

As in the previous experiment, repeated-measures ANOVA showed no clear evidence of behavioral priming or habituation effects overall, as order of treatment presentation and day of trial were not significant predictors of female responses: order of treatment (latency to orient F 2,73 = 0.096; p = 0.909; number of approaches F 2,73 = 317; p = 0.729; comprehensive receptivity score F 2,73 = 0.343; p = 0.711); day of trial (latency to orient: F 2,219 = 2.070; p = 0.129; number of approaches F 2,219 = 1.214; p = 0.299; comprehensive receptivity score F 2,219 = 0.669; p = 0.513). As above, data were pooled across time periods and the analysis was collapsed around treatment as the main effect.

Latency of orientation to stimuli did not vary significantly with temporal disparity treatment (ANOVA: F 2, 219 = 0.427, p = 0.669). Likewise, female approaches to the stimuli did not vary significantly with treatment (ANOVA: F 2,219 = 2.546, p = 0.0807) (Fig. 6). As the p value of 0.0807 might be considered marginal, additional testing was done. Post hoc analyses with Dunnett’s test of comparison of IL and OS with control (IS) showed no significant differences between IL and IS (p = 0.155), or OS versus IS (p = 0.788). Bayesian analysis showed values consistent with acceptance of the null hypothesis for comparisons of interleaved versus out-of-synchrony (IL vs. OS BF10 = 0.343) as well as out-of-synch versus control (IS vs. OS BF10 = 0.219), but intermediate marginal support for interleaved versus control (IL vs. IS BF10 = 0.773). While some females were receptive to all three stimulus treatments, frequency of receptivity was not independent of temporal synchrony (Friedman’s χ 2 = 6.25, df = 2, p = 0.0439). Female receptivity score (measured as sum of receptivity displays) varied significantly with treatment (ANOVA: F 2, 219 = 3.556, p = 0.030). Females displayed significantly higher levels of receptivity (Fig. 7) to the IS (synchronous) stimulus over both the OS (out-of-synch) and IL (interleaved) stimuli (Tukey’s post hoc tests, α < 0.05). Subsequent analyses with Dunnett’s test of comparison of IL and OS with control (IS) showed significant differences between IL and IS (p = 0.044), but not OS versus IS (p = 0.069). Bayesian analyses showed strong support for a significant difference between the control and interleaved treatment (IL vs. IS BF10 = 18.82), an intermediate result for the out-of-synchrony treatment (IS vs. OS BF10 = 1.314), and strong support for no difference between the two experimental treatments (IL vs. OS BF10 = 0.280).

Fig. 6
figure 6

Mean number of approaches females made to temporally disparate or synchronous male courtship signals (N = 78). Vertical error bars one SEM. There were no significant differences

Full size image
Fig. 7
figure 7

Mean comprehensive receptivity score to multimodal video-vibratory playback for temporal disparity stimulus treatments (N = 78). Vertical error bars one SEM (different letters significance by Tukey’s post hoc test)

Full size image

Discussion

Results of these studies strongly suggest that female S. ocreata demonstrate cross-modal integration of spatially and temporally disparate visual and vibratory components of multimodal signals. It has previously been demonstrated in this species that while females are receptive to either courtship signal when unimodal (visual alone or vibratory alone), they exhibit greater levels of receptivity (enhancement) to multimodal signals (Uetz et al. 2009). Here, there was no significant difference in the mean level of receptivity directed to the visual signal in the 45° treatment and to either signal in the 0°/multimodal treatment, strongly indicating that females perceived the 45° visual signal as being multimodal. If this signal was not perceived as multimodal, there likely would have been reduced receptivity to the visual signal, and/or more behaviors would have been directed to the vibratory signal in that treatment. The standard test of a hypothesis of cross-modal binding, suggested by the “ventriloquism effect,” is based on the prediction that disparate signals will be bound to the visual signal as the stronger stimulus (Alais and Burr 2004; Pages and Groh 2013) and that response behaviors will be directed to the origin of the visual signal, which is stronger for humans. Here, female S. ocreata directed the majority of their responses in the 45° treatment to the visual signal and responded to that signal as though it were multimodal. Females thus behaved in a manner indicating cross-modal binding of spatially separate signals, as suggested by the ventriloquism effect and previous tests for cross-modal integration in animals (Narins et al. 2005).

While there are no neurophysiological studies of visual acuity in this species (but see Barth 2002; Land and Nilsson 2012 for details on other spider species), these data strongly suggest that female S. ocreata appeared to recognize signals separated by ≥90° as arising from distinct individuals. Females oriented to and approached both signals, indicating signal disparity did not affect detection or recognition of signals. They approached the visual and vibratory signal with similar frequency, and there were no significant differences in the level of receptivity directed to either signal. Compared to the multimodal signal, females displayed reduced receptivity to spatially disparate signals in a pattern similar to that seen with isolated unimodal (visual alone or vibratory alone) male courtship signals (Uetz et al. 2009). This suggests that females perceive spatially separate signals as coming from different sources, which is consistent with other two-choice studies (Uetz and Norton 2007; Stoffer and Uetz 2015, 2016; Stoffer et al. 2016).

With respect to temporal synchrony of signal modes, female responses are more difficult to interpret, as both signals originated from the same location. In this case, any differences in orientation or approach responses to individual signal modes would be lost. However, there is some indication that a temporal equivalent of the ventriloquism illusion might be in effect, even though there were no significant differences in female orientation and approach behaviors across treatments. It is clear that temporal binding affects the way females perceive male courtship signals, as females were significantly more receptive to signals with temporally synchronous components (IS) than to those with alternating (IL) signals. However, females showed no differences between the IS and OS treatments, suggesting that temporal binding was in effect for the OS treatment. However, in this case it is uncertain whether the overlap of visual and vibration signals might be perceived as a slightly longer multimodal signal (perhaps with an “echo”) or as an atypical or even novel signal. Future experiments might include comparing treatments with overlapped signals with the visual component leading versus one with the vibration component leading to fully parse out female perception of temporally disparate signals.

Signalers and receivers must both contend with environmental complexity, and it is possible that this may have influenced the evolution of cross-modal integration. Environmental complexity presents a challenge to animals attempting to communicate, as signal components may be occluded or altered, and thus the perception and/or interpretation of signals may be affected. A male whose signals reach the female without occlusion or alteration by the environment, or interference from another individual, would definitely have an advantage over males whose signals do. On the other hand, it is essential that a female be able to discriminate among multiple males, and in order to choose the best possible mate, must correctly attribute signals to the appropriate male.

To our knowledge, this is the first study to demonstrate cognitive binding of multimodal signals in an invertebrate, although evidence is mounting that spiders and other invertebrates possess more cognitive ability than given credit for. Previous studies have shown behavioral plasticity in this species (Taylor et al. 2006), as well as both learning and risk-balancing decision-making in other spider species (Jackson et al. 2001; Skow and Jakob 2006). Taken together, results strongly indicate that spiders are capable of more complex perceptual and cognitive processes than had previously been thought.