Introduction

When moving through an earth fixed environment we are exposed to visual and inertial (e.g., vestibular and somatosensory) stimuli. Both the visual, in the form of optic flow (Gibson 1950; Warren and Hannon 1988), and inertial (Guedry 1974; Telford et al. 1995; Gu et al. 2007; Crane 2012) senses provide cues to heading direction. Previous work has examined how visual and inertial cues are combined to yield a unified heading perception. These experiments have introduced small offsets between the visual and inertial heading while changing the relative reliability of the stimuli (Fetsch et al. 2009, 2012; Gu et al. 2008). In such situations, the visual and inertial stimuli are integrated near optimally based on their relative reliabilities or Bayesian ideal (Knill and Pouget 2004; Fetsch et al. 2010; Crane 2017).

Sensory systems are continually processing stimuli from multiple sensory modalities and sources. In some situations, it is appropriate to integrate sensory signals while in other situations separate consideration is more appropriate. For multisensory integration to occur the sensory signals must be perceived to have a common causation, which has led to a causal inference model of multisensory perception (Körding et al. 2007; Sato et al. 2007; Wozny et al. 2010). In prior studies of visual–inertial heading integration a multisensory integration strategy has been shown to occur (Fetsch et al. 2009; Angelaki et al. 2011; Butler et al. 2010, 2011). In these prior visual–inertial heading perception experiments the offset between the stimuli was small. Even without an offset, during eccentric gaze, the visual and inertial headings were perceived to be about 15° different when presented separately suggesting physiological offsets and integration occur during such circumstances (Crane 2017). It is not always appropriate to perceive optic flow and inertial heading in the same direction: Although inertial information almost always represents self-motion, in situations such as moving through a crowd during a or snow storm visual optic flow cues may represent external motion rather than self-motion through an earth fixed environment. Thus, there are situations in which visual and inertial headings should be perceived as having independent causation and not integrated. One study demonstrated when visual and inertial headings were separated that subjects tended to use only one sensory modality to indicate heading, and detection thresholds were larger than in the visual-only condition (de Winkel et al. 2010). Suggesting that multisensory integration does not always occur in some situations. In that particular case it may be because the motion profiles of the visual and inertial stimuli were perceived as dissimilar in part because of the characteristics of the stimuli used.

Causal inference strategies have been examined in visual–inertial heading integration (Acerbi et al. 2018). Human subjects were presented with visual and inertial heading stimuli within ± 25° of straight ahead with cue disparity of up to ± 40°. They were asked to judge if stimuli were in the same or different directions (explicit causal inference) or if the inertial stimulus was to the right or left of the visual stimulus (implicit causal inference). They found causal inference was best fit by a model that was independent of visual cue reliability (coherence). The study only included headings near straight ahead and did not use a cue disparity greater than 40°, which did not allow the limits of integration to be fully explored. Furthermore, the subjects were only asked to judge the angle of the inertial heading only as left or right. At the largest cue disparity tested (40°) subjects still felt the cues were the same in about 25% of trials, and at all other disparities tested it was 50% or greater. Thus, the range of cue disparities where the cues can reliably be identified as different was not explored. A second limitation was that subjects were only asked relative headings, so that disparities of 10°–40° had a similar effect and the influence on heading estimation or absolute heading direction was unknown.

Current knowledge of the effect of cue disparity on visual–inertial heading integration is limited. Some studies have limited it to ± 40° (Acerbi et al. 2018) or ± 20° (Ramkhalawansingh et al. 2018) which is insufficient to explore the limits of common causation. One study demonstrated that visual–inertial heading offsets of up to 90° were still perceived as having common causation in most subjects (de Winkel et al. 2015). A later experiment by this group looked at effects of stimulus duration, velocity, and acceleration on perception of common causation in visual–inertial headings, but the maximum difference between visual and inertial headings was limited to ± 90° (de Winkel et al. 2017). In those experiments subjects were instructed to estimate self-motion direction, but were not told to consider visual and inertial headings separately. Thus, the instructions given to the subjects implied that the visual and inertial headings should not be considered independently, which may have influenced perception. The current study looks at synchronous visual and inertial headings and asked subjects to separately report the direction of each in separate trial blocks. The effect of the range of headings presented and visual stimulus polarity (i.e., reporting environmental vs self-motion) on perception are considered.

Methods

Ethics statement

The research was conducted according to the principles expressed in the Declaration of Helsinki. Written informed consent was obtained from all participants. The protocol and written consent form were approved by the University of Rochester research science review board prior to the study being conducted.

Human subjects

Ten subjects participated in the experiments. Six of the subjects were female (#1, 3, 4, 6, 9, and 10). Mean age was 43 ± 19 years (mean ± SD, range 24–70). Subjects had no history of dizziness or vestibular disease, and they had vision that was normal or corrected to normal. Because the experiments were done over a period of time, not all subjects were available to participate in each experiment.

Equipment

Inertial stimuli were delivered using a 6-degree-of-freedom motion platform (Moog, East Aurora, NY, model 6DOF2000E) similar to that used for human motion perception studies in other laboratories (Fetsch et al. 2009; MacNeilage et al. 2010; Grabherr et al. 2008; Cuturi and Macneilage 2013) and described previously for the current laboratory in past heading estimation studies (Crane 2012, 2014). Subjects sat in a padded racing style seat which included a four-point safety harness. A helmet was used to couple the head to the motion platform.

During visual, inertial, and combined stimulus conditions, an audible white noise was reproduced from two platform-mounted speakers on either side of the subject as previously described (Roditi and Crane 2012). The noise from the platform was similar regardless of motion direction or test condition. Tests in the current laboratory previously demonstrated that subjects could not predict the platform motion direction based on sound alone (Roditi and Crane 2012).

Stimuli

Visual and inertial stimuli were presented concurrently and represented the same motion profile. The inertial stimuli consisted of a single cycle 2 s (0.5 Hz) sine wave in acceleration. This motion profile has been previously used for heading estimation in the horizontal plane (Crane 2012). The visual component was presented on a color LCD screen with a 1920 × 1080 pixel resolution at 50 cm from the subject which fills 98° horizontal field-of-view. The visual component consisted of a star field which simulated the movement of the observer through a random-dot cloud with binocular disparity, described previously (Crane 2012). Each star was a yellow triangle with a height of 0.5 cm and width at the plane of the screen adjusted for distance. Star density was 0.01 per cm3. The furthest stars were 150 cm and the nearest stars were 20 cm (a depth of field of 130 cm). Visual coherence was set to 70 percent for each trial block to provide a similar relative reliability to both the visual and inertial cues in typical subjects (Crane 2017). Coherence was defined as one minus the fraction of points that were randomly repositioned between 60 Hz visual frames.

Experimental procedure

Four experiments were conducted with varying offsets between visual and inertial heading stimuli. The experiments differed in terms of the direction and offsets that were possible as well as the instructions given to subjects. All the experiments were divided into two blocks, one reporting inertial heading and another reporting visual heading although both stimuli were always presented concurrently. Subjects reported heading using a mechanical dial that was centered in front of them below the screen. The dial could be freely rotated about a vertical axis without any discontinuities. After orienting the dial, subjects pressed a button on either side of it to report they were finished. Subjects were asked to report their perception in three different ways in different trials (Fig. 1): (1) the direction of inertial motion (I), (2) the direction of self-motion implied by the visual stimulus (Vs) which can be described as the direction the stars were coming from, and (3) the direction of implied environmental motion (Ve) which can be described as the direction the stars were moving. A negative angle corresponds to the left and a positive angle corresponds to the right. Sound cues were used to denote start of trials as previously described (Crane 2014).

Fig. 1
figure 1

Experimental setup and diagram demonstrating the relationship between the stimulus directions subjects were asked to report. a The subject sat in front of a viewing screen with a dial to report the perceived heading direction. b Example of paired visual–inertial stimuli in which the inertial stimulus (I) is straight ahead. c The associated visual stimulus (Vs) corresponding to self-motion through a fixed environment is shifted 30° to the right. This same visual stimulus can also be interpreted as environmental motion in the opposite direction relative to a fixed observer (Ve)

Full size image

Experiment 1

The inertial stimuli had a range of ± 30° in 10° increments and the visual offset had a range of ± 60° in 20° increments so that the maximum visual heading was 90°. The visual and inertial headings were delivered in random order throughout the trial block with each visual–inertial combination (a total of 49 unique stimuli) delivered twice. In one block subjects were asked to report the inertial heading (I) and in the other block the self-motion heading implied by the visual stimulus (Vs). Experiment 1 included subjects: 1, 2, 3, 9, and 10. There were 196 stimulus presentations per subject.

Experiment 2

These experiments were similar to experiment 1, however the inertial stimulus range was expanded to ± 140° in 35° increments and a visual offset range was ± 120° in 30° increments. Subjects were again instructed to report I and Vs in separate trial blocks. This experiment was done because it was found that the range of stimuli used in experiment 1 was not sufficient to demonstrate conditions in which there was a complete dissociation in how visual and inertial stimuli were perceived. Experiment 2 included subjects: 1, 3, 4, 5, 6, 9, and 10. There were 324 stimulus presentations per subject, with 81 unique stimuli combinations presented.

Experiment 3

The same range of inertial and visual stimuli as experiment 1 were used. However, the visual stimuli were in the opposite direction relative to experiment 1 and subjects were asked to report the direction of environmental visual motion (Ve). Thus when compared with experiment 1, the visual stimuli were in the opposite direction but because of the instructions to the subjects the ideal responses were the same. As with experiment 1, they also reported the direction of inertial motion, I, in some trials. This experiment investigated the potential for the reported stimulus direction rather than actual direction to influence perception. Experiment 3 included all subjects. There were 196 stimulus presentations per subject.

Experiment 4

The same range of inertial and visual stimuli in experiment 2 was used, but with the same instructions given in experiment 3, i.e., environmental visual motion (Ve) was reported. Unlike experiment 3, in this experiment the visual and inertial stimuli covered nearly the full range so there was no need to reverse the direction of the visual stimuli. Experiment 4 included subjects: 1, 2, 4, 5, 9, and 10. There were 324 stimulus presentations per subject.

Data analysis

Perceived headings (both inertial and visual) were expressed as ± 180° relative to the delivered inertial heading. The experiment began with an exploration of the influence of visual stimuli on inertial heading perception and so a paired two-tailed t test was run between the visual and inertial responses at each inertial angle (p < 0.05). The visual offsets relative to the inertial offsets had a similar effect regardless of the direction of the offset (e.g., 30° and − 30° had similar influences, but in opposite directions). The two opposite offsets were combined by reflecting the responses of the negative offsets. In other words, the responses when the visual offset was to the left were inverted so they could be combined with the condition when the visual stimulus was to the right of the inertial heading. The combined positive and negative offsets led to the use of one-way ANOVA for comparisons of all subjects, as there were twice as many offset responses compared to the 0° offset. Individual subject responses were compared using a rmANOVA and significance was determined using a Bonferroni test. All statistical analysis and linear regressions were performed using Kaleidagraph 4.5 (Synergy Software, Reading, PA, USA). To normalize the level of influence across offsets, both the visual responses and the inertial responses were subtracted from the inertial headings. Thus the ideal normalized visual response was always zero and the ideal visual response was always the offset.

Results

Performance of a single subject which demonstrates the typical variation between trials is shown (Fig. 2).

Fig. 2
figure 2

Subject 3 was asked to report direction of suggested self-motion through a fixed environment (Vs, solid arrow). The inertial stimulus direction is shown on the x-axis (abscissa) while the visual stimulus is shown relative to the inertial stimulus on the y-axis (ordinate). Solid lines are drawn to show the ideal performance for inertial headings (solid line at 0) and visual headings (solid line at the offset). Trials in which the visual stimulus was reported are shown as an open circle, inertial trials are shown as an gray diamond. Each visual and inertial trial response is shown (small diamonds and circles) and averaged. Dashed lines represent a linear regression fit to the actual responses. This example is from experiment 1, but similar variation was seen in each experiment. The results for each experiment at reflective offset angles (e.g., − 60° and 60°) were combined and averaged for each offset: a 20°. b 40°. c 60°

Full size image

Experiment 1

Inertial stimuli were limited to the range of near straight ahead (± 30°). Visual stimuli were offset ± 20°, 40°, or 60° relative to inertial heading. With all conditions it was found that when subjects were asked to report the direction of the inertial stimulus there was a strong bias in the direction of the visual stimulus (Fig. 3). Headings were expressed such that the ideal visual response was the offset relative to the inertial heading and the ideal inertial response was defined as zero. For the smallest offset of 20°, the perception was similarly independent of whether subjects were asked to report the visual (22 ± 17°; mean ± SD) or inertial direction 14 ± 15°. For offsets of 40° and 60° the mean visual offset remained near ideal at 46° and 62°. With all offsets, the inertial stimulus had minimal influence on perceived direction when subjects were asked to report the visual stimulus (Vs). The perception of inertial heading was influenced by the visual stimulus at all offsets. With the largest 60° visual offset the inertial heading was offset 35° ± 25° towards the visual stimulus. Performance was varied between subjects with subjects 1, 2, and 3 perceiving the inertial close to visual, subject 9 demonstrating a smaller influence of visual on inertial, while subject 10 perceived them relatively independently (Fig. 4). In most subjects, this experiment demonstrated the perceived inertial heading was strongly biased such that it was perceived as near the visual heading over the range of offsets tested. During each condition, the inertial heading perception was significantly shifted towards the visual heading (p < 0.05 for 0° vs 20°, 40°, and 60°; one-way ANOVA; Bonferroni test). The visual heading was slightly overestimated as being away from the inertial heading. One potential confounding factor in these experiments was that subjects may have determined that inertial stimuli only fall within a relatively narrow range of straight ahead (± 30°) while the visual stimulus could be well outside of this narrow range, and such knowledge may have influenced perception.

Fig. 3
figure 3

Aggregate results from experiment 1. Inertial stimuli were limited to the range of ± 30° with visual offsets of ± 20, ± 40°, or ± 60°. Data were plotted similar to Fig. 2 with the inertial stimulus direction on the abscissa and the visual offset on the ordinate. Error bars represent the standard error of the mean (SEM). Ideal performance is shown as a solid line while fits to the perceived heading are shown as a dashed line. Fits were performed by linear least squares regression using Kaleidagraph software. For each offset (20°, 40°, 60°), visual R values were: 0.49, 0.14, and 0.12, respectively, while inertial R values were: 0.73, 0.38, and 0.09, respectively. Inertial angle has been subtracted from the visual and inertial responses

Full size image
Fig. 4
figure 4

Results of experiment 1 averaged by subject across the range of inertial heading offsets. Error bars represent ± 1 SEM. *Significant shift in heading perception vs 0° offset in both positive and negative offsets (i.e., ± 20°). †Significant shift in heading perception vs 0° offset in positive offset (i.e., + 20°), but not negative offset (i.e., − 20°). ‡Significant shift in heading perception vs 0° offset in negative offset (i.e., − 20°), but not negative offset (i.e., + 20°). Significance measured using rmANOVA; Bonferroni test

Full size image

Experiment 2

To address the potential effects of the narrow range of stimuli and relatively small offsets between visual and inertial stimuli tested in experiment 1, the range of inertial stimuli was expanded to ± 140° in 35° increments and the range of visual offsets was increased to ± 30°, 60°, 90°, and 120°. With a ± 30° offset the visual and inertial stimuli were perceived to have a similar direction (p = 0.1407, paired t test) which was intermediate between the two (Fig. 5a). With larger offsets the visual and inertial headings were perceived as different (p < 0.001 at every heading) but were both significantly influenced by the other modality such that the perceived differences were smaller than the actual offsets (Fig. 5b–d). In contrast to experiment 1, the mean visual heading always underestimated the actual heading suggesting that visual heading perception was influenced by the inertial heading. However, at every offset the perceived inertial heading was shifted towards the visual heading by a greater amount suggesting the visual heading still had the larger influence. During each condition, the inertial heading perception was significantly shifted towards the visual heading (p < 0.05 for 0° vs 30°, 60°, 90°, and 120°; one-way ANOVA; Bonferroni test). Visual and inertial stimuli influenced each other such that during the combined stimulus presentation the perceived headings (dashed lines) were closer together than the actual difference (solid lines). In this experiment there was variation in the perception between subjects (Fig. 6) with subjects 4, 5, and 10 perceiving the visual and inertial stimuli relatively independently. Subject 6 perceived the visual stimulus similar to the inertial heading, while subjects 1 and 3 tended to perceive the inertial heading closer to the visual heading. Subject 9 perceived inertial and visual heading near a combination of the two stimuli, except at a ± 120° offset in which case inertial and visual headings were perceived independently.

Fig. 5
figure 5

Aggregate results from experiment 2. Subjects reported inertial (diamond) or visual (circle) stimulus similar to experiment 1. The inertial stimulus range was expanded to ± 140° in 35° increments with a visual stimulus offset of ± 30 to ± 120° in 30° increments. Error bars represent ± 1 SEM. Fits to the perceived heading are shown as a dashed line. Fits were performed by linear least squares regression using Kaleidagraph software. For each offset (30°, 60°, 90°, 120°), visual R values were: 0.18, 0.33, 0.07, and 0.23, respectively, while inertial R values were: 0.34, 0.25, 0.14, and 0.003, respectively. Inertial angle has been subtracted from the visual and inertial responses

Full size image
Fig. 6
figure 6

Results of experiment 2 by subject. The values represent the average of all inertial stimulus offsets. These results represent a variation in perception by subject: subjects #3, 4, and 10 tended to perceive the visual and inertial stimuli in different directions while the remaining subjects reported them in a similar direction. Error bars represent ± 1 SEM. *Significant shift in heading perception vs 0° offset in both positive and negative offsets (i.e., ± 30°). †Significant shift in heading perception vs 0° offset in positive offset (i.e., + 30°), but not negative offset (i.e., − 30°). ‡Significant shift in heading perception vs 0° offset in negative offset (i.e., − 30°), but not negative offset (i.e., + 30°). Significance measured using rmANOVA; Bonferroni test

Full size image

Experiment 3

Examined the potential effect of reporting a visual heading direction even if it was not consistent with moving through a fixed environment. The limited range of inertial and visual stimuli in experiment 1 were used, but the direction of the visual stimuli was shifted 180°. Subjects were asked to report inertial heading, I, in one trial block (as before) and the direction of environmental visual motion (Ve) in the other. Thus, the ideal response for the visual stimulus was the same as the response in experiment 1, although the visual stimulus was in the opposite direction. Perception of visual motion and inertial motion was near ideal (Fig. 7), while the visual and inertial stimuli having minimal influence on how the other was perceived. Inertial heading perception was significantly shifted (relative to 0°) at the 60° condition (p < 0.05; one-way ANOVA; Bonferroni test). Performance across subjects was relatively uniform (Fig. 8).

Fig. 7
figure 7

Aggregate results from experiment 3: The inertial stimuli were similar to experiment 1, but the visual stimuli moved in the opposite direction and subjects were asked to report the direction of environmental motion (Ve). Inertial stimuli covered the range of ± 30° and visual stimuli were offset by ± 160°, ± 140°, or ± 120°. These offsets were chosen such that the reported direction of Ve would be ± 20°, ± 40°, or ± 60° from I, similar to the report of Vs in experiment 1. Subjects reported the direction of inertial motion as the direction they were going; thus, the range of stimuli were such that the direction of self-motion implied by visual motion would never be in the range of inertial headings tested. Perception of both inertial and visual motion was close to ideal performance. Error bars represent ± 1 SEM. Fits were performed by linear least squares regression using Kaleidagraph software. For each offset (20°, 40°, 60°), visual R values were: 0.02, 0.13, and 0.03, respectively, while inertial R values were: 0.38, 0.63, and 0.09, respectively. Inertial angle has been subtracted from the visual and inertial responses

Full size image
Fig. 8
figure 8

Results of experiment 3 by subject. Values represent the average of all inertial offsets tested. All subjects perceived the visual and inertial stimuli in separate directions. Error bars represent ± 1 SEM and if not shown are smaller than the plot symbol. ‡Significant shift in heading perception vs 0° offset in negative offset (i.e., − 30°), but not negative offset (i.e., + 30°). Significance measured using rmANOVA; Bonferroni test

Full size image

Experiment 4

Examined the effect of reporting the direction of visual environmental motion (Ve) and inertial heading (I) when a wider range of stimuli were considered. These stimuli were the same as experiment 2 with the subject essentially asked to report Ve which was opposite Vs. Unlike experiment 3, the visual stimulus headings were not reversed, so the only change relative to experiment 2 was in the instructions given the subject. This change in how visual stimulus direction was reported had no effect on the inertial stimulus perception, which continued to have a deviation towards Vs, even though subjects were asked to report the opposite. The perception of Ve was shifted towards I by an average of 12° for the largest offset (Fig. 9a), and 13–36° for smaller offsets (Fig. 9b, c). Inertial heading perception was significantly shifted (p < 0.05) for 0° vs 60°, 90°, and 120° (one-way ANOVA; Bonferroni test). Examination of the individual subjects reveals much of this deviation was due to subject 1, while the inertial stimulus had little influence on Ve in most subjects (Fig. 10).

Fig. 9
figure 9

Aggregate results from experiment 4: Visual and inertial stimuli were similar to experiment 2. Subjects reported the direction of inertial motion as the direction they were going while reporting the visual motion as the direction of environment motion (i.e., opposite the direction implied by motion through a fixed environment). In each case the perception of the visual environmental motion (Ve) direction was less (closer to the inertial heading) than ideal. The direction of inertial motion continued to be biased towards Vs (180° from Ve). Fits were performed by linear least squares regression using Kaleidagraph software. For each offset (30°, 60°, 90°,120°), visual R values were: 0.03, 0.16, 0.35, and 0.04, respectively, while inertial R values were: 0.66, 0.42, 0.43, and 0.02, respectively. Inertial angle has been subtracted from the visual and inertial responses

Full size image
Fig. 10
figure 10

Results of experiment 4 by subject. Values represent the average of all inertial offsets tested. *Significant shift in heading perception vs 0° offset in both positive and negative offsets (i.e., ± 30°). †Significant shift in heading perception vs 0° offset in positive offset (i.e., + 30°), but not negative offset (i.e., − 30°). ‡Significant shift in heading perception vs 0° offset in negative offset (i.e., − 30°), but not negative offset (i.e., + 30°). Significance measured using rmANOVA; Bonferroni test

Full size image

Although the experiments included subjects from a wide age range (20–70 years), the experiments were not designed to look at effects of aging (e.g., having groups defined by age) and effects seen in some subjects like segregation of visual and inertial perception appeared to occur across the age range and were unlikely to be age related.

Discussion

The current findings extend those of previous experiments that have demonstrated perception of common causation with relatively large offsets between visual and inertial headings. Previous work examined visual–inertial offsets of up to ± 90° with subjects reporting their overall perception from the combined stimulus (de Winkel et al. 2015, 2017). They concluded that visual and inertial stimuli being perceived as a single heading (i.e., common causality) was part of the multisensory integration process, but the nature of the experiment also forced subjects to report only a single heading. Some prior work has reported perception of individual sensory modalities but this work focused on a more limited range of offsets that did not explore the range of offsets where stimuli were reliably perceived independently, for instance, other studies have used offsets limited to only 10° (Butler et al. 2010), 20° (Ramkhalawansingh et al. 2018) or 40° (Acerbi et al. 2018). The current experiments contribute to this area by examining a wider range of headings, collecting independent reports of visual and inertial headings, and separating effects of visual environmental motion from the appearance of self-motion through a fixed environment.

Integration of visual–auditory cues has been studied much more extensively relative to visual–inertial heading cues. The best example of visual–auditory cue integration is the “ventriloquism” effect in which the localization of sound is perceived as coming from a different location when there is a visual stimulus consistent with common causation. The angular limit for the “ventriloquism” effect is about 30° in the horizontal plane (Recanzone 2009; Thurlow and Jack 1973; Witkin et al. 1952). The current study, and other recent studies of visual–inertial heading integration (de Winkel et al. 2015, 2017) demonstrate the binding window is at least threefold larger. This begs the question as to why visual–inertial integration is relatively insensitive to large offsets. One possibility is that visual heading perception occurs in a retina based coordinate system such that gaze offsets cause visual heading estimates to be biased (Crane 2015, 2017), while inertial headings are perceived relative to body coordinates (Chen et al. 2013; Yang and Gu 2017). One way to deal with such a situation would be to reconcile these into a common coordinate system, but this does not occur (Crane 2017; Yang and Gu 2017). In the absence of a common coordinate system, it might be appropriate to allow both sensory modalities to influence each other even with a large offset.

The visual–inertial heading integration strategy was variable between subjects in some conditions. When the headings were close together and limited to near straight ahead (experiment 1), or subjects were asked to report visual headings as the direction of environmental motion (Ve, experiments 3 and 4) the strategy was relatively uniform between subjects. However, when a full range of inertial headings was possible and paired with potentially large offsets in the visual stimulus direction, performance was varied (Fig. 6). Subjects 1 and 3 tended to exhibit a visual capture strategy in which both the inertial and visual stimulus were perceived closer to the visual heading. This was similar to the strategy used by most subjects in experiment 1 when the range of stimuli and offsets were limited. Subject 6 had an opposite inertial capture strategy in which both stimuli were perceived closer to the inertial stimulus. The remaining subjects perceived the visual and inertial stimuli separately even with relatively small offsets. Thus, it seems a full range of strategies are potentially used when the range of possible stimuli and offsets is large. In a subset of subjects heading differences as great as 120° are perceived as similar (Fig. 5d, subjects 3 & 6) while others perceive them as different with differences as small as 20° (Fig. 4a, subject 10). When the same subject was tested across conditions, they tended to show similar trends across the offsets tested. The reason for this variation between subjects is unknown, as all subjects received similar instructions and we did not include individuals with motion sickness or migraine history. We believe all subjects had a similar expectation that the visual and inertial stimuli would not necessarily be in a similar direction, but subjects were not told of the types of offsets that would be possible in each trial block and expectations could have varied. Previous work in this area that considered individual subject performance has also demonstrated that strategies can be different across participants (de Winkel et al. 2015, 2017), which makes it difficult to generate a theory that predicts individual performance.

These experiments used a task in which visual and inertial stimuli were presented together and the direction of each was reported in separate trial blocks. This is not the only way to approach the question of common causality. Others have addressed the issue by looking at responses during the unisensory condition and comparing this with the perception during the combined stimulus presentations (de Winkel et al. 2015, 2017). Asking subjects to judge only one sensory modality when two are presented might imply that the two modalities are separate, while asking for a combined perception might imply that they are the same or force subjects to consciously choose an intermediate heading even if they were perceived as different. Another option would be to ask subjects to report if the sensory modalities represent the same or different headings or to report if one modality is to the left or right of the other. To our knowledge no such experiments have been done with visual–inertial integration.

In conclusion, limiting the range of inertial headings presented allowed visual headings, with a larger possible range, to dominate the perception of the combined stimulus (Fig. 11). With a wider range of stimuli there was significant variation between how subjects perceived simultaneous visual and inertial headings. In most subjects the visual and inertial headings influenced each other such that perception was shifted towards an intermediate heading even with large offsets. When asked to report visual motion as the environmental motion direction, the inertial heading had little influence.

Fig. 11
figure 11

White bars are the percentage inertial heading perception changed due to visual stimuli. Black bars are the percentage visual heading perception changed due to inertial stimuli. a Experiment 1: as visual–inertial offset increased, the influence of visual stimuli on inertial perception decreased. b Experiment 2: visual influence on inertial perception continued to decrease with larger offsets. a (-R tag) Experiment 3: negligible visual influence on inertial perception. c Experiment 4: visual influence on inertial perception decreased as effective visual offset increases

Full size image