Introduction

The haptic sense allows us to skillfully interact with objects in our environment. Reaching towards an object as well as grip scaling requires accurate visuo-haptic transfer of spatial object information. Haptic perception refers to tactual perception in which both the cutaneous sense and kinesthesis convey significant information about distal objects and events (Loomis and Lederman 1986). Whereas the haptic (active tactile) sense exceeds the visual sense when it comes to the extraction of material object properties, it is less suited for the extraction of spatial information (Lederman and Klatzky 2004).

Systematic errors occured when blindfolded participants were asked to match the orientation of a manually explored reference bar by turning a test bar to a parallel orientation, using the contralateral hand (Kaas and Van Mier 2006; Kappers 2003, 1999; Newport et al. 2002). In principle, veridical performance on this parallel matching task can only be achieved in an allocentric reference frame, preserving the metric of extrapersonal space. Yet, error size and direction showed a high correlation with the natural orientation of the hands at the bar locations. Error size increased with larger horizontal distances between reference and test bar. In addition, errors showed a typical directionality, that is, when matching a reference orientation parallel to the midsagittal plane (i.e. 90°), the end of the test bar that was farthest away from the participant was turned away from the midsagittal plane (positive error see Fig. 4). This indicates that the haptic system cannot factor out the biasing effect of hand orientation (Kappers 2004).

In contrast, a similar spatial task, haptic mirror matching, induced much smaller errors. In this task, participants turned a test bar to an orientation, which was the mirror image of a reference orientation with respect to the midsagittal plane of the body (Kaas and van Mier 2006; Kappers 2004; Newport et al. 2002). Kappers showed that the parallel and mirror matching performance can be described in a common reference frame. This so-called intermediate reference frame is a weighted average of an egocentric, hand-centered frame, and an allocentric, world-based frame, in which the position of an object is coded with respect to other objects. The accuracy difference between parallel and mirror matching might arise because mirror matching can take advantage of the mirror symmetry of the hands and arms with respect to the midsagittal plane. Hence, while parallel matching is characterized by a systematic positive bias caused by the transfer of information from one spatial location to another, accurate performance in the mirror task can be achieved by exact reproduction of the bar orientation with respect to the contralateral hand.

Imaging data from our lab did not show any differences in the neural substrates underlying parallel and mirror matching (Kaas et al. 2007). Similar sensorimotor, frontal and parieto-occipital areas were found in the delay between haptic exploration for subsequent parallel or mirror matching. Comparable parieto–occipital areas have previously been reported to respond to changes in visual orientation (Valyear et al. 2006) and to purely cutaneous discrimination of tactile gratings passively applied to the finger pad (e.g. Zangaladze et al. 1999).

Even though haptic parallel and mirror matching might be supported by the same brain areas and a common intermediate reference frame, Newport et al. (2002) demonstrated that these tasks were differentially affected by noninformative visual cues from the surrounding room. The term “noninformative vision” indicates that the visual cues were not explicitly relevant to the task at hand. Whereas parallel matching improved when participants could freely view the region above the haptic workspace, mirror matching was unaffected or even deteriorated. This finding was interpreted as evidence for a preferential use of extrinsic, visual coordinates in the parallel task, and intrinsic, limb-based coordinates in the mirror task.

The beneficial effects in the Newport study might be partly explained by gaze orienting towards the site of haptic stimulation, which has been shown to improve parallel matching (Zuidhoek et al. 2004). However, over and above the effect of gaze orienting, an effect of noninformative vision remained in Zuidhoek’s study. This might be caused by a resolution-enhancing reorganization of tactile receptive fields due to feedback connections from multimodal brain regions to unimodal tactile areas (Kennett et al. 2001). On the other hand, Millar and Al-Attar (2005) found no beneficial effect on a tactile landmark task when the visual stimulation consisted of diffuse light, eliminating all spatial cues. They argue that spatial and non-spatial aspects have to be distinguished when considering the effect of vision on haptics, and propose that vision only improves haptic spatial performance to the extent that it adds cues with potential relevance to spatial discrimination and reference. In the noninformative vision studies by Newport (2002) and Zuidhoek (2004) and colleagues the visual surroundings offered an abundance of vertical and horizontal cues. These contextual spatial cues, together with the presence of a fixed haptic start orientation parallel to the midsagittal plane in Newport´s study, might have helped participants to build a stable visuo-haptic spatial reference frame. Hence, the availability of visual input might have changed the strategies used for parallel and mirror matching in the visual condition.

The interaction between vision and touch has been a matter of debate for centuries. In 1709, Berkeley postulated that there are no necessary connections between the visual and haptic modality. In modern day cognitive neuroscience there is theoretical and empirical support for the view that input from different sensory modalities is initially processed in parallel streams, and that direct cortical connections between somatosensory and visual areas are sparse (e.g., Mesulam 1998). This leads to the hypothesis that cross modal associations might to a large extent depend on experience. Still, some hard-wired neuronal circuits might detect stimulus relations that are particularly frequent and important. For instance, the ability to visually recognize an object that was previously presented to the (right) hand has been demonstrated already in newborns (Streri and Gentaz 2004; Streri 2003), testifying to an early, though fragile, ability to extract tactile information and transform it into a visual format. In adulthood, humans are capable of reversibly transferring object and scene information between the tactile and visual domain (Newell et al. 2005; Woods et al. 2004). Neuroimaging studies indicate that the insula-claustrum and anterior intraparietal sulcus might support the exchange of visuo-haptic three-dimensional shape information (Grefkes 2002; Hadjikhani and Roland 1998), whereas Saito et al. (2003) found posterior intraparietal sulcus, near the parieto-occitital sulcus to show higher activation when participants performed a visuo-tactile matching of two-dimensional shapes. A nearby region in parieto-occipital cortex was found to be active in both tactile and visual discrimination of finger pad-sized grating orientations (Kitada et al. 2006; Zangaladze et al. 1999; Sathian et al. 1997).

Visual input has been shown to change the strategy used on haptic sorting tasks (Klatzky et al. 1987). Other studies have shown the importance of vision for motor planning and the calibration of somatosensation. For instance, Rock and Victor (1964) showed that haptic perception of shape was changed in participants wearing minifying lenses. In addition, it has been shown that the felt finger position is recalibrated according to the retinal position in prism adaptation studies (Clower and Boussaoud 2000), that visual context shapes pointing errors (Battaglia-Mayer et al. 2003), and that the transformation between eye- and hand-centered reach locations is achieved in an eye-centered reference frame (Buneo et al. 2002). Visual effects on somatosensation might be especially salient in situations of uncertainty, as indicated by a series of classic experiments by Shimojo and Shams (2001). They have shown that uncertainty in one modality is resolved by using temporally contingent information from other modalities.

The aim of the current study was to investigate the effect of presenting visual orientation cues during the exploration phase of haptic parallel and mirror matching tasks on signed error, response precision (standard deviation) and matching time. The visual cues were either congruent or incongruent with respect to the haptic reference orientation. We assumed that parallel matching favors the use of visual coordinates while mirror matching promotes a somatosensory frame. We therefore expected that congruent and incongruent visual orientation cues would differentially affect performance during parallel and mirror matching. Our first hypothesis was that parallel matching accuracy would parametrically vary with the direction of the visual incongruent orientation cues. Concretely, we expected that negatively incongruent visual orientations, counteracting the typical positive bias, would lead to smaller errors than congruent visual orientations, which in turn would induce smaller errors than positively incongruent visual orientations (H1a). On the other hand, we expected mirror accuracy to show no such dependence on visuo-haptic incongruence level (H1b). Finally, if vision per se indeed has an influence on haptic performance, we expected to find a general accuracy increase not only for visuo-haptic congruent but also for visuo-haptic incongruent parallel matching compared to purely haptic parallel matching (H2a), while no such general accuracy increase was anticipated for visuo-haptic (in)congruent mirror matching compared to purely haptic mirror matching (H2b).

Materials and methods

Participants

Thirty right-handed participants (6 males and 24 females, average age 21.9 years, sd 2.6) were recruited through advertisements posted in several University buildings. Handedness was assessed with a Dutch translation of the Edinburgh Handedness Inventory (Oldfield 1971). Participants had either normal or corrected-to-normal visual acuity. Participation was rewarded with a gift certificate or credit points to fulfill course requirements. The study was approved by the local ethics committee and was performed in accordance with the declaration of Helsinki. Prior to the experiment, participants were naive with respect to experimental goals and setup. Before starting the experiment, they received training on the different tasks. It was not until they fully understood the entire procedure that the actual experiment began.

Apparatus

The setup consisted of two turn bars (10 × 1 cm) with a blunt plastic needle at each end, mounted on plate, which was attached to a table. The bars’ centers were 60 cm apart and positioned at 11.5 cm from the edge (of the participant’s side) of the table. The orientation of the bars could be read with an accuracy of 0.5°. In the following, the bar used to present the reference orientation will be referred to as “reference bar”, and the bar, which participants had to turn to a matching orientation will be called “test bar”. A large button box was positioned directly behind the test bar (see Fig. 1a). Two small round patches of cloth were attached near the edge of the table, just in front of the bars’ centers, indicating the resting positions for the fingers when participants were not touching the bars. The hands or belly did not touch the edges of the table.

Fig. 1
figure 1

A schematic overview of the experimental set-up, a seen from above and b from aside. Participants were blindfolded in the haptic runs. The response button was always located behind the test bar; that is, on the right side when the right hand was used for matching, and vice versa. The curtain used in the visuo-haptic runs is displayed as transparent grey in the picture. However, it was black and opaque in reality

Full size image

Participants did not receive any direct visual information from the two turn bars. A black vertical board (height 75 cm × width 90 cm) was attached to the table top directly in front of the participant, preventing any visual information of the bars and the hands. The board contained two square indentations (width 25 cm, height 15 cm) for the hands in the lower left and right corners, allowing participants to comfortably touch the turn bars without seeing their hands. Another circular opening (diameter 20 cm) was located in the middle of the board, its center 50 cm above the table top. A black tube (diameter 20 cm × length 50 cm) was attached with one end to the opening in the board and the other end to a computer screen (40 cm diagonal). This ensured that the participant could only see the screen itself, and would not be influenced by the edges of the computer screen or other external cues. Participants were seated on a chair with adjustable height, making sure that each participant was able to view the screen in a comfortable way. A black curtain, glued to the top of the board was placed over the participant’s head and back, eliminating all visual input except for the stimuli presented on the computer screen (see Fig. 1b). Participants were blindfolded in the haptic runs. Auditory instruction cues were presented to the participants through headphones.

Procedure

Participants were sitting in front of the table supporting the two turn bars, their midsagittal plane coinciding with the centre plane of the stimulus plate, halfway between the bars. An auditory cue (1,000 Hz) indicated that participants had to feel the orientation of the reference bar, in a time interval corresponding to the length of the tone (1,500 ms; see Fig. 2). In the visuo-haptic runs, a visual stimulus was presented in the same time interval. Half of the participants used the left hand for exploration and the right hand for matching; the other half used the right hand for exploration and the left hand for matching. When the tone ended the visual stimulus was removed, and participants returned their fingers to the resting position on a piece of cloth in front of the bar. After a delay of 5,000 ms another tone (2,000 Hz, 100 ms) announced that participants had to turn the test bar, either to a parallel or a mirrored version of the reference orientation, depending on the task instruction. The trial ended when participants pressed the response button behind the test bar. They were instructed to respond as quickly and accurately as possible. After finishing the trial, participants returned their fingers to the resting position, on the round patches of cloth in front of the bar. The inter trial interval (ITI) lasted for 6,000 ms, during which the experimenters wrote down the orientation of the test bar and prepared the orientation of the reference bar and the start orientation of the test bar for the next trial. The reference bar and test bar, at 60 cm distance from each other, were explored with either left or right hand and turned with the contralateral hand. The use of left and right hand was varied across participants.

Fig. 2
figure 2

A sketch of the stimulus timing. The period during which the participant heard the auditory cues is indicated by a grey bar. In the visuo-haptic runs, visual input was added during the 1.5 s haptic exploration phase

Full size image

The experiment consisted of a haptic and a visuo-haptic part, containing parallel and mirror runs. In the visuo-haptic condition of the experiment the participants received either congruent or incongruent visual cues during the exploration phase. This resulted in the following four visuo-haptic runs: parallel congruent, mirror congruent, parallel incongruent, and mirror incongruent. The order of visuo-haptic and haptic parts and the order of the runs within each part were counterbalanced across participants. Trials were presented in pseudorandom order.

Haptic stimuli

Twelve different orientations were used for the reference bar to control for orientation. The orientations were all within 10° from 0°, 45°, 90° and 135° (i.e., −10, 0, 10; 35, 45 55; 80, 90, 100; 125, 135, 145). This orientation jitter was added to reduce weariness in the participants and to avoid standard “clock time” orientations, which would allow the use of simple mnemonic strategies. The midsagittal plane of the participant’s body was parallel to the 90° orientation. The start orientation of the test bar (the orientation to which the test bar was set before the participant started the matching phase of the next trial) was varied between 0° and 90°, making sure that, within each run, the average turning distance was the same for all four reference orientations. Participants performed 12 (reference orientation) × 2 (tasks)  × 2 (start orientations of the test bar) = 48 haptic trials, 24 trials per haptic run.

Visual stimuli

Each visual stimulus consisted of a grey stripe (10 × 1 cm; visual angle approximately 10°) clearly visible on a black background, presented on a 15 in. computer screen (screen resolution 1,024 × 768). The stripe was grey instead of white to avoid visual after effects. In the congruent runs, the visual orientations could be transformed into haptic orientations by a translation, followed by a single rotation around an axis perpendicular to the midsagittal plane (90°). The transformation for this operation was invariant for all congruent trials. In the incongruent condition, visual stimuli differed from the haptic reference by an additional rotation around an axis perpendicular to the frontoparallel plane of 16 or 8° in either clockwise or counterclockwise direction, varying across trials (see Fig. 3). Incongruence direction was defined with respect to the test hand. Incongruence size was defined with respect to the reference orientation, because the 5 s delay before matching ruled out any direct influence of the visual cue on the haptic response. A visual orientation is called positively incongruent when it deviates from the haptic reference orientation in the same direction as the typical bias in haptic parallel matching. Hence, a visual orientation with a clockwise deviation from the haptic reference is called positively incongruent when the right hand is used for matching, but negatively incongruent when the left hand is used for matching. A pilot showed that participants were only able to visually discriminate orientation differences of 8° or more.

Fig. 3
figure 3

An example of the different visual stimuli presented during exploration of a particular haptic orientation in the visuo-haptic congruent (0 degree deviation) and incongruent (±8° and ±16° deviation) runs

Full size image

Participants performed 12 (reference orientations) × 2 (task) × 2 (start orientations of the test bar) = 48 trials in the two congruent visuo-haptic runs, and 12 (reference orientations) × 2 (task) × 4 (visual incongruence level) = 96 trials in the two incongruent runs. In the incongruent runs, the start orientation of the test bar was varied in such a way that the average turning distance to the correct parallel or mirrored orientation was constant for each reference orientation and amount of incongruence (8° or 16° difference) within each run.

Data analysis

Signed errors were calculated for every trial by subtracting the orientation of the reference bar from the orientation of the test bar. In the mirror condition, matching errors were expressed as deviations from the mirrored reference orientation (i.e. 60° = >120°). Errors were subsequently normalized to lie between −89° and +90° and averaged within conditions. Finally, the errors on right-hand matching trials were multiplied by −1, to make sure that errors made with the left and right hand, which were expected to have opposite signs, would not cancel out (see Fig. 4). For each participant, averages were subsequently computed over haptic orientations. Standard deviations were computed separately within subjects for each of the four orientation bins (containing orientations within of 0°, 45°, 90° and 135°). These values were then averaged across bins. This procedure was adopted to optimally control for the effect of orientation. The main interest of the current study was the effect of visual orientation cues on haptic parallel and mirror matching. Therefore, reference bar orientation was not included as a statistical factor in the data analyses. Different orientations were mainly included to maintain attention of the participants and to encourage them to carefully explore the reference bar.

Fig. 4
figure 4

Examples of positive and negative errors, which were defined with respect to the test hand. Black line reference bar; Grey line test bar 

Full size image

Matching time was computed by subtracting the time of the second auditory signal from the time of the button press response. The correlation between the matching times and the signed errors was computed for each individual participant. After Fisher-transforming them, a t test was performed to see if the correlation was significantly different from zero across participants. This analysis revealed a mild but significant positive correlation (0.17) between matching time and signed errors (t(29) = −2.27, P = 0.031), indicating that participants made larger errors with longer response times. Therefore, modulations with respect to accuracy in the different experimental conditions cannot be explained by a speed-accuracy trade-off.

Two repeated-measures ANOVAs were performed on the signed matching errors, the standard deviations of the errors and the matching times. The first repeated-measures ANOVA evaluated all runs together, with task (parallel or mirror) and condition (haptic, visuo-haptic congruent, positive visuo-haptic incongruent, negative visuo-haptic incongruent) as within-subjects factors. The second repeated-measures ANOVA was performed on the visuo-haptic incongruent runs only, with task (parallel or mirror) and incongruence level (−16, −8, +8 and +16) as within-subjects factors, testing the potential systematic variation due to incongruence level (H1a and H1b). Incongruence direction was defined with respect to the test (response) hand; that is, trials with a visual deviation of 8° clockwise were assigned to the −8° condition for participants matching with the left hand and to the +8° condition for participants matching with the right hand, etc. Finally, hypotheses 2a and 2b were directly tested for each dependent variable with one-sided paired t tests, using a Bonferroni corrected alpha-level of 0.05/4 = 0.0125.

Results

Signed matching error

Due to missing values, data of one participant from the right-hand exploration group had to be disregarded in this analysis. The condition (4) by task (2) repeated-measures ANOVA on the signed matching errors showed a significant main effect for condition and task (F(3,84) = 11.75, P < 0.0005; F(1,28) = 87.96, P < 0.0005), as well as a significant interaction (F(2.99,83.59) = 9.43; P < 0.0005, Greenhouse-Geisser (G-G) corrected for non-sphericity). The parallel task elicited larger errors in all conditions, as can be seen in Fig. 5a. The main effect of condition was caused by significant differences between haptic and visuo-haptic congruent matching and between haptic and visuo-haptic negatively incongruent matching within-subjects contrasts F(1,28) = 6.41, P = 0.017; F(1,28) = 19.55, P < 0.0005). The interaction arose because of the decreased accuracy for mirror matching in the congruent visual condition, as opposed to the slight improvement in parallel matching performance in the visual congruent condition (within-subjects contrast: F(1,28) = 15.40; P = 0.002). In a separate additional repeated measures ANOVA including only the parallel task, we found that only visuo-haptic negative incongruence significantly improved matching accuracy compared to the haptic condition (19° vs. 22° within-subjects contrasts: F(1,28) = 12.04, P = 0.002), while visuo-haptic positive incongruence significantly reduced the accuracy compared to the haptic condition (22° vs. 26°; F(1,28) = 5.34, P = 0.028). A similar additional repeated measures analysis on the mirror task showed that performance in the haptic condition was significantly more accurate than in the visuo-haptic congruence condition (3° vs. 9°; within-subjects contrast: F(1,28) = 20.53, P < 0.0005).

Fig. 5
figure 5

Average signed matching errors (a) and matching times (b) and their standard errors, for each condition and task. While for the parallel task accuracy increased during VIC-, the mirror task was unaffected by VIC- or VIC+, with errors increasing during VC. Matching times were longer for the parallel task, and in runs with incongruent visual cues compared to haptic runs. Haptic = blindfolded haptic matching; VC = haptic matching with visual congruent input; VIC+ = haptic matching with visual positively incongruent input; VIC- = haptic matching with visual negatively incongruent input

Full size image

Matching time

The repeated-measures ANOVA with condition and task as within-subjects factors revealed a significant main effect of condition (F(1.98,55.52) = 15.96, G-G corrected; P < 0.0005) and task (F(1,28) = 5.41, P = 0.027). Longer matching times were found in the parallel task as shown in Fig. 5b. The effect of condition was due to significantly longer matching times for all visuo-haptic conditions compared to haptic matching (within-subjects contrast: F(1,28) = 11.75, P = 0.002 for congruent visuo-haptic matching; F(1,28) = 24.21, P < 0.0005 for positive incongruent matching; and F(1,28) = 29.22, P < 0.0005 for negative incongruent matching).

Standard deviation

There were no significant main effects in the repeated-measures ANOVA on the standard deviations with condition and task as within-subjects factors. However, we did find a significant interaction between condition and task (F(1.81, 50.73) = 3.51, P = 0.041, G-G corrected). The interaction was caused by the fact that the standard deviations in the mirror task were almost equal or higher during the incongruent visuo-haptic conditions compared to the haptic condition while they were lower in the parallel task (Table 1; within subjects contrasts: F(1,28) = 6.13, P = 0.020; F(1,28) = 6.26, P = 0.018; for positive and negative incongruency, respectively).

Table 1 Average standard deviation of the haptic response in degrees
Full size table

The effect of visuo-haptic incongruence level

Signed matching error

The task (2) by incongruence level (4) analysis on the signed error revealed significant main effects for task and incongruence level (F(1,29) = 76.00, P < 0.0005; F(3,87) = 13.71, P < 0.0005, respectively), as well as a significant interaction between task and incongruence level (F(3,87) = 9.74; P < 0.0005). As indicated by Fig. 6, the parallel task showed a reduced error for negatively incongruent compared to positively incongruent visual cues, whereas incongruence level did not affect performance on the mirror task. These findings are in line with hypotheses 1a and 1b. The smallest parallel matching error occurred when a −8° visual stimulus was shown during haptic stimulus exploration. For each task, we performed an additional repeated measures ANOVA including the haptic condition as well (see dotted line in Fig. 6). For the parallel task we found that performance significantly decreased during the +16° visual incongruent cues while significantly increasing during the −8° visual incongruent cues compared to haptic parallel matching (within-subject contrasts: F(1,29) = 34.89, P < 0.0005; F(1,29) = 10.58, P = 0.003; Note that the direction of visual deviation was defined with respect to the test hand. A positive direction corresponds with the typical egocentric bias, whereas, the negative direction is opposite to the typical error direction for parallel matching. In contrast, the mirror task remained unaffected by visual incongruent cues.

Fig. 6
figure 6

Average signed matching errors for each congruence level and task. The parallel and mirror task were differentially affected by the levels of visuo-haptic incongruence, which were defined with respect to the test hand. The average parallel and mirror matching errors in the haptic condition are plotted for comparison (striped lines)

Full size image

Matching time

The task (2) by incongruence (4) analysis revealed only a significant effect of task on matching time (F(1,29) = 7.27; P = 0.012). Parallel matching was performed significantly slower than mirror matching.

Standard deviation

We found only a significant effect of task in the task (2) by incongruence level (4) repeated measures ANOVA on the standard deviations (F(1,29) = 6.98, P = 0.013), which was caused by a significantly higher standard deviation for the mirror task than for the parallel task with incongruent visual information.

The effect of vision per se on haptic matching

Using Bonferroni-corrected paired t-tests, hypothesis 2a, concerning the positive effect of visuo-haptic cues on parallel matching was rejected for both congruent and incongruent cues. Visual congruent cues did not significantly improve parallel matching compared to blindfolded haptic matching with respect to signed errors. Visuo-haptic congruent cues were associated with a significant increase in matching time (t(28) = −2.85, P = 0.004).

The visuo-haptic incongruent conditions did not show a significant accuracy increase either. Matching times were significantly longer in the parallel incongruent runs (t(28) = −5.24, P < 0.0005). However, the standard deviation did show a significant decrease in the visual incongruent versus the haptic condition (9.26° vs. 12.58°; t(28) = 3.57, P = 0.001).

Hypothesis 2b was not rejected; performance on the mirror task was indeed reduced in the presence of visual congruent cues, as reflected by a significant increase in the signed errors (t(28) = −4.51; P <0.0005) and a significant increase in average matching time (t(28) = 2.89, P = 0.0035). Standard deviations were not significantly affected by visual congruent cues.

Visuo-haptic incongruent cues did not affect performance in the mirror task compared to the haptic condition; neither the accuracy nor the standard deviations significantly differed between these conditions. However, matching time significantly increased when visuo-haptic cues were presented (t(29) = −4.23, P =  < 0.0005).

Discussion

Our objective was to investigate the effect of visual orientation cues on delayed haptic orientation matching. We provided congruent and parametrically varied incongruent visual cues during the exploration phase of a haptic parallel and mirror task. With respect to the mirror task, our findings are in accordance with both our hypotheses. Mirror matching error increased during runs with congruent visual input and was unaffected in runs with incongruent visual input (H1b and H2b). Regarding the parallel task, contrary to our hypothesis 2a, we found no general beneficial effect of visual input on parallel matching accuracy, neither for congruent visual cues, nor for positively incongruent visual cues as compared to purely haptic parallel matching. However, in agreement with our hypothesis 1a, we found that parallel matching error varied as a function of deviation size and the direction of incongruent visual orientation cues. Negatively incongruent visual cues significantly improved parallel matching accuracy. That is, parallel matching error was smallest when the visual orientation slightly counteracted the typical positive bias in haptic parallel matching.

Potential effects of (in)stable visuo-haptic correspondence

Our results clearly show that the effect of visual input on haptic performance depends on the type of task and the level of congruency of the visual cues. Congruent visual cues were detrimental to the mirror task, which allowed a purely somatosensory strategy. Incongruent visual input had no effect on this task, but did affect the parallel task, which requires an allocentric reference frame for veridical performance.

Because participants were not informed explicitly on whether they were in the congruent or incongruent visuo-haptic condition, the objectively defined (in)congruency does not necessarily correspond to the subjective experience of a given participant in a given trial. Informal comments from the participants support the view that the incongruent condition is in fact experienced as a mixed condition, in which the visual deviations in the same direction as the egocentric bias were experienced as congruent and the visual deviations counteracting the egocentric bias as incongruent. The subjective experience of visual incongruence might interact with the objective incongruence level, explaining our finding that the beneficial effect of the visual incongruence is maximal when it moderately counteracts the egocentric bias.

The visual and haptic orientations in the current study were presented at different locations, in different spatial planes. Congedo et al. (2006) showed that spatial misalignment of visual and haptic information promotes the dominance of visual information when judging a rotation angle. In the congruent runs of the current study, the spatial transformation to establish the congruence of the visual and haptic orientation was stable, and in accordance with proprioceptive information on the position of the head and eyes with respect to the hand. Therefore, in both the parallel and the mirror task, the correspondence might be established more or less automatically and reliably. This might bias the weighting of visual and haptic input even more in favor of the visual modality and consequently affect the strategy used for the mirror task. The increased matching times for mirror matching in the visual congruent runs, equivalent to those for parallel matching, might support the view that a different, less somatosensory matching strategy is used in the matching phase. Mirror matching performance apparently no longer benefits from the mirror symmetry of the hands and arms with respect to the midsagittal plane when an attempt is made to integrate the visual information.

In the incongruent runs, an additional rotation around an axis perpendicular to the plane of the visual or haptic bar is needed to equate the visual and haptic orientation on a trial-by-trial basis. The orientation of the body midsaggital plane might then be experienced as deviating in a direction opposite to the visual incongruence direction in the horizontal plane, causing a change in the estimated orientation of the test bar. This would be similar to what happens in the classical rod-and-frame task, where participants who fail to reproduce the true vertical when confronted with a tilted luminous frame in an otherwise dark room might experience a body tilt in the opposite direction, which is then used as a faulty reference (Witkin 1949; Witkin and Asch 1948). Luyat et al. (2005a) showed that a tilted visual context affects the (visual) oblique effect, which is supposedly caused by a change in the subjective vertical. Our results are also in agreement with previous research showing that haptic contextual cues affect the haptic perception of orientations, causing haptic rod-orienting responses to deviate in the direction of distractor stripes (Luyat et al. 2005b). Similarly, the availability of (vertical) visual cues has been shown to influence the visual perception of spatial orientation (e.g., Groen et al. 2002; Howard and Hu 2001; Luyat et al. 1997), producing an error in the direction of the textured background when visual contextual cues (tilted frame or tilted lines) diverted the perception of the vertical.

Estimation of the position and orientation of the midsagittal plane in space probably also occurs in blindfolded haptic parallel matching with the contralateral hand, where it is used as a reference to factor out the difference in the orientation of the hands at different locations. However, the perceived orientation of the arms and hands with respect to the midsagittal plane is subject to errors also in haptic matching, as demonstrated by the large systematic errors occurring in this condition. Depending on the direction of the incongruence of the visual cues, the error due to this faulty reference-estimation is either compensated or aggravated when the perceived orientation of the midsagittal reference plane is adjusted to the incongruent reference orientations. Indeed, the visuo-haptic incongruent cues did affect performance in the parallel task, depending on the direction and size of the visual deviation with respect to the hand used for matching. Interestingly, the lowest error was found for negatively incongruent orientations, which counteracted the typical positive bias in haptic parallel matching. The +8 incongruent orientation led to a similar error reduction. This might be due to the fact that this orientation still deviates less from the haptic reference orientation than the average haptic parallel matching response.

The extra processing step incorporating the orientation of the hands with respect to the midsagittal plane is not specifically needed for accurate mirror performance, and therefore, a less computationally demanding, more hand-centered somatosensory strategy might prevail when trial-by-trial adjustments are required to relate the visual and haptic cues, as is the case in the incongruent run. Therefore, the greater reliance on a more hand-centered strategy might have resulted in active blocking of visual input in the incongruent condition. This is exemplified by the performance level, which was comparable to the level for purely haptic mirror matching. The increased matching times most likely reflect the active inhibition of the visual matching strategy.

General implications

We did not replicate a general improvement in the parallel task with the presentation of visual input, as reported in earlier studies (Zuidhoek et al. 2004; Newport et al. 2002)? This might be due to the fact that in the current study, the way the visual orientations were presented did not enable participants to construct a veridical allocentric, world-based reference frame, and did not provide additional visual information on the position of the hands in space. The visual cues were only presented during exploration, therefore only allowing establishing the transformation between one hand and the external visual reference (presenting the visual orientation cues during all task phases would have changed the task to a visual orientation matching task). This might not suffice to improve the computation of the spatial transformation of the orientation to the other hand.

Whereas the visual input in previous studies of non-informative vision consisted of the more or less coincidental visuo-spatial layout of the environment in the visual periphery (Zuidhoek et al. 2004; Newport et al. 2002), the present study used constructed cues, which were presented focally. There is evidence that different cortical systems are involved for reaching in central and peripheral vision, with a more extended cortical network including the parieto-occipital junction associated with the latter (Prado et al. 2005). Peripheral vision has been shown to have considerable impact on the perception and stabilization of the body in space (e.g. Berensci et al. 2005).

Our results are in accordance with the modality appropriateness hypothesis, which states that the interference by a task-irrelevant modality is reduced if processing accuracy of the task-relevant modality is high (Rock and Victor 1964). Our findings are also in agreement with results reported by Ernst and Banks (2002), showing that visuo-haptic integration occurs in a statistically optimal fashion, minimizing the variance in the final perceptual estimate. This results in a task-dependent weighting of visual and haptic input, determined by the reliability of the information obtained by vision or haptics for the task at hand. Hence, since visual size information is more reliable, size perception shows visual capture. However, changing the reliability of visual size information by adding noise increased the weight of the haptic input in determining the eventual percept.

In the mirror task, somatosensory processing accuracy is high, therefore the incongruent visual input has no effect. The congruent input, on the other hand, showing a stable relationship with the haptic input and body proprioception, did interfere with mirror matching performance, most likely because it biased the system to use a less task-efficient visual strategy.

In the parallel task, transfer of the haptic orientation to the contralateral hand is associated with low somatosensory accuracy. Performance on this task was systematically affected by incongruent visual information. We propose that haptic tasks requiring an allocentric, world-based representation might use a broader reliability range for visuo-haptic congruency, also integrating visual input which is actually incongruent.

Finally, visual information per se did not significantly improve the veridicality of parallel matching, demonstrating that vision alone is not enough to allow the computation of an allocentric reference frame. However, our results do suggest that moderately incongruent visual input counteracting the haptic bias might improve performance on haptic allocentric tasks. Apart from the visual orientation cues, other visual spatial cues might be needed to help create an allocentric reference frame to infer the relative position of the body in space.

We conclude that haptic task performance in the presence of vision is not determined by a single strategy or reference frame, but flexibly adapts the weighting of visual and somatosensory input, according to task demands and the level of congruency and reliability of the respective input.