Abstract
Vibro-tactile interfaces were proposed as an alternative to enhance human-machine communication in information-rich domains. The current study aims to examine the effectiveness of two levels of tactile alerts when combined with visual alerts, in MUM-T (Manned UnManned Teaming) setup. In MUM-T, aside from their primary mission, mounted operators are responsible for supportive unmanned systems and must attend to their health. On the simple level, the alert provides information about a threat or a failure in the supportive unmanned systems, while in the complex level, the alert includes more specific information about the source of failure, that may require more effort to interpret. The experiment simulates an operational mission in which participants ride an autonomous ground patrol vehicle while identifying threats and targets in the area and being supported by two unmanned systems. Response accuracy to alerts and threat identification rates were measured. Results indicate that tactile alerts given in addition to visual alerts in a visually loaded and auditory noisy scene, improve task performance. Moreover, the complex level of tactile alerts did not impair performance compared to the simple level of tactile alerts and led to higher rate of identification in specific cases. Nevertheless, relatively high rates of false alarms (FA) for threats were observed, especially when tactile alerts were present, which can be explained by the payment matrix (no penalty) or by the assumption that adding tactile alerts may lead participants to be more vigilant, which can lead to higher correct identifications, but also to higher FA rates.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Military operational activities require obtaining large quantities of data from multiple sources in a short period of time while running multiple tasks simultaneously [1]. Operators are expected to detect changes quickly and respond systematically and fast [2]. Unmanned systems are potent force multipliers, in the MUM-T (Manned UnManned Teaming) operational concept, they are controlled from a moving ground or aerial platform, and are used for tasks that otherwise would have been taken over by other manned or remotely operated platforms. MUM-T can cause significant task load increase for its operating crew and key design elements are necessary for achieving the workload reduction necessary to facilitate it [3]. We focus on the use of tactile alerts to improve operators’ ability to detect threats and attend to failures. Tactile alerts may enhance attentional abilities by distributing information between several resources [4], as visual alerts alone are less suited for this level of complexity [5]. The tactile channel can be used to alleviate workload and attract operators’ attention to mission related notifications at various levels and events [6].
Yet, like the visual and auditory channels, frequent use of tactile alerts may increase workload and impair performance [7]. Moreover, multiple alerts can lead to neglect, where the alerts are ignored [8], mainly when the task is unclear and the transfer of information to the operator is impaired [9]. Therefore, the tactile alerting system must be used properly. Tactile cues can be used as signals with lower information processing requirements [10]. According to Elliot et al. [11], when tactile cues are added to existing visual cues, performance improves. An important question is content related what kind of tactile messages should be transferred by a tactile display.
The current study was aimed to examine the level of tactile alert (simple/complex) that should be used in a MUM-T operational setup. For this, a burdening, visually and auditory loaded operational mission was examined. We defined two levels of tactile alerts - simple and complex: A simple alert provides information that is easy to interpret about the occurrence of a pre-determined event. It is defined as a binary happened/did-not-happen alert; A complex alert requires more effort to interpret, but includes more specific informative information. It provides the operator with at least one more layer of information, e.g., source, direction, distance, etc. [6]. Two types of tactile alerts were given; A “danger” alert which indicates upon the presence of hostile targets, and a “failure” alert which indicates upon a technical failure in one of the supportive unmanned systems. “Danger” alert was defined as more immediate and required faster responses.
It was hypothesized that H: Identification rate of alerts will be higher when tactile alerts are given in addition to visual ones, more so, for the tactile alerts provided in the “complex” level for both “danger” and “failure” alert types.
2 Method
The study aimed to examine the effectiveness of two levels of tactile alerts when combined with visual alerts. Each participant was placed in a simulated workstation of an autonomous ground patrol vehicle that navigated autonomously. While driven along the route, the operator was required to identify threats and targets in the mission area and manage two unmanned vehicles (UVs), an unmanned ground vehicle (UGV) and a drone, that support the mission in a MUM-T setup. Occasionally a failure occurred in one of the supporting vehicles and the operator had to attend to it while continuously seeking for threats and targets. Each participant executed three different scenarios, one scenario for each level of tactile alerts (none, simple, complex). Visual alerts were presented on a computer screen (#3 in Fig. 1b); Tactile alerts were transmitted through a wearable tactile display located on the soldiers’ forearms (Fig. 1a).
Three females and 23 males, students aged 24–30 (M = 27.1, SD = 1.6) all military reserve soldiers on active duty at least once in the year prior to the experiment participated in the study. Recruitment was via social media and compensation was 40 NIS for one hour. Nine participants’ objective performance data was withdrawn from the study due to a technical problem in saving their data, leaving a total of 17 participants. These withdrawn participants did not change the population’s parameters.
2.1 Apparatus
Experimental Environment.
The study was conducted at the Human Performance Lab. The simulator consists of three visual displays in different sizes: screen #1 - 22′, screen #2 - 42′, screen #3 - 13.3′, running a simulation of the operational mission (see Fig. 1b). Participants sat about one meter away from the main screen (#2 in Fig. 1b). The laboratory is temperature and noise controlled.
Operational Tasks.
To accomplish the mission three task types were required. Ongoing: the task was to detect targets and mark them by pressing a “target” button on visual display (#3). The target button was blinking in red three times when a target was identified by one of the supportive UVs (Fig. 2), 14–16 targets per session with a variation of 10–85 s between two consecutive targets. UV status: the participant was asked to identify failures in the UVs and report them, 11–13 failure alerts per session, with a variation of 5–40 s gap between two alerts. Gray and blue buttons were blinking when there was a failure in one of UVs (gray for drone and blue for UGV) and stopped blinking when the operator pressed the corresponding button on the visual display (#3; Fig. 2). A secondary task was to report upon identification of vehicles via the communications device: “car identified”. The purpose of this task was to serve as a distracting element to the main mission tasks.
Headphones.
(Samson SR850) were used to transmit typical military radio communication, presented at a range of 40 dB with short peaks of up to 80 dB, to add to the realism of the simulation and increase mental demand.
Tactile Interface.
Military applications strive for simplicity on one hand, and redundancy on the other hand for robustness, therefore a two-tactor system was chosen but with one signal [12]. The two C2 tactors (i.e., vibrating tactile actuators) powered by the Eval 2.0 controller (Engineering Acoustics Inc (EAI)) and were stitched to two elastic-fiber straps. The straps were worn around the forearms, one on each hand, over the participant’s skin (see Fig. 1a). The tactile alerts included four different temporal patterns (on-off and duration modulation) through the tactile system (see Tactile alerts’ design). The gain scale ranges from 1–255 units, as determined by the EAI apparatus.
Tactile Alerts’ Design.
Two types of tactile alerts were chosen: one for conveying “danger” and one for conveying “failure”. The tactile signals for each alert type were chosen following a pre-experiment. For the pre-experiment five tactile cues were designed according to a list of criteria such as the alerts’ overall length, gain and pulse rate (see Fig. 3a). The experimental part of the pre-experiment was divided into two. The first part included a navigation task, in which each participant was exposed to all five different cues once in random order. Time gap between two cues was at least two minutes. Participant had to choose which kind of meaning better describes the alert that has just been executed: “danger” or “failure”. Participants responded while the navigation mission continued. In the second part, participants were able to feel all five cues for an unlimited period of time, by pressing on the alert buttons. They were asked to choose the most appropriate alert for “danger” and afterwards for “failure”. From the pre-experiment, for the “danger” alert, all five alert types were chosen almost at the same rate in the first part, while in the second part alerts C, D and E were hardly chosen at all, therefore, it was decided not to use them for the context of danger. Among alerts A and B, it was decided to choose alert B for “danger” although the rates of the two alerts were close, with a small favor for B in both parts of the pre-experiment. For the “failure” alert, alerts A and B were less chosen by participants in the first part of the pre-experiment, and there was almost an equality between alerts C, D and E. To choose the fitted alert for “failure”, we used the second part of the pre-experiment, where there was an unequivocal majority for alert C. It was decided to choose alert C as “failure” alert.
Thus, alert B was chosen for “danger” and alert C for “failure” for the current experiment. The “simple” tactile alert condition consisted of alerts B for danger/threat and C for “failure” in any one of the UVs. The “complex” tactile alert condition consisted of the same alert B for danger/threat and two alerts for “failure” depending on the source of the failure: alert C1 for failure in the UGV and alert C2 for failure in the drone (see Fig. 3b).
2.2 Procedure
Participants were invited solely to the lab for one-hour long sessions. At the beginning of the session, each participant read the instructions and signed informed consent. Following a short verbal briefing, each participant got familiarized with the task (the way targets were marked and how to detect them, how to identify failures of the vehicles and how to report them). A separate briefing about the tactile and visual displays was given in order to familiarize participants with the alerts.
The experimental part of the study included a trial scenario (of two minutes) in which participants received both visual and tactile alerts (“simple” condition), and three operational scenarios, each participant performed all scenarios. Participants completed each scenario with one of the three modality conditions. The scenarios’ order was fixed, but the modality conditions were balanced across participants. The length of each section was approximately nine minutes, with a break of a few minutes between sessions to prepare participants for the next session.
3 Results
3.1 Detection of Danger, Threats and Targets
Missed Alerts.
Data of 782 danger alerts were collected. All in all, danger was detected 709 times. Thus, there were 73 (9.3%) missed cases. A Generalized Linear Mixed Model (GLMM) with binominal family was used with random effects of participants and scenarios to account for differences among scenarios and individual differences among participants. The dependent variable was defined as a binary variable: 1 for reacting to danger, and 0 for missing a danger, and the independent variable was the alert level (none, simple, complex). The final model includes a main effect of alert level (χ2 (df = 2) = 7.037, p < .01) and two random effects, for subject and scenario. Post-Hoc (Tukey-HSD) analyses revealed a significant difference between the “none” condition and the “complex” level of tactile alert; 15% missed cases without tactile alerts out of all miss alerts, and 5% with “complex” level of tactile alerts. Results remain stable in perspective of missing dangers per person according to the different conditions as shown in Fig. 4b.
False Alarms.
There was no penalty for FA. On average, there were 8.87 (SE = 1.54, ME = 7.50) incorrect responses (clicking on the “danger” button when no target was present) per participant, of those 2.25 occurred in the “none” condition, 3.12 occurred during the “simple” condition, and 3.5 in the “complex” condition.
3.2 Identification of Failure Alerts
Missed Alerts.
Data from 672 failure alerts were collected. There were 13 (1.9%) missed notifications (no response), only 2 of which with a tactile alert. A Generalized Linear Mixed Model (GLMM) with binominal family was used with random effects of participants and scenarios to account for differences among scenarios and individual differences among participants. The dependent variable was defined as a binary variable: 1 for reaction for a failure alert, and 0 for missing a failure alert, and the independent variable was the alert level (none, simple, complex). The final model includes main effect of alert level (χ2 (df = 2) = 5.797, p < .01) and two random effects, for subject and scenario. Post-Hoc (Tukey-HSD) analyses revealed that there was a significant difference between the “none” condition to the “simple” level of tactile alert; 5% missed cases without tactile alerts out of all miss alerts and 0.48% missed cases with “simple” level of tactile alerts out of all missed alerts, and also between the “none” condition to the “complex” level of tactile alert; 5% missed cases without tactile alerts out of all miss alerts, and 0.49% with “complex” level of tactile alerts. Results remain stable in perspective of missing failure alerts per person according to the different conditions as shown in Fig. 4a.
False Alarms.
There were no incorrect responses (clicking on one of the “failure” buttons when no failure occurred).
4 Discussion
The aim of the experiment was to examine the effectiveness of two tactile alert levels in a multi-task operation with high workload. Previous studies have shown that tactile cues can capture attention, which may be helpful under high workload [13]. How to define the level of detail and complexity necessary of the tactile alerts is important for this operational research field. The results of the identification rate of both “danger” and “failure” alerts were significantly higher when tactile alerts were added than when only visual alerts were shown. Thus, hypothesis H was confirmed. Rates were 95.6% identification of tactile notifications compared to 91.1% for visual notifications only (without tactile alerts). Moreover, the “complex” level of tactile alerts provided higher rates of “danger” identification compared to the “none” condition, while the rate of “failure” identification for both the “simple” and the “complex” levels was significantly higher than for the “none”. This indicates that having higher granularity of tactile alerts did not interrupt operators with performing their main task, and moreover, improved task performance. More so, “complex” tactile alerts can improve performance better than the “simple” ones when looking at the identification of “danger” alerts.
False alarm (FA) for a presence of a target was higher when tactile alerts were given. Incorrect responses (report about a target when there is no target in the field) in the military context is a severe mistake. One explanation for the high rate of FA is that due to the experimental design, operators pressed the “target” button right when they received a “danger” alert, even though this kind of alert is designed to warn the operator of a close danger/target observed by the supporting unmanned vehicles, and not an immediate one. Another explanation may lie on the “payment matrix” [14], in which when gain or loss for a specific behavior are not defined, people tend to pay less attention to false alarms. Accordingly, participants were not told that they will be fined for incorrect responses, which may have caused higher rates of FA. No fine for FA could make them prefer to react in more leniently in order to miss fewer targets. Further study should use more specific instructions regarding the fine for incorrect responses in order to avoid high rate of FA. Moreover, the data regarding the higher rate of FA in the tactile alert conditions can testify that operators did not distinguish between the alert types (“danger” and “failure”) and pressed on the “target” button even when a “failure” alert was activated. However, it may also indicate that the additional tactile alerts condition led participants to be tenser and more vigilant, which led to higher correct identifications, but also to higher FA rates.
Alongside the supportive results to use the tactile alerts there are few limitations. First, in the “complex” level condition, the three tactile alerts shared a similar initiation pattern, which forces the operator to wait until the middle or the end of the alert before responding, or to use the visual display right at the beginning of the tactile alert without attending to it until the end. By that participants actually treat the “complex” level of tactile alerts as the “simple” level. Second, participant had limited time to learn and train on the tactile alert patterns, while the visual alerts are more common and straight forward. A longer learning session of the tactile patterns may lead to different results and should be taken into consideration in further researches. A follow-up study should also include a simpler level of only one simple tactile cue with no specific information regarding “danger” or “failure” (a basic level), that will be compared to the levels tested here. Possibly, the “complex” tactile alerts may create a burden on the operator and a basic level would be more effective in multi-task operation.
In conclusion, in the current study it was found that tactile alerts can be combined with visual alerts, in a multi-task operation in MUM-T setup without reducing performance. Moreover, tactile alerts in this kind of operation reduce the missing of “danger” alerts, which is important for operational setups where missing alerts can lead to severe consequences.
References
Katzman, N., Oron-Gilad, T.: Towards a taxonomy of vibro-tactile cues for operational missions. In: EuroHaptics, no. 2018 (2018)
Oron-Gilad, T., Redden, E.S., Minkov, Y.: Robotic displays for dismounted warfighters: a field study. J. Cogn. Eng. Decis. Making 5(1), 29–54 (2011)
Strenzke, R., Schulte, A.: Human-automation cooperation issues in manned-unmanned teaming. In: 2012 AUVSI Unmanned Systems North America Conference, vol. 2, pp. 851–870 (2012)
Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(3), 449–455 (2008)
Sklar, A.E., Sarter, N.B.: Good vibrations: tactile feedback in support of attention allocation and human-automation coordination in event-driven domains. Hum. Factors J. Hum. Factors Ergon. Soc. 41(4), 543–552 (1999)
Katzman, N., Oron-Gilad, T., Salzer, Y.: Tactile interfaces for dismounted soldiers: user-perceptions on content, context and loci. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting (2015)
Ferris, T., Stringfield, K., Sarter, N.: Tactile ‘change blindness’ in the detection of vibration intensity, pp. 1316–1320 (2010)
Marsh, E.B., Hillis, A.E.: Dissociation between egocentric and allocentric visuospatial and tactile neglect in acute stroke. Cortex 44(9), 1215–1220 (2008)
Mortimer, B.J.P., Elliott, L.R.: Identifying errors in tactile displays and best practice usage guidelines. In: Chen, J. (ed.) Advances in Human Factors in Robots and Unmanned Systems. AISC, vol. 595, pp. 226–235. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60384-1_22
Baldwin, C.L., et al.: Multimodal cueing: the relative benefits of the auditory, visual, and tactile channels in complex environments. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 1431–1435 (2012)
Elliott, L.R., Coovert, M.D., Prewett, M., Walvord, A.G., Saboe, K., Johnson, R.: A review and meta analysis of vibrotactile and visual information displays, September 2009
Elliott, L.R., Schmeisser, E.T., Redden, E.S.: Development of tactile and haptic systems for U.S. infantry navigation and communication. In: Smith, M.J., Salvendy, G. (eds.) Human Interface 2011. LNCS, vol. 6771, pp. 399–407. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21793-7_45
Spence, C.: Crossmodal spatial attention. Ann. N. Y. Acad. Sci. 1191, 182–200 (2010)
Zeeman, E.C.: Population dynamics from game theory. In: Nitecki, Z., Robinson, C. (eds.) Global Theory of Dynamical Systems. LNM, vol. 819, pp. 471–497. Springer, Heidelberg (1980). https://doi.org/10.1007/BFb0087009
Acknowledgement
This work was supported by the US Army Research Laboratory through the GDLS subcontract no: GDLS PO 40253724 (B.G. Negev Technologies and Applications Ltd) under Prime Contract no W911MF-10-2-0016 (Robotics Consortium), Robotics CTA 2015-2020, T2C1S3C, Michael Barnes, Technical Monitor. The views expressed in this work are those of the authors and do not reflect an official Army policy. The work is unclassified, Approved for public release.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this paper
Cite this paper
Rosenblum, D., Katzman, N., Oron-Gilad, T. (2020). Identification Rate of Simple and Complex Tactile Alerts in MUM-T Setup. In: Nisky, I., Hartcher-O’Brien, J., Wiertlewski, M., Smeets, J. (eds) Haptics: Science, Technology, Applications. EuroHaptics 2020. Lecture Notes in Computer Science(), vol 12272. Springer, Cham. https://doi.org/10.1007/978-3-030-58147-3_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-58147-3_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58146-6
Online ISBN: 978-3-030-58147-3
eBook Packages: Computer ScienceComputer Science (R0)