Examples of Haptic System Development

Abstract

In this section, several examples of task-specific haptic systems are given. They give an insight into the process of defining haptic interactions for a given purpose and illustrate the development and evaluation process outlined in this book so far. Examples were chosen by the editors to cover different basic system structures. Section 14.1—Tactile You-Are-Here-Maps illustrates the usage of a tactile display in an assistive manner, enabling a more autonomous movement of people with visual impairments. Section 14.2—User Interface for Automotive Applications presents the development of a haptic interface for a new kind of user interaction in a car. It incorporates touch input and is able to simulate different key characteristics for intuitive haptic feedback. Section 14.3—HapCath describes a comanipulation system to provide additional haptic feedback in cardiovascular interventions. The feedback is intended to reduce exposure for both patient and physician and to permit new kinds of diagnosis during an intervention.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Tactile You-Are-Here Maps

Limin Zeng and Gerhard Weber

1.1 Introduction

You-are-here (YAH) maps as one of the truly ubiquitous tools are installed on walls or wooden boards to allow sighted persons quickly acquire the information about where they are within their surroundings [6]. In addition to various geographic features rendered on a YAH map, a YAH symbol indicating readers’ current position on the map must be represented. In recent years, a large number of location-based maps on handheld devices become available and provide digital YAH maps, which represent dynamic data about position and orientation.

However, for visually impaired people, these visual YAH maps are inaccessible, neither YAH maps installed on walls nor ubiquitous electronic YAH maps. Although people having visual impairments are able to use global positioning system (GPS)-based navigation devices to query their current location and follow turn-by-turn guidance to reach unfamiliar destinations, they cannot explore the surroundings and acquire more related spatial geographic information, such as the layout of street networks and the shape of a complex crossing, which might help them understand an unfamiliar point of interest better.

To improve accessibility features of map exploration applications for the visually impaired, a number of haptic devices have been employed in previous studies. The TouchOver Map application indicates that visually impaired users find it was challenging to find out the details of short streets, e.g., direction, connection on regular touchscreen displays, due to lack of explicit haptic feedback [18]. Even if visually impaired users could distinguish several simple tactile patterns (e.g., circles, squares) on electrostatic haptic displays [22, 25] that offer better haptic sensation than common touchscreen displays, it is still unclear whether they can read a complex city map with various geographic features, because at present, there is no map-rendering system developed for such an electrostatic haptic display. Recently, Zeng and Weber implemented a city map system on a pin-matrix display with 7,200 pins, which is suitable for desktop use [26]. Haptic sensation of raised and lowered pins on the pin-matrix display allows individuals with visually impairments to explore streets and points of interest (POIs) interactively. However, the desktop-based tactile map system does not render the current users’ location.

Considering the good haptic sensation of a pin-matrix display, we develop a ubiquitous tactile YAH map system (named TacYAH map). A touch-enabled pin-matrix display with 960 pins allows mobile usage if a map can be rendered accordingly. In addition to a series of tactile map symbols rendered on the TacYAH map, we design a set of tactile YAH symbols consisting of raised and lowered pins to indicate users’ updated location and heading orientation. Moreover, the visually impaired users can interact with the TacYAH map by panning and zooming through a haptic controller.

1.2 The TacYAH Map Prototype

(a) Mobile HyperBraille Display

The employed tactile display is a mobile version of the HyperBraille display [23], consisting of a matrix of $30 \times 32$ piezoelectric refreshable pins (see Fig. 14.1). The innovative array cells reduce the size of the display considerably compared to existing Braille displays consisting of one row of Braille cells. The weight of the display is about 600 g, and its pin matrix area covers $7.0 \times 8.1\,\text {cm}$. Each pin has only two statuses, that is, either raised or lowered. The height of a raised pin is about 0.7 mm, and the space between each pin reaches 2.5 mm; thus, the raised pins can be distinguished by fingertips. Tactile force strength of each pin is more than 30 cN, which ensures the raised pins would stand up always even if under high finger pressure. The whole screen refreshes rapidly with the help of its high-speed data bus and piezoelectric actuators (up to 5 Hz refresh rate). There is a capacitive layer on the top of the pin-matrix area that makes the display touch enabled. Furthermore, due to its low power consumption, the tactile display makes use of a common USB cable to supply power and data.

Fig. 14.1

The mobile HyperBraille display (a matrix of $30 \times 32$ pins)

(b) System Components

In addition to the portable pin-matrix display, the prototype consists of several additional components (see Fig. 14.2). A smartphone equipped with a GPS sensor and a digital compass provides users’ updated location and orientation accordingly. A WiiCane whose handle is enhanced by mounting a Wii remote control is used to input commands (e.g., panning, zooming) via its built-in keys, such as the key “A” for querying where users are, and the keys “$-$” and “$+$” for zooming out and zooming in, respectively. Note that the WiiCane offers tactile feedback via a vibration to inform on the completion of processing the input commands. Users acquire additional auditory information through an earphone, like the street name or POI name. Besides, a light laptop is required to run the main program, and it is connected to all other devices by Bluetooth or wired connections.

Fig. 14.2

The overview of the TacYAH Map prototype (left the system architecture; right the system worn by a user)

(c) Map Data Processing

At present, most current commercial map providers (e.g., Google Maps, OpenStreetMap) offer tile-based map data as images to various clients, like Web browsers or map applications on mobile phones. Considering the challenging tasks to generate tactile maps from image processing methods [24], the TacYAH map parses the vector-based map data in the format of Geography Markup Language (GML, ISO 19136:2007). To generate GML map data according to users’ input commands, an open-source Geographic Information System (GIS) software, namely GeoServer, has been employed in the prototype. As illustrated in Fig. 14.3, there are seven main steps to prepare a response to users’ input (e.g., panning, zooming, and acquiring auditory information). After pressing a key in the second step, users’ commands are translated into Web Feature Service (WFS) requests. The WFS utilizes HTTP to express and transport detailed geographic features. Visually impaired people touch the tactile representation and listen to auditory information within the output modules. The TacYAH map prototype can easily import worldwide city map data from the OpenStreetMap repository.

Fig. 14.3

Flowchart of map data processing

(d) Tactile Map Symbols

Generally, on a visual map, various geographic features can be rendered by different icons, styles, and colors, even on overlapping layers. However, it is challenging to render a city map on such a low-resolution pin-matrix display (10 pins per inch). Thus, to represent various geographic features, a set of distinguishable tactile map symbols has been designed in the TacYAH map system, as shown in Fig. 14.4.

Furthermore, in addition to the set of geographic features, a set of YAH symbols has been designed for rendering not only users’ location but also users’ heading orientation. During the design process, 7 visually impaired individuals (3 female and 4 male, mean age 30 years) were invited to choose the best set among 3 candidate sets (see Fig. 14.5). Five of them preferred the second set, and 2 participants chose the third one. Therefore, the TacYAH map system employs the second set. It is important that the tactile symbols should not be changed (like their size) when the map is zoomed in or zoomed out. Otherwise, the visually impaired might be confused with the modified symbols that are different to the ones they learned before.

The tactile map symbols and the YAH symbols make up a city map on a pin-matrix display as shown in Fig. 14.6. Visually impaired individuals learn the layout of streets and complex crossings, as well as the surrounding buildings and POIs. Note, it is necessary to keep enough space between each symbol (e.g., about 2 pins), to ensure the symbols can be identified correctly. However, the bus stop symbol is an exception, as it can be recognized even if close to a street [26].

Fig. 14.4

Examples of tactile map symbols (the black dots refer to raised pins)

Fig. 14.5

The three candidate sets of tactile YAH symbols [27]

Fig. 14.6

A circle crossing rendered on the pin-matrix display (left a screenshot of the crossing from OpenStreetMap; right a representation on the pin-matrix display)

(e) Map Interaction

Convenient exploration of maps requires support for a variety of user strategies. Regardless of the specific information, the WiiCane vibrates for a short amount of time (the vibration duration: 300 ms) to inform users on the completion of rendering output, and then, the users can touch the updated screen. Note that, the north is always at the top of the map (north-up map).

“You Are Here”   It informs users about where they are, which renders the surrounding map on the tactile display, and the updated YAH symbol in the center of the display. When a user presses the key “A” on the WiiCane, the function will be triggered.

Panning   Users can pan a map to the left/right/up/down side by pressing the appropriate button on the WiiCane. Each panning operation will update one-third of the screen vertically or horizontally, rather than the whole screen. Otherwise, the visually impaired users might become confused after panning due to losing the previous context.

Zooming   In the TacYAH map, two zoom levels are supported. At the first level, only street networks and YAH symbols are rendered (i.e., Street View), while at the second level, it renders streets and POIs at the same time (i.e., POI View). The whole screen represents an area covering $150 \times 160\,\text {m}$ (5 m/pin) in Street View, and covers an area of $100 \times 106$ m (3.3 m/pin) in POI View. Certainly, many more zoom levels can be designed and implemented, such as a building view to represent the exact shape of the building.

Auditory geographic information   To acquire the related auditory information about the surrounding geographic features, like the name of a street or a building, and the bus lines offered at a bus station, a visually impaired user needs to touch the corresponding map symbol with one finger on the tactile display and press the key “1” or “2” on the WiiCane simultaneously. The auditory information is generated by Text-to-Speech (TTS) software and based on the information stored in the GIS, such as names of streets or POIs and bus lines at bus stations.

In summary, the TacYAH map prototype provides not only audible geographic information, but also tactile representation of street maps via the predesigned tactile map symbols and YAH symbols. It is necessary to validate whether visually impaired people are able to benefit from the proposed system, under the hypothesis that end users can locate themselves independently, even in unknown areas.

1.3 Evaluation

In order to investigate the performance of the TacYAH map prototype, 5 blind participants were invited in a pilot study. After training the tactile symbols, the two map views, and usage of the system, the participants were guided to two unknown outdoor test sites and asked to locate themselves, seek the surrounding streets and nearby POIs within 100 m with the help of the TacYAH map system. The precision of identifying the YAH symbols is 95 % after a short amount of training time (mean: 75.0 s). In the 10 outdoor field tests, all the participants can locate themselves successfully and find out the surrounding streets and the targeted POIs (mean recall: 100 %, mean precision: 80 %). Specifically, they would estimate the distance (mean error: 9.1 m; ST. Dev.: 3.82) and the orientation (mean error: 18$^\circ$; ST. Dev.: 0.25) between themselves and the nearby POIs in 100 m.

1.4 Conclusion and Outlook

The results of the conducted pilot study with blind users indicate through the proposed haptic YAH map system individuals having visually impairments are able to locate themselves on the move and explore the surroundings in unknown areas. In addition to many more evaluations with end users, the functionalities of the TacYAH map as a proof of concept prototype should be improved in the future toward a final product, like including a head-up map. It will be helpful to extend the TacYAH map to render indoor floor plan, specifically in airports and railway stations. Through this new assistive navigation system, the visually impaired will be able to be guided with turn-by-turn instructions by an appropriate low-cost app in their mobile phones and explore the surroundings easily both.

2 Automotive Interface with Tactile Feedback

Ingo Zoller, Peter Lotz, Thorsten A. Kern

Jörg Reisinger

Please note that this Sect. 14.2 is a reworked version of [28, 29].

2.1 Context

Products with active tactile feedback inside a car cover all elements which are in direct contact with its occupants. The more classic applications are active steering wheels with tactile and/or kinaesthetic support [14] and break or throttle pedals [16]. Research additionally focuses on tactile signals to create situative awareness or give assistive clues [15]. In addition to these feedback loops directly related to the task of driving, comfort functions like multimedia and climate control elements are also subject to active tactile feedback. Numerous solutions were presented for tactile feedback on automotive touchscreens, but lately faceplates (Fig. 14.7a) and remote control touch devices like touchpads (Fig. 14.7b) are also equipped with active haptics. The kind of feedback generated by these devices is usually designed to mimic the impression of real push buttons.

Fig. 14.7

Faceplate with active haptic feedback (a); stand-alone touchpad with handwriting recognition and haptic feedback (b)

The intention to create a situative and configurable tactile feedback to confirm the activation of a function is not new at all. Different technologies are in application to create such feedback. The type of technology in use is always strongly influenced by the volume available. In mobile application, vibrotactile solutions range from standard rotary motors with eccentric discs over high-current versions of such motors for the creation of shorter and sharper pulses (TouchSenseby Immersion) to voice coil systems which allow a broader frequency response. Lately, other systems can be seen on the market reducing building volume and costs to create vibrotactile systems with broad perceptional bandwidth by an intelligent way to mix just few oscillating frequencies [7]. Besides these classic actuators based on rotary or linear movement, lately mobile devices can be noted based on piezoelectric actuators, or electroactive polymers. Ultrasonic [1] and electrostatic [8] devices gain increased attention, too. Originally, they were not invented for confirmation of a function, but for tactile texture simulation. However, they show some fascinating performance in that point.

Although this bunch of technology exists and some of those already have reached a series level in mobile devices, only few solutions can be applied to automotive applications too. This fact is due to market requirements and technical reasons: Automotive industry has a certain delay in the application of new technologies. Typical design sequences for automotive products last 3 years, which makes them less interesting for young emerging companies which are required to make profitable business within a short period of time. In addition, quality requirements are high, making an entrance into this market almost impossible for anyone but an established Tier 1 or Tier 2 supplier. Besides these general challenges, automotive operating conditions differ in two significant aspects from mobile devices: Their operation is truly one-handed, which cancels any approach to create the haptic feedback in the holding hand. In addition, the feedback has to be very precise and strong to create a good signal-to-noise ratio in the vibrating surrounding of a moving car. This makes actuators for haptic feedback which are still suitable for automotive to become very special.

2.2 The Floating TouchPad of Mercedes Benz

Fig. 14.8

Applied automotive haptic device—touchpad in the Mercedes Benz C-Class, Year 2014

The floating touchpad of Mercedes Benz is an example for such an actuated automotive touch control (Fig. 14.8). Comparable to a smartphone, several functions of the headunit can be controlled by finger gestures and handwriting recognition. At all it offers a haptic feedback on the touch surface to confirm input. A vertical movement of the touch area helps to increase feedback quality and reduces working gaps compared to lateral movement. The flat building actuator enables the realization of a raised esthetic shape. Regarding to this exposed position of the touchpad above the rotary COMAND®Controller, the touchpad acts also as a hand rest. This doubled functionality requires a switchable haptic input effect. To achieve the differentiation between valid and invalid touch, a robust sophisticated 3-dimensional algorithm is developed (patent pending). Thus, the haptic effect only is released if a valid finger interaction is recognized.

The best hardware and algorithms mean nothing, if user interaction and perceptive quality are not sufficient. Aside of hard environmental requirements, the whole user interaction has its own rules in automotive applications. In addition to a straining primary driving task, strong vertical oscillations take effect on the whole human body, especially arm, hand, and finger. Thus, the serial mechanical linkage holding and controlling the relative position between fingertip and user interface has to be reduced and optimized. In the past, this led to separated user interaction concepts [21]. Anyway—probably regarding to cost effects—touchscreen interfaces are more widely spread, while separated interaction concepts usually are found in upper-class cars.

At all these disturbances require a suiting, not only a stronger, haptic feedback. Automotive devices usually are not handheld, and the remaining contact point between the user and the device is the fingertip. The haptic sensing area of the thenar eminence which often casually is used additionally as a very sensitive human input channel [3] is not available. At all, a clear and distinctive feedback is required. Even standard actuators which are often used in the handheld market do not provide a sufficient haptic feedback, even working under all automotive required environmental conditions.

But how should a perfect automotive haptic feedback look like? Initially, it has to be stated that both interaction certainty and the impression of “liking” are criteria for perceptive quality. Regarding to information, interaction certainty of user interfaces needs to provide a robust signal transfer. For that purpose, the interface has to use a known perceptive alphabet to prevent a not wanted strenuous learning. This simply means that the displayed haptic effect has to show known haptic effects, which are correlating to commonly known haptic experiences. Thus, new effects must be able to differentiate information and have to be understood intuitively. Moreover, the effect has to show a positive impression. The number of such effects seems unfortunately small and ordinary. At all in a context of Mercedes Benz push buttons surrounding the touchpad, which show all described quality aspects, such a push feedback of the touchpad seems to be a valuable and promising feedback. The expectations are exceeded if the user does not realize that the touchpad’s feedback is artificial.

But at all how these requirements can be realized? To answer that question, aspects of perceiving a push button have to be taken into account. Maybe a little strange, but it can be supposed that human beings like being lazy. Several, non-published studies show that people like low actuation force combined with highest precision impression (see [19]). Comparing with energy effort, low actuation force means energy saving as well as high precision means high sensing efficiency, because no energy has to be wasted while clarifying a sensual uncertainty.

Our haptic sensing mechanism is extremely efficient, and the difference between vibrating effect and a “snap” is small, and even some strange frequencies denote aberrance from the expected. So besides a well-tuned mechanical system, the electrical control plays an important role. A filtered actuation allows the reduction of disturbing frequencies [4].

A more macrostructural look at haptic quality means the homogeneity of the haptic effect distributed over the whole touch surface as well as different actuation speed. A uniform movement of the surface is required as well as a subjectively identical haptic effect in each point of the touchpad and at different interaction speeds, which is called homogeneity. The interaction-based measurement described later in this chapter plays a key role for an efficient development of the haptic effect and homogeneity reaching a high quality level. Figure 14.9a, b shows the development progress regarding to one dynamic feature. A strong inhomogeneity regarding location and speed can be seen in Fig. 14.9a, reflecting subjective impression. Figure 14.9b shows the effect of the optimization to achieve a homogeneous dispay of haptic effects.

Fig. 14.9

Optimization of the homgeneity of the haptic effect on the surface of the touchpad. Considered measure is the signal energy. The abscissa shows different measuring points distributed over the touch surface. The ordinate shows the parameter values. The different colors show different measurement speed. a measurement before optimization, b after optimization [28]

As mentioned in the beginning, besides the perceptive quality, also the passive behavior of the touchpad plays an important role. A so-called phantom feedback, a hard reaching of the end stop that causes unwanted impression of a haptic snap, has to be prevented. Included in the actual COMAND Online®-HMI, the whole system can be controlled easily also while driving. Thus, it is an important step introducing touch holistically to a drivers’ environment, to reach best acceptance and comfort.

2.3 Actuator Design

The design requirements for an actuator to create precise and strong haptic feedback are not at all obvious. Analysis of the performance of real push buttons indicate that the actuator has to create extremely sharp oscillations $({>}3\,\text {g})$ which are short in time (damped, see Fig. 14.10 left). In addition, a mechanical switch always operates at a certain pretension in force (e.g., 2–5 N) and after a clearly defined path of travel (e.g., 0.25–0.5 mm, see Fig. 14.10 right). Accordingly, the functional elements for an actuator to create haptic feedback on a decorative surface can be identified as follows:

• Force source to create an acceleration of a mass weighting approximately 100 g

• Parallel guidance to match the force–displacement curve independent from the area touched

• Force-sensing mechanism to guarantee a reproducible switching point

For all components, packaging requirements are formulated to limit the thickness of the device to below 6 mm. For this application, an actuation system compromising the following components has been designed:

• Electromagnetic actuator based on punch-bended pole and anchor with PCB-based coils

• Parallel guidance mechanism based on flexible metallic hinges

• Optical SMD displacement sensors

Fig. 14.10

Acceleration measurement and force–distance requirements of a mechanical push button

2.3.1 Electromagnetic System

The magnetic system consists basically of two steel plates and one PCB located between the steel plates. The PCB contains the electromagnetic coils; the coil cores are supplied by one of the steel plates. Thus, a simple yet effective electromagnet is formed (Fig. 14.11). This system offers significant advantages compared with a conventional electromagnet. The inductance of the coils is small, thus allowing for very short raise and fall times and very short magnetic pulses. The PCB-based coils are capable of carrying effective current densities in the range of 10 Å mm$^{-2}$ over the whole PCB thickness, while the direct connection between PCB and supportive metal parts ensures an effective heat transfer and thus prevents overheating of the coil system.

Fig. 14.11

Basic actor design—PCB between two steel plates. The springs represent the elastic component of the module, at the same time they provide the parallel guiding system

Electromagnetic simulation was used to calculate expected force displacement curves. Measurements showed a good agreement between these calculated curves and real measurements (Fig. 14.12). The influence of mechanical tolerances and magnetic imperfections were included into the simulation of the system, and after several optimization loops, the magnetic system was finalized to be manufactured by well-established, cost-efficient processes and capable technologies suitable for automotive use. The current density/force displacement curves allow for significant dynamic ranges in feedback both due to force profile and acoustic feedback options.

Fig. 14.12

Distance/magnetic force measurement using pole shoe plates out of different manufacturing processes—sample 4 is closest to CAD target. Spring is excluded from measurement; thus, only the magnetic snap force $\Delta$F (see Fig. 14.10b) depending on distance is measured. Distance is measured from mechanical rest position

2.3.2 Parallel Guidance

Customers explicitly ask for a homogeneous movement of the surface, with a preferred movement range of less than 0.3 mm. The intention behind this is to visually inhibit actual movement of the input device. For a high-quality feel of the context-sensitive feedback, it is required to activate the haptic pulse at the same user input force anywhere inside the active area. Both aspects are achieved by a parallel guiding system free from play (Figs. 14.11 and 14.13b). Two parallel springs guide the anchor plate with respect to the pole shoe plate. At the same time, they provide the return force for the actuator module. The transfer rod between the two spring arms transfers a share of the force to the other side of the guiding system to prevent a rotation and tilting of the anchor plate and stabilize the parallel guiding system. As a result, homogeneous force–displacement curves are reached almost independent of the location on the actuator surface (Fig. 14.13). The system leads to an almost vertical movement of the anchor plate, and a homogeneous force at the switching point over the active area.

2.3.3 Sensors

Sensing the user input is a crucial issue within the whole system. Additionally, the automotive context requires a high robustness against electromagnetic interference, operation in a broad temperature range, and reliability during a lifetime which is much higher than of any consumer device. In contrast, for a valuable haptic impression, high sensitivity and precision of the sensors are necessary. Table 14.1 summarizes these requirements. A user gets a constant and valuable impression at repeated inputs if the traveling of the surface varies less than 10 % around the predefined switching point. In combination with the small distance, the surface can travel at all (see Sect. 14.2.3), and the resolution of the ADC of the used microcontroller, the sensitivity of the sensor has to be better than 700 mV/mm. To detect fast movements, the sensor has to provide a cutoff frequency above 500 Hz.

Fig. 14.13

FEM results of simulated force displacement curves at several locations on actuator surface (a) compared with real measurement results (b). Only mechanical spring force is shown; snap force $\Delta$F (see Fig. 14.10b) is excluded

Table 14.1 Sensor requirements

To fulfill all of these different and opposed requirements as much as possible, an integrated reflective interrupter is used in this haptic input device. The most important advantages are as follows: The optical functional principle of the distance measurement does not interfere with any electromagnetic fields; the sensitivity can be adjusted by two resistors $R_F$ and $R_L$ (Fig. 14.14a) and the design of the reflective surface (Fig. 14.14b). The elevation of the reflective surfaces from the anchor plate level controls the operating point of the sensor. Hence, the output of the sensor is almost linear for the whole movement of the input surface (Fig. 14.15).

Fig. 14.14

Equivalent circuit of the reflective interrupter (a); assembly of sensor and reflective surface (b)

Fig. 14.15

Sensor output characteristics

2.3.4 Acceleration Amplitude Range

Simulation results lead to the expectation that the maximum acceleration achievable is mostly determined by the air gap remaining at the switching point. Accelerations of more than 10 g are achievable. Smaller accelerations can be controlled by limiting the available current. Initial customer evaluations showed that this performance exceeds the actual needs. A nominal acceleration of 3 G corresponds to user expectations (Fig. 14.16) in the intended context.

Fig. 14.16

Examples of haptic output characteristic with a real finger as a load [28]

2.4 Evaluation

With this actuator being used in high-volume series-production devices, extraordinary requirements on its quality and the measurement of its performance are essential.

Fig. 14.17

Static measurement for a push button: measurement device (left); force–displacement curve and technical features derived from the curve (right) [28]

Fig. 14.18

Two push buttons of the same type (left), and the corresponding static force-displacement measurements (right). One has an added mass of 2.5, Both curves seem the same [28]

Fig. 14.19

New C-class Touchpad with functional sketch of its electromechanical actuation system (left), static force-displacement curve (right) [28]

Fig. 14.20

Acceleration in the interaction during pushing a button is simulated (Interaction I). If the probe parameters are changed in the simulation, the interaction will be changed significantly. Interaction II: Probe damping is 3 times greater; Interaction III: Probe stiffness is 10 times greater; Interaction IV: Probe mass is 10 times greater [28]

As described in [28] the existing static measurement principles are not sufficient to describe human haptic perception and even the new, artificially actuated user interfaces cannot be described correctly. Figure 14.17 shows measuring force versus displacement with a rigid probe and the resulting characteristic curve plotted force versus displacement. Zhou shows several examples where clearly perceived differences cannot be seen in static measurement, such as different inertia by simply adding masses (see Fig. 14.18) or different viscoelasticities by influencing the frictional pairings of the control’s bearings. Electromechanically driven switches usually do not show typical snap-behavior as a rapid force reduction, which is one of the key features in the static measurements (see Fig. 14.19).

To solve this problem, [28] show that the switch itself and the human finger influence each other so strongly that the resulting dynamic behavior changes strongly if, for example, the finger’s impedance is changed as shown in Fig. 14.20. This expounds the missing information, or in other words: the reduced bandwidth when the impedance rises to a rigid measurement finger that gets dominant to the switches’ impedance.

Fig. 14.21

Interaction model when pushing a button (left); finger-like measurement device (right) [28]

The described method of “interaction-based-dynamic measurement” as shown in Fig. 14.21 considers both impedances of the switch and the human finger to measure the mechanical interaction-data like acceleration versus time. In comparison to static or also the opposite contactless measurement, the mechanical behavior of a perceived haptic feeling and not the standalone mechanical parameters of the system are measured as perceived. Regarding its interaction-perceiving principle its biggest advantage out of a specifications’ view is that these parameters and their tolerance ranges can be transferred to different components, because perception will be constant for all. To get to the point, the perception, and not the device, is specified.

Measurements comparing the finger-like measurement tip with real human fingers in Fig. 14.22 show similarity. Returning to the previously described examples the new interaction-based-measurement shows differences with changed inertia in Fig. 14.23 as well with different active haptic feedback in Fig. 14.24.

The haptic features that connect the objective with subjective perception are the focus of current research.

2.5 Discussion and Outlook

Any engineering work focuses on designing a product. A successful industrialization of haptic actuation technology in any industrial context, however, requires always two things:

• a technology with high performance and optimized quality-to-cost ratio for the intended market

• a measure for this performance and quality

Fig. 14.22

Three measurements of acceleration (left, in time domain; right, in frequency domain) between three different fingers and a push button (red curves) are compared to one measurement of acceleration between the probe and the same button (blue curve). The two groups of the curves are very similar [28]

Fig. 14.23

Dynamic measurement of two push buttons with the same force–displacement curve, but different masses [28]

Fig. 14.24

Dynamic measurement of two touchpad samples with the same stiffness, but different active haptic feedback [28]

Both items are not to be underestimated. A successful product is nothing without a corresponding quality control. In particular, in the area of haptic technology, this is something which always needs to be developed according to the application at hand. The Mercedes C-class touchpad is an excellent example of this conjunction of research and development and is proposed as an example to extend the real product range of active haptic devices available to the market.

3 HapCath: Haptic Catheter

Thorsten Meiss, Thomas Opitz, Tim Rossner, and Nataliya Stefanova

3.1 Introduction

Catheterizations are a conventional process for diagnostic and interventional treatment of blood vessels, which suffer from atherosclerotic depositions, diminishing the blood flow and causing infarcts. In the USA, diagnostic and interventional catheterizations of the heart were performed approximately 1,059,000 times in 2007 (Data base 2011 [20], and about the same amount in Germany in 2009 [2]). In many cases, catheterization is a simple process for well-trained cardiologists: A guide wire is inserted into an artery, usually the arteria fermoralis at the upper leg, and is slid toward the heart. By rotating the proximal end of the wire (the end in the physicians hand), the physician leads the tip at the wire’s distal end into the coronary arteries. To visually control the guide wire movement, short-time 2D-X-ray video is used. By sliding a catheter over the guide wire, the physician can lead contrast fluid into the vessels to visualize the course of the arteries for diagnostic purposes. Through this hollow catheter, the physician can lead tools to the upper branches of the coronary vessels or may change and reposition the guide wire very quickly. To reopen totally closed or occluded vessels, the physician can thread a balloon catheter over the proximal end and slide it over the guide wire, through the catheter, into the occluded part of the vessel. Then, he can widen the affected vessel by inflating a balloon (dilatation) and optionally expand a stent to prevent the vessel from contracting again.

However, in many cases, the vessel is totally closed and/or penetrating the obstruction with the guide wire tip becomes very difficult. Additionally, navigation of the wire often turns into a challenging task. Due to the small diameter and the low stiffness of the wire, the physician cannot feel the forces at the wire’s tip; the 2D-X-ray imaging linked with the legal limitation of the amount of noxious contrast fluid requires well-trained operators and a long training phase of new cardiologist. To overcome these challenges originating from a lack of intuitive usable information from the guide wire’s tip, the HapCath system provides haptic feedback of the forces acting on a guide wire’s tip during vascular catheterization [12] (Fig. 14.25). In order to achieve this, force measurement and signal transmission out of the patient’s body is realized. Thus, the transmitted signals are used to control actuators within a special haptic display to provide a scaled, amplified force, which is coupled back onto the guide wire. These scaled forces surpass the friction forces, which arise along the length of the guide wire in the vessel and catheter, enabling the user to feel the tip interacting with the walls and obstructions inside the vessel. This shall simplify and accelerate the navigation of the wire and reduce the risk of punctuating the vessel or damaging plaque (depositions). The aim of providing haptic feedback is to enable grasping the right way through the vessels just like with a blind man’s cane. For this purpose, very small force sensors have been designed, fabricated, tested, and integrated into guide wires. Special electronics to calculate the 3D force vector acting at the tip have been designed, and a haptic display with a translational and a rotational degree of freedom to couple the forces back onto the guide wire to amplify the measured forces has been constructed and tested.

Fig. 14.25

Schematic of the assistive system HapCath: The forces $F_0$ at the tip of the guide wire are measured by means of a small force sensor. The signal $S_{F0}$ is transmitted out of the patient’s body over the wire. Within a haptic display, the signal $S_{F0}$ is reconverted into a scaled force $n \cdot F_0$ by means of amplifiers and actuators, thereby overcoming the friction force $F_F$ within the catheter and vessels. This force is displayed to the surgeon’s hand as the amplified force $F_H$

3.2 Deriving Requirements

To our knowledge, the exact forces at the guide wire tip during catheterization have been unknown up to recently. For this project, detailed analysis of the advancing of the guide wire within the vessels [13] with simulation [5] and experimental measurements of the guide wire interactions have been performed.

3.3 Design and Development

3.3.1 Force Sensor Design

Figure 14.26 shows selected relevant scenarios of the interaction of the guide wire with the vessel walls or with stents inside the vessel.

Fig. 14.26

Pictures of different interactions of the guide wire with the vessel, with different kind of plaque (a), within a heavily wriggled vessel path (b), and with a stent (c) and (d)

A force sensor can be integrated at the tip of the guide wire or with some distance to the tip. To allow for the measurement of the interaction forces when the guide wire is advanced with the tip backward (Fig. 14.26d), the sensor needs to be integrated with several centimeters of distance from the tip. This will lead to additional friction forces in the sensor signal and will result in lower frequency response due to higher mass and damping. The most beneficial location to integrate a force sensor therefore is directly into the tip of the wire, due to higher amplitude and frequency resolution of the contact force measurement.

Simulations haven been performed were the guide wire is modeled as distributed elastic elements interacting with viscous elastic walls of the arteries with  [5]. Additionally, experiments to determine the buckling load of different types of guide wires where conducted [13]. Both methods reveal a maximum force in axial direction of the wire of around 100–150 mN, e.g., for penetration occlusions, depending on the kind of guide wire used. The forces during advancing and navigation as well as for detecting surface properties, e.g., roughness or softness—for diagnostic purposes—were estimated to be in the range of 1–25 mN.

To allow for force measurement at the tip, two types of microforce sensors have been designed and fabricated [10, 11, 13] (Fig. 14.27). They are built from monocrystalline silicon with implanted boron p-type resistors. This technology was chosen to fulfill the requirements on microscale manufacturing with its high level of integration, a relatively high voltage output for robust external readout as well as high mechanical stiffness to fulfill the requirements on high-frequency resolution up to 1000 Hz as well as the need for quasi-static measurements when the guide wire remains in static contact with a constriction.

3.3.2 Guide Wire and Sensor Packaging

Guide wires are disposable medical products, manufactured with technologies of precision engineering. The guide wire requires a maximum torsional stiffness and a variable bending stiffness along the wire. To integrate an electrical connection of the sensor over or within the guide wire is a challenging task. A loss in rotational stiffness due to softer materials of the conductors than stainless steel or nickel–titanium will result in less mechanical performance. This is the main reason why the space for the integration of electrical wires is very sparse. The electrical connection is established with four insulated robust copper wires, each with a small diameter of 27 $\upmu$m.

The sensors are glued onto the tip of the wire with UV curable medical adhesive. They are enclosed into a flexible polyurethane polymer [9] and covered with medically compatible parylene C. Figure 14.28 gives an insight into the assembly.

Fig. 14.27

Two types of monocrystalline silicon force sensors: Both are designed to resolve the full force vector in amplitude and angles. Their size is compared to an ant

Fig. 14.28

The integration of the sensor into the guide wire tip encompasses several steps of precision mounting, dispensing, and covering with glues and cover polymers

3.3.3 Haptic Display Design

The guide wire is navigated through the vessels with two degrees of freedom: translationally, to advance the guide wire, and rotationally, to choose the relevant wire branch. The haptic display is designed to provide these forces and motions (Fig. 14.29) [17].

Fig. 14.29

Basic design of the haptic user interface with the translational degree of freedom and side view of the implementation [17]

The haptic display supports the generation of static forces to display the penetration of occlusions. To give feedback of surface roughness and to reflect the dynamic amplitudes during penetration or when the wire is moved over the grid of stents or rough depositions, the haptic interface is designed with low mechanical inertia to generate high-frequency feedback as well. To optimize the dynamic performance of the haptic interface, the equivalent circuit representation of the electromechanical setup with guide wire and passive user impedance is used (Fig. 14.30) [17].

Fig. 14.30

Network model of the haptic user interface including the guide wire and the user’s passive mechanical impedance

3.3.4 Electronic Design

The system is powered with one single electronic system. The electronics provide power supply to the force sensor and a unique six channel high-resolution analog front-end. A microcontroller with floating point unit is used to control the sensor readout and to calculate the 3D force vector of the contact forces. It provides angle measurement and PWM control for two brushless DC motors for the haptic feedback interface as well. Force signals are transferred to a PC and to a display for information purposes. The control loop is implemented in the microcontroller itself without the need for time-critical communication with the PC. This allows for a fast control loop with up to 10 kHz s$^{-1}$ control rate for smooth haptic feedback [13].

3.4 Verification and Validation

To validate the function of the whole system, the tactile guide wire, the electronics, and the haptic display are connected [12]. The guide wire is advanced into a model of the arteries built from silicone tubes (smooth, healthy arteries) partially filled with epoxy glue mixed with sand to model rough depositions (calcified plaque). Figure 14.31 shows the sensor signal over time when the guide wire is moved in the model and Fig. 14.32 over plates with defined surfaces. The haptic feedback at the interface allows for discrimination of smooth and rough surfaces. When touching different surfaces, the contact force is amplified and different surfaces can be distinguished clearly.

Fig. 14.31

Moving the guide wire with a minimal contact force. The physician maneuvers the guide wire by moving the handle by rotating, pushing, and pulling (a). Measurements with a prototype of the tactile guide wire within a model of the arteries with artificial plaque (b). The measurement shows the sensor signal during inserting, moving the wire forward, rotating the tip, and going into the right and then into the left vessel branch

Fig. 14.32

Force signal over time recorded from packaged sensors integrated into a guide wire prototype to evaluate different surface roughness. Glass (a), paper (b), and sand in epoxy glue (c). Notable is the reproducible, nearly periodic output of the packaged sensor on paper (b). Increasing roughness of the surfaces leads to increasing output signals (a), (b) to (c)

By increasing the amplification factor of the forces, the sensation of soft or elastic surfaces changes to that of much more rigid features due to the higher stiffness emulated by the system. This makes soft, fragile surfaces much easier to detect. They become virtually harder, whereby elongation due to force is reduced. We assume that this can lead to much less ruptures of vulnerable features, leading to fewer complications during catheterizations in the future, as well as a higher success rate for complicated interventions with wriggled arteries.

3.5 Conclusion and Outlook

The project involves several technical challenges, encompassing sensor design and sensor integration, as well as adapted haptic feedback. The current field of research is focused on transferring the results into application by refining the design and the technical implementation of the sensor, electrical wire, and guide wire assembly. The application will benefit from ongoing research regarding optimized filtered signal feedback from touch scenarios of the tip with different surfaces. The project is funded by the German Research Foundation DFG which supports this project under Grant No. WE 2308/3-3.