Keywords

1 Introduction

There is a wide range of research about sensory substitution including feel the sound by tactile stimuli. For instance, presenting pitch of the sound on the tip of finger like Braille patterns or transferring rhythms through vibrations [1,2,3]. A tactile vocoder [4] that allows listening to audio with vibration arrays placed on the fingers or a haptic chair [5] used for speech training that allows one to feel the vibration stimulation by sitting on a chair-shaped stool are examples of research that enables sensation by converting sound information into touch information.

We can assume that the vibration information aids a hearing impaired people in terms of providing the missing auditory information. However, the techniques above are difficult to implement in a regular environment because it requires a large scale system and the actuator is expensive so there is a demand for a simpler structure and an equipment that can change and transmit audio information into vibration using the regular actuator technique.

We developed hearing device VibGrip [6] it converts audio information into vibrations that can be felt with the use of the 5 fingers. With 5 actuators, it is possible to feel the different musical part on each finger. For example, thumb is mapping for keyboard sounds, forefinger is mapping for vocal sounds and middle finger is mapping for drum sounds. On the other hand, VG has no sensor and it is difficult to control the intensity of vibration. We improved VG by using pressure sensor. The VG+ allows controlling the intensity of each finger’s vibration and also controlling filter-effect. With this function, people with hearing impaired can feel the vibration of each musical part more clearly and enjoy in the concert.

2 System Description

The VG+ has piezoelectric actuators mounted on a small housing. The piezoelectric actuators used in this device can generate frequencies between 100 Hz–40 kHz and since it is suitable for producing low frequencies, it is possible to feel the audio information as vibrations. This is done by holding the device in one hand and having all fingers come into contact with the piezoelectric actuator.

The connection of the VG+ with a PC outputs a mono audio signal (5ch) via the audio interface and done using an amplifier. The design concept of the housing includes a third ear on the device which has been set to mimic the shape of an ear which serves as a hearing expansion. In addition, it is easy to hold with a grip shape similar to the ones in a bicycle. There are 5 pressure sensors mounted on the actuator, it detects how strong grasping the device with each finger. In current prototype we used Arduino and pressure sensor: Interlink 402. Figure 1 shows the shape and structure of VG+.

Fig. 1
figure 1

Structure and shape of VG+

Full size image

The integration of the system that we are proposing for this time (Fig. 2). VG+ device works as haptic display and controller. VG+ application consists of two modes. One is controlling the intensity of each finger’s vibration, the other is controlling center frequency of low-pass filter. With this application, people with hearing impaired can adjust the vibration more intuitively.

Fig. 2
figure 2

VG+ system diagram

Full size image

3 Experience

The presenting condition are Audio+Vibration (AV) and vibration (V). In condition V, participant will wear the headphone has noise cancelling function: Bose QC25 to simulate sensorineural hearing loss. At first, participant sits on the chair and grasp the VG+, then system starts automatically plays rock music. They move each finger to control the vibration in mode1: intensity control and mode2: low pass filter control in condition AV. After that, participant try mode1 and 2 in condition V.