Keywords

1 Introduction

1.1 Eye-Tracker Overview

An eye tracker is a device used to analyze eye movements, as the eye scans the environment or fixates on particular objects in the scene, and to simultaneously localize the eye position in the image and track its movement from one location to another over time for determining the gaze direction. Eye-tracking technology is a specialized application of computer vision to do such analyses. However, this technology needs sophisticated software to process each image, captured by a high-speed camera, so that the gaze direction (estimated by the pupil center) is obtained, and the mapping from the gaze direction in the image to the screen is calculated to predict the gaze location on the screen. The mapping between the image and the screen is depicted in Fig. 1. The original gazing point A on screen, captured in the image I, constitutes vector GP, which is mapped back to obtain vector OS on the screen. The coordinates of S(sx,sy) and mapping error Err° (in angle degree) between S and A can be calculated from the coordinates of G(x,y). The mapping calculation is discussed in more detail in the next section [1].

Fig. 1
figure 00501

Mapping from the pupil center to screen

Full size image

1.2 Cameras and Open-Source Software

Observing from the eye-tracker overview, the high-speed camera and the R&D in software integration might be the main costs to keep the commercial eye-tracker devices at a high price in the market. However, rather cheap commercial over-the-shelf (COTS) cameras and open-source software are now available in the web; some of these are listed as follows:

COTS camera

QuickCam Pro 4000, MS LifeCam VX-1000, Sony PS3 Eye cam

Open source

ITU Gaze [2], CL Eye Platform Driver [3], AForge.NET, OpenCV

By minor modifications of the COTS camera and integration with open-source software, some low-cost eye-tracker devices have been developed, as shown in Table 1.

Table 1 Low-cost and commercial eye trackers
Full size table

This chapter presents a low-cost (US$100) eye tracker using Sony PS3 Eye camera combining the open-source Code Laboratories [3], which achieved high-speed image capturing of 187 frames per second (fps); a more detailed description is given in the next section. The chapter is organized as follows. Section 2 describes the hardware configuration and background technology. Section 3 describes the integration software. Section 4 presents the application examples. Finally, Section 5 presents the conclusion as well as future works.

2 Hardware Configuration and Background Technology

2.1 Hardware Configuration

The Sony PS3 Eye is reconstructed to become infrared (IR) sensitive only and two IR-LEDs (850 nm) are appended for the dark pupil tracking process. USB 2.0 was used for the communication with the computer. Figure 2 shows our glass-like head-mounted tracking hardware.

Fig. 2
figure 00502

Modified Sony PS3 eye-tracker hardware

Full size image

2.2 Background Technology

Glint, Bright Pupil, and Dark Pupil. The light reflected from the IR-LED beside the camera caused a white speck on the iris called glint. If the IR light source coax with the camera, most of the light is reflected back, causing a bright pupil; otherwise the pupil is rather dark [5] as shown in Fig. 3.

Fig. 3
figure 00503

Bright pupil, dark pupil, and corneal reflection of an infrared camera image

Full size image

Glint–Pupil Difference Vector. A camera was used to track the location of the pupil center with reference to the glint of the IR light. Since the cornea of an eye is nearly spherical, the position of the glint remained more or less fixed as an eye moves to focus on different points of regard (POR) and the difference vector between glint and pupil is resulted as shown in Fig. 4. Eye tracker uses these difference vectors (GPs) in the image to determine the gaze vectors (OSs) on screen; a more detailed description of screen mapping is presented next.

Fig. 4
figure 00504

Three glint–pupil difference vectors (GPs)

Full size image

Screen Mapping. After the point P(x,y) is calculated in the image, our design used the following least-square polynomial equations to map the estimated gaze point S(sx,sy) on the screen as shown in Fig. 1:

$$ sx={a_0}+{a_1}x+{a_2}y+{a_3}xy+{a_4}{x^2}+{a_5}{y^2} $$
(1)
$$ sy={b_0}+{b_1}x+{b_2}y+{b_3}xy+{b_4}{x^2}+{b_5}{y^2} $$
(2)

Calibration. The coefficients a 0–a 5 and b 0–b 5 of the aforementioned equations have to be determined before tracking is started. Therefore, at the beginning of the eye-tracking process, the user is asked to look at a set of known position points on the screen, to make the values of (sx,sy) and (x,y) become known, in Eqs. 1 and 2, such that the coefficients a 0–a 5 and b 0–b 5 can all be solved.

3 Integration Software

A sophisticated software system should be developed to control the camera operation in accordance with the related fast image capturing, glint–pupil detection, calibration, and mapping calculation. The above-mentioned open source offered such integrating services.

With minor modifications of the PS3 Eye camera as the main hardware component and proper coding adjustments in the software interface to the CL Eye Platform Driver, offered by the open-source Code Laboratories [3], our low-cost eye tracker runs at the speed of 187 fps in 320 × 240 resolution, which is at least three times faster than the ordinary low-cost eye trackers usually running between 30 and 60 fps [4, 5].

4 Application Examples

4.1 High-Speed Demonstration in ISO Test

ISO test requires that if the participant can make eye movement between two predefined locations on screen within 2,000 ms, it achieves a correct response trial and its response time is recorded. There are 5 sessions with 16 trials each. The rate of correct response trials and its average (correct) response time are displayed in Fig. 5.

Fig. 5
figure 00505

(a) Response time comparison, (b) Correct response rate comparison

Full size image

After having developed the ISO test application program, our design was then compared with the low-cost KSL-240 and commercially available high-price (over US$5,000) Tobii under the ISO 9249-Part 9 eye-tracking test [4, 6]. The test performance favored ours in terms of response time and correct response rate as shown in Fig. 5.

4.2 Gaze Replay, Heat Map, and AOI

The applications Gaze Replay, Heat Map, and Areas of Interests (AOI) were also developed, whose performances are explained next.

Gaze Replay. Gaze Replay is able to display dynamically the sequence of gaze path as shown in Fig. 6a, where the circle denotes the gaze position while the size of circle is proportional to the gaze time. The numbers in all circles represent their timing sequence.

Fig. 6
figure 00506

(a) Gaze replay, (b) Heat map, (c) AOI

Full size image

Heat Map. Heat Map is able to display the total gaze time at each gaze area using “d” magnitude levels as shown in Fig. 6b. The basic gaze unit is 5 ms. By using “d = 3” in this experiment, the area obtained the maximum number of accumulated gaze units assigned red color, the minimum assigned green, and the middle assigned yellow. The number of “d” and the related colors are users’ choices.

AOI. AOI is able to select the areas according to the user’s interests for further processing by using squares, as shown in Fig. 6c, where the data in each AOI are then transformed to Microsoft Excel for further processing in terms of saccade and fixation.

5 Conclusion and Future Works

We have been building a low-cost and high-speed eye tracker, which is a sort of hardware and software integration, and some application programs have been running in the system. Three application results are displayed in Figs. 5 and 6. Currently, we are developing a tracking accuracy application test.

The future work will include head movement estimation. This will require the development of control software involving the complicated issues concerning the measurement of distances and angles through image processing [8]. However, if it is successfully implemented, then the chin rest used in the experiments would become unnecessary.