Keywords

1 Introduction

Online travel agencies (OTA) vigorously compete on the Internet for the attention, visits and ultimately bookings of travellers of the services offered through these intermediaries. One of the essential differentiators of the services provided by OTAs—besides the breadth of coverage of travel companies—is the ease of using these online travel portals, their usability. Poor usability directly translates into low conversion rates, i.e. low revenues and profits. For OTAs to grow, it is critical to remove any usability problems for the target customer groups, who demand the high margin travel products. Clearly, for improving the financial performance of an OTA, one needs to start with identifying high-margin travel products. Then for these products, the most promising customer groups must be characterized. And finally, the user experience for these identified customer groups when they shop on the platform must be continuously improved.

This chapter has several objectives. The first one is to identify usability problems for a real-life existing OTA (onlineweg.de), which prevent potential users who visit the page from booking a travel service. Problems that are the result of poor software and web design require changes to the existing structure. For communicating the case for change compellingly to the team members in marketing and IT of the firm, the study must generate objective, quantitative data. Eye-tracking generates such objective, quantitative data. The second objective is to quantify the economic losses that the usability problems cause for the firm, or, to express this in another way: the study will try to ascertain the economic gain that conversion improvements will be generating for the firm.

The third objective goes beyond this individual case study. Our aim is to make the process transparent, which leads to conversion improvements because understanding this process and learning the necessary steps involved is important in general for all the conversion improvements yet to come. Then our fourth objective concerns a commonly heard prejudice against eye-tracking that claims this method would be too advanced, sophisticated and time consuming to be of any practical use outside of the university. Therefore, it is our objective to prove that the educational demands for staff members of a firm, who are conducting usability studies with eye-tracking are at reasonable levels. A team of 14 young university students at the University of Applied Sciences Munich conducted many of the steps involved in gathering the qualitative data (through the so-called “Retrospective Think Aloud” technique) and the quantitative data through the eye-tracking method.Footnote 1 While they did receive training and on-going support for their project work from the authors of the present chapter, their work results used for the present study were substantial enough to prove that the necessary knowledge and skills can be learned and acquired within a relatively short period of time.

In order to achieve these objectives, our study will show in chapter “A Review of Eye-Tracking Methods in Tourism Research”, a brief overview of the OTA onlineweg.de and its market environment in terms of the growth of e-tourism related to OTAs and the intensity of dynamic competition among them. We also focus on one of the key customer groups for onlineweg.de, which results from the identification of some of the travel products with higher margins. Thereby, the two essential elements of improving usability have been put in place: the travel products and the relevant customer groups. Chapter “Best Practices for Eye-tracking Studies: DOs and DON’Ts” gives a literature review of the relevant sources for conducting the applied research which follows in the subsequent chapters. The methodological issues are discussed in chapter “Eye Tracking: Evaluation, Potential and Limitations of Field Applications”, which concern selecting and recruiting the test users and the process of gathering the qualitative and quantitative data through RTA and eye-tracking, respectively. Two different use cases are employed in our study and we present them in chapter “Knowledge Co-Creation through Eye Tracking in Tourism”. The results of the eye-tracking analysis are discussed in detail in chapter “The Relevance of Eye-Tracking to Understand Users’ Practices and Content Interpretation in Tourism-related Online Navigation” for both use cases separately. In addition to these results, in chapter “Areas of Interest on Destination Websites: A Generation Y’s Perspective” we summarize the general assessment of the quality of onlineweg.de by the test users. From these results, we give in chapter “Measurement of Visual Attention to Advertising Using Eye-Tracking Techniques” a number of recommendations on how to improve the conversion rate. This chapter answers the question of whether we did achieve our objectives.

2 The Market Environment of the Multichannel E-Commerce Company Onlineweg.de

Onlineweg.de is not just a self-service online portal for booking travels. It also serves travel agents, who work in the firm’s network of 1280 travel agent bureaus nationwide and who sell directly offline to customers. Such a multichannel approach has several advantages as compared to a pure online company. First of all, travel agents can help to sell higher margin products through bundling travel services, which reduces price transparency but also increases the recreational value for customers. Secondly, customers cannot search for truly innovative travel services because—by definition—they do not know the appropriate search terms yet. Papathanassis (2011), for example, discusses many of these innovations in the travel industry. Especially for innovative travels, do-it-yourself travels are oftentimes risky—in terms of quality uncertainty—and disappointing for travellers. Thirdly, travelling is information intensive in terms of quality characteristics that customers are often not aware of before the start of the travel. And the suitability of a travel depends very much on personal preferences. Thereby, travel agents can add real value to travellers’ vacation experience. Another countervailing trend for do-it-yourself travels which supports instead the latent demand for organized travels is the ageing of our society and the rising loneliness of elderly people, who can meet people with similar interests and mindsets through travel. These advantages of a multichannel approach to OTAs notwithstanding, the present study focuses on the market of younger unassisted self-service customers, who can easily switch between competing OTAs. The underlying assumption here is that they would be less forgiving when it comes to poor design of OTAs.

2.1 The Growth of E-Tourism in Germany

Our study will use a technology for improving conversion rates which is still rather novel for the tourism industry as will become obvious from our literature review in chapter “Best Practices for Eye-tracking Studies: DOs and DON’Ts”. For a number of years, the academic community has been researching the potential and actual impact of information and communication technologies on tourism. For example, the articles in the yearly conference proceedings of the International Federation for IT and Travel & Tourism (IFITT) document the lively academic discussions among scholars from various academic fields. In the 2015 edition edited by Tussyadiah and Inversini (2015), two of the articles deal with eye-tracking applications in tourism.

Overall, the growing research in this area is supported by the growth of the travel and tourism industry worldwide and by the information intensity of travel choices. The latter offers a large number of productive applications for ICT. What seems simple is anything but: Few consumer choices are as complex and information intensive as holiday trips. And the internet is ideally suited to serve these information needs globally. As users’ experiences with online information sources grow, so does the use of specialized tourism information sources like, for instance, Tripadvisor.de.

2.2 Dynamic Competition Among Online Travel Agencies

After gathering information about the next tourist destination, consumers can stay on the Internet and book their hotel and trip through online travel agencies (OTAs). Since many businesses in the travel and tourist industry like airlines apply price discrimination based on passenger data to optimize their revenues, consumers use Internet technology to minimize their expenses by comparing rates provided by OTAs. Since online booking has grown into a large market, there are several providers that compete vigorously for market share.

Figure 1 shows the Internet search volume on Google for three of the largest OTAs in Germany from the beginning of 2004 to September 2017: HRS (Hotel Reservation Service), Expedia and Booking. The search volume is an indicator of their market shares. The graph shows that the market leader position changed three times during the time period covered by the data. Expedia was the leader among the three up until August 2008. Then between September 2008 and March 2013, HRS became the market leader, and after that Booking, which had a rather late start but a very consistent growth path, left its competitors behind and took the undisputed Number One position. This shows how intense dynamic competition is in the OTA market and how difficult it is to maintain market leadership.

Fig. 1
A graph of google search index versus date depicts dynamic competition among online travel agencies in Germany as measured by google search volume for e-tourism. The following three curves are plotted with an increasing trend. 1. H r s. 2. Expedia, 3. Booking. The peak value for booking is 100.

The growth and decline of three OTAs in Germany

Full size image

The set of hotels that are included in each of the OTAs are very much the same, but what differentiates them are dimensions like the information provided and their usability. For an OTA, improving the usability is not just a “nice to have” feature, but a matter of competitiveness of the highest priority. The owners demand an honest and blunt analysis of their site’s usability and do not welcome the systematic underreporting of usability problems, which might please the management but hurt the bottom line of the firm. The best way to prevent the systematic underreporting of usability problems is the application of eye-tracking, instead of exclusively relying on the so-called “think aloud” technique, which can result in underreporting of usability problems.

2.3 Highlighting a Key Target Group for Onlineweg.de

For the process of continuously improving the usability of a website it makes sense to focus on a group of more internet savvy users because if they have problems, all others will. Therefore, for the target group of the present study “Generation Y”, also called Millennials, was chosen. This population cohort was born between 1980 and 1999 and raised during unprecedented levels of peace and prosperity. They largely lack any direct personal experiences with major economic or political crises in Germany like the famine after WWII or the cold war. They also could not gain such experiences indirectly through the eye-witness reports of their parents and grandparents anymore. This makes it difficult for them to properly comprehend the fragility of peace and democracy outside of their home country.

At the same time, they are more actively using the Internet, social media and computer technologies than older generations do. According to a recent market study by the German market research firm GfK (2017), this group is relatively higher in booking travel online. They book more than 50% of their trips online, and they prefer shorter and cheaper trips. Travel packages are less popular among this age group than among older people. They prefer to organize their travel themselves.

3 Literature Review

While the use of eye-tracking in academic research has been going on for decades, its application in tourism research is still somewhat recent. Yet for usability studies, eye-tracking has become a well-established method. An important issue for hotels is the online available reviews of former customers and their impact on new bookings. Aicher et al. (2016) show with their eye-tracking study that rating symbols are given higher attention and therefore priority on hotel review platforms than written comments, and the ratio of positive and negative reviews only partially impacts customers’ booking decisions. The dark side of reviews of prior customers is fake reviews and their impact on competitors. Lappas et al. (2016) addressed in their study the impact that different attack strategies using fake reviews can have on the ranking of hotels. Using a large database, they show the enormous vulnerability of the competitiveness of individual hotels to fake reviews.

How can tourist destinations that invested heavily in new attractions and infrastructure but that are not yet trendy attract new customers? Search engine optimisation is quite powerless here because it completely depends on what people are searching for. The only way out is to use online advertising that draws the attention of potential customers. Unfortunately, the eye-tracking evidence about the ineffectiveness of banners is so unanimous that it resulted in coining a new term: “banner blindness”. Despite these negative results, Hernández-Méndez and Muñoz-Leiva (2015) conducted an eye-tracking study about the effectiveness of different designs of online advertisements for the hospitality industry. Their results show differential attention to text versus images and static versus animated banner advertising. Another source of inspiration that travellers receive for visiting new destinations is social media sources. In using eye-tracking, Marchiori and Cantoni (2015) studied tourism-related web browsing behaviour on social media pages that is formative for travellers’ preferences for travel destinations.

Noone and Robson (2014a, b) examined individuals’ eye movements during the two major stages while booking a hotel online, which are browsing and deliberation of actually booking a hotel. Their studies reveal the differences in attention that customers devote to the different information items during different phases of the purchase choice process. Pan et al. (2013) and Pan and Zhang (2016) studied the factors that influence consumers’ attention when selecting a hotel, such as the page position of hotels, the number of options and the images presented. It turned out that images reduced the problem of information overload and helped to increase the hotel’s appeal to customers when it does not excel in terms of other criteria. Hao et al. (2015) also used eye-tracking for identifying the elements (a large main picture and little text) that make hotel websites visually appealing to Chinese generation Y customers.

Vryona (2014) studied in his doctoral dissertation, where he used eye-tracking, the process of customers’ decision-making when they make hotel online reservations. He identified the so-called Customers’ Critical Information Requirements (CCIRs) which impact customers’ decision-making process when they book their hotel online. Aldi (2015) examined in her master thesis the usability of the booking procedure on three four-star hotel websites, which are all located in Lugano (Switzerland). She derives from the results of her eye-tracking analysis a series of usability guidelines that are applicable to the design of the hotel booking process.

On the methodological side, it is important to note that the common use of the so-called “think aloud” technique in usability studies can have very serious shortcomings if it is used without eye-tracking. These shortcomings can result in systematically underreporting the true usability problems for the target users. The methods’ shortcomings can arise, for example, from (i) a high-power distance between the usability analyst and the user, or from (ii) problems of users’ lacking reading comprehension, which falls short of the level of reading competencies required for the text. The first problem has been pointed out, for instance by Email and Ahmad (2014), who conducted usability research with and without eye-tracking for websites in Malaysia. They (2014, p. 268) write:

Due to the power distance that is already present in the Malaysian culture, the user during the Think Aloud (TA) process sees the moderator as a supervisor and hence has a tendency to be afraid in disagreeing in the effectiveness, efficiency and satisfaction of degree of usability of a website under test. This is one reason why Think Aloud (TA) technique alone may not be suitable and reliable in usability studies in Malaysia.

The German society has become increasingly diverse over the last two decades, and the share of people from countries with a much higher power distance than the one which is common among German-born university graduates has been growing continuously. That share is even higher among younger population groups like the millennials than among the total population.

The magnitude of problem (ii) is largely overlooked in public as well as academic discussions. Usability professionals typically have a university degree and most of their friends and acquaintances have received a similarly advanced education. This often results in selected attention towards the groups to which they belong themselves and lack of understanding for other groups. According to the so-called “Level One” study (Grotlüschen and Riekmann (2012)), there are about 20 million people in Germany whose reading competencies are too low to comprehend relatively simple texts, which typically can be found in written information published by businesses and government authorities. In other words, one in four people in Germany cannot understand the texts on a normal webpage! The complexities of the German language are a usability barrier of the first order. And that is true qualitatively and quantitatively. Yet highly educated UX professionals should not expect to receive an honest answer from test users when they ask the question: “Now, did you understand the text well?” If they did, they will say “Yes.”, but if they did not because they lack the reading competency, they will rarely if ever admit this since such inabilities are highly stigmatized in a society that is built on education. Both problems (i) and (ii) result in systematic underreporting of usability problems, but eye-tracking will reveal them truthfully for obvious reasons.

However, the potential applications of eye-tracking in the tourism industry far exceed its use for usability studies. There are at least five other applications that immediately come to mind here. One of the recent technological advances is the use of augmented reality through smart glasses in museums. And eye-tracking is used in smart glasses for the control of the augmented reality presented. Another application is visual merchandising and the design of hotel rooms and hotel entry halls. As is well known, the first impression of the interior architecture has an important and lasting impact on the visual appeal and, therefore, on the brand image of luxury hotels. Eye-tracking also records the sequence of fixations and thereby tracks peoples’ very first looks. Tourism services are experiential services that address the emotions, and the emotional value of these services translates into financial value. Emotional reactions can be directly observed through the changes of the pupil with eye-trackers. A further application of eye-tracking in the tourism industry can solve a common problem that travellers encounter in airports and railway stations, which is spatial orientation. Eye-tracking with mobile eye-trackers has been successfully employed for improving the way-finding usability of the interior design of these buildings. And finally, eye-tracking has been used for improving the design and usability of geographic maps. Clearly, eye-tracking applications in the tourism industry are not limited to usability studies. However, our chapter is focused on the use of eye-tracking for improving the usability of online travel agencies, and we, therefore, limit our literature review to publications in this area.

4 Methodological Approach

For our study, we applied qualitative sampling where only a small number of test users are included, yet for each of them, detailed qualitative information was gathered through the Retrospective Think Aloud (RTA) technique (i.e. showing the respondents their gaze replay and discussing this aloud) and eye-tracking, which was then analysed in depth. Such an approach is suitable for usability studies because their aim is not to infer averages of important characteristics of the total population, but to identify as many usability problems as possible, since any such problem can result in a lost conversion.

Our methodological approach combined the Retrospective Think Aloud (RTA) technique for gathering subjective data as well as eye-tracking for quantitative, objective data in order to reveal as many usability problems as possible. Before we go into the details of applying these two methods, we cover the selection of test users, their recruitment and the process of gathering data from them.Footnote 2

4.1 Selecting the Test Users

For selecting the test users, we applied qualitative sampling, where we made sure that the personal characteristics of the people selected coincide with the characteristics of the target group. For achieving this, we used three socio-demographic characteristics:

  • Age

  • Gender

  • The highest educational degree achieved

We included 21 test users (TUs) in our sample with the following distribution of the socio-demographic characteristics:

  • Gender: 11 female and 10 male TUs

  • Age: Born between 1980 and 1989, 11 TUs and 1990–1999, 10 TUs

  • Highest educational degree achieved:

    • 14.3% of the millennials are still in school (3 of 21 TUs)

    • 14.3% of the millennials have 9 years of school (3 of 21 TUs)

    • 33.3% of the millennials have 10 years of school (7 of 21 TUs)

    • 38.1% of the millennials have a high school diploma (8 of 21 TUs)

4.2 Recruiting the Test Users

Test users were recruited through social networks and by directly approaching individuals, who might be suitable for inclusion. Each of them filled out a questionnaire, which determined the suitability of the person for becoming a test user.

4.3 Data Collection

The test persons were invited to the Digital Lab of the Tourism Department of the University of Applied Sciences Munich. In order to provide a comfortable and relaxed seating position, height-adjustable seats were used. Before beginning the eye-tracking test, the eye-tracker had to be calibrated by applying the so-called five-point calibration. Following this, the gaze points were analysed, and in case the calibration turned out to be imprecise, the procedure was repeated. The screen then showed a description of the task. Furthermore, the test users received printouts describing the tasks.

The use cases were always conducted in the same sequence. After completion, the Retrospective Think Aloud (RTA) technique was applied and each test user then watched the recorded video of their own session including their own gaze replay. Using some written instructions test users were asked questions—including two open questions—about suspected usability problems. The video recordings during the task completion and the discussion of the RTA were recorded.

5 The Use Cases Studied

Specific user tasks are important for eye-tracking studies due to the fact that gaze behaviour depends essentially on users’ intentions using the website. These use cases are tested and analysed via eye-tracking in the digital lab of the faculty of tourism. In addition, and to support the results of the eye-tracking study, more qualitative aspects of the user experience have been obtained through interviews. The findings are summarized in suggestions on how to improve usability, conversion rate and time on site of the website onlineweg.de.

The selection of user tasks applied two criteria: relevance for the target group of millennials and profitability for the OTA per customer. Thereby, two user tasks were set: (1) a city trip with the railroad including a musical ticket and (2) a travel package below € 300. These tasks were checked in a pre-test. Because the original tests took too much time, use case (1) was simplified to a more straight-forward hotel booking process.

5.1 Characterization of Task (1)

The test users were tasked with booking a city trip to Hamburg. The choice of a particular arrival and departure airport were excluded (because flight bookings are irrelevant for this OTA). The city trip was to take place in July because this month is ideal for such trips. Furthermore, the test user is supposed to make a reservation for two travellers in a hotel with at least four stars and should select the first hotels that appear to keep the case simple. The task ends as soon as the request for personal data appears. For the first task, test users needed about 15 min:

  • Theme: city trip

  • Destination: Hamburg

  • Time: July

  • Number of overnight stays: 1

  • Quality of the hotel: at least 4 stars

  • Number of travellers: 2

Usability Challenge

When booking a hotel, the selection of the search frame (search mask) is decisive. On the start page of onlineweg.de appears a search mask, which should be used for booking travel packages. At this point, it is important that the user finds the search mask for hotels. Otherwise, he will be offered flights only, which does not correspond to the task. The second source of problems arises when choosing a hotel to select the option “dates and prices” instead of “hotel description”. Here, the user finds no option for booking the selected hotel but has to return to the list of hotels.

5.2 Characterization of Task (2)

The task set here was to book the cheapest travel package for two people below € 300. The travel should go to Turkey since there are right now—for obvious reasons—many extremely cheap offers. The time window was set to be September, the number of overnight stays at least five, and the quality of the hotel at least three stars. The departure airport was left open since this added flexibility offers the option to realize a lower price. The meals option included breakfast and dinner. The test user was tasked with selecting the cheapest offer suggested. Again, the use case ended as soon as the input of personal data is requested, and the time requirement to complete the case was also about 15 min:

  • Theme: cheapest travel package

  • Destination: Turkey

  • Time: September

  • Number of overnight stays: at least 5

  • Quality of the hotel: at least 3 stars

  • Number of travellers: 2

Usability Challenge

As for the first task, when starting onlineweg.de the search mask for travel packages opens. The test user then has either the option of using this search mask through selecting the appropriate price range, or can use the search mask of the category “travels up to € 300”. One hurdle in this task is the selection of the period and the duration of the travel. Both items have to be entered separately and during our initial analysis, the travel duration was often overlooked and not changed. This can make it difficult for the test user to find a travel with five overnight stays. At the end of the booking process, the total price is shown for both travellers, while up to then only the price per person is displayed, which can lead to confusion.

6 Results of the Eye-Tracking Analysis

In order to conduct an eye-tracking analysis, so-called “areas of interest” (AOIs) have to be specified.Footnote 3 These areas are set manually and are placed at specific regions within the video recordings. Thereby, the attention different users give these AOIs can be objectively recorded, compared and statistically analysed. On the main page of onlineweg.de, seven such AOIs were defined, which are highlighted by colours in Fig. 2: the logo (green), the upper list with options (orange) with the selection option “hotel” (orange arrow), the lower list with options (pink) with the selection option “hotel” (pink arrow), the search field (yellow), and the button for “more search criteria” (blue). The latter is especially relevant for detecting problem no. 1. Several different eye-tracking metrics were chosen: time to first fixation, fixation time, visit duration, time to first mouse click and mouse click count.

Fig. 2
A snapshot of the webpage online weg dot d e in which the logo, the upper main menu options, the search box, and the selection options are highlighted with arrow marks.

Areas of interest on the main page

Full size image

6.1 Task (1)

Task (1) did consist of 21 test users. According to the recorded eye-tracking data, all noticed the search field (yellow). This AOI had been discovered for the first time after 3.6 s on average and was selected with a mouse click after 9 s. Only 10 test users noticed the upper list with options (orange) after 14 s. The average fixation lasted 0.65 s, but only three test users looked at the button “hotel” (orange arrow), and only they clicked on this button and did not move through the search field (yellow).Footnote 4 Out of 20 test users, 18 noticed the lower list with options (pink) and had at least one fixation in this AOI with the fixation lasting 0.4 s. Only five test users paid attention to the selection option “hotel” (pink arrow) and only one of them clicked on this button.

Consequently, 17 test users (80%) worked their way through the search field, and only 4 were choosing the hotel button. The RTA confirms this problem. Seventy-five percent of the test users went through the search field (yellow) and said they had not noticed the hotel buttons. The rest did not pay any attention to it. One-quarter of the test users which searched via the search field (yellow) then later noticed that by following this path they could not finalize their purchase. The short fixations on the upper and lower list with options (orange and pink, respectively) and the comments during the RTA session suggest that they did not notice the hotel buttons and, therefore, went through the search field. It is, however incorrect to view this as a “user mistake” because users have expectations on how to proceed on such e-commerce websites, and when the designers did not anticipate the users’ expectations and how they are accustomed to navigate such sites, then one should rather speak of a “designer mistake”, especially if the majority of users proceeded this way.

Another problem occurred because the button for booking can be accessed only through dates/prices but not through the hotel description. In summary, four test users terminated the booking task because of problem no. 1 and eight test users arrived at an unintended result. Therefore, 60% of the users would not have booked any travel.

6.2 Task (2)

The total number of test users in task (2) is 20 because the data from the test user no. 17 had to be excluded from the analysis. None of them used the button “Travels below 299”, which would require the use of the upper list with options (orange) including the selection option “hotel” (orange arrow). Eighty percent of the test users began the task via the search mask for travel packages. Nineteen out of 20 noticed the search field after about 3 s. There were 10 and 12 test users with fixations at the upper and lower list with options, respectively, which is less than in use case (1). Probably because of the task of booking a travel package many went immediately to this search field.

Fifteen percent of the test users used the sequence of buttons holiday travel → package travel. One test user completed the wrong task and booked a hotel instead of a holiday package. Problem no. 3 did concern the entry of the travel dates and the travel duration. One-quarter of the test users filled out only one field. Some of the test users did notice the field “travel duration”, but did not click on it. This leaves 75% of the test users who filled out the travel dates and the travel duration, but 46% of them did not completely fill out the required fields (for example, instead of precisely 5 nights, the input read 5–8 nights), or test users were initially confused by the system. For the selection “travel duration” fixations from all test users were recorded. Figures 3 and 4 give screenshots of these fields in the dropdown menu that cause these problems when choosing the travel dates and the travel duration, respectively.

Fig. 3
A snapshot of the web page online weg dot d e, in which travel dates within the dropdown menu are selected.

Areas of interest on the main page—selection travel dates within the dropdown menu for the travel duration and the travel dates

Full size image
Fig. 4
A snapshot of the web page online weg dot d e, in which travel duration within the dropdown menu for the travel duration and the travel dates are selected.

Areas of interest on the main page—selection travel duration within the dropdown menu for the travel duration and the travel dates

Full size image

When we look at the heat map for the use case (2) (Fig. 5), we notice that the fields “travel dates” and “travel duration” (Fig. 6)—which open up as a dropdown when clicking on these field in the search field (yellow) on the first page—belong to those AOIs that received the second largest attention. This is surprising because the data showed that only 75% of the test users clicked on these fields.

Fig. 5
A snapshot of the web page online weg dot d e, in which the heat map for the travel dates within the dropdown menu is selected.

Corresponding heat map for the travel dates within the dropdown menu

Full size image
Fig. 6
A snapshot of the web page online weg dot d e, in which the heat map for the travel duration within the dropdown menu is selected.

Corresponding heat map for the travel duration within the dropdown menu

Full size image

Several intensive centres of fixations within the dropdown menu of the travel duration become visible in the heat map (Fig. 6). The largest such centre occurs at the default option of 7 days in the upper highlighted area. The second-largest centre is the selection option 5–8 nights, which leads to one of the major problems. Much lower attention was given to the option of 5 nights, which was a part of task (2). The apparent complexity of the choice of the travel dates and the travel duration was not only confirmed through erroneous data input and overlooked fields but also through the concurrent RTA. Another potential problem, which was no. 4—lacking comprehension of the total costs of the travel—was not relevant because all test users viewed this as easily comprehensible. All in all, 25% of the test users could not complete task 2 or reached a wrong result in the end.

Since all test users first completed use case (1) and then use case (2), one might suspect some learning effects could reduce usability problems. Eye-tracking data also records time data such as the time to the first mouse click and the visit duration, and one can detect such learning effects from changed time data recordings between the two cases. And indeed, from the eye-tracking time data it was apparent that the test users noticed all lists, buttons or the search fields on the first page faster in task (2) than in task (1), i.e. the time to first fixation was lower.

However, for one of the usability problems (the choice of travel duration) the time data from the eye-tracking recordings, which are shown in Table 1, does not reveal any obvious differences between the two tasks. The time data of the eye-tracking recording clearly reveals from the longer visit duration within the dropdown menu (3.12 s) as compared to the visit duration for the selection of the travel duration (1.85 s) that test users were confused, resulting in the wrong data entry which resulted from these comprehension problems.

Table 1 Areas of interest metrics
Full size table

7 General Assessment of the Quality of Onlineweg.de by the Test Users

Besides the information obtained from the test users through eye-tracking and RTA, their judgement of the quality of this OTA and their willingness to book a travel with onlineweg.de is, of course, also important for the owners of the firm. Besides these standard answers as reported in Tables 2 and 3, test users also gave open text answers assessing the strong and weak points of the website.

Table 2 Judgement of the quality of onlineweg.de
Full size table
Table 3 Judgement of the quality of onlineweg.de
Full size table

The most important objective of the present study is the improvement of conversion rates. For achieving this objective, every negative statement, complaint or critical point needs to be taken very seriously. For different people, different problems might be a condition sine qua non. If, for example, 40% of test users were voicing problems that are all different, then the average share of users for each complaint will be rather small, but disregarding each of the problems would then result in a loss of conversion of 40%, which is anything but small. Consequently, maximizing conversion rates requires to “win’em all”, i.e. to win all customers and to listen to each of them.

Positive (3 test users)

  • Clarity of the website

  • Very good pictures and customer reviews for the hotels

  • No banner ads

Negative

  • Confusing (30% of the TUs).

  • Too many different options/paths lead to the same result (2 TUs).

  • Could not find the price selection option (2 TUs).

  • The search criteria do not adjust automatically (2 TUs).

  • The travel duration did not adjust automatically from the input of the travel dates (2 TUs).

  • The input of the dates is easier on competing OTAs (1 TU).

  • It takes too much time to finish the booking process (1 TU).

  • No choice option for the hotel category was given (1 TU).

  • The hotel button was not clear enough (1 TU).

  • Some of the fields’ functions are unclear (1 TU).

Other eye-tracking recordings indicate that in both tasks test users only looked at the upper third of the start page (Fig. 7). Yet this might be also due to the fact that the test users did not freely surf around but were completing a given task. More importantly, 20% of the test users could not remember the logo or the name of the firm, which is confirmed through the eye-tracking data because in use case (1) and (2) just 6 test users, respectively, looked at the logo with a fixation duration of 0.25 s.

Fig. 7
Two screenshots of the website onlineweg dot com illustrate the start page along with heat maps.

Heat map of the start page

Full size image

8 Recommendations for Improvements

First of all, one should emphasize that web-form design is anything but trivial. There are many details that make it or break it. Yet user-friendly web-form design is the key step for a high conversion rate, or put another way around: A user-unfriendly web-form design costs firms a lot of money. It therefore really pays off, to first learn about the dos and don’ts of web-form user-friendliness, before designing a web-form. Or, if there are already web-forms in place at the firm’s website, one must evaluate the existing ones in a customer-oriented, self-critical way, comparing the realized design with the recommendations given in the literature about the principles of web-form design principles, such as Wroblewski (2008).

Several studies have used eye-tracking for examining the usability of web-forms. For example, Bergstrom and Schall (2014) devote an entire chapter (Chapter 5: Forms and Surveys) to web-forms in their book about eye-tracking for the improvement of user experience design. While there are many guidelines for web-form designs, much of it is just based on personal opinions of usability professionals and anecdotal evidence from usability testing, as pointed out by Bojko and Schumacher (2008). They systematically evaluate the layouts of five different web-forms through a combination of usability and eye-tracking. Thereby, they measure objectively user behaviour and their eye movements and gather users’ subjective self-reports. Seckler et al. (2013, 2014) also used eye-tracking for evaluating the usability improvements of 20 guidelines for the design of interactive online forms. Their studies show that improving the design of web-forms results in faster completion times, fewer form submission trials and fewer eye movements. They also combine their eye-tracking study with user interviews, which show increased satisfaction. Another issue of usability of website design besides web-forms, which is especially relevant for websites with many sections like onlineweg.de, is navigation. In using eye-tracking, Leuthold et al. (2011) evaluated how different navigation systems impact user performance, navigation strategy and subjective preference. The different navigation systems they considered in their eye-tracking study varied with respect to their navigation designs (vertical versus dynamic menus) and the complexity of the tasks they have to serve (simple versus complex navigation tasks). It turned out that vertical menus were more user-friendly since eye-tracking recorded objectively that users needed fewer eye fixations and completed tasks faster and more successfully.

In our own study, we used the RTA technique and the eye-tracking method, which revealed a number of usability problems of the web-forms of onlineweg.de. These problems unnecessarily reduce the conversion rate. Two problem major areas have been identified and for those, we present more in an exemplary than in a comprehensive fashion some recommendations for improvements.

Hotel Booking

The booking process was frequently terminated because by using the wrong search mask, test users received no or wrong results. A button for “hotels” should be highly visible in the upper and lower search mask. Thereby, visitors would be directly guided towards the appropriate search mask and the booking process would become much easier.

Travel Package

The location of the search mask for travel packages on the start page simplifies the booking process for such travels. However, the input of the travel time and the travel duration is not user-friendly because these two fields have to be filled out separately. This is an unnecessary complication since the system can be programmed to calculate the travel duration automatically once the dates have been specified. Often one field is overlooked or disregarded, which causes results that customers do not want.

Impatient customers will terminate the booking process. They do not want to learn the website’s booking system, but correctly expect that the website’s designers and developers should learn instead how customers typically proceed. This is a good example of how incomplete software development reduces the financial results of the firm. There are numerous ways to complete the unfinished software development task and to bring the user-friendliness up to the standards of its competitors. Here, the old word: “Customer is king” truly holds. It is not the customer who has to adjust to the software developer, but vice-è-versa.

9 Conclusion

Now, did we achieve our objectives which we outlined in the introduction? Our first objective is the identification of usability problems for a real-life existing OTA, the company onlineweg.de. The results of the two set tasks unambiguously show that the very beginning of the booking process is already hampered because users are typically unaware of the problem that they are in the wrong search mask. Another problem arises from the difficulty of choosing the duration and time period of the travel, which caused confusion and frustration with most users. This confirms the presumed problem no. 1 (wrong search field) and problem no. 3 (wrong input of duration and time period of the travel). Even users from the group of millennials, who are the most Internet savvy got lost in the booking process because of these problems and would have most likely terminated their booking process. In conclusion, we might say that a simple, immediately transparent booking path is critical for a high conversion rate on OTAs. Even the very beginning of the booking process should minimize users’ efforts and must avoid any frustration or confusion in order to keep the termination rate as low as possible.

Our second objective is to quantify the economic gain quantitatively that conversion improvements will generate for the firm. Our study shows that these gains will be very substantial. In task 1, 60% of the users would not have booked any travel! And in task 2, 25% of the test users could not complete the booking process successfully or reached a wrong result in the end.

The term “conversion rate optimisation” brings the attention of the business community to the simple fact that “search engine optimisation” is certainly not enough to guarantee online business success. We understand “conversion rate optimisation” as “conversion rate improvement” and it is the ongoing process of continuing improvements that matter. Therefore, the issue is not to “optimize” the conversion rate because the optimum cannot be really specified. The task is rather to continuously improve the conversion rate. Customer groups change as new and innovative tourist services enter the offered product portfolio. Navigation experience and expectations of customers also change over time due to changes in competitors’ website design. And technologies change over time like the growing use of mobile devices with their small displays for purchasing online. Using eye-tracking research to continuously improve the conversion rate is therefore adamant for any online shop, and specifically of course for the tourism industry.

With our third objective, we made the steps for administering a usability study with the eye-tracking method transparent, since it is this process that is important for future applications of usability studies with eye-tracking for OTAs, rather than the individual case. All in all, the results show that it is the combination of the Retrospective Think Aloud (RTA) technique and the eye-tracking method which reveals:

  • Where users stumble upon usability problems

  • Where users became aware of such problems

  • What users noticed and what they overlocked and therefore, were unaware and could not voice them in the RTA session

What makes the individual case of a real-life usability study with eye-tracking valuable (like the one presented here) is the fact that it reveals typical problems of imperfect websites and suboptimal business practice—instead of discussing ideal and imaginary business worlds that are so often far away from business realities. Our case study shows that the eye-tracking method is an efficient and effective solution for studying users’ behaviour and perception. From the data gathered (fixation, duration, and their sequence) one can infer the visual perception of users. In this case study, the eye-tracking data recorded clearly revealed how most of the users overlooked in task (1) the button for hotels in the upper as well as in the lower list of commands. This led to erroneous booking results. Moreover, task (2) nobody booked through the option “travels up to € 299”, which the eye-tracking explained by the lack of any recorded fixations there. Additionally, fixations of the dropdown menu for the travel duration was also nearly zero.

Finally, with our fourth objective, we sought to demonstrate convincingly that the research technology and methodology for usability studies using eye-tracking has sufficiently matured so that entry-level researchers with a rather limited time budget can generate productive results. Such studies will be adopted into business practice more often if incumbent employees participate in this work. We report here about the study results of a group of 14 university students, which shows realistically what novice eye-tracking researchers can accomplish at the beginning of their learning paths within a limited amount of time.

Last but not least, during our eye-tracking study, the conversion bottleneck of this particular OTA became apparent: the usability problems of its web-forms. Thereby, our study adds to the academic literature devoted to improving the usability of web-forms that are pivotal for conversion rates by using eye-tracking as a research method.