Abstract
Aiming at development of intelligent service on mobile device, this paper proposes a new travel information query method, which combines image acquisition device, image recognition, and recommendation technologies. The framework of information query consists of four components, including passive information query, active information query, trip scheduling, and information management. A prototype application is designed to demonstrate the feasibility of smart tourism guidance by mobile device. User can browses stationary information through the application program. Furthermore, the user takes pictures and transmits it to cloud server. The cloud server hosts image recognition and delivers the corresponding information to the user. For trip scheduling, the prototype recommends the proper trips to guide user easily, which is referred to user’s preferences with location based service. The resultants will demonstrate that the prototype is implemented by an application program runs on mobile device. It is powerful to search for travel information and to generate trip schedules.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
In the tourism domain, mobile and wireless technologies have been pointed out as one of the most influential technological innovation because of the fast growing of intelligent devices are in a huge user base [1]. In the past, tourists got traveling information only from few stationary channels such as tour guide books, television programs, and discussion among friends, so the information retrieving process could be monotonous and time wasting [2]. After the late 1990s, thanks to the popularization of the computer and the developing of the internet, most people can easily search for and get the traveling information by using internet connected computers. In the kind of Internet Content Provider (ICP) environment, traveling information were usually shared by formal content provider or website owner, causing the insufficient richness of information. After 2000, wireless communication and smart mobile device technology have grown rapidly, and mobile services have turned into mainstream and changed the traditional information environment from ICP into Wireless Content Provider (WCP). The change not only makes people interact with each other and get the information easily, but also revises the traditional tourists behavior and information service model, allowing the tourists to grasp the real-time and personal information anytime, anywhere [3].
In the era that mobile devices are taking over, more and more tourists search for travel information and make travel plans via mobile devices [4]. Mobile technology with the properties of ubiquity, timely, flexibility and localization frees the tourists from being restricted to passive and fixed-point information gathering [5], dramatically changing the information service model to be directly and personalized. There is a variety of mobile travel information services in the market. The services are categorized into before-travel, during-travel, and after-travel services based on when the users use them listed in Table 1. And the most emphasis is how to enhance the information interactive services, to fulfill tourists’ demand at the right place and right time. The smart mobile device is therefore become the most important connecting channel to this purpose [6].
The main task that most users perform via mobile devices is information search, especially for those which are in urgently needed. However, restricted by the screen size and the operation mode, searching for and browsing information via mobile devices is not as easy as via the traditional computers [7]. To resolve the difficulties in operating on mobile devices, the user interfaces, including information inputting, should be simple and easy enough [8]. In the past, researches have been focused on optimal design for inputting texts and searching via mobile devices. Recently, as hardware technology to smart mobile devices improves, non-contact inputting (such as, microphone, camera, or NFC wireless sensor technology) is becoming an alternative to text inputting and the new solution for information retrieving on mobile devices [9]. This paper will investigate the new travel information querying method, which combines mobile phone camera, image recognition and recommendation technology.
Almost all mobile phones come with an integrated camera or image acquisition device. Camera is typically used for taking pictures for posterity purposes; however, there are many other applications for which the images may be applied [10]. Instead of typing the key word to search for and get the information on mobile phone, tourists can now use a powerful image-based mobile search service, which functions by sending an image acquired by a phone camera to a server. The server hosts visual-based recognition and personalized recommendation engines, returns the personalized, appropriate search results back to the user, and helps the tourists know the attraction or finishes a suitable itinerary in convenience. The visual input in the real environment is a new and rich interaction modality between a mobile user and vast information [11], and it brings advantages as follows:
-
Compared to traditional text searching, the visual input saves a lot of time that typing may cost.
-
In most of the traveling conditions, the tourists are not familiar with the searching object. Visual input by camera can solve the main problem that users have no idea to come out an appropriate key word or suffer from the language problem.
-
Unique feature of image makes the searching results more precise than ever.
The tourism industry acknowledges that the launch of value-added mobile services is clearly identified as the main factors affecting the competitiveness of the tourism market [12]. Mobile phones with cameras present new opportunities and challenges for mobile information association and retrieval, and it is clear that visual information query will eventually be integrated with other mobile traveler supported services and change the way the tourism industry works today.
In this paper, we propose the framework of information query system, which is implemented by the prototype application (APP). The system consists of passive information query, active information query, trip scheduling, and information management. In the beginning, user browses information on APP. Furthermore, he/she can take a picture to get the corresponding information. Combined user’s preferences with location based service (LBS), the prototype recommends the proper trips to guide user easily. The rest of this paper is organized as follows: design and prototype implementation are described in Sects. 2 and 3, respectively. The resultants will be shown in Sect. 4, and the concluding remarks will be drawn in Sect. 5.
2 Design
The proposed prototype implements the framework of information query system for tourism. Figure 1 shows the block diagram of the proposed prototype. The details of functions in the prototype are introduced as follows:
-
Passive Information Query. In the conventional applications, theme-based query and context menu are common approaches to obtain information. Considerations of user’s preference and time consumption, seven types of themes have been set in the prototype. Those themes include “culture & heritage”, “gourmet guide”, “offshore islands”, “ecotourism”, “hot springs”, “LOHAS”, and “night markets”. User acquires information by pressing the theme of interest. For instance, when user presses the button of “gourmet guide”, he/she can browse the introductions of restaurants in Taiwan. Similarly, user clicks the item in the context menu to acquire the corresponding information.
-
Active Information Query. In order to provide friendly query, active information query are realized by two approaches, namely LBS-based query and recognition-based query. User clicks the button of “Nearby”, then, landmarks, restaurants, and exhibitions around user are marked on Google Map. He/she clicks the icon on the map to acquire the information of the spot. For recognition-based query, user clicks the button of “Scan” to take picture. Cloud server analyzes the shoot picture and then delivers the related information to mobile device.
-
Trip Scheduling. Pressing the button of “Quick Plan”, user picks the dates, the locations, and the themes of interest, and the prototype creates a new trip schedule against different periods. Shaking the mobile device, a new schedule is created. Furthermore, the other way to create trip schedule is referred to the result of recognition-based query. The recognition result is treated as a seed point of interest (POI) to create trip schedule.
-
Information Management. The tasks of information management have two: (1) systematically retrieval information via defined metadata, (2) flexibly update information. Especially for implementation of active information query, we flexibly add/delete the reference images and the information without altering the system.
3 Prototype Implementation
3.1 Information Management
For collecting and updating POI related information, a POI table is built and maintained. Figure 2 shows a screenshot of a partial POI table. This table integrates information collected from multiple sources and is used for further analysis and recommendation.
The structure of a record in POI table is shown in Table 2. For the x-th POI (denoted as poi x ), the denotations id(poi x ), name(poi x ), add(poi x ), and des(poi x ) represent the identity, the name, the address, and the description of poi x , respectively. The denotation ts(poi x ) is the expected number of hours that tourists spend on poi x , where ts(poi x )∈{1,2,3}. Each POI has its own business hours. In this system, a day is divided into three time intervals, i.e., period 1(8:00 ~ 11:59), period 2(13:00 ~ 16:59), and period 3(18:00 ~ 20:59). Three Boolean variables, namely vp 1(poi x ), vp 2(poi x ), and vp 3(poi x ), are used to indicate appropriate periods for visiting poi x , where vp i (poi x )∈{“true”,“false”}. The value of vp i (poi x ) will set to be “true”, if poi x is appropriate to be visited in period i ; otherwise, this value will set to be “false”. Each POI belongs to a kind of preference themes, such as foods, arts, natural, and shopping. We use prefer(poi x ) to denote the theme assigned to poi x and collect ratings, reviews, and tags for scoring POIs. The score of poi x denoted as score(poi x ) is a weighted sum of its number of browsed, saved, reviewed, and checked in on social media.
3.2 Active Information Query
The prototype actively collects information of landmarks, restaurants, and exhibitions which are located around user. As we above mentioned, LBS-based query and recognition-based are two approaches for implementation of active information query. Figure 3 illustrates the block diagram of recognition-based query, and the query process to mobile device is described as follows:
-
User takes a picture which is called as query image, and then the image is transmitted to cloud server.
-
After implementing image recognition, the application receives the information with respect to query image from the second database in cloud server.
The objectives of cloud server have two: (1) finding a reference image in database as like as the query image, and (2) delivering the information to the application. The query process to cloud server is described as follows:
-
Corner detection is applied to reference images and query image to find key pixels.
-
Extract features of key pixels. Histogram of oriented gradient (HOG) is employed to represent feature descriptor of key pixel [13, 14].
-
The first database stores feature descriptors of all reference images, and the information corresponded to the reference image are stored in the second database.
-
Compute feature similarity between the query image and the reference images. The most similar reference image is found with the maximum feature similarity.
-
The identity of the selected reference image (which is denoted as id(poi x ) in Table 2) is an index to acquire the related information in the second database. Then, the information is delivered to the application.
3.3 Trip Scheduling
Given date, location and user’s preference, the system recommends schedule of trip with respect to opinions on social media. The structure of schedule of trip is listed in Table 3. Each hour is regarded as a time slot. The three periods, period 1(8:00 ~ 11:59), period 2(13:00 ~ 16:59), and period 3(18:00 ~ 20:59), consist of 4, 4, and 3 time slots, respectively. Let slot(i,j) be the j-th time slot in the i-th period, and slotnum(period i ) be the total number of time slots in period i , where i∈{1,2,3} and j∈{1,2,3,4}. The rec(i,j) denotes the recommendation POI at slot(i,j). In our definition, a recommendation POI crosses one or more time slots. For each period, the seed POI is given by either recognition-based query or user’s selection.
For example, the seed POI, “Taipei National University of the Arts”, puts into period 1, and it derives from the result of recognition-based query. Let tc and TH be, respectively, target county and preference themes, where TH = {theme k1, theme k2, …}. In period 2, given tc and TH, the system selects a seed POI (denoted as poi s ) which belongs to tc and TH with the highest social score. The selection of poi s follows the four conditions:
-
(1)
add(poi s ) ⊂ tc,
-
(2)
vp i (poi s ) = true,
-
(3)
∃ theme k ∈ TH such that theme k (poi s ) = true,
-
(4)
∀ poi x , poi x satisfies (1) ~ (3) ∧ poi x ≠ poi s → score(poi x ) ≤ score(poi s ).
Whenever poi s is selected, ts(poi s ) time slots will become occupied. For example, assuming that “Taipei Water Park” is the seed POI selected for period 2 and ts(“Taipei Water Park”) = 2. After selecting “Taipei Water Park” as the seed POI in period 2, the time slots slot (2,1) and slot (2,2) become occupied, and the values of rec (2,1) and rec (2,2) become “Taipei Water Park”.
The system continues selecting subsequent POIs for filling the schedule. As the last occupied time slot in the schedule of period i is slot(i,l), and rec(i,l) = poi a . The next POI, denoted as poi b , is selected according to the following conditions:
-
(5)
add(poi b ) ⊂ tc ∧ distance(poi a , poi b ) ≤ 3 km,
-
(6)
vp i (poi b ) = true,
-
(7)
∃ theme k ∈ TH such that theme k (poi b ) = true,
-
(8)
l + ts(poi b ) ≤ soltnum(period i ),
-
(9)
∀ other unselected poi x , poi x satisfies (5) ~ (8) ∧ poi x ≠ poi b → score(poi x ) ≤ score(poi b ).
The function distance(poi a , poi b ) is defined as the geographical distance between poi a and poi b . The condition (5) makes sure that the distance between two POIs is acceptable. The condition (8) ensures that the total number of time slots spent by all recommendation POIs will not exceed slotnum(period i ). The system continues selecting subsequent POIs until either all time slots in period i are occupied or there is no POI satisfying the conditions of (5) ~ (9).
4 The Resultants
The prototype application employed the Qualcomm Vuforia software development kit (SDK) to implement feature extraction and feature matching [15]. The APPs for Android and iOS can be downloaded from [16, 17], respectively. Three resultants were emphasized on and presented in this paper, including LBS-based query, recognition-based query, and trip scheduling. In what follows, the resultants are introduced in detail.
4.1 LBS-Based Query
Pressing the button of “Nearby” in the main frame, the application immediately presented the related information of landmarks, restaurants, and exhibitions around the user, which is realized based on LBS. Figure 4(a) shows the map marked the icons after pressing “Nearby”. Figure 4(b) and (c) depict the detailed information of two spots corresponded to the red icon and the blue icon in Fig. 4(a), respectively.
4.2 Recognition-Based Query
Pressing the button of “Scan” to shoot a picture, the application transmits the shot picture to cloud sever and then receives the related information after image recognition. Figure 5(a) and (b) show the query image and the corresponding information, respectively. Figure 5(a) is the image shot in the activity handbook. This function was practically verified through the activity of Taipei National University of the Arts in October 2014, and the activity called as “Kuandu Story Guidance”.
4.3 Trip Scheduling
Given date, location, and user’s preference theme, the application planned a schedule of trip. Furthermore, the prototype designed to generate another trip schedule by shaking mobile device. In the case of Fig. 6(a), the user set a two-day trip in two locations, and he/she was interested in the theme of gourmet. Figure 6(b) shows two different schedules of trip under the same initial settings. It is obvious that the right-side schedule has one spot more than the left-side schedule has.
5 Conclusions and Future Work
This paper introduces the framework of information query system, and it is realized by a prototype application. We propose a new travel information query scheme, which combines image acquisition device, image recognition, and recommendation technology. The resultants demonstrate that four functions run efficiently to provide user travel information, and those functions include passive information query, LBS-based query, recognition-based query, and trip scheduling. The future work will focus on how to actively provide user more appropriate information by analyzing user’s query behavior.
References
Buhalis, D., Law, R.: Progress in information technology and tourism management: 20 years on and 10 years after the internet—the state of eTourism research. Tour. Manag. 29(4), 609–623 (2008)
Liao, J.-M.: A usability study on a real-time travel information system in 3G mobile phone applications. Master’s thesis, National Cheng Kung University, Taiwan (2005)
Lu, M.-P.: Market research of taiwan internet social networking system mobilization development. Master’s thesis, National Chiao Tung University, Taiwan (2009)
Kray, C., Baus, J.: A survey of mobile guides. In: Workshop on HCI in Mobile Guides, 5th International Symposium on HCI with Mobile Devices and Services, pp. 1–5 (2003)
Siau, K., Lim, E.P., Shen, Z.: Mobile commerce: promises, challenges and research agenda. J. Database Manage. 12(3), 4–13 (2001)
Carlsson, C., Walden, P., Yang, F.: Travel MoCo: a mobile community service for tourists. In: The 7th International Conference on Mobile Business, pp. 49–58 (2008)
Langelund, S.: Mobile travel. Tourism Hospitality Res. 7(3/4), 284–286 (2007)
Ali, A., Ouda, A., Capretz, L.F.: A conceptual framework for measuring the quality aspects of mobile learning. Bull. IEEE Tech. Committee Learn. Technol. 14(4), 31–34 (2012)
Russell-Rose, T., Tate, T.: Chapter 8: Mobile search. In: Designing the Search Experience, pp. 219–251 (2013)
Neven, H., Neven Sr., H.: U.S. Patent No. 7,962,128, Washington, DC (2011)
Lim, J.H., Li, Y., You, Y., Chevallet, J.P.: Scene recognition with camera phones for tourist information access. In: IEEE Conference on Multimedia and Expo, pp. 100–103 (2007)
European Commission. Final report of working group on mobile services for tourism (2003)
Szántó, B.M., Pozsegovics, P., Vámossy, Z., Sergyán, S.: Sketch4match - content-based image retrieval system using sketches. In: IEEE 9th International Symposium on Applied Machine Intelligence and Informatics, pp. 183–188 (2011)
Hu, R., Collomosse, J.: A performance evaluation of gradient field HOG descriptor for sketch based image retrieval. J. Comput. Vis. Image Underst. 117(7), 790–806 (2013)
Qualcomm Vuforia software development kit. https://developer.vuforia.com/
Android-based Smart Tourism Application. https://play.google.com/store/apps/details?id=tw.org.iii.ari.smarttourism
iOS-based Smart Tourism Application. https://appsto.re/tw/nE_tM.i
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Zeng, YC., Chan, YH., Lin, TY., Shih, MJ., Hsieh, PY., Chao, GL. (2015). Scene Feature Recognition-Enabled Framework for Mobile Service Information Query System. In: Yamamoto, S. (eds) Human Interface and the Management of Information. Information and Knowledge in Context. HIMI 2015. Lecture Notes in Computer Science(), vol 9173. Springer, Cham. https://doi.org/10.1007/978-3-319-20618-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-20618-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20617-2
Online ISBN: 978-3-319-20618-9
eBook Packages: Computer ScienceComputer Science (R0)