1 Introduction

Past few years, there is great advancement of camera phone and digital camera to share media on web services and as well as social media such as Flickr, Facebook, and YouTube etc. Users share their travel experience like photo video on these social media web services. Photo sharing web service contains billion images accessible everywhere taken on earth. Increases volume of these images is define different form including geo-tagged information, photographs, time and other variety of textual information. The increase the volume geo-reference and social media resource such as photo video documents together with geo-tagged facilities. For example more than 40 million geo referenced photo on Flickr and over the 1 million geo tagged article on Wikipedia. Textual Meta data and temporal references, these enhanced the multimedia provided wealth data to solve the vision and media task. To discover the graphic related information and knowledge about human societies and open the new opportunities provide by multimedia. In computer vision research works most of people at single location make rich signified from image and also recognize the image where image was taken as well as image contents [1, 2].

The users are capable to share media anytime anywhere to cooperative communicate other users. The mobile base and web service enhanced the social media and increase abilities human exploration. The email, messages other forms and various application supported by social network and it allows human being to create communication their business, work, group of people or all around the world. Photo share web site provides the great photography information for travelers like longitude and latitude information and allow picture display online map, using GPS service many geo tagged photos and location aware device. The photo taken time and visit place for tourist and textual tags can considered very useful geographical information on web site. Most of research work focus on point of interest, geo-tagged photo, recommendation system and personalized travel recommendation system. The estimate the viewing direction of geo tagged photo to identify the landmarks and geo information of photos also detect the errors viewing direction up to 30 meter errors [3, 4].They recognized the objects such as video, robot vision, mobile device and electronic games, matching two images, texture, motion, match land marks and so on [5].To improve the landmarks, location based search and image quality also provide the better digital image then early designed system. The reduce number of images visual manner in to small using metadata and tags and number of coherent subset solve processing problem cheap [6].The multimedia like video, photo as well as contain textual information like notes, title description and tags but also tagged with temporal context such as what time was photo taken and spatial context like where was photo taken. In recently year’s tourist attraction on interesting popular locations or landmarks are that place often photography, photo sharing community and match tourist landmarks interest it has been a hot research topic. Due the GPS technology especially auto-navigation system to improve location recommendation and activity recommendation and research not far when they precise geographical information for image like longitude and latitude because of geo coordinates or geo information new up data mining opportunity for better and accurate reorganization image retrieval personal collection. User can accomplish their travel experience through GPS traces and share travel information to other users. They considered predicting user’s interest and location information because they see user past location history and calculate user location have been visited and their travel experience to recommend new city according to user interest. Estimating geographical information from the image it very challenges and high level computer vision problem and they estimate image location on surface of earth. The various methods proposed to find the famous or new place or representative travel sequence to address travel related queries and they achieved better performance to detect the travel sequences. Day by day there are many new opportunities and challenges multimedia, information retrieval and data mining research areas [7–12].

User can choice close list trip planning like time, family, his/her local culture etc. The user stays time visit one location; it could be estimate according to location user visited. Users always prefer that system which can recommend tourist locations to match the tourists’ interest because time is key component of trip planning and trip planning is a time consuming task [13, 14]. The quality of recommendation system is measured by relevance of recommendation to the request and this relevance based on distinct properties of recommendation that includes content, authority and context and the query that includes intents, needs and context. Travel recommend on different city to user is best tourist assisting service should consider these aspects of relevance while constructing and recommending tourist locations from social media. The recommender systems that use ideas of a community of users to help individuals in that community more effectively by identifying contents of interest from a potentially overwhelming set of choices [15–17]. In this paper we propose a system that considers following aspects in making tourist locations recommendations:

The goal of this paper is to study community contributed collections of geo-tagged photos that come with temporal and spatial context, in combination with historical weather data to derive their weather context, for recommending context driven personalized semantic tourist locations. The key contributions of this paper are:

  • We present architecture of a system that is capable to address dynamic queries that could include any or all of the contexts in a query that are: temporal, spatial and weather, for semantically meaningful recommendation system significant tourist locations recommendations in geo-tagged

  • From user supplied photos collection, we show how to group photos using their associated geo-tags to sense semantically meaningful tourist locations where the photos were taken.

  • Given user’s travel preferences acquired from his/her traveling history, we predict his/her tourist preferences for tourist locations in new city. We provide an extensive evaluation of proposed method on Flickr dataset.

This paper is organized as follows: we begin by discussing the related work in Sect. 2; a formal definition of our problem and framework for solution is given in Sects. 3 and 4; we describe the system architectures. Evaluation is presented in Sects. 5 and 6 concludes the paper.

2 Background and Related Work

In this section, we give the classification background and related works in to three parts such as Geo tags visualization, browsing and exploitation for landmarks detection, Travel recommendation for tourist and Personalized recommendation for tourist.

2.1 Geo Tags Visualization, Browsing and Exploitation for Landmarks Detection

There are many different methods have been purposed to explore landmarks using the user contributed in social media. To discover specific geo-tags from Flickr used tag associate with geo-tagged photos on world map or any other region at any zoom level [1]. Landmarks also detect the location of an image, to detect landmarks method direct match feature points or unknown image to image known geo-location [2]. Another work based on clustering point of interest detect to location, segmentation and geographical spatial knowledge [3].They estimate the view direction of images to browse landmarks and different view also they detect the location of landmarks further estimated direct view where were photo taken [4]. They investigate landmarks having three kind of information such as landmarks position but also angle, and feature vector. Compare angle and feature, a landmarks image correspond few number of landmarks another image [5].They improve the user interaction, geographical query from visual summary used landmarks represent on social media [6].They provide geo location information about the pictures and their geographical coordinates with the help of map interface [7]. They used HITS (Hypertext Induced Topic Search)-based inference model to user experience and concentration travel location geo spatial region also search the query depend rank algorithm for web retrieval information [8].They purpose the matrix factorization method to give location suggestion according the activity of current state of user and GPS traces for the extract location data [9]. To predict the geo locations of photos various methods have been purposed and divided map each location cell represented by grid. They purposed model language show the estimate efficiency analysis and image taken by specific location [10]. In selected metropolitan area k-means algorithm used to find the spatial clustering geo tagged photo. The value K-mean predefined it’s very difficult. To find the arbitrary shape of clustering K-mean did not suitable [11]. The bases on photo tags and its features to predict the photos location by nearest neighbor method [12].

2.2 Travel Recommendation for Tourist

They focus on temporal information with query and photos in term of trip duration, the weather information and current temporal they did not considered. Solving automatic router planning problem to compute the visit time of site and discover trip [13].They develops the recommendation system for a tourist that can provide the best travel route to user as well as popular landmarks. Travel best routes to destination in the city using geo tagged photos. The length of road and popularity assessment both are considered to routes recommendation. Maximal tourist’s popularity and minimizes distance its best router recommendation for tourist. User have good travel plan and recommend suitable routes in the city,find out the best road set the routing for tourist trip [14].They purposed method for tourist preference location to predict from Flickr photos, using probabilistic Bayesian method used to individual user favorite location and calculate the similarities geo-tagged photos for different users [15].Travel recommendation for tourist such as travel time, reach ability, distance and sequential between locations, like trip duration, planning, trip cost and number of factor take from account perform travelling recommendation for tourist The unfamiliar city information tourist know due to short period of time journey minimal effective while save a lot time. HITS (Hypertext Induced Topic Search)-based inference model between user and location and travel sequence recommendation [8].Two kind of user to discover interest trajectory pattern one those who have interest in most necessary trajectory patterns like new city have many famous location most tourists have interest and other those who interest discover the new location in diverse way, not much interest router areas [16].

2.3 Personalized Recommendation for Tourist

The various approaches and algorithm have been purposed to make the personalized recommendation system for tourist from geo tagged photo. They purposed new method for personalization travel to get tourist his/her location history and their preference in one city recommend tourists location to another city. To more precise predict tourist’s location preferences in unknown or new city by personalized method. The planning for tourist’s trip various and unknown locations used personalized recommendation tourist locations and consider spatial, weather content for tourist [17].The user travel preferences location matching provide individual user by personalized recommendation. To predict tourist preference unknown location based on visited location histories provide by collaborative filtering model. User create the travel experience location visited depend on multiple GPS traces and user also view location through GPS by personal travel recommendation [8]. Personalized recommendation system for tourist provides the user histories information and also travel recommends preferences. The researchers have advantage similar trajectory because they calculate similar routing of individual user which obtain from track user histories of the other users for personalized recommendation system [14].They purpose recommendations user preferences travel location to predict famous location according to user choice base on his previous travel history geo tagged photos and they recommend personalized travel location specific preferences of user. The similarities between user and any other user mix popularity and personalized score together based on their location popularity. Tourist try find the similar another user who having same travelling histories to attractive another location [15]. They focus on personalized recommendation system geo tagged photos quality in photo share to online website. Popularity of user photo estimation and detect tourist attracts to geo tag. User personal preference shows the visual and textual information in photo. User travel behavior to discover the spatial fluctuation to attract popular and distance by user photos time taken [18]. Personal recommendation system depend on user personal interest by categories like park, restaurant and famous place etc. recently research user location histories and social environment user to make recommendation and user preferences using preference to selected user algorithm top-k rank location are reappearance recommend as the user [19]. They purpose to personalized recommendation system for tourist to attract unfamiliar cities and satisfy the users according to their preferences. Tourist plan number of categories such as choice attracts location, accommodation and destination at most present in travel recommendation. They focus on Information for tourist location and travel destination, Bayesian network select estimation the tourist prefers activities. Personalized recommendation two aspects one finds the location tourist that attract the tourists and find the route direction two attractive first, to calculate the distance between original points to destination and second provide the best direct to reach the destination [20, 21].

Number of methods, techniques and applications are purposed above research interests for tourist guides most based on generalized also emphasizes have been obtained personalized recommendation service for tourist.

3 Problem Definition

Before we formally define the problem, we give definitions of some basic concepts and terms.

Definition 1

Geo tagged photo: A geo-tagged photo p can be defined as p\(\,=\,\){idp, tp, gpxp, up} containing a unique photo ID, idp; its geo-tags, gp; photo’s temporal context, tp; and the ID of the user that contributed the photo, up. Each photo p can be annotated with a set of tags xp. Geo-tags gp of each photo p, represented by latitude and longitude, are the coordinates of the geographical region where it was taken.

Definition 2

Photo collection: Collection of all photos, contributed by all tourists can be represented as P\(\,=\,\){PU1, PU2, PU3, ..., PUn} where PUi(i\(\,=\,\)1...n) is the collection of photos contributed by user i.

Definition 3

Location: A location \(\mathcal{L}\) can be viewed as geographical region within a city like park, lake or museum, which is popular for tourists to visit and take photos. A clustering algorithm is required to find tourist locations using geo-tags associated with photos.

Definition 4

Travel time sequence: A travel time can be taken as a trip made by a tourist to visit locations according to a temporal order.

The problem of location recommendation for trip planning with geo-tagged social media is formulated as; given a collection of geo-tagged photos P\(\,=\,\){PU1, PU2, PU3, ..., PUn}, how to locate tourist locations and build traveling history of each user to derive his/her travel preferences to generate and address query Q. We aim to address problem of recommendation by trying to exploit travel history of user to recommend that best fit his/her interest.

4 Our Approach

We describe the overall architecture behind our approach that supports two kinds of tasks. First, tasks for performing time consuming computations that includes finding tourist locations from geo-tag photos, locations’ semantic enrichment, locations’ and users’ profiling, building database of tourist locations and modeling users’ preferences and similarities among users based on preferences. Second are tasks that recommendation and generation location and query processing. Figure 1 depicts the overall architecture of our system.

Fig. 1
figure 1

Architecture of tourists location recommendation system

Full size image

4.1 Discovering Location

Density based clustering algorithms like DBSCAN [22] have several advantages over other types of clustering algorithms: they require minimum domain knowledge to determine the input parameters and can discover clusters with arbitrary shape. In addition, they can filter outliers and work effectively when applied to large databases. DBSCAN requires only two parameters: \(\varepsilon \)(epsilon) and the minimum number of points required to form a cluster (minPts).

4.2 Semantic Enrichment

To describe the locations extracted by clustering photos geographically, we enrich the locations with semantic in terms of name and category by a method described. It contains three steps: (1) each photo in location cluster has an associated set of tags contributed by user. In first step, we use method described in [2] to derive representative tag for each location. Considering each location cluster LC\(\,=\,\)(Pc, gc) and set of tags Xc that are associated with the photos of cluster Pc, they used TF-IDF method to score each tag x\(\,\in \,\)Xc. At the end of this step, for cluster Pc we have a list of tags Xc and each tag x\(\,\in \,\)X’chas a score Score(Pc, x). The higher the score, the more distinctive the tag is within a cluster. (2) In the second step, we use Web services available online, like Google Places to extract the information about the POIs in a certain geographical area.

4.3 Location and Users’ Profiling

Once the photos have been clustered using their associated geo-tags to find the tourist locations where they were taken, and the locations are labeled with semantic, we are interested in building the profile of locations and users in order to create user-location matrix and user-user similarity matrix that we will use for personalized tourist recommendations

4.4 Model User Preference and User Similarities

In a social media website, there is a huge amount of users who contribute their data, and through these users generated data, we can mine social context information about the user. In this section, we analyze photos’ geo location information to model the similarity between two users, which will be later used in personal recommendation. We use kernel density estimation to model a use’s travel preference. The kernel density estimation algorithm is nonparametric way of estimating an unknown probability distribution based on a set of data samples which are independently drawn from the unknown distribution. We assume a user’s travel preference is a 2 dimensional distribution over the geographical locations.

A user u’s travel preference can be estimated by:

$$\begin{aligned} P_u (\vec {g})=\frac{1}{hL}\sum _{i=1}^L {K\left( \frac{\vec {g}-\vec {g}_i }{h}\right) } \end{aligned}$$

Then the similarity between the user u and the user can be calculated by symmetric Kullback-Leibler

Divergence(KL-divergence):

$$\begin{aligned} UserSim(u,w)=\frac{1}{2}(D_{KL} (P_u \parallel P_w )+D_{KL} (P_w \parallel P_u )) \end{aligned}$$

4.5 Customize Trip Plan

In this section we suggest the travel router and travel plan for tourist. Travel short trip, long trip according the tourist have a time. Our system also able to suggest the identify tourist plan according their time and preferences.

4.6 User Preference

We get user travel history what they visit city and recommends another city according to user preference it not depend on famous location but also depend on food and user own interest.

4.7 How Much Time User has to Visit the Location

First our system asks how much time user has to visit the location in that city and inside the city place. We make the categories such as short term user, long term user, one day user, 1 h user etc., show details in Fig. 2.

Fig. 2
figure 2

Number of point of user interest and time categories

Full size image

In above we already explain that our system asks how much user have then according to time we can made the categories our system give the response to your first what are available cities for them to visit and how much it take time to reach the location

5 Experimental Evaluation and Results

5.1 Data Acquisition

We get dataset from using public API from Flickr, it consists two kinds of data first Geo-tagged photo collection and other Historical weather data. Geo-tagged photos. We used name of cities in different languages as query text for searching public geo-tagged photos. We download crawled dataset comprises of 1,376,886 photographs with their spatial and temporal context. All these were taken in the eight different cities of China between January 01, 2000 and November 17, 2013. We downloaded weather historical data using underground weather API that is also publically available.

5.2 Data Preprocessing

To clean the photos’ data, we removed two types of photos from data set (1) photos that were collected in the result of search based on text containing name of a city in their metadata (i.e. tags, title, user description etc) but their spatial context (latitude, longitude) did not match the geographical context of that city. (2) Photos with incorrect temporal context show in Table 1.

Gives the information about locations found by applying density based clustering algorithm to geo-tags associated with photos. It also summarizes the information regarding the popularity of locations based on unique number of visits and visitors. To detect visits from photo taken activities we use value of visit duration threshold dvisitthr = 6 h in Table 2.

Table 1 Data set summary
Full size table
Table 2 Give the overview of location distribution and popularity in different temporal and weather context
Full size table

Baseline methods

We compared the following baseline methods to show the effectiveness of context ranking (CR), which is our proposed one.

Popularity Rank (PR).

Users created contents like photos or videos can be measured in popularity while some users want to see popular photos that many other users have seen, others may want to see photos that are regarded to have high expertise. The number of unique visits made to those locations, we can determine rank the location on the general popularity score [17, 23]. The probability of locations L given a visits v and context c can be written as:

$$\begin{aligned} \mathcal {P}\left( {\frac{L}{v},c} \right) \propto /\mathcal {P}(L) \end{aligned}$$

The approach results in static ranking, equal for all users. The user ranks can be passed into probability density functions to produce biased user scores. We analyze the popularity of locations and visits, it high-ranked expertise or popularity unique visits and it gets higher in low-ranked.

Collaborative Filtering Rank (CFR).

The second type of feature, similar to the traditional collaborative filtering [24], considers the user for recommending Location. The equation becomes:

$$\begin{aligned} \mathcal {P}\left( {\frac{L}{v},c} \right) \propto /\mathcal {P}\left( \frac{L}{v},\right) \end{aligned}$$

We compare with our approach is the state of the art user based collaborative filtering method that exploits evaluations or ratings (derived from the visits) of other tourists with similar interests, and potentially provide a ground for the cooperative production of tourist travel recommendations [17, 25].

Classic Rank (CLR).

The Hypertext Induced Topic Search (HITS) based Inference method to calculate the location’s interest and user’s travel experiences in terms of authority score and hub score by exploiting the reinforcement relationship between users and locations [22].

Figure 3 depicts the performance of prediction recommendation of our proposed context rank (CR) and other baseline methods in terms of precision (P). Popularity based ranking and classic rank gives better prediction results as compared to collaborative filtering method, the reasons that can be are; many users do not have single preferences but visit locations of many types and those locations which are popular and significant when they come to visit new city.

Fig. 3
figure 3

Performance comparison of proposed method and baseline with different number of test locations per user in terms of precision (P)

Full size image

In Fig. 4 depicts the computing complexity of different methods in calculating a prediction. Clearly, our Method (context rank) is much more efficient than the different baseline methods and also personalized context method. In short, our proposed method is as effective as the model different baseline methods and personalized context method.

Fig. 4
figure 4

Performing time consuming computations prediction

Full size image

Recommend popular places. we assuming that the numbers of most popular(and therefore irrelevant) landmarks may be different from case to case and by letting this number vary between 1 and 10, we can observe the performance of the proposed context rank and its relative improvement over the baseline and personalized context recommendation approaches, as shown in Tables 3 and 4. Note that we use 1 (in the first column) to denote the case where no assumption of the relevance of most popular landmarks is made, and we measure the performance according to the ground truth in the test set.

Table 3 MAP comparison between context rank and baseline approaches and personalized context (first column) most popular places in different cities in China
Full size table
Table 4 MRR comparison between context rank and baseline approaches and personalized context (first column) most popular places in different cities in China
Full size table

Benefit ratio. Benefit ratio (BR) is the ratio of number of users who get an improved prediction to number of users who get a deteriorated prediction in terms of precision over the baseline. Precision, as discussed before, provides an insight into the recommendation capability of ranking methods in terms of prediction at each visit level. To check the effectiveness of recommendation methods in terms of prediction at user level, we compute BR over all baselines using equation.

$$\begin{aligned} \hbox {BR }=\frac{\textit{number}\, \textit{of} \,\textit{user} \,\textit{with} \,\textit{improved} \,\textit{precision} \,\textit{in} \,\textit{prediction}}{\textit{number} \,\textit{of} \,\textit{user} \,\textit{with} \,\textit{deteriorated} \,\textit{precision} \,\textit{in}\, \textit{prediction}} \end{aligned}$$

BR results plotted in Fig. 5 show that exploiting context for personalized recommendation can give improved recommendations for most users.

Fig. 5
figure 5

Benefit ration of proposed context rank method over baseline and personalized context aware methods

Full size image

Mean average precision. Mean average precision (MAP@n) is a widely used evaluation metric to measure the ranking effectiveness that is mean over the precision values after each correct recommendation in the top-n. To calculate MAP@100, we recommend 100 locations considering each visit made by each test user in test city as a query and the location visited as one relevant item. We get average precision (AP) for each query AP = 1/r, where r is the position of relevant item in ranked list. We obtain the MAP using equation.

$$\begin{aligned} \hbox {MAP}=\frac{\sum _{i=1}^{Nq} {\hbox {AP}_i } }{N_q } \end{aligned}$$

where Nq is the total number of queries and APi is AP for query i. Figure 6 gives the performance of ranking ability of different ranking methods.

Fig. 6
figure 6

Ranking ability of different method in term of MAP@100 across different number of test per user

Full size image

6 Conclusion and Future Work

In this paper we gave an approach to extract semantically recommendation for tourist locations from geo tagged social media like photos for tourist travel recommendations. We contributed a method that applies collaborative filtering and context rank in scalable way by eliciting tourist preferences with exploiting a user’s publically contributed photos and takes into account current context of user for recommendation system for tourists. We presented the evaluation of our methods on a sample of publically available

Flickr data set containing photos taken in several cities of China and results show that our recommendation method is able to predict tourist’s preferences in new city more precisely and generate better recommendations as compared to other state-of-the art landmark recommendation methods. From results we can conclude that people preferences with short and targeted visits is easier to predict by methods based on popularity and performance of collaborative filtering methods based on tourist preferences gets better in the case of long and real tourist visits. In the future, we plan to investigate the recommendation system in combination with context awareness in trips for tourist recommendations.