Abstract
In this poster, we propose an evaluation framework that investigates the integration of the user context (interests, location and time) into the evaluation process of mobile IR. Our approach is based on a diary study where users are asked to log their queries annotated by their location and time. Users’ interests are explicitly acquired or implicitly learned based on users’ relevance judgments for the retrieved documents answering their queries. We propose two evaluation protocols namely training/test in chronological order and k-fold cross validation. We exploit this framework in order to evaluate the performance of our context-based personalized mobile search approach. Experimental results show the stability performance of our approach according to the proposed evaluation protocols and demonstrate the viability of the diary approach as a means to capture context in evaluation.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Brown, P.J., Jones, G.J.F.: Context-aware retrieval: Exploring a new environment for information retrieval and information filtering. Personal Ubiquitous Computing 5(4), 253–263 (2001)
Tamine-Lechani, L., Boughanem, M., Daoud, M.: Evaluation of contextual information retrieval effectiveness: Overview of issues and research. KIS 24(1), 1–34 (2009)
Bouidghaghen, O., Tamine, L., Daoud, M., Laffaire, C.: Contextual evaluation of mobile search. In: Doan, B.-L., Jose, J., Melucci, M., Tamine, L. (eds.) Workshop on Contextual Information Access, Seeking and Retrieval Evaluation, Milton Keynes, CEUR Workshop Proceedings, vol. 569 (2010)
Mizzaro, S., Nazzi, E., Vassena, L.: Retrieval of context-aware applications on mobile devices: How to evaluate? In: Proceedings of IIix, pp. 65–71. ACM, NY (2008)
Göker, A., Myrhaug, H.: Evaluation of a mobile information system in context. Information Processing and Management 44(1), 39–65 (2008)
Borlund, P., Ingwersen, P.: Measures of relative relevance and ranked half-life. In: 21st Annual International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 324–331. ACM, NY (1998)
Mountain, D., MacFarlane, A.: Geographic information retrieval in a mobile environment: evaluating the needs of mobile individual. JIS 33(5), 515–530 (2007)
Rieman, J.: The diary study: a workplace-oriented research tool to guide laboratory efforts. In: INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems, pp. 321–326. ACM, NY (1993)
Sohn, T., Li, K.A., Griswold, W.G., Hollan, J.D.: A diary study of mobile information needs. In: CHI, pp. 433–442. ACM, New York (2008)
Voorhees, E.M.: The Philosophy of Information Retrieval Evaluation. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 355–370. Springer, Heidelberg (2002)
Bouidghaghen, O., Tamine-Lechani, L., Boughanem, M.: Dynamically personalizing search results for mobile users. In: Andreasen, T., Bulskov, H. (eds.) FQAS 2009. LNCS, vol. 5822, pp. 99–110. Springer, Heidelberg (2009)
Daoud, M., Tamine, L., Boughanem, M.: Towards a graph based user profile modeling for a session-based personalized search. KIS 21(3), 365–398 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bouidghaghen, O., Tamine, L., Boughanem, M. (2010). A Diary Study-Based Evaluation Framework for Mobile Information Retrieval. In: Cheng, PJ., Kan, MY., Lam, W., Nakov, P. (eds) Information Retrieval Technology. AIRS 2010. Lecture Notes in Computer Science, vol 6458. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17187-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-17187-1_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17186-4
Online ISBN: 978-3-642-17187-1
eBook Packages: Computer ScienceComputer Science (R0)