Abstract
The paper discuss fundamentals of semantic evaluation of information retrieval systems. Semantic evaluation is understood in two ways. Semantic evaluation sensu stricto consists of automatic global methods of information retrieval evaluation which are based on knowledge representation systems. Semantic evaluation sensu largo includes also evaluation of retrieved results presented using new methods and comparing them to previously used which evaluated unordered set of documents or lists of ranked documents. Semantic information retrieval methods can be treated as storing meaning of words which are basic building blocks of retrieved texts. In the paper, ontologies are taken as systems which represent knowledge and meaning. Ontologies serve as a basis for semantic modeling of information needs, which are modeled as families of concepts. Semantic modeling depends also on algorithmic methods of assigning concepts to documents. Some algebraic and partially ordered set methods in semantic modeling are proposed leading to different types of semantic modeling. Then semantic value of a document is discussed, it is relativized to a family of concepts and essentially depends on the used ontology. The paper focuses on sematic relevance of documents, both binary and graded, together with semantic ranking of documents. Various types of semantic value and semantic relevance are proposed and also some semantic versions of information retrieval evaluation measures are given.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Berry, M.W., Browne, M.: Understanding Search Engines. Society for Industrial and Applied Mathematics (2005)
Breitman, K.K., Casanoca, M.A., Truszkowski, W.: Semantic Web: Concepts, Technologies and Applications. Springer, Heidelberg (2007)
Buitelaar, P., Ciminao, P. (eds.): Ontology Learning and Population: Bridging the gap between Text and Knowledge. IOS Press (2008)
Butcher, S., Clarke, C.L.A., Cormack, G.V.: Information Retrieval: Implementing and Evaluating Search Engines. Massachusetts Institute of Technology Press (2010)
Cleverdon, C.W.: The Cranfield tests on index language devices. In: Aslib Proceedings, vol. 19, pp. 173–192 (1967); Reprinted in Readings in Information Retrieval, Sparck-Jones, K., Willett, P. (eds.) Morgan Kaufmann (1997)
Colomb, R.M.: Ontology and the Semantic Web. IOS Press (2007)
Cuadra, C.A., Katter, R.V.: Opening the black box of relevance. Journal of Documentation 231(4), 291–303 (1967)
Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer, Heidelberg (1999)
Jarvelin, K., Kekalainen, J.: Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20(4), 422–446 (2002)
Manning, C.D., Raghavan, P., Schutze, H.: Intorduction to Information Retrieval. Cambridge University Press (2008)
Najork, M.A., Zaragoza, H., Taylor, M.J.: HITS on the Web: How does it com- pare. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 471–478 (2007)
Robertson, S.: On GMAP and other transformations. In: Proceedings of the 15th ACM International Conference on Information and Knowledge Management, pp. 78–83 (2006)
Staab, S., Studer, R. (eds.): Handbook on Ontologies. Springer, Heidelberg (2009)
Taylor, R.S.: Process of Asking Questions. American Documentation 13, 291–303 (1967)
van Rijsbergen, C.J.: Information Retrieval, 2nd edn. ch. 7. Butterworths (1979)
Voorhees, E.M.: The philosophy of information retrieval evaluation. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 355–370. Springer, Heidelberg (2002)
Voorhees, E.M., Harman, D.K. (eds.): TREC. Experiment and Evaluation in Information Retrieval. Massachusetts Institute of Technology Press (2005)
Wille, R.: Restructuring lattice theory. In: Rival, I. (ed.) Ordered Sets. Reidel (1982)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag GmbH Berlin Heidelberg
About this chapter
Cite this chapter
Wasilewski, P. (2012). Towards Semantic Evaluation of Information Retrieval. In: Bembenik, R., Skonieczny, L., Rybiński, H., Niezgodka, M. (eds) Intelligent Tools for Building a Scientific Information Platform. Studies in Computational Intelligence, vol 390. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24809-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-24809-2_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24808-5
Online ISBN: 978-3-642-24809-2
eBook Packages: EngineeringEngineering (R0)