Abstract
In digital libraries semantic techniques are often deployed to reduce the expensive manual overhead for indexing documents, maintaining metadata, or caching for future search. However, using such techniques may cause a decrease in a collection’s quality due to their statistical nature. Since data quality is a major concern in digital libraries, it is important to be able to measure the (loss of) quality of metadata automatically generated by semantic techniques. In this paper we present a user study based on a typical semantic technique used for automatic metadata creation, namely taxonomies of author keywords and tag clouds. We observed experts assessing typical relations between keywords and documents over a small corpus in the field of chemistry. Based on the evaluation of this experiment, we focused on communalities between the experts’ perception and thus draw a first roadmap on how to evaluate semantic techniques by proposing some preliminary metrics.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Bao, S., Xue, G., Wu, X., Yu, Y., Fei, B., Su, Z.: Optimizing web search using social annotations. In: WWW 2007: Proceedings of the 16th international conference on World Wide Web. ACM Press, New York (2007)
Bischoff, K., Firan, C.S., Nejdl, W., Paiu, R.: Can all tags be used for search? In: CIKM 2008: Proceeding of the 17th ACM conference on Information and knowledge management. ACM Press, New York (2008)
Chan, S.: Tagging and Searching – Serendipity and museum collection databases. In: Proceedings of Museums and the Web 2007. Archive & Museum Informatics 2007, Toronto (2007)
Cimiano, P., Handschuh, S., Staab, S.: Towards the self-annotating web. In: Int. Conf. on the World Wide Web (WWW). ACM, New York (2004)
Diederich, J., Balke, W.-T.: The Semantic GrowBag Algorithm: Automatically Deriving Categorization Systems. In: Kovács, L., Fuhr, N., Meghini, C. (eds.) ECDL 2007. LNCS, vol. 4675, pp. 1–13. Springer, Heidelberg (2007)
Diederich, J., Balke, W.: Automatically Created Concept Graphs using Descriptive Keywords in the Medical Domain. In: Methods of Information in Medicine (METHODS), Schattauer, vol. 47(3) (2008)
Fuhr, N., Hansen, P., Mabe, M., Micsik, A., Sølvberg, I.T.: Digital Libraries: A Generic Classification and Evaluation Scheme. In: Constantopoulos, P., Sølvberg, I.T. (eds.) ECDL 2001. LNCS, vol. 2163, p. 187. Springer, Heidelberg (2001)
Fuhr, N., Tsakonas, G., Aalberg, T., Agosti, M., Hansen, P., Kapidakis, S., et al.: Evaluation of digital libraries. In: Int. J. on Digital Libraries, vol. 8(1) (2007)
Gangemi, A., Catenaccia, C., Ciaramita, M., Lehmann, J.: Qood grid: A meta-ontology-based framework for ontology evaluation and selection. In: Proc. of the 4th International Workshop on Evaluation of Ontologies for the Web (EON 2006), Edinburgh, Scotland (2006)
Golder, S.A., Huberman, B.A.: The structure of collaborative tagging systems (2005) CoRR abs/cs/0508082
Gonçalves, M.A., Moreira, B.L., Fox, E.A., Watson, L.T.: What is a good digital library? In: A quality model for digital libraries. Inf. Process Manage, vol. 43(5) (2007)
Gonçalves, M.A., Fox, E.A., Watson, L.T., Kipp, N.A.: Streams, structures, spaces, scenarios, societies (5s): A formal model for digital libraries. ACM Trans. Inf. Syst. 22(2) (2004)
Halpin, H., Robu, V., Shepherd, H.: The complex dynamics of collaborative tagging. In: WWW 2007: Proceedings of the 16th international conference on World Wide Web. ACM Press, New York (2007)
Hearst, M.A.: Automatic Acquisition of Hyponyms from Large Text Corpora. In: Int. Conf. on Computational Linguistics, Nantes, France (1992)
Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: Information Retrieval in Folksonomies: Search and Ranking. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 411–426. Springer, Heidelberg (2006)
http://www.nlm.nih.gov/pubs/factsheets/mesh.html (last accessed on 25.03.2009)
http://www.nlm.nih.gov/pubs/factsheets/medline.html (last accessed on 25.03.2009)
Khoo, M., Pagano, J., Washington, A., Recker, M., Palmer, B., Donahue, R.A.: Using web metrics to analyze digital libraries. In: JCDL (2008)
Krestel, R., Chen, L.: The art of tagging: Measuring the quality of tags. In: Domingue, J., Anutariya, C. (eds.) ASWC 2008. LNCS, vol. 5367, pp. 257–271. Springer, Heidelberg (2008)
Kruk, S.R., Woroniecki, T., Gzella, A., Dabrowski, M.: JeromeDL - a Semantic Digital Library. In: Semantic Web Challenge (2007)
Kruk, S.R., Kruk, E., Stankiewicz, K.: Evaluation of Semantic and Social Technologies for Digital Libraries. In: Semantic Digital Libraries. Springer, Heidelberg (2009)
Li, Y., Bandar, Z.A., Mclean, D.: An approach for measuring semantic similarity between words using multiple information sources. IEEE Transactions on Knowledge and Data Engineering 15(4) (2003)
Lozano-Tello, A., Gómez-Pérez, A.: OntoMetric: A method to choose the appropriate ontology. Journal of Database Management, Special Issue on Ontological analysis, Evaluation, and Engineering of Business Systems Analysis Methods 15(2) (2004)
Rada, R., Mili, H., Bicknell, E., Blettner, M.: Development and application of a metric on semantic nets. IEEE Transactions on Systems, Man and Cybernetics 19(1) (1989)
Razikin, K., Goh, D.H.-L., Chua, A.Y.K., Lee, C.S.: Can social tags help you find what you want? In: Christensen-Dalsgaard, B., Castelli, D., Ammitzbøll Jurik, B., Lippincott, J. (eds.) ECDL 2008. LNCS, vol. 5173, pp. 50–61. Springer, Heidelberg (2008)
Sanderson, M., Croft, B.: Deriving concept hierarchies from text. In: Proc. of Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Berkeley, CA, USA. ACM, New York (1999)
Saracevic, T.: Digital library evaluation: toward evolution concepts. Library Trends 49(2) (2000)
Tartir, S., Aroinar, I.B., Moore, M., Sheth, A.P., Aleman-Meza, B.: OntoQA: Metric-based ontology analysis. In: Proceedings of IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge sources (2005)
Vrandečić, D., Sure, Y.: How to design better ontology metrics. In: Franconi, E., Kifer, M., May, W. (eds.) ESWC 2007. LNCS, vol. 4519, pp. 311–325. Springer, Heidelberg (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tönnies, S., Balke, WT. (2009). Using Semantic Technologies in Digital Libraries – A Roadmap to Quality Evaluation. In: Agosti, M., Borbinha, J., Kapidakis, S., Papatheodorou, C., Tsakonas, G. (eds) Research and Advanced Technology for Digital Libraries. ECDL 2009. Lecture Notes in Computer Science, vol 5714. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04346-8_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-04346-8_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04345-1
Online ISBN: 978-3-642-04346-8
eBook Packages: Computer ScienceComputer Science (R0)