Abstract
In this paper we investigate the retrieval capabilities of six Internet search engines on a simple query. As a case study the query “Erdos” was chosen. Paul Erdos was a world famous Hungarian mathematician, who passed away in September 1996. Existing work on search engine evaluation considers only the first ten or twenty results returned by the search engine, therefore approximation of the recalls of the engines has not been considered so far. In this work we retrieved all 6681 documents that the search engines pointed at and thoroughly examined them. Thus we could calculate the precision of the whole retrieval process, study the overlap between the results of the engines and give an estimate on the recall of the searches. The precision of the engines is high, recall in very low and the overlap is minimal.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
AltaVista. (1996).Help for Advanced Query. [Online]. Available: http://altavista.digital.com/cgi-bin/query?pg=ah [November 1996].
Alta Vista. (1997a)About AltaVista Search. [Online]. Available: http://www.altavista.digital.com/av/content/about.htm [December 1997].
AltaVista. (1997b)About AltaVista Search. [Online]. Available: http://www.altavista.digital.com/av/content/about_our_story_2.htm [December 1997].
Almind, T. C. &Ingwersen, P. (1997). Informetric Analyses on the World Wide Web: Methodological Approaches to ‘Webometrics’.Journal of Documentation, 53(4), 404–426.
Bar-Ilan, J. (1997). The ‘Mad Cow Disease’, Usenet Newsgroups and Bibliometric Laws.Scientometrics, 39(1), 29–55.
Cailliau, R. (1995).A Little History of the World Wide Web. [Online]. Available: http://www.w3.org/History.html [December 1997].
Chu, H. & Rosenthal, M. (1996). Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology.ASIS96. [Online]. Available: http://www.asis.org/annual-96/Electronic-Proceedings/chu.htm [December 1997].
Courtois, M. P. (May/June 1996) Cool Tools for Searching the Web— An Update.Online, 29–36.
DeZelar-Tiedman, C. (1997). Known-Item Searching on the World Wide Web.Internet Reference Services Quarterly, 2(1), 5–14.
Ding, W. & Marchionini, G. (1996). A Comparative Study of Web Search Service Performance.ASIS96. [Online]. Available: http://www.glue.umd.edu/~weid/asis/fulltext.htm [December 1997].
Dong, X. &Su, L.T. (1997). Search Engines on the World Wide Web and Information Retrieval from the Internet: A Review and Evaluation.Online & CDROM Review, 21(2), 67–81.
Excite. (1996).How to use Excite search. [Online]. Available: http://www.excite.com/Info/searching.html?an [November 1996].
Excite. (1997).What We Do. [Online]. Available: http://corp.excite.com/Company/what.html [December 1997].
Feather, J. &Sturges, P. (Eds). (1997).International Encyclopedia of Information and Library Science. London: Rutledge, 1997, 263–265.
Feldman, S. (1997a). ‘It Was Here a Minute Ago!’: Archiving the Net.Searcher, 5(9), 52. [Also Online] Available:http://www.infotoday.com/searcher/oct/story4.htm [December 1997]
Feldman, S. (1997b). ‘Just the Answers, Please’: Choosing a Web Search Service.Searcher, 5(5), 44–57. [Also Online]. Available:http://www.infotoday.com/searcher/may/story3.htm [December 1997]
Haskin, D. (1997). Power Search.Internet World, 8(12), 79–92.Hypertext Markup Language—2.0— The HTML Coded Character Set. (1997). [Online]. Available:http://www.w3.org/MarkUp/html-spec/html-spec_13.html [December 1997]
Infoseek. (1996a).About Ultraseek. [Online]. Available: http://guide.infoseek.com/Help?pg=AboutUltra.html&sv=N3 [November 1996].
Infoseek. (1996b).Feature Comparison. [Online]. Available: http://guide.infoseek.com/doc?pg=comparison.html&sv=N3 [November 1996].
Kahle, B. (March 1997). Preserving the Internet.Scientific American, 82–83.
Krippendorff, K. (1980).Content Analysis— An Introduction to Its Methodology, Beverly Hills: Sage Publications, 1980.
Krol, E. (1992).The Whole Internet Guide, New York: O'Reilly, 1992.
Lycos. (1996).Lycos Inc. Information. [Online]. Available: http://www.lycos.com/help.html [November 1996].
Larson, R. (1996). Bibliometrics of the World Wide Web: An Exploratory Analysis of the Intellectual Structure of Cyberspace.ASIS96. [Online]. Available: http://sherlock.berkeley.edu/asis96/asis96.html [December 1997].
Lancaster, F. W. &Fayen, E.G. (1973).Information Retrieval On-Line, Los Angeles: Wiley-Becker. chapter 6.
Leighton, H. V. & Srivastava, J. (1997).Precision among World Wide Web Search Services (Search Engines): Alta Vista, Excite, Hotbot, Infoseek, Lycos. [Online]. Available: http://www.winona.msus.edu/is-f/library-f/webind2/webind.html [December 1997].
Magellan. (1996).Magellan's Frequently Asked Questions. [Online]. Available: http://www.mckinley.com/feature.cgi?fag_bd[November 1996].
McClure W. L. &Stan A. H. (1995). Communicating Globally: The Advent of Unicode.Computers in Libraries, 15(5), 19–24.
Opentext. (1996a).The Open Text Index— Frequently Asked Questions. [Online]. Available: http://index.opentext.net/main/help.htm [November 1996].
Opentext. (1996b).The Open Text Index— Search Help. [Online]. Available: http://index.opentext.net/main/help.htm [November 1996].
Oudet, B. (March 1997). Multilingualism on the Internet.Scientific American, 77–78.
Rousseau, R. (1997). Sitations: an Exploratory Study.Cybermetrics, [Online], 1(1). Available: http://www.cindoc.es/cybermetrics/articles/vlilpl.htm [November 1997].
Tomaiulo N. G. &Packer, J. G. (1996). An Analysis of Internet Search Engines: Assessment of Over 200 Search Queries.Computers in Libraries, 16(6), 58–62.
The Unicode Homepage on the Web. (1997). [Online]. Available: http://www.unicode.org [December 1997].
Venditto, G. (1996). Search Engine Showdown.Internet World, 7(5), 79–86.
Wired Cybrarian. (1997). [Online]. Available: http://www.wired.com/cybrarian/frame/reference/stats.html [December 1997].
Woodruff, A. et al. (1996). An Investigation of Documents from the World Wide Web.Proceedings of the Fifth International World Wide Web Conference. 963–980.
Zorn, P. et al. (May/June 1996). Searching— Trics of the Trade.Online, 15–28.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bar-Ilan, J. On the overlap, the precision and estimated recall of search engines. A case study of the query “Erdos”. Scientometrics 42, 207–228 (1998). https://doi.org/10.1007/BF02458356
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02458356