Abstract
After a review of developments in the quantitative study of science, particularly since the early 1970s, I focus on two current main lines of ‘measuring science’ based on bibliometric analysis. With the developments in the Leiden group as an example of daily practice, the measurement of research performance and, particularly, the importance of indicator standardisation are discussed, including aspects such as interdisciplinary relations, collaboration, ‘knowledge users’. Several important problems are addressed: language bias; timeliness; comparability of different research systems; statistical issues; and the ‘theory-invariance’ of indicators. Next, an introduction to the mapping of scientific fields is presented. Here basic concepts and issues of practical application of these ‘science maps’ are addressed. This contribution is concluded with general observations on current and near-future developments, including network-based approaches, necessary ‘next steps’ are formulated, and an answer is given to the question ‘Can science be measured?’
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Research Performance
- Science Citation Index
- Citation Analysis
- Bibliometric Analysis
- Bibliometric Indicator
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Adam, D. (2002). The counting house. Nature, 415, 726–729.
Albert, M.B., Avery, D., Narin, F., MacAllister, P. (1991). Direct validation of citation counts as indicators of industrially important patents. Research Policy, 20, 251–259.
Arunachalam, S., Srinivasan, R., Raman, V. (1994). International collaboration in science-participation by the Asian giants. Scientometrics, 30, 7–22.
Bar-Ilan, J. (2001). Data collection methods on the Web for informetric purposes — A review and analysis. Scientometrics, 50, 7–32.
Bayer, A.E., Smart, J.C., McLaughlin, G.W. (1990). Mapping intellectual structure of a scientific subfield through author cocitations. Journal of the American Society for Information Science, 41, 444–452.
Beaver, D. de B., Rosen, R. (1978). Studies in scientific collaboration, 1: Professional origins of scientific co-authorship. Scientometrics, 1, 65–84.
Björneborn, L., Ingwersen, P. (2001). Perspectives of webometrics. Scientometrics, 50, 65–82.
Borgman, C.L. (ed.) (1990). Scholarly Communication and Bibliometrics. Newbury Park: Sage.
Braam, R.R., Moed, H.F., van Raan, A.F.J. (1991a). Mapping of science by combined cocitation and word analysis, I: Structural Aspects. Journal of the American Society for Information Science (JASIS), 42, 233–251, and, II: Dynamical Aspects. Journal of the American Society for Information Science (JASIS), 42, 252–266.
Braun, T., Glänzel, W., Schubert, A. (1988). World flash on basic research — The newest version of the facts and figures on publication output and relative citation impact of 100 countries 1981–1985. Scientometrics, 13, 181–188.
Braun, T., Glänzel, Grupp, H. (1995). The scientometric weight of 50 nations in 27 science areas, 1989–1993. 1: All fields combined, mathematics, engineering, chemistry and physics. Scientometrics, 33, 263–293; and 2: Life sciences. Scientometrics, 34, 207–237.
Brooks, T.A. (1986). Evidence of complex citer motivations. Journal of the American Society for Information Science, 37, 34–36.
Butler, L. (2003). Modifying publication practices in response to funding formulas. Research Evaluation, 17, 39–46.
Callon, M., Bauin, S., Courtial, J.P., Turner, W. (1983). From translation to problematic networks: an introduction to co-word analysis. Social Science Information, 22, 191–235.
Cole, S., Cole, J.R., Dietrich, L. (1978). Measuring the cognitive state of scientific disciplines. In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H. (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op. cit..
de Candolle, A. (1873, 2nd. edition 1885). Histoire des sciences et des savants depuis deux siècles. Genève/Basel: H.Georg. Reprint in 1987 by Fayard.
Egghe L., Rousseau, R. (2000). The influence of publication delays on the observed aging distribution of scientific literature. Journal of the American Society for Information Science, 51, 158–165.
Elkana, Y., Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H. (Eds.) (1978). Toward a metric of science: the advent of science indicators. New York: John Wiley.
Etzkowitz, H., Leydesdorff, L. (2000). The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university-industry-government relations. Research Policy, 29, 109–123.
Garfield, E. (1979). Is citation analysis a legitimate evaluation tool? Scientometrics, 1, 359–375.
Garfield, E. (1980). Premature discovery or delayed recognition — Why? Current Contents, 21, May 26, 5–10.
Gilbert, G.N. (1978). Measuring the growth of science-review of indicators of scientific growth. Scientometrics, 1, 9–34.
Glänzel, W. (1996). A bibliometric approach to social sciences, national research performances in 6 selected social science areas, 1990–1992. Scientometrics, 35, 291–307.
Glänzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics, 51, 69–115.
Glänzel, W., Meyer, M. (2003). Patents cited in the scientific literature: An exploratory study of ‘reverse’ citation relations. Scientometrics, 58, 415–428.
Glänzel, W., Schlemmer, B., Thijs, B. (2003). Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon. Scientometrics, 58, 571–586
Glänzel, W., Debackere, K. (2003). On the opportunities and limitations in using bibliometric indicators in a policy relevant context. In: Bibliometric analysis in science and research. Applications, Benefits and Limitations. Second Conference of the Central Library, Forschungszentrum Jülich, (pp. 225–236). (ISBN 3-89336-334-3).
Gläser, J., Laudel, G. (2001). Integrating scientometric indicators into sociological studies: methodical and methodological problems. Scientometrics, 52, 411–434.
Grupp, H., Schmoch, U., Hinze, S. (2001). International alignment and scientific regard as macro-indicators for international comparisons of publications. Scientometrics, 51, 359–380.
Haitun, S.D. (1982). Stationary scientometric distributions. 1: Different approximations. Scientometrics, 4, 89–104.
Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44, 193–215.
Holton, G. (1978). Can science be measured? In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H. (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op. cit.
Horrobin, D.F. (1990). The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association (JAMA), 263, 1438–1441.
Kamerlingh Onnes, H. (1882). De betekenis van kwantitatief onderzoek in de natuurkunde (The meaning of quantitative research in physics). Inaugural Address as Professor of Physics, Leiden University.
Koenig, M.E.D. (1983). Bibliometric indicators versus expert opinion in assessing research performance. Journal of the American Society for Information Science, 34, 136–145.
Kostoff, R.N. (1995). Federal research impact assessment — Axioms, approaches, applications. Scientometrics, 34, 163–206.
van Leeuwen, Th.N., Moed, H.F., Tijssen, R.J.W., Visser, M.S., van Raan, A.F.J. (2001). Language biases in the coverage of the Science Citation Index and its consequences for international comparisons of national research performance. Scientometrics, 51, 335–346.
van Leeuwen, Th.N. (2004). Second generation bibliometric analysis. Ph.D. Thesis Leiden University.
Lewison, G. (2001). The quantity and quality of female researchers: a bibliometric study of Iceland. Scientometrics, 52, 29–43.
Lewison, G. (2002). Researchers’ and users’ perceptions of the relative standing of biomedical papers in different journals. Scientometrics, 53, 229–240.
Lotka, A.J. (1926). The frequency distribution of scientific productivity. J. Washington Acad. Sci., 16, 317–323.
MacRoberts, M.H., MacRoberts, B.R. (1996). Problems of citation analysis. Scientometrics, 36, 435–444.
MacRoberts, M.H., MacRoberts, B.R. (1988). Author motivation for not giving citing influences — A methodological note. Journal of the American Society for Information Science, 39, 432–433.
Martin, B.R., Irvine, J. (1983). Assessing basic research: some partial indicators of scientific progress in radio astronomy. Research Policy, 12, 61–90.
May, R.M. (1997). The scientific wealth of nations. Science, 275, 793–796.
McCain, K.W. (1984). Longitudinal author cocitation mapping — The changing structure of macroeconomics. Journal of the American Society for Information Science, 35, 351–359.
McCain, K.W. (1990). Mapping authors in intellectual space — A technical overview. Journal of the American Society for Information Science, 41, 433–443.
Melin, G., Persson, O. (1996). Studying research collaboration using co-authorships. Scientometrics, 36, 363–377.
Moed, H.F., van Leeuwen, Th.N. (1995). Improving the accuracy of the Institute for Scientific Information’s Journal Impact Factors. J. of the American Society for Information Science (JASIS), 46, 461–467.
Moed, H.F., van Leeuwen, Th.N. (1996). Impact factors can mislead. Nature, 381, 186.
Moed, H.F., Luwel, M., Nederhof, A.J. (2002). Towards research performance measurement in the humanities. Library Trends, 50, 498–520.
Moravcsik, M.J. (1975). Phenomenology and models of growth of science. Research Policy, 4, 80–86.
Moravcsik, M.J., Murugesan, P. (1979). Citation patterns in scientific revolutions. Scientometrics, 1, 161–169.
Moxham, H., Anderson, J. (1992). Peer review. A view from the inside. Science and Technology Policy, February 1992, 7–15.
Narin, F. (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity. Washington D.C.: National Science Foundation.
Narin, F. (1978). Objectivity versus relevance in studies of scientific advance. Scientometrics, 1, 35–41.
Narin, F. (1994). Patent bibliometrics. Scientometrics, 30, 147–155.
Narin, F., Hamilton, K.S., Olivastro, D. (1997). The increasing linkage between US technology and public science. Research Policy, 26, 317–330.
National Science Board (1973). Science Indicators 1972. Washington DC: Government Printing Office.
Nederhof, A.J. (1988). The validity and reliability of evaluation of scholarly performance. In: A.F.J. van Raan (ed). (1988), Handbook of Quantitative Studies of Science and Technology (pp. 193–228). Amsterdam: Elsevier/North-Holland, (ISBN 0-444-70537-6).
Noma, E. (1982). An improved method for analysing square scientometric transaction matrices. Scientometrics, 4, 297–316.
Noyons, E.C.M., van Raan, A.F.J. (1998). Monitoring scientific developments from a dynamic perspective: self-organized structuring to map neural network research. J. of the American Society for Information Science and Technology (JASIST), 49, 68–81.
Noyons, E.C.M., Luwel, M., Moed, H.F. (1999). Combining mapping and citation analysis for evaluative bibliometric purpose. A bibliometric study on recent development in microelectronics. Journal of the American Society for Information Science and Technology (JASIST), 50, 115–131.
Noyons, E.C.M. (1999). Bibliometric mapping as a science policy and research management tool. Ph.D. Thesis Leiden University. Leiden: DSWO Press (ISBN 90-6695-152-4).
Noyons, E.C.M., Buter, R.K., van Raan, A.F.J., Schmoch, U., Heinze, T., Hinze, S., Rangnow, R. (2003). Mapping excellence in science and technology across Europe (Part 1: Life sciences, Part 2: Nanoscience and nanotechnology). Report to the European Commission by the Centre for Science and Technology Studies (CWTS), Leiden University, and the Fraunhofer Institute for Systems and Innovation Research (Fraunhofer-ISI), Karlsruhe.
Noyons, E.C.M. (2004). Science Maps within in a Science Policy Context. This Handbook.
OECD (1963). The measurement of scientific and technological activities, ‘Frascati Manual’, Paris: Organization for Economic Co-operation and Development (OECD).
Peritz, B.C. (1983). A classification of citation roles for the social sciences and related fields. Scientometrics, 5, 303–312.
Porter, A.L., Chubin. D.E. (1985). An indicator of cross-disciplinary research. Scientometrics, 8, 161–176.
De Solla Price, D.J. (1978). Toward a model for Science Indicators. In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H., (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op.cit.
De Solla Price, D.J. (1981). The analysis of scientometric matrices for policy implications. Scientometrics, 3, 47–53.
Prime, C., Bassecoulard, E., Zitt, M. (2002). Co-citations and co-sitations: A cautionary view on an analogy. Scientometrics, 54, 291–308.
Prpić, K. (2002). Gender and productivity differentials in science. Scientometrics, 55, 27–58.
van Raan, A.F.J. (ed). (1988). Handbook of Quantitative Studies of Science and Technology. Amsterdam: Elsevier/North-Holland (ISBN 0-444-70537-6).
van Raan, A.F.J. (1990). Fractal dimension of co-citations. Nature, 347, 626.
van Raan, A.F.J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36, 397–420.
van Raan, A.F.J. (1997). Scientometrics: State-of-the-Art. Scientometrics, 38, 205–218.
van Raan, A.F.J. (1998). In matters of quantitative studies of science the fault of theorists is offering too little and asking too much. Scientometrics, 43, 129–139.
van Raan, A.F.J. (2000a). The Pandora’s box of citation analysis: measuring scientific excellence, the last evil? In: B. Cronin and H. Barsky Atkins (eds.). The Web of Knowledge. A Festschrift in honor of Eugene Garfield. Ch. 15, p. 301–319. Medford (New Jersey): ASIS Monograph Series, 2000 (ISBN 1-57387-099-4).
van Raan, A.F.J. (2000b). On growth, ageing, and fractal differentiation of science. Scientometrics 47, 347–362.
van Raan, A.F.J. (2001). Two-step competition process leads to quasi power-law income distributions. Application to scientific publication and citation distributions. Physica A, 298, 530–536.
van Raan, A.F.J., Noyons, E.C.M. (2002). Discovery of patterns of scientific and technological development and knowledge transfer. In W. Adamczak, A. Nase (Eds.), Gaining Insight from Research Information. Proceedings of the 6th International Conference on Current Research Information Systems, University of Kassel, August 29–31, 2002 (pp. 105–112). Kassel: University Press, (ISBN 3-933146-844).
van Raan, A.F.J., van Leeuwen, Th.N. (2002). Assessment of the scientific basis of interdisciplinary, applied research. Application of bibliometric methods in nutrition and food research. Research Policy, 31, 611–632
van Raan, A.F.J. (2003). Reference-based publication networks with episodic memories. E-print ArXiv cond-mat/0311318.
van Raan, A.F.J. (2004). Sleeping Beauties in Science. Scientometrics, 59, 461–466.
van Raan, A.F.J., van Leeuwen, Th.N. (2004). Statistical aspects of research group performance, journal impact, and peer judgement. To be published.
Rinia, E.J., van Leeuwen, Th.N., van Vuren, H.G., van Raan, A.F.J. (1998). Comparative analysis of a set of bibliometric indicators and central peer review criteria. Evaluation of condensed matter physics in the Netherlands. Research Policy, 27, 95–107.
Rinia, E.J., van Leeuwen, Th.N., van Vuren, H.G., van Raan, A.F.J. (2001). Influence of interdisciplinarity on peer-review and bibliometric evaluations. Research Policy, 30, 357–361.
Rip, A., Courtial, J.P. (1984). Co-word maps of biotechnology — An example of cognitive scientometrics. Scientometrics, 6, 381–400.
Schmoch, U. (1993). Tracing the knowledge transfer from science to technology as reflected in patent indicators. Scientometrics, 26, 193–211.
Schwechheimer, H., Winterhager, M. (2001). Mapping interdisciplinary research fronts in neuroscience: a bibliometric view to retrograde amnesia. Scientometrics, 51, 311–318.
Schubert A., Glänzel, W. (1983). Statistical reliability of comparisons based on the citation impact of scientometric publications. Scientometrics, 5, 59–74.
Seglen, P.O. (1992). The skewness of science. Journal of the American Society for Information Science, 43, 628–638.
Seglen, P.O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45, 1–11.
Small, H. (1973). Co-citation in the Scientific Literature: A New Measure of the Relationship Between Publications. Journal of the American Society for Information Science, 24, 265–269.
Small, H., Greenlee, E. (1980). Citation context analysis of a co-citation cluster-recombinant DNA. Scientometrics, 2, 1980.
Small, H., Sweeney, E. (1985). Clustering the Science Citation Index using co-citations, I: A Comparison of Methods. Scientometrics, 7, 393–404.
Small, H., Sweeney, E., Greenlee, E. (1985). Clustering the Science Citation Index using cocitations, II: Mapping Science. Scientometrics, 8, 321–340.
Small, H. (1999). Visualizing science by citation mapping. Journal of the American Society for Information Science, 50, 799–813.
Swanson, D.R. (1986). Fish oil, Raynaud’s syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine, 30, 7–18.
Swanson, D.R. (1987). Two medical literatures that are logically but not bibliographically connected. Journal of the American Society for Information Science, 38, 228–233.
Sullivan D., Koester, D., White, D.H., Kern, R. (1980). Understanding rapid theoretical change in particle physics — a month-by-month co-citation analysis. Scientometrics, 2, 309–319.
Thelwall, M., Smith, A. (2002). Interlinking between Asia-Pacific University Web sites. Scientometrics, 55, 363–376.
Thelwall, M., Harries, G. (2003). The connection between the research of a university and counts of links to its web pages: An investigation based upon a classification of the relationships of pages to the research of the host university. Journal of the American Society for Information Science, 54, 594–602.
Vinkler, P. (1993). Research contribution, authorship and team cooperativeness. Scientometrics 26, 213–230.
Vinkler, P. (1998). Comparative investigation of frequency and strength of motives toward referencing, the reference threshold model-comments on theories of citation? Scientometrics, 43, 107–127.
Vlàchy, J. (1979). Mobility in science. Bibliography of scientific career migration, field mobility, international academic circulation and brain drain. Scientometrics, 1, 201–228.
Weingart, P. (2003). Evaluation of research performance: the danger if numbers. In: Bibliometric analysis in science and research. Applications, Benefits and Limitations. Second Conference of the Central Library, Forschungszentrum Jülich (pp. 7–19). (ISBN 3-89336-334-3).
White, H.D., Griffith, B.C. (1981). Author cocitation— a literature measure of intellectual structure. Journal of the American Society for Information Science, 32, 163–171.
White, H.D., McCain, K.W. (1998). Visualizing a discipline: An author co-citation analysis of information science, 1972—1995. Journal of the American Society for Information Science, 49, 327–355.
Wouters, P.F. (1999), The Citation Culture, PhD thesis, University of Amsterdam.
Ziman, J. (1978). From Parameters to Portents-and Back. In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H., (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op.cit.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
van Raan, A.F. (2004). Measuring Science. In: Moed, H.F., Glänzel, W., Schmoch, U. (eds) Handbook of Quantitative Science and Technology Research. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2755-9_2
Download citation
DOI: https://doi.org/10.1007/1-4020-2755-9_2
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-2702-4
Online ISBN: 978-1-4020-2755-0
eBook Packages: Humanities, Social Sciences and LawHistory (R0)