"skipMainNavigation"
Search
Quick Search in Journals
Quick Search anywhere
Advanced Search
Radiology
Menu
  • Information
    • Previous
    Next
    REVIEWS AND COMMENTARY

    Bibliometric Indicators: Quality Measurements of Scientific Publication

    Published Online:Apr 8 2010https://doi.org/10.1148/radiol.09090626

    Abstract

    It is essential to simultaneously consider multiple indicators when evaluating the quality of any scientific output and to refer to expert opinion to interpret them.

    Bibliometrics is a set of mathematical and statistical methods used to analyze and measure the quantity and quality of books, articles, and other forms of publications. There are three types of bibliometric indicators: quantity indicators, which measure the productivity of a particular researcher; quality indicators, which measure the quality (or “performance”) of a researcher’s output; and structural indicators, which measure connections between publications, authors, and areas of research. Bibliometric indicators are especially important for researchers and organizations, as these measurements are often used in funding decisions, appointments, and promotions of researchers. As more and more scientific discoveries occur and published research results are read and then quoted by other researchers, bibliometric indicators are becoming increasingly important. This article provides an overview of the currently used bibliometric indicators and summarizes the critical elements and characteristics one should be aware of when evaluating the quantity and quality of scientific output.

    © RSNA, 2010

    References

    • 1 Whitehouse GH . Citation rates and impact factors: should they matter?. Br J Radiol 2001;74(877):1–3.
      MedlineGoogle Scholar
    • 2 Rhen C , Kronman U . Bibliometric handbook for Karolinska Institutet. Stockholm, Sweden: Karolinska Institutet, 2006.
      Google Scholar
    • 3 de Solla Price DJ . Networks of scientific papers: the pattern of bibliographic references indicates the nature of the scientific research front. Science 1965;149(3683):510–515.
      MedlineGoogle Scholar
    • 4 Pritchard A . Statistical bibliography or bibliometrics. J Doc 1969;25(4):348–349.
      Google Scholar
    • 5 Rahman M , Haque TL , Fukui T . Research articles published in clinical radiology journals: trend of contribution from different countries. Acad Radiol 2005;12(7):825–829.
      MedlineGoogle Scholar
    • 6 Rahman M , Fukui T . A decline in the U.S. share of research articles. N Engl J Med 2002;347(15):1211–1212.
      MedlineGoogle Scholar
    • 7 Al-Shahi R , Will RG , Warlow CP . Amount of research interest in rare and common neurological conditions: bibliometric study. BMJ 2001;323(7327):1461–1462.
      MedlineGoogle Scholar
    • 8 Henderson SO , Brestky P . Predictors of academic productivity in emergency medicine. Acad Emerg Med 2003;10(9):1009–1011.
      MedlineGoogle Scholar
    • 9 Itagaki MW , Pile-Spellman J . Factors associated with academic radiology research productivity. Radiology 2005;237(3):774–780.
      Google Scholar
    • 10 Itagaki MW . Impact of the National Institutes of Health on radiology research. Radiology 2008;247(1):213–219.
      Google Scholar
    • 11 Mussurakis S . Financial support for research in radiology: a survey of original investigations published in the AJR and Radiology. AJR Am J Roentgenol 1994;163(4):973–979; discussion 981–982.
      MedlineGoogle Scholar
    • 12 Lundberg J . Bibliometrics as a research assessment tool: impact beyond the impact factor Stockholm, Sweden: Karolinska Institutet, 2006.
      Google Scholar
    • 13 Kermarrec AM , Faou E , Merlet JP , Robert P , Segoufin L . Que mesurent les indicateurs bibliométriques? INRIA. http://www.inria.fr/inria/organigramme/documents/ce_indicateurs.pdf. Published 2007. Accessed October 18, 2008.
      Google Scholar
    • 14 Moed HF , De Bruin RE , Van Leeuw TN . New bibliometric tools for the assessment of national research performance: database description, overview of indicators and first applications. Scientometrics 1995;33(3):381–422.
      Google Scholar
    • 15 Garfield E , Sher IH . New factors in the evaluation of scientific literature through citation indexing. Am Doc 1963;14(3):195–201.
      Google Scholar
    • 16 Garfield E . Citation indexes for science; a new dimension in documentation through association of ideas. Science 1955;122(3159):108–111.
      MedlineGoogle Scholar
    • 17 van Leeuwen TN , Moed HF . Development and application of journal impact measures in the Dutch science system. Scientometrics 2002;53(2):249–266.
      Google Scholar
    • 18 Seglen PO . Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314(7079):498–502.
      MedlineGoogle Scholar
    • 19 Lowy C . Impact factor limits funding. Lancet 1997;350(9083):1035.
      MedlineGoogle Scholar
    • 20 Garfield E . Journal impact factor: a brief review. CMAJ 1999;161(8):979–980.
      MedlineGoogle Scholar
    • 21 Thomson Reuters. Journal Citation Reports, 2008 Science Edition. http://isiknowledge.com/ . Accessed June 23, 2009.
      Google Scholar
    • 22 Not-so-deep impact. Nature 2005;435(7045):1003–1004.
      Google Scholar
    • 23 Chew FS , Relyea-Chew A . How research becomes knowledge in radiology: an analysis of citations to published papers. AJR Am J Roentgenol 1988;150(1):31–37.
      MedlineGoogle Scholar
    • 24 Dong P , Loh M , Mondry A . The “impact factor” revisited. Biomed Digit Libr 2005;2:7.
      MedlineGoogle Scholar
    • 25 Rousseau R . Journal evaluation: technical and practical issues. Library Trends. http://findarticles.com/p/articles/mi_m1387/is_3_50/ai_88582623. Published 2002. Accessed February 15, 2009.
      Google Scholar
    • 26 Chew M , Villanueva EV , Van Der Weyden MB . Life and times of the impact factor: retrospective analysis of trends for seven medical journals (1994–2005) and their Editors’ views. J R Soc Med 2007;100(3):142–150.
      MedlineGoogle Scholar
    • 27 Lundberg J . Lifting the crown-citation z-score. J Informetrics 2007;1(2):145–154.
      Google Scholar
    • 28 Althouse M , West JD , Bergstrom TC , Bergstrom CT . Differences in impact factor across fields and over time. J Am Soc Inf Sci Technol 2008;60(1):27–34.
      Google Scholar
    • 29 Garfield E . Is citation analysis a legitimate evaluation tool?. Scientometrics 1979;1(4):359–375.
      Google Scholar
    • 30 Rogers LF . Impact factor: the numbers game. AJR Am J Roentgenol 2002;178(3):541–542.
      MedlineGoogle Scholar
    • 31 Smith R . Journal accused of manipulating impact factor. BMJ 1997;314(7079):461.
      MedlineGoogle Scholar
    • 32 Neuberger J , Counsell C . Impact factors: uses and abuses. Eur J Gastroenterol Hepatol 2002;14(3):209–211.
      MedlineGoogle Scholar
    • 33 Agrawal AA . Corruption of journal impact factors. Trends Ecol Evol 2005;20(4):157.
      MedlineGoogle Scholar
    • 34 Thomson Reuters. Cited journal graph. Journal Citation Reports Web site. http://admin-apps.isiknowledge.com/JCR/help//h_ctdjrnlg.htm. Published 2008. Accessed June 23, 2008.
      Google Scholar
    • 35 Saha S , Saint S , Christakis DA . Impact factor: a valid measure of journal quality?. J Med Libr Assoc 2003;91(1):42–46.
      MedlineGoogle Scholar
    • 36 Thomson Reuters. Five-year journal impact factor. Journal Citation Reports Web site. http://admin-apps.isiknowledge.com/JCR/help/h_impfact.htm. Published 2008. Accessed June 23, 2008.
      Google Scholar
    • 37 Jacso P . Five-year impact factor data in the Journal Citation Reports. Online Inf Rev 2009;33(3):603–614.
      Google Scholar
    • 38 Thomson Reuters. Glossary of Thomson Scientific. http://scientific.thomsonreuters.com/support/patents/patinf/terms/. Published 2008. Accessed November 23, 2008.
      Google Scholar
    • 39 Thomson Reuters. Immediacy index. Journal Citation Reports Web site. http://admin-apps.isiknowledge.com/JCR/help/h_immedindex.htm. Published 2007. Accessed November 23, 2008.
      Google Scholar
    • 40 Thomson Reuters. Cited half-life. Journal Citation Reports Web site. http://admin-apps.isiknowledge.com/JCR/help/h_ctdhl.htm. Published 2007. Accessed November 23, 2008.
      Google Scholar
    • 41 Van Raan AF . Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics 2006;67(3):491–502.
      Google Scholar
    • 42 Bergstrom CT , West JD . Assessing citations with the Eigenfactor metrics. Neurology 2008;71(23):1850–1851.
      MedlineGoogle Scholar
    • 43 Bergstrom CT , West JD , Wiseman MA . The Eigenfactor metrics. J Neurosci 2008;28(45):11433–11434.
      MedlineGoogle Scholar
    • 44 Bergstrom C . EigenFACTOR.org: ranking and mapping scientific knowledge. http://www.eigenfactor.org/methods.html. Published 2007. Accessed November 15, 2008.
      Google Scholar
    • 45 Lundberg J , Frannsson A , Brommels M , Skar J , Lundkvist I . Is it better or just the same? Article identification strategies impact bibliometric assessments. Scientometrics 2006;66(1):183–197.
      Google Scholar
    • 46 Hirsch JE . An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 2005;102(46):16569–16572.
      MedlineGoogle Scholar
    • 47 Batista PD , Campiteli MG , Kinouchi O , Martinez AS . Is it possible to compare researchers with different scientific interests?. Scientometrics 2006;68(1):179–189.
      Google Scholar
    • 48 Braun T , Glanzel W , Schubert AA . Hirsch-type index for journals. Scientist 2005;19(22):8.
      Google Scholar
    • 49 Egghe L , Rousseau R . An informetric model for Hirsch-index. Scientometrics 2006;69(1):121–129.
      Google Scholar
    • 50 Bornmann L , Daniel HD . Does the h-index for ranking of scientists really work?. Scientometrics 2005;65(3):391–392.
      Google Scholar
    • 51 Bornmann L , Daniel HD . What do we know about the h index?. J Am Soc Inf Sci Technol 2007;58(9):1381–1385.
      Google Scholar
    • 52 Thomson Reuters. Web of Science overview. ISI Web of Knowledge Web site. http://isiwebofknowledge.com/products_tools/multidisciplinary/webofscience/. Published 2009. Accessed January 9, 2009.
      Google Scholar
    • 53 Thomson Reuters. Journal Citation Reports overview. Thomson Reuter Web site. http://www.thomsonreuters.com/products_services/scientific/Journal_Citation_Reports. Published 2009. Accessed January 09, 2009.
      Google Scholar
    • 54 Rowlands I . The missing link: journal usage metrics. Aslib Proc 2007;59(3):222–228.
      Google Scholar
    • 55 Moed HF . The impact-factors debate: the ISI’s uses and limits. Nature 2002;415(6873):731–732.
      MedlineGoogle Scholar
    • 56 Yang K , Meho LI . Citation analysis: a comparison of Google Scholar, Scopus, and Web of Science. Proc Am Soc Inf Sci Technol 2007;43(1):1–15.
      Google Scholar
    • 57 Jacso P . Dubious hit counts and cuckoo’s eggs. Online Inf Rev 2006;30(2):188–193.
      Google Scholar

    Article History

    Received April 19, 2009; revision requested May 29; revision received July 14; accepted July 24; final version accepted July 29; final review by P.A.G. December 11.
    Published online: Apr 8 2010
    Published in print: May 2010