Original Article| Volume 63, ISSUE 11, P1205-1215, November 2010

Science mapping analysis characterizes 235 biases in biomedical research

  • David Chavalarias
    Centre de Recherche en Épistémologie Appliquée, École Polytechnique-CNRS, 32 Bd Victor, 75015 Paris, France

    Institut des Systèmes Complexes de Paris Ile-de-France, 57–59 rue Lhomond, 75005, Paris, France
    Search for articles by this author
  • John P.A. Ioannidis
    Corresponding author. Tel: + 30-265-1007807; fax: + 30-265-1007867.
    Department of Hygiene and Epidemiology, University of Ioannina School of Medicine and Biomedical Research Institute, Foundation for Research and Technology-Hellas, Ioannina 45110, Greece

    Tufts Clinical and Translational Science Institute and Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Department of Medicine, Tufts University School of Medicine, Boston, MA 02111, USA
    Search for articles by this author



      Many different types of bias have been described. Some biases may tend to coexist or be associated with specific research settings, fields, and types of studies. We aimed to map systematically the terminology of bias across biomedical research.

      Study Design and Setting

      We used advanced text-mining and clustering techniques to evaluate 17,265,924 items from PubMed (1958–2008). We considered 235 bias terms and 103 other terms that appear commonly in articles dealing with bias.


      Forty bias terms were used in the title or abstract of more than 100 articles each. Pseudo-inclusion clustering identified 252 clusters of terms. The clusters were organized into macroscopic maps that cover a continuum of research fields. The resulting maps highlight which types of biases tend to co-occur and may need to be considered together and what biases are commonly encountered and discussed in specific fields. Most of the common bias terms have had continuous use over time since their introduction, and some (in particular confounding, selection bias, response bias, and publication bias) show increased usage through time.


      This systematic mapping offers a dynamic classification of biases in biomedical investigation and related fields and can offer insights for the multifaceted aspects of bias.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ioannidis J.P.
        Why most published research findings are false.
        PLoS Med. 2005; 2: e124
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Vist G.E.
        • Falck-Ytter Y.
        • Schunemann H.J.
        • GRADE Working Group
        What is "quality of evidence" and why is it important to clinicians?.
        BMJ. 2008; 336: 995-998
        • Sackett D.L.
        Bias in analytic research.
        J Chronic Dis. 1979; 32: 51-63
        • Chavalarias D.
        • Cointet J-P.
        Bottom-up scientific field detection for dynamical and hierarchical science mapping—methodology and case study.
        Scientometrics. 2008; 75: 37-50
        • Palla G.
        • Farkas I.J.
        • Pollner P.
        • Derényi I.
        • Vicsek T.
        Directed network modules.
        New J Physics. 2007; 9: 186
      1. Clusters & communities: overlapping dense groups in networks. Available at:

      2. Chavalarias D, Cointet J-P. The reconstruction of science phylogeny. arXiv:0904.3154v2 [physics.soc-ph].

      3. Bastian M, Heymann S, Jacomy M, Gephi: an open source software for exploring and maintaining networks. International Conference on Weblogs and Social Media, 2009. Available at:

        • Cointet J.-P.
        • Chavalarias D.
        Multi-level science mapping with asymmetric co-occurrence analysis: methodology and case study.
        Networks and Heterogeneous Media. 2008; 3: 267-276
      4. MOMA. Module Mapping for large electronic corpora and social media. Available at:

      5. Pajek Wiki. Available at:

      6. GraphViz Graph Visualization Software. Available at:

        • Kempthorne O.
        A simple approach to confounding and fractional replication in factorial experiments.
        Biometrika. 1947; 34: 255-272
        • Barnes E.H.
        Response bias and the MMPI.
        J Consult Psychol. 1956; 20: 371-374
        • Wallin P.
        Volunteer subjects as a source of sampling bias.
        Am J Sociol. 1949; 54: 539-544
        • Shinkman P.G.
        • Kornblith C.L.
        Comment on observer bias in classical conditioning of the planarian.
        Psychol Rep. 1965; 16: 56
        • Zajonc R.B.
        • Burnstein E.
        Structural balance, reciprocity, and positivity as sources of cognitive bias.
        J Pers. 1965; 33: 570-583
        • Snyder J.D.
        Race bias in hospitals. What the Civil Rights Commission found.
        Hosp Manage. 1963; 96: 52-54
        • Binder A.
        The Rorschach test: a perceptual bias.
        Percept Mot Skills. 1964; 18: 225-226
        • Blackwell D.
        • Hodges J.L.
        Design for the control of selection bias.
        Ann Math Statistics. 1957; 28: 449-460
        • Callon M.
        • Courtial J.
        • Laville F.
        Co-word analysis as a tool for describing the network of interactions between basic and technological research the case of polymer chemistry.
        Scientometrics. 1991; 22: 155-205
        • Qin H.
        Knowledge discovery through co-word analysis.
        Library Trends. 1999; 48: 133-159
        • Vineis P.
        History of bias.
        Soz Prev Med. 2002; 47: 156-161