Journal of Clinical Epidemiology
Volume 59, Issue 8 , Pages 842-848 , August 2006

Peering at peer review revealed high degree of chance associated with funding of grant applications

  • Nancy E. Mayo

      Affiliations

    • Division of Clinical Epidemiology R4.29, McGill University Health Center, RVH Site, 687 Pine Avenue West, Montreal, H3A 1A1, Canada
    • Corresponding Author InformationCorresponding author. Tel.: 514 934 1934 ext 31564; fax: 514 843 1493.
    • ,
  • James Brophy

      Affiliations

    • Health Technology Assessment Unit, R4.14, McGill University Health Center, RVH Site, 687 Pine Avenue West, Montreal, H3A 1A1, Canada
    • ,
  • Mark S. Goldberg

      Affiliations

    • Division of Clinical Epidemiology R4.29, McGill University Health Center, RVH Site, 687 Pine Avenue West, Montreal, H3A 1A1, Canada
    • ,
  • Marina B. Klein

      Affiliations

    • Divisions of Infectious Diseases/Immunodeficiency, Royal Victoria Hospital, McGill University Health Centre, 687 Pine Avenue West, Montreal, H3A 1A1, Canada
    • ,
  • Sydney Miller

      Affiliations

    • Department of Psychology, Science Pavilion, Concordia University, 7141 Sherbrooke Street West, Montreal, Quebec H4B 1R6, Canada
    • ,
  • Robert W. Platt

      Affiliations

    • Department of Pediatrics and of Epidemiology and Biostatistics, Montreal Children's Hospital Research Institute, 4060 Ste Catherine Street West, #205, Westmount, QC H3Z 2Z3 Canada
    • ,
  • Judith Ritchie

      Affiliations

    • McGill University Health Centre, Room D6-156, MGH site, 1650 Cedar Avenue, Montreal Quebec H3G 1A4 Canada

,Accepted 12 December 2005.

References 

  1. Robin ED, Burke CM. Peer review in medical journals. Chest. 1987;91(2):252–257
  2. Swift M. Innovative research and NIH grant review. J NIH Res. 1996;8:18–20
  3. Horrobin DF. Referees and research administrators: barriers to scientific research?. Br Med J. 1974;2(912):216–218
  4. Resch K, Ernst E, Garrow J. A randomized controlled study of reviewer bias against an unconventional therapy. J R Soc Med. 2000;93:164–167
  5. Anonymous. Bad peer reviewers: small proportions of referees are undermining the scientific process, especially in biology. Some of the problems are getting worse, partly because of changes in scientific publishing. Nature. 2001;413:93
  6. Greenberg DS. Another step towards reshaping peer review at the NIH. Lancet. 1999;354(9178):577
  7. Horrobin DF. Peer review of grant applications: a harbinger for mediocity in clinical research. Lancet. 1996;34B:1293–1295
  8. Horrobin DF. The philosophical basis of peer review and the suppression of innovation. JAMA. 1990;263(10):1438–1441
  9. Grant D. Peer review is a two way process. Nature. 1997;388:822
  10. Stehben WE. Basic philosophy and concepts underlying scientific peer review. Med Hypotheses. 1999;52(1):31–36
  11. Zuckerman HMR. Patterns of evaluation in science. Minerva. 1971;9:66–100
  12. Weber EJ, Katz PP, Waeckerle JF, Callaham ML. Author perception of peer review: impact of review quality and acceptance on satisfaction. JAMA. 2002;287(21):2790–2793
  13. Sweitzer BJ, Cullen DJ. How well does a journal's peer review process function? A survey of authors' opinions. JAMA. 1994;272(2):152–153
  14. Garfunkel JM, Lawson EE, Hamrick HJ, Ulshen MH. Effect of acceptance or rejection on the author's evaluation of peer review of medical manuscripts. JAMA. 1990;263(10):1376–1378
  15. Pupique M. Peer review my foot!. J Biol Rhythms. 2002;17(2):194
  16. Brenner S. Moron peer review. Curr Biol. 1999;9(20):R755
  17. Horrobin DF. Something rotten at the core of science?. Trends Pharmacol Sci. 2001;22(2):51–52
  18. Kronick DA. Peer review in 18th-century scientific journalism. JAMA. 1990;263(10):1321–1322
  19. Glantz SA, Bero LA. Inappropriate and appropriate selection of “peers” in grant review. JAMA. 1994;272(2):114–116
  20. Vincent PC. Galileo's peers. Pathology. 2000;32(3):165
  21. Judson HF. Structural transformations of the sciences and the end of peer review. JAMA. 1994;272(2):92–94
  22. Wessely S. Peer review of grant applications: what do we know?. Lancet. 1998;352(9124):301–305
  23. Godlee F, Jefferson T. Peer review in health sciences. London, England: BMJ Publishing Group; 1999;
  24. Katz D. Who reviews the peer review? Mackinac Center for Public Policy; Midland, MI: 2002.
  25. Jefferson TO, Alderson P, Davidoff F, Wager E. Editorial peer-review for improving the quality of reports of biomedical studies. (Cochrane Methodology Review). The Cochrane Library 2004;1.
  26. Gilbert JR, Williams ES, Lundberg GD. Is there gender bias in JAMA's peer review process?. JAMA. 1994;272(2):139–142
  27. Wenneras C, Wold A. Nepotism and sexism in peer-review. Nature. 1997;387(6631):341–343
  28. Policy Research in Science and Medicine (PRISM) . Women and peer review: an audit of the wellcome trust's decision-making on grants. 8th ed.. London NW: The Wellcome Trust; 1997;
  29. Grant J, Burden S, Breen G. No evidence of sexism in peer review. Nature. 1997;390(438):
  30. Cole S, Cole JR, Simon GA. Chance and consensus in peer review. Science. 1981;214(4523):881–886
  31. Hodgson C. How reliable is peer review? An examination of operating grant proposals simultaneously submitted to two similar peer review systems. J Clin Epidemiol. 1997;50(11):1189–1195
  32. Demichell V, Pietrantonj C. Peer review for improving the quality of grant applications. (Cochrane Methodology Review). The Cochrane Library 2004;1.
  33. NIH. Peer Review at the NIH. www.csr.nih.gov/REVIEW/peerrev.htm. 2004.
  34. MRC. Peer review in Great Britain. www.mrc.ac.uk/index/funding/funding-specific_schemes/funding-assessment_process.htm. 2004.
  35. CIHR. Peer review in Canada. http://www.cihr-irsc.gc.ca/e/services/4656.shtml. 2004.
  36. Hodgson C. Evaluation of cardiovascular grant-in-aid applications by peer-review—influence of internal and external reviewers and committees. Can J Cardiol. 1995;11(10):864–868
  37. Thorngate W, Faregh N, Young M. Mining the archives: analyses of CIHR research grant adjudications. Ottawa, Ontario: Carleton University; 2002;
  38. Henneberg M. Peer review: the holy office of modern science. Natural Sci. 1997;1(2):
  39. Nunnaly JC. sychometric theory. 2nd ed.. New York: McGraw-Hill Book Company; 2004;
  40. Kassirer JP, Campion EW. Peer review. Crude and understudied, but indispensable. JAMA. 1994;272(2):96–97
  41. Bailar JC, Patterson K. The need for a research agenda. N Engl J Med. 1985;312(10):654–657

PII: S0895-4356(06)00005-9

doi: 10.1016/j.jclinepi.2005.12.007

Journal of Clinical Epidemiology
Volume 59, Issue 8 , Pages 842-848 , August 2006

Article Tools

* Image Usage & Resolution