Advertisement
Covid-19 Article| Volume 125, P164-169, September 2020

Pandemics and methodological developments in epidemiology history

  • Alfredo Morabia
    Correspondence
    Corresponding author. Barry Commoner Center, Queens College, City University of NY Remsen Hall, Room 311, 65-30 Kissena Blvd, Flushing NY 11365, USA. Tel.: +17186704109.
    Affiliations
    Barry Commoner Center for Health and the Environment, Queens College, City University of New York, Flushing, NY, USA
    Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA
    Search for articles by this author

      Highlights

      • Catastrophic pandemics since the 17th century appear to have spurred innovative methods, concepts, and institutions in epidemiology.
      • The plague, cholera, tuberculosis, influenza, and HIV/AIDS left a longstanding imprint on epidemiologic methods and concepts.
      • Pandemics may generate an urgent need for methods that overcome the inadequacy of older methods.
      • COVID-19–specific contribution could be a greater understanding of population thinking beyond academic and professional circles.

      Abstract

      The crisis spurred by the pandemic of COVID-19 has revealed weaknesses in our epidemiologic methodologic corpus, which scientists are struggling to compensate. This article explores whether this phenomenon is characteristic of pandemics or not. Since the emergence of population-based sciences in the 17th century, we can observe close temporal correlations between the plague and the discovery of population thinking, cholera and population-based group comparisons, tuberculosis and the formalization of cohort studies, the 1918 Great Influenza and the creation of an academic epidemiologic counterpart to the public health service, the HIV/AIDS epidemic, and the formalization of causal inference concepts. The COVID-19 pandemic seems to have promoted the widespread understanding of population thinking both with respect to ways of flattening an epidemic curve and the societal bases of health inequities. If the latter proves true, it will support my hypothesis that pandemics did accelerate profound changes in epidemiologic methods and concepts.

      Keywords

      What is new?

        Key findings

      • This brief retrospective of catastrophic pandemics since the 17th century suggests that pandemics did accelerate profound changes in epidemiologic methods and concept.

        What this adds to what was known?

      • The possible impact of pandemics on epidemiologic methods and concept has not been explored as systematically in the past.

        What is the implications and what should change now?

      • The public at large may have a better understanding of epidemiologic findings if the covid-19 pandemic has promoted the widespread understanding of population thinking both with respect to ways of flattening an epidemic curve as well as the societal bases of health inequities.
      Is there a causal link between pandemics and methodological developments in epidemiology? This article explores whether pandemics of the past have been accompanied by major advances in epidemiologic research methods. Pandemics are particular because they threaten the established order, they challenge our way of life and of exerting power, and they appear as a failure of human societies against nature. They create a sense of urgency to find solutions to problems that older methods failed to handle properly. Thus, it is reasonable to wonder whether they also represent watersheds, discrete moments in the refinement and improvement of the methods used in a discipline which, etymologically, is the science of epidemics.
      Assuming, as I have proposed elsewhere [
      • Morabia A.
      Enigmas of health and disease: how epidemiology helps unravel scientific mysteries.
      ,
      • Morabia A.
      Epidemiology's 350th anniversary: 1662-2012.
      ,
      • Morabia A.
      Epidemiology: an epistemological perspective.
      ], that there is no epidemiologic science before the end of the 17th century, this is a brief retrospective of catastrophic pandemics since the 17th century, that is, the plague in the 17th century, cholera in the 19th century, tuberculosis and influenza in the early 20th century, and HIV/AIDS in the 1980s.

      1. Plague

      Recurrent plague outbreaks between the 14th and the 17th centuries disrupted urban life and unsettled civic powers. In London, the pandemics of plague progressively led to the systematic collection of death counts [
      • Heitman K.
      Authority, Autonomy and the first London bills of mortality.
      ]. Quantitative plague reports were introduced in 1518 and expanded to other individual causes of death in 1554 and 1555. Serious episodes of plague in 1592 and 1625 were associated with the first broadside Bills and the regular, weekly handbills with parish-by-parish counts of plague deaths. At some points, these Bills of Mortality allowed for, as shown in Fig. 1, an organized evacuation of the city when the persistent increase in plague deaths in some parishes indicated that a new outbreak was unfolding.
      Figure thumbnail gr1
      Fig. 1Scenes in London during the plague of 1665. Facsimile reproduction from a pictorial broadside of 1665–1666. Some of the scenes show an ordered retreat away from London by water and by land. Wellcome in Creative Commons.
      When John Graunt analyzed this continuous series, week after week, for several decades of mortality data, he saw, as expected, that plague deaths waved across time, but he also discovered a phenomenon never reported before: deaths from some causes other than the plague occurred with great constancy [
      • Graunt J.
      Observations Upon the Bills of Mortality. Fifth Edition of 1676.1256.
      ]. For instance, the number of deaths from “Consumption and Cough,” that is, mostly tuberculosis, that would occur any year in London could be predicted from the number of deaths observed during the previous year: about 2000. Graunt discovered that occurrence of events in populations is predictable and potentially comparable and can therefore be a source of scientific knowledge. The properties of events in populations could not even have been suspected before the 17th century because there were no data to observe them. A comparative health science had remained elusive because the life course of individuals, the only observation available to physicians, is unpredictable and incomparable. Thus, the surveillance system built up to restrain the disruptive consequences of the plague, the catastrophic pandemic of the time, appears to have provided the foundation to the most decisive methodological advance in the history of social and human sciences: the discovery of population thinking.

      2. Cholera

      Before the 19th century, cholera had been endemic in certain parts of Asia but remained localized until faster means of transportation and routes made it feasible for sick travelers to survive the trip from, say Bombay, India, to Marseille, France, or from Rotterdam, Netherlands, to New York or Cartagena, Colombia. Since then, cholera spread in waves or pandemics. The second pandemic (1826–1837) reached Europe, North Africa, and North America. The third (1841–59) also reached Central and South America. These pandemics were scary. During the 1831–1832 pandemic, the case fatality ratio in Vienna, Austria, was 49.02% of 4,360 cases [
      • Farr W.
      On prognosis.
      ]. In an official report about the 1832 cholera pandemic in Paris and its surroundings, 48.6% of 4,907 medically certified cholera deaths of all ages occurred in the first 24 hours [
      • Farr W.
      On prognosis.
      ] ([
      Commission nommée par MM. les préfets de la Seine et de police.
      ], p.70). It was speculated that the stench emanating from rotten material caused cholera if inhaled by susceptible persons. Susceptibility was an individual trait, and there was no way of proving it existed. However, John Snow believed there was some kind of still unobserved living thing that could multiply in the water supply and that caused the cholera syndrome when ingested from water or food or from sleeping in infected linen. He conducted a well-known survey—different from his investigation of the outbreak related to the water pump in Broad Street—that allowed him to compare Londoners residing in the same parishes, equally exposed to stench and air pollution, but drinking water of different degrees of cleanliness. Leaden pipe belonging to private companies conveyed water from sections of the Thames, one contaminated and one not, to tanks and then from these tanks to pumps situated in the individual houses ([
      • Snow J.
      ], p. 16, 46–47). Thus, the pandemics of cholera, the greatest pandemic of the 19th century, stimulated the first use at a population scale of a comparative study design to explore a causal association. Group comparison has become the other defining trait of epidemiology besides population thinking.

      3. Tuberculosis

      Tuberculosis has affected human populations probably since the Neolithic, as the Bacillus is thought to have adapted to humans from domesticated cattle in dwellings shared promiscuously by humans and animals [
      • McNeill W.H.
      Plagues and peoples.
      ]. In 1900, many in public health believed that susceptibility to tuberculosis was hereditary. Persons with tuberculosis in 1900 affluent Western societies did not die consumed by their disease anymore as had been the case until the first half of the 19th century. They could have offspring. Eugenicists feared that, at some point, every human being would become susceptible to tuberculosis, leading to the extinction of the human species. The perception of tuberculosis then was analogous to a pandemic. Of all diseases, it had become the main killer. However, no methods existed to study whether persons with tuberculosis had a survival advantage: the latency between infection and diagnosis was long, and the cross-sectional surveys that had been used in the 19th century, by John Snow, for example, were biased because they comprised only selected survivors. Precisely for this reason, the German physician, Wilhelm Weinberg, invented a new design, referred to today as a cohort study, never used as neatly and at this scale before. Using population registries of Stuttgart, in Germany, he followed up 25,786 children from birth to the age of 20 years, comparing those from families in which a parent had died from tuberculosis with children from families in which the parent had not died from tuberculosis [
      • Morabia A.
      • Guthold R.
      Wilhelm Weinberg's 1913 large retrospective cohort study: a rediscovery.
      ]. Children from tuberculous parents lived shorter lives. Tuberculosis conferred no selection advantage and did not threaten the human species with extinction. Weinberg's report is equivalent to a textbook of epidemiology, which described in detail the new design and analytic methods of his study. His review of hundreds of thousands of records, individual follow-up of dozens of children, and complex computations without a calculator was a huge endeavor unlikely to have been deemed worthwhile if it were not for the hope of an enormously important outcome. Comparative cohort studies will turn out to be a key determinant for the achievements of the subsequent chronic disease epidemiologic research.

      4. Influenza 1918

      Until 1918, public health was overseen by health officers and civil servants. In April 1918, Wade Hampton Frost was still working for the U.S. Public Health Service when Surgeon General Rupert Blue made him responsible for his Influenza Task Force. Frost conducted a massive epidemiological house-to-house survey in 18 U.S. communities and established the prevalence of infection in the population in the fall of 1918. Contemporaneously, the Johns Hopkins University School of Hygiene and Public Health (SHPH) admitted its first students on October 1, 1918. William Henry Welch, who founded the SHPH, approached Frost in the summer of 1919 and made him Chair of the newly formed epidemiology department in September 1919. This was the first such department ever. Frost later became the first “Professor of epidemiology” in the United States. I am not aware of documents establishing the connection between the pandemic and the academization of epidemiology, and, as an epidemiologist, Frost is primarily remembered for his work on tuberculosis, and not for his work on influenza. Nonetheless, both Welch and Frost had been actively involved in the response to the 1918 flu and for the years 1919–1920 all of Wade Hampton Frost's publications [
      • Frost W.
      The epidemiology of influenza.
      ,
      • Frost W.
      The epidemiology of influenza.
      ,
      • Frost W.
      • Sydenstricker E.
      Epidemic influenza in foreign countries.
      ,
      • Frost W.
      • Sydenstricker E.
      Influenza in Maryland. Preliminary statistics of certain localities.
      ,
      • Frost W.
      Statistics of influenza morbidity. With special reference to certain factors in case incidence and case fatality.
      ] are about influenza. Thus, with hindsight, we can appreciate that the 1918 pandemic may have spurred the emergence of an academic parallel to the public service dedicated to public health and epidemiology. The later development of an epidemiologic corpus of methods and concepts mostly came from these schools of public health and academic epidemiology [
      • Morabia A.
      Has epidemiology become Infatuated with methods? A historical perspective on the place of methods during the classical (1945-1965) phase of epidemiology.
      ].

      5. HIV-AIDS

      The HIV pandemic unsettled epidemiology at first because epidemiologists had become used in the 20th century to studying chronic diseases and were taken by surprise by this resurgence of infectious scourges. Since World War II, the epidemiologic corpus of study designs had been substantially refined but the tools to study causes of disease were restricted to fixed exposures and outcomes well defined in time (e.g., to tobacco and lung cancer) [
      • Hill A.B.
      The environment and disease: association or causation?.
      ]. The epidemiologic usage of the causal concepts of confounding, interaction, and mediation was still in their infancy [
      • Morabia A.
      History of the modern epidemiological concept of confounding.
      ]. When the first cases of Kaposi's sarcoma were reported in 1981, epidemiologists focused on noninfectious exposures such as recreational drugs, which fitted the tobacco-lung cancer model [
      • Vandenbroucke J.P.
      • Pardoel V.P.
      An autopsy of epidemiologic methods: the case of "poppers" in the early epidemic of the acquired immunodeficiency syndrome (AIDS).
      ,
      • Marmor M.
      • Friedman-Kien A.E.
      • Laubenstein L.
      • Byrum R.D.
      • William D.C.
      • D'Onofrio S.
      • et al.
      Risk factors for Kaposi's sarcoma in homosexual men.
      ]. The viral nature of the disease was established by virologists in laboratories [
      • Barré-Sinoussi F.
      • Chermann J.C.
      • Rey F.
      • Nugeyre M.T.
      • Chamaret S.
      • Gruest J.
      • et al.
      Isolation of a T-lymphotropic retrovirus from a patient at risk for acquired immune deficiency syndrome (AIDS).
      ]. A major question from then on was the assessment of the efficacy of HIV/AIDS time-varying treatments. Contemporaneously, a fresh approach to causality, now known as the potential outcome framework and the corresponding causal models, was occurring, inspired by work in the analysis of randomized controlled trials [
      • Splawa-Neyman J.
      On the application of probability theory to agricultural experiments. Essay on principles. Section 9 (1923).
      ,
      • Copas J.B.
      Randomization models for the matched and unmatched 2x2 tables.
      ,
      • Rubin D.B.
      Estimating causal effects of treatments in randomized and nonrandomized treatments.
      ]. The first application of these developments in epidemiology was unrelated to the HIV/AIDS epidemic [
      • Greenland S.
      • Robins J.M.
      Identifiability, exchangeability, and epidemiological confounding.
      ]. They looked appropriate to handle the “healthy worker effect,” an up-to-then intractable methodological issue in occupational epidemiology: when studying the effect of an occupational exposure on a disabling illness, sick workers, who terminate employment early, are at an increased risk of death although no further exposed, resulting in a biased assessment of the association among the healthier workers remaining in the study. These causal models might have remained of limited usage in epidemiology for some time if it had not been shown that they were also appropriate for estimating the effect of time-varying treatment on the time to clinical AIDS among HIV-infected subjects: the fact that sicker patients were more likely to both be treated and die, an issue referred to as treatment by indication, generated selection and confounding issues analogous to those of the healthy worker effect [
      • Robins J.
      A new approach to causal inference in mortality studies with sustained exposure periods-application to control of the healthy worker survivor effect.
      ]. The watershed happened with James Robins's application of “causal models” to “the Analysis of Randomized and Nonrandomized AIDS Treatment Trials Using A New Approach to Causal Inference in Longitudinal Studies” [
      • Robins J.
      The analysis of randomized and nonrandomized AIDS treatment trials using A new approach to causal inference in longitudinal studies.
      ]. These methods are now used routinely in health and social science research, including epidemiology, and are textbook matter [
      • Hernan M.
      • Robins J.M.
      Causal inference: what if?.
      ]. Thus, the deadly HIV/AIDS pandemic appears to have accelerated the integration of a new conceptual framework that resulted in the refinement of causality-related concepts in epidemiology and more generally in human and social sciences.

      6. Discussion

      The cases of the plague, cholera, tuberculosis, influenza, and HIV/AIDS seem to indicate that these pandemics have accelerated major methodological developments in epidemiology. These changes would have occurred but at a later time and differently. However, one may argue that pandemics and development of methods are in reality two separate, independent narratives that, by chance, appear to be chronologically associated.
      For example, the recollection of specific methodological developments may, in retrospect, make us associate them with specific pandemics that occurred at about the same time. However, this type of reverse causality would not be true for the 1918 influenza pandemic. The great influenza long appeared, as Major Greenwood put it, “epidemiology's crux” because of the lack of major work or influential epidemiologic work that emerged from it ([
      • Greenwood M.
      Epidemics and crowd diseases: introduction to the study of epidemiology.
      ], p.137). I am not aware of anyone connecting the creation of the school of public health at The Johns Hopkins University to the 1918 influenza [
      • Fee E.
      Disease & discovery. A history of the johns hopkins school of hygiene and public health 1916-1939.
      ], but today, with hindsight, the connection seems very likely, as I have tried to show previously, and its historical implications were huge: the creation of an academic arm of epidemiology established the foundations for the later development of a corpus of formal and rigorous methods for the conduct of epidemiologic research.
      In favor of the causal link is the difficulty of finding counterexamples of minor epidemics that led to methodological breakthroughs of the same magnitude as those described previously for pandemics. In the 18th century, the Royal Society innovated in designing a comparative study of the mortality associated with natural smallpox infection and that provoked by variolation [
      • Boylston A.W.
      Defying providence. Smallpox and the forgotten 18th century medical revolution.
      ]. It involved a subtle analysis of the bills of mortality and an intercontinental physician survey. Natural smallpox was ten times more lethal than variolation, a finding leading to a wave of smallpox inoculation, at least in the United Kingdom. In his 1747 nonrandomized trial on board, a naval warship doctor James Lind allegedly compared 6 treatments, a single one of which (citrus fruits) was effective in curing scurvy [
      • Lind J.
      A treatise of scurvy.
      ]. But these 18th century methodological breakthroughs were minor compared with the discovery of population thinking which gave birth to all population-based sciences we know today, such as demography, statistics, sociology, evolutionary genetics, and so on. In the 19th century, the researches of Pierre-Charles-Alexandre Louis [
      • Louis P.C.A.
      Researches on the effects of bloodletting in some inflammatory diseases, and on the influence of tartarized antimony and vesication in pneumonitis.
      ] and Ignaz Semmelweis [
      • Semmelweis I.P.
      Die Aetiologie, der Begriff und die Prophylaxis des Kindbettfiebers [The etiology, concept, and prophylaxis of childbed fever].
      ] were innovative in their own ways, but their long-term methodological legacy is not comparable to that of Snow. In the 20th century, the refinement of the case-control study design was induced by etiologic research on cancer [
      • Lane-Claypon J.
      A further report on cancer of the breast, with special reference to its associated antecedent conditions.
      ] and cardiovascular diseases [
      • English J.P.
      • Willius F.A.
      • Berkson J.
      Tobacco and coronary disease.
      ], which were on the rise and progressively replacing acute infectious diseases and tuberculosis as public health priorities. The epidemic of pellagra also led to methodologically innovative work by Joseph Goldberger [
      • Mooney S.J.
      • Knox J.
      • Morabia A.
      The Thompson-McFadden Commission and Joseph Goldberger: contrasting 2 historical investigations of pellagra in cotton mill villages in South Carolina.
      ]. But none of this work rivals the contribution of Wilhelm Weinberg [
      • Weinberg W.
      Die Kinder der Tuberkuloesen.
      ], which opened the way to the study of chronic disorders, including the elucidation of the health effects of smoking [
      • Doll R.
      • Hill A.B.
      Smoking and carcinoma of the lung; Preliminary report..
      ], by rigorously establishing the methodological basis of cohort studies and survival analysis. Finally, no disease or epidemic has shaken the conceptual foundations of causal inference in epidemiology as much as the HIV/AIDS epidemic.
      It may be that the critical conditions of a pandemic stimulate an urgent fresh look at older methods that failed to prevent the disasters. Above all, the common trait of pandemics is the unusual, immediate, surrounding, unavoidable presence of death, in neighbors, relatives, and friends. Similar situations did not occur in other epidemics, such as smallpox in the 18th century, any of the acute infectious diseases such as diphtheria, measles, and so on in the 19th century [
      • Hardy A.
      The Epidemic streets. Infectious disease and the rise of preventive medicine. 1856-1900.
      ], or the cancer and cardiovascular epidemics of the 20th century. These were all real epidemics, with their toll of deaths and suffering, but they did not halt movement in society and make people urgently reflect on the limits and the dead ends of the available methods.
      However, my argument would be weakened if one could argue that the link with pandemics is not specific and that innovative methods and concepts in epidemiology have followed other types of happenings. My thesis could also be disproved by COVID-19, if the pandemic does not shatter an important dimension of epidemiologic methods. I would speculate that its consequences will be comparable to those of the 1918 influenza. We have now schools of public health all over the world, and academic epidemiology is thriving. Still, population thinking remains esoteric outside of academia and public health. This may be changing. Never in the past have we collectively considered what an epidemic curve is, what herd immunity is, and how we can act together to affect the progression of the epidemic. Similarly, Covid-19 may have accelerated the understanding that health inequities can be better studied and corrected at a population rather than at an individual level. Population thinking may become mainstream and, if it does, this will have huge consequences on the society we will build when the ordeal is over.

      Acknowledgments

      The author is grateful to Kristin Heitman, Iain Donaldson, Daniel Fox, Michael C. Costanza, and Jan Vandenbroucke for comments on an earlier version of this commentary.

      References

        • Morabia A.
        Enigmas of health and disease: how epidemiology helps unravel scientific mysteries.
        Columbia University Press, New York, NY2014: 296
        • Morabia A.
        Epidemiology's 350th anniversary: 1662-2012.
        Epidemiology. 2013; 24: 179-183
        • Morabia A.
        Epidemiology: an epistemological perspective.
        in: Morabia A. History of epidemiological methods and concepts. Birkhaeuser, Basel2004: 1-126
        • Heitman K.
        Authority, Autonomy and the first London bills of mortality.
        Centaurus. 2020; 62https://doi.org/10.1111/1600-0498.12305
        • Graunt J.
        Observations Upon the Bills of Mortality. Fifth Edition of 1676.1256.
        in: Hull V.H. The Economic Writings of Sir William Petty. Cambridge University Press, Cambridge, UK1899: 313-435
        • Farr W.
        On prognosis.
        in: Morabia A. History of epidemiological methods and concepts, 2004. Supplement Part 1 (pages 199-208). Birkh,user, Basel1838: 159-178
      1. Commission nommée par MM. les préfets de la Seine et de police.
        Rapport sur la marche et les effets du choléra-morbus dans Paris et les communes rurales du département de la Seine: année 1832. Imprimerie Royale, Paris1834
        • Snow J.
        Delta O. On the mode of communication of cholera. Churchill, London1855
        • McNeill W.H.
        Plagues and peoples.
        Anchor Books, New York, NY1977
        • Morabia A.
        • Guthold R.
        Wilhelm Weinberg's 1913 large retrospective cohort study: a rediscovery.
        Am J Epidemiol. 2007; 165: 727-733
        • Frost W.
        The epidemiology of influenza.
        Public Health Rep. 1919; 34: 1823-1836
        • Frost W.
        The epidemiology of influenza.
        JAMA. 1919; 73: 313-318
        • Frost W.
        • Sydenstricker E.
        Epidemic influenza in foreign countries.
        Public Health Rep. 1919; 34: 1361-1376
        • Frost W.
        • Sydenstricker E.
        Influenza in Maryland. Preliminary statistics of certain localities.
        Public Health Rep. 1919; 34: 491-504
        • Frost W.
        Statistics of influenza morbidity. With special reference to certain factors in case incidence and case fatality.
        Public Health Rep. 1920; 35: 584-597
        • Morabia A.
        Has epidemiology become Infatuated with methods? A historical perspective on the place of methods during the classical (1945-1965) phase of epidemiology.
        Annu Rev Public Health. 2015; 36: 69-88
        • Hill A.B.
        The environment and disease: association or causation?.
        Proc R Soc Med. 1965; 58: 295-300
        • Morabia A.
        History of the modern epidemiological concept of confounding.
        J Epidemiol Community Health. 2011; 65: 297-300
        • Vandenbroucke J.P.
        • Pardoel V.P.
        An autopsy of epidemiologic methods: the case of "poppers" in the early epidemic of the acquired immunodeficiency syndrome (AIDS).
        Am J Epidemiol. 1989; 129: 455-457
        • Marmor M.
        • Friedman-Kien A.E.
        • Laubenstein L.
        • Byrum R.D.
        • William D.C.
        • D'Onofrio S.
        • et al.
        Risk factors for Kaposi's sarcoma in homosexual men.
        Lancet. 1982; 1: 1083-1087
        • Barré-Sinoussi F.
        • Chermann J.C.
        • Rey F.
        • Nugeyre M.T.
        • Chamaret S.
        • Gruest J.
        • et al.
        Isolation of a T-lymphotropic retrovirus from a patient at risk for acquired immune deficiency syndrome (AIDS).
        Science. 1983; 220: 868-871
        • Splawa-Neyman J.
        On the application of probability theory to agricultural experiments. Essay on principles. Section 9 (1923).
        Stat Sci. 1990; 5: 465-480
        • Copas J.B.
        Randomization models for the matched and unmatched 2x2 tables.
        Biometrika. 1973; 60: 467-476
        • Rubin D.B.
        Estimating causal effects of treatments in randomized and nonrandomized treatments.
        J Educ Psychol. 1974; 66: 688-701
        • Greenland S.
        • Robins J.M.
        Identifiability, exchangeability, and epidemiological confounding.
        Int J Epidemiol. 1986; 15: 413-419
        • Robins J.
        A new approach to causal inference in mortality studies with sustained exposure periods-application to control of the healthy worker survivor effect.
        Math Model. 1986; 7: 1393-1512
        • Robins J.
        The analysis of randomized and nonrandomized AIDS treatment trials using A new approach to causal inference in longitudinal studies.
        in: Sechrest L. Freeman H. Mulley A. Health service research methodology: a focus on AIDS. U. S. Public Health Service, National Center for Health Services Research, Washington DC1989: 113-159
        • Hernan M.
        • Robins J.M.
        Causal inference: what if?.
        Chapman & Hall/CRC, Boca Raton, FL2020
        • Greenwood M.
        Epidemics and crowd diseases: introduction to the study of epidemiology.
        Ayer Company Publishers, Incorporated, North Stratford, NH1935
        • Fee E.
        Disease & discovery. A history of the johns hopkins school of hygiene and public health 1916-1939.
        Johns Hopkins University Press, Baltimore, MD1987
        • Boylston A.W.
        Defying providence. Smallpox and the forgotten 18th century medical revolution.
        CreateSpace Independent Publishing Platform, North Charleston, SC2012
        • Lind J.
        A treatise of scurvy.
        University Press, Edinburgh1753
        • Louis P.C.A.
        Researches on the effects of bloodletting in some inflammatory diseases, and on the influence of tartarized antimony and vesication in pneumonitis.
        Hilliard, Gray, Boston1836
        • Semmelweis I.P.
        Die Aetiologie, der Begriff und die Prophylaxis des Kindbettfiebers [The etiology, concept, and prophylaxis of childbed fever].
        Hartleben's Verlags-Expedition, Wien1861
        • Lane-Claypon J.
        A further report on cancer of the breast, with special reference to its associated antecedent conditions.
        His Majesty's Stationary Office, London, UK1926
        • English J.P.
        • Willius F.A.
        • Berkson J.
        Tobacco and coronary disease.
        J Am Med Assoc. 1940; 115: 1327-1328
        • Mooney S.J.
        • Knox J.
        • Morabia A.
        The Thompson-McFadden Commission and Joseph Goldberger: contrasting 2 historical investigations of pellagra in cotton mill villages in South Carolina.
        Am J Epidemiol. 2014; 180: 235-244
        • Weinberg W.
        Die Kinder der Tuberkuloesen.
        S Hirzel, Leipzig1913
        • Doll R.
        • Hill A.B.
        Smoking and carcinoma of the lung; Preliminary report..
        Br J Cancer. 1950; 2: 739-748
        • Hardy A.
        The Epidemic streets. Infectious disease and the rise of preventive medicine. 1856-1900.
        Clarendon Press, Oxford, UK1993