A new method for assessing drug causation provided agreement with experts' judgment

Article Outline

• Abstract
• 1. Introduction
• 2. Materials and methods
  • 2.1. Data collection
  • 2.2. Statistical method
  • 2.3. Coherence of the method
• 3. Results
  • 3.1. Descriptive analysis of data
  • 3.2. Weighting of criteria
  • 3.3. Coherence of the method
• 4. Discussion
  • 4.1. Main results
  • 4.2. Validity of results
    • 4.2.1. Data collection
    • 4.2.2. Judgment bias
    • 4.2.3. Calibration
    • 4.2.4. Coherence of new method
• 5. Conclusions
• Acknowledgments
• Appendix. Case reference report
• References
• Copyright

Abstract 

Background and Objective

The many methods proposed for causality assessment of adverse drug reaction (ADR) generally rely on algorithms. They have no clear relationship to probabilities, however, a situation we attempted to improve.

Study Design and Setting

Thirty ADR cases corresponding to 32 suspect drugs were randomly selected from the French pharmacovigilance database. The statistical weighting was performed by using a multilinear regression with logit(p) as the dependent variable and seven judgment criteria as independent variables. The best model (i.e., giving the best correlation with the gold standard) was retained for the new causality assessment method.

Results

The weights [logit(p)] for the 21 choices, on average three for each of the seven criteria, ranged from −3.95 to 0.86, secondarily rounded to multiples of 0.5. The correlation between the probability obtained from the final method and the gold standard was quite good (R² = .92).

Conclusion

This method based on the rational weighting of seven causality criteria is straightforward to use and provides very good agreement with experts' judgment. Moreover, unlike most classical algorithms, it respects one basic rule of probabilities—namely, a symmetrical probability distribution for drug causation around the .5 neutral position (maximum uncertainty).

Keywords: Drug safety, Epidemiology, Causality, Adverse effects, Consensus, Methods

Back to Article Outline

1. Introduction 

About 20 methods have been proposed so far to assess the causal relationship between a drug treatment and the occurrence of an adverse event at the individual level (i.e., in a given patient). These belong to roughly three main categories: expert judgment, probabilistic approaches, and algorithms [1].

In expert judgment or global introspection, an expert expresses judgment about possible drug causation after taking into account all available and relevant data on the considered case. This approach, which is usually not standardized, is quite similar to clinical diagnosis. Therefore, expert judgment suffers the same main limitation (i.e., subjectivity), leading to poor reproducibility and to inter- and intrarater disagreements [2], [3], [4], [5], [6].

Almost all probabilistic approaches have been derived from Bayes' theorem. These have several indisputable advantages [7]: (i) setting the starting point (i.e., the prior probability or odds) according to the background information on the drug–event association being considered (e.g., incidence of the adverse event with and without treatment); (ii) taking into account any relevant information for the considered case (e.g., probability of a positive test result among drug-caused event population and among a non-drug-caused event population) by transforming it into a multiplying factor (the likelihood ratio), thereby making it possible (iii) to obtain a precise estimate of drug causation on a continuous scale ranging from 0 to 1 for probability, or from 0 to infinity for odds. Moreover, the process respects the basic rule of probability theory, which states that in the absence of any relevant information, one should obtain a neutral estimate (i.e., a probability of .5 or an odds of 1) [8]. Bayesian approaches are not well suited to routine use, however, because they require precisely quantified information to model probability distributions for each parameter—even if, in certain cases, assumptions can be made [1], [8].

By contrast, algorithmic approaches are widely used for the operational assessment of adverse drug reactions (ADRs) because of their appealing simplicity: successive evaluation of criteria (sum of scores) or decision trees. The main argument put forward for their use is the increase obtained in inter- and intrarater agreement [9], [10]; however, assessments produced by a given algorithm strongly depend upon the weight of each criterion, which is fixed more or less arbitrarily by the authors of the method [8], [11]. Few algorithms have been weighted on the basis of an ad hoc statistical approach and therefore [10]

Our objective here is to propose a new approach based on (i) the logistic function to model expert judgment and (ii) a scientific weighting using a multilinear regression.

Back to Article Outline

2. Materials and methods 

2.1. Data collection 

The basic principle of scientific weighting is reference to a gold standard. In the present case, a large set of ADR cases would have been required in which the probability of responsibility of each suspected drug was precisely known. Unfortunately, an operational and indisputable tool providing such an exact probability does not exist [10]. As stated above, a Bayesian approach could have been used for this purpose, but for our study it was not realistic, given the complexity of computing prior odds and likelihood ratios for each case. Moreover, it was likely that in many cases the information required would have been lacking, thereby running the risk of making subjective assumptions. Benichou et al. [12], [13] used an original approach, selecting only cases for which the certainty of drug or nondrug causation was attested by positive or negative rechallenges (the response when the drug is subsequently readministered). That approach presents the major drawback of weighting the method only for the two extreme zones of the probability scale. For these reasons, we used expert consensus, as in the design of some existing causality assessment methods [14], [15], [16]. Indeed, global introspection can be considered as an indisputable reference if, and only if, (i) it involves several experts having a wide experience of the problem considered, (ii) these experts are provided with clear and complete information on the case, and (iii) a step-by-step approach, such as the Delphi method, is used to resolve disagreements.

In order to start from the reality of routine pharmacovigilance practice, a sample of 30 ADR cases was randomly selected from the French pharmacovigilance database. For each suspect drug, the probability (p) of drug causation was assessed by a first group of five senior experts, from five pharmacovigilance centers (Group 1). By using global introspection, each expert was asked to express separately his or her judgment on the responsibility of each suspect drug on a 100-mm visual analog scale, secondarily converted into a probability of drug causation ranging from 0 to 1. These assessments were then discussed during a consensus meeting. The causes of discrepancies were discussed until a consensual agreement was obtained for each case on a final probability, which was then considered as the gold standard for drug causation for the given case.

A second group of five experts, two from the pharmaceutical industry and three from pharmacovigilance centers (Group 2), was asked to assess drug causation for the same set of case reports, and for each case to assess the seven criteria retained for the new method: (i) time to onset, (ii) dechallenge (the response when the drug is discontinued), (iii) rechallenge, (iv) search for non drug-related causes, (v) risk factors for drug reaction (underlying disease or state or the drug interaction will increase the responsibility of the drug whose effects or toxicity are very increased), (vi) reaction at site of application, or relevant and reliable laboratory test strongly in favor of the drug responsibility, and (vii) previous reports of similar drug–event associations and symptoms evocative of a drug causation.

For the sample of 30 observations, 39 drug–event pairs were assessed by the five experts of the group 2.

For each criterion assessment (39 × 7 = 273), we first retained those which were identical for at least three out of the five experts. When such an agreement was not obtained, a third group of five senior experts, from the Pharmacovigilance Center in Bordeaux (Group 3), was consulted about these contradictory options. Their opinions were then sent to the members of Group 2, and the Delphi method was used again until a consensus was reached.

To avoid repeated effects, we ruled out, for a given case, concomitant drugs having the same probability of drug causation and the same assessments for the seven criteria as the suspected drug; 32 observations were thus retained for the final analysis.

2.2. Statistical method 

The main originality of the proposed method consists in the use of the logistic function

to predict the probability p of drug causation from a set of variables X₁, X₂, …, X_j, the values of which result from the assessment of the j = 7 selected criteria. We did not consider the α constant, so as to obtain a probability p of .5 when no criterion was assessable or discriminant (i.e., when The weighting process consisted in tuning the values of the parameters β₁, β₂, …, β_j resulting, for the 32 drug–event pairs, in the best fit between the probability p produced by the model and the probability of drug causation expressed by the experts in Group 1.

For this purpose, we used a multilinear regression model derived from the logistic function

The criteria assessed by Group 2 were used as dummy variables and the values of β parameters were obtained with the least squares method.

For each criterion, the nondiscriminant option was used as the reference (e.g., “not attempted or not interpretable” for the criterion ‘rechallenge’).

All statistical analyses were done with SAS statistical software (release 8.01; SAS Institute, Cary, NC).

To make the method more straightforward to use, the values of β produced by the multilinear model were secondarily rounded as multiples of 0.5 (e.g., 0.5 for 0.41). Moreover, in the test sample, some situations were never or rarely met, making their statistical weighting impossible. This was the case of “against the role of drugs” for the criterion ‘dechallenge’ and of “negative” for the criterion ‘rechallenge.’ In these cases, the value opposite of the positive option for the criterion was arbitrarily given.

Finally, we tested the effect produced by increasing or decreasing each rounded value by an increment of 0.5. In the end of the process, we retained the model which gave the best correlation with the consensual judgment of the first group of experts.

2.3. Coherence of the method 

In the end, the final model was compared to the guideline proposed by the World Health Organization (WHO) [17] for assessing the probability of drug causation. This guideline considers four main operational criteria (time relationship to drug administration, dechallenge, rechallenge, and search for other explanations). By using the combinations of these criteria considered by the WHO to define four main degrees of causality (certain, probable or likely, possible, and unlikely), it was simple to derive the corresponding probabilities from the proposed method.

Back to Article Outline

3. Results 

3.1. Descriptive analysis of data 

The initial individual assessment of the 32 drug–event pairs by the five experts in the first group showed a relatively high degree of disagreement between experts: for only 7 cases (22%), the judgment expressed by the five experts was included in an interval of the 25 mm; the same was observed for four experts in 13 additional cases (41%), the fifth expressing a quite different judgment. In consequence, the results and reasons for disagreement were discussed during a one-day meeting until a probability of drug causation (p) was reached for each event–drug pair.

The probability distribution of these assessments (Fig. 1) shows that (i) clear-cut judgments (i.e., probabilities < .05 or > .95) were relatively scarce and (ii) the probability distribution was shifted toward the right (i.e., >.5) in 84% of cases.

• 
  • View Large Image
  • Download to PowerPoint

• Fig. 1. 

  Distribution of probabilities of drug causation in gold standard; 84% of probabilities were greater than .5.

Table 1 shows the distribution of assessments of each of the seven criteria and the corresponding gold standard probability of causation.

Table 1. Distribution of answers for each criterion in test sample of adverse drug reaction cases

aReaction type according to Rawlins and Thompson, 1977 [22].

3.2. Weighting of criteria 

In the multilinear regression analysis, most of the variability of logit(p) was explained by the final model (R² = .866) (Table 2).

Table 2. Crude results obtained from the multilinear regression on logit(p) and rounded weighting for the final model

aReaction type according to Rawlins and Thompson, 1977 [22].

Avoiding computation of the logistic model for each drug–event pair, Table 3 allows a quick transform of the sum of the scores of seven criteria into the final probability.

Table 3. Probabilities of drug causation corresponding to sum of scores (by logistic transform)^a

aProbability of drug causation p = 1/[1 + exp(−final score)].

The correlation coefficient of the probability calculated with the new method (Table 2, Table 3) and the gold standard was .86 (P < .001).

3.3. Coherence of the method 

Table 4 shows the coherence between the proposed method and the WHO guideline; certain corresponds to a probability of .95, probable to .88, possible to .62, and unlikely to .05.

Table 4. Correspondence between WHO guideline and new method

Back to Article Outline

4. Discussion 

4.1. Main results 

The development of a new causality assessment method could appear a challenge, given that ∼20 such have been designed since the pioneering work of N.S. Irey in 1974 [18]. These are mainly represented by operational algorithms that are much simpler to use, because they are by nature restricted to a rough scoring of drug causation and are devoted to routine practice. Most of them combine the answers to simple questions about temporal relationship, clinical and biological patterns, alternative etiologic candidates, and notoriety to categorize causality into four or five levels (e.g., not related, doubtful, possible, and probable). The method described here provides an original weighting process based on a statistical regression (logit-linear model). The method conserves the basic principle and simplicity of algorithms, but expresses the drug causation as a probability (see Appendix) that can vary in a quasi-continuous manner from .01 to .99.

When several drugs are prescribed simultaneously, the assessment is made independently for each one of them. Consequently, the sum of the individual probabilities obtained for each drug can exceed 1. To our knowledge, no method including the Bayesian approaches, addresses this problem satisfactorily.

In routine practice, this is not of major concern, because the method makes it possible to identify the most suspect drug (i.e., the one with the highest probability of causation).

The originality of this method stems from the use of the logistic model to transform the sum of the scores resulting from the assessment of the seven criteria into the final probability. The logistic model was chosen because it appears to mimic acceptably the basic process of an expert's judgment. The absence of any relevant information produces a neutral probability of .5. When the aggregation of the negative or positive evidence in favor of drug causality makes the probability tend towards 0 or 1, respectively, the increment becomes asymptotic as the two extreme zones are reached. The most critical part of any causality assessment method is to determine the weight of each criterion in the model, in order to produce the final assessment. The reliability of the final method depends highly upon this critical development phase. This weighting has to reproduce the implicit reasoning of an expert during causality assessment.

The basic hypothesis of the new method was that the probability (p) of drug causation can be predicted by the assessment of seven causality criteria by the logistic model. Multilinear regression using the logit(p) as dependent variable made it possible to weight the method by a rigorous statistical approach. Indeed, multilinear regression on logit(p) gave very good results in our study sample.

4.2. Validity of results 

4.2.1. Data collection 

The assessment profile (Table 1) of the 32 drug–event pairs (no case with negative rechallenge) shows that our study sample was randomly selected from routine spontaneous reports. Indeed, in this type of passive surveillance, it is expected that the prescribers report only cases for which the event seems a priori attributable to the drug. The weighting process was based on 32 drug–event pairs. This number may appear to be relatively small, because some situations were rarely or never met in this sample, thereby precluding their statistical weighting. In such cases, it was possible to test several arbitrary values and to retain those producing the best fit with the gold standard. The sample size was limited for an obvious practical reason; it would have been unrealistic to ask high-grade experts to work on a number of ADR cases significantly greater than 30. Per se, 32 drug–event pairs corresponded to 160 visual analog scales and to 1,120 criteria assessments.

The shift toward the right (84% being greater than 0.5) of probabilities given by group 1 was an expected result, because one prerequisite of the weighting process was to use actual case reports selected from routine practice in which a priori only suspect drug–event pairs were assessed.

4.2.2. Judgment bias 

Our gold standard was obtained from the consensus of five senior experts using global introspection, which largely depends upon the knowledge and experience of the selected experts. Those called upon for the study were highly experienced, with more than 10 years of daily practice in drug causality assessment using both global introspection and an operational algorithm, mainly that used in France [19], [20]. Although it is impossible to know whether the latter point influenced their judgment by making an implicit reference to this method, it is likely that the approach used (i.e., the Delphi method) and the balanced professional background of these experts minimized this potential bias.

4.2.3. Calibration 

Although the final weighting produced a very satisfactory fit with the gold standard, some of the figures in Table 2 may initially appear surprising. For example, in most of the existing methods, positive or negative rechallenge is per se a decisive criterion to validate or to rule out the existing causal role of the drug. Our statistical weighting based on only two cases produced a value of 0.41 for negative rechallenge subsequently rounded up to 0.5, which is one of the smaller weights in Table 2. A comparable result was found by Salamon and Peytour [21], who also used a statistical regression model to produce the weight of various criteria. They showed that rechallenge was a poorly discriminant criterion. In fact, rechallenge is by definition assessable only if the other chronological criteria (i.e., time to onset and dechallenge) are discriminant—that is, if the event occurs during a drug treatment and resolves after withdrawal or dose reduction. It is thus reasonable to consider that in our method a positive rechallenge leads per se to a final probability of .82 (sum of weights = 1.5): that is, compatible time to onset (weight = 0.5), suggestive dechallenge (weight = 0.5), and positive rechallenge (weight = 0.5). This probability rises to .88 (sum of weights = 2) in the case of a highly suggestive time to onset (weight = 1).

Table 2, Table 3 show that the sum of the weighting scores can vary between −4.5 and 4.5. This symmetry around 0 is satisfactory with regard to the rules of probability and to the logistic model used as reference (Fig. 2). The transformation of these extreme values by the logistic formula leads to probabilities of .01 and .99, respectively, which gives scope for a large set of values of drug causation probability.

• 
  • View Large Image
  • Download to PowerPoint

• Fig. 2. 

  Using the logistic function to obtain the probability of drug causation from the sum of scores.

In our method, the thresholds of .05 and .95, which are generally admitted in biostatistics to define what is unlikely or very likely, are obtained for final scores of −3 and 3, respectively. Unlike other previously published approaches, no single criterion alone can enable reaching these thresholds.

4.2.4. Coherence of new method 

Even if it has been scientifically designed, the final step of the development of a new diagnostic tool is to check its validity in a large set of cases for which the truth is definitely known. As stated in the Introduction, such references are extremely difficult to establish for drug causation. Even if the WHO guideline does not in itself constitute such a reference, our method showed good agreement with this operational diagnostic tool used worldwide (Table 4).

Back to Article Outline

5. Conclusions 

Although further developments could be made to improve this weighting tool for widespread routine use, the present method has several advantages: (i) it is based on the logistic function which offers rational modeling of expert judgment; (ii) the weighting of the criteria was obtained by a statistical approach, with reference to a consensus of two groups of senior experts; and (iii), although straightforward to use, it directly provides a probability of drug causation ranging from .01 to .99.

The availability of this method in a computerized form makes the causality assessment easier. Thanks to these advantages, this new method should allow a better use of information in order to perform individual causality assessments in routine pharmacovigilance systems or in a framework of clinical trials, as well as a better interpretation of the causality assessment obtained (imputability given in the form of probability).

Back to Article Outline

Acknowledgments 

This study was funded as a research project by grants from the non-profit-making association ARME-Pharmacovigilance, the French Health Product Safety Agency (AFSSAPS) and the Direction Générale de la Santé (French Ministry of Health). We gratefully acknowledge the contributions of Jacques Caron (Centre régional de Pharmacovigilance, Lille), Georges Lagier (Centre régional de Pharmacovigilance, Paris Fernand Widal), Louis Merle (Centre régional de Pharmacovigilance, Limoges), Thierry Vial (Centre régional de Pharmacovigilance, Lyon), Rose Marie Chichmanian (Centre régional de Pharmacovigilance, Nice), Agnès Lillo-Le Louët (Centre régional de Pharmacovigilance, Paris, HEGP), Francis Wagniart (Institut de Recherche Internationales SERVIER [IRIS]), Gaby Danan (Aventis Pharma), Philippe Tréchot (Centre de Pharmacovigilance, Nancy), Rajaa Lagnoui (Centre régional de Pharmacovigilance, Bordeaux), and Caroline Roussillon (Centre régional de Pharmacovigilance, Bordeaux).

Back to Article Outline

Appendix. Case reference report 

An 86-year-old woman had a medical history of chronic lymphoid leukemia, allergy to amoxicillin, and hematuria attributed to aspirin 15 years ago. She was hospitalized from March 6 to 14 for acute myocardial infarction and treated with aspirin 250 mg/day, ramipril, furosemide, potassium chloride, and molsidomine.

On March 18, she reported a melena but said that the symptoms occurred when the therapy was begun. The etiologic search confirmed anemia (hemoglobin 8.3 g/dL) and a duodenal ulcer.

The issue is to determine if the occurrence of the melena is related to the aspirin.

Time to onset. The time to onset criterion refers to the temporal relationship existing between the drug administration and the occurrence of the adverse event.

Aspirin was started on the day of the event but prior to its occurrence. Therefore, the time to onset is considered compatible (score: 0.5).

Rechallenge and dechallenge. The rechallenge and dechallenge criteria are not relevant for this case; both are considered “not conclusive or not available” (score: 0 for each criterion).

Search for non-drug-related causes. Assessing the search for non-drug-related causes consists in reviewing the main possible nondrug causes of the adverse event. Another alternative could have been a gastric carcinoma; this was ruled out by the endoscopy which showed the presence of a duodenal ulcer. There is no reasonable alternative non-drug-related explanation for this adverse event. This criterion is quoted as “nondrug causes ruled out” (score: 1).

Risk factors for drug reaction. The occurrence of hematuria with aspirin 15 years ago and the age of this patient (86 years) are well-validated risk factors for melena with the same medication. This criterion is considered “well-validated and present” (score: 0.5).

Reaction at site of application, or relevant and reliable laboratory test strongly in favor of the drug responsibility. There is neither reaction at site of application nor validated laboratory test in favor of responsibility for the drug (score: 0).

Previous reports of similar drug–event associations and symptoms evocative of a drug causation. Melena is a well-known adverse effect of acetylsalicylic acid; moreover, it is an expected effect of the aspirin, owing to its pharmacological properties. This criterion is considered “labeled reaction and/or type A reaction” (score: 0.5).

Transform to final probability. The sum of the weighting scores is 2.5. Transformation of this value by the logistic formula leads to a probability of .92 (Table 3).

Back to Article Outline

References 

1. Kramer MS. Imputabilité des effets indésirables: individu (analyse du cas) versus groupe (épidémiologie). [in French] In: 3es entretiens Jacques Cartier. Lyon: Centre Jacques Cartier; 1989;p. 31–44
   • View In Article
2. Blanc S, Leuenberger P, Berger JP, Brooke EM, Schelling JL. Judgments of trained observers on adverse drug reactions. Clin Pharmacol Ther. 1979;25:493–498
   • View In Article
   • MEDLINE
3. Karch FE, Smith CL, Kerzner B, Mazzullo JM, Weintraub M, Lasagna L. Adverse drug reactions-a matter of opinion. Clin Pharmacol Ther. 1976;19:489–492
   • View In Article
   • MEDLINE
4. Kramer MS. Difficulties in assessing the adverse effects of drugs. Br J Clin Pharmacol. 1981;11(Suppl 1):105S–110S
   • View In Article
5. Koch-Weser J, Sellers EM, Zacest R. The ambiguity of adverse drug reactions. Eur J Clin Pharmacol. 1977;11:75–78
   • View In Article
   • MEDLINE
   • CrossRef
6. Feinstein AR. Clinical biostatistics: XXX. Biostatistical problems in ‘compliance bias.’. Clin Pharmacol Ther. 1974;16:846–857
   • View In Article
   • MEDLINE
7. Lanctôt KL, Naranjo CA. Comparison of the Bayesian approach and a simple algorithm for assessment of adverse drug events. Clin Pharmacol Ther. 1995;58:692–698
   • View In Article
   • MEDLINE
   • CrossRef
8. Auriche M. Approche bayésienne de l'imputabilité des phénomènes indésirables aux médicaments. [In French] Therapie. 1985;40:301–306
   • View In Article
   • MEDLINE
9. In:  Bégaud B editors. Dictionary of pharmacoepidemiology. Chichester, UK: John Wiley & Sons; 2000;
   • View In Article
10. Hutchinson TA, Lane DA. Assessing methods for causality assessment of suspected adverse drug reactions. J Clin Epidemiol. 1989;42:5–16
    • View In Article
    • MEDLINE
    • CrossRef
11. Péré JC, Godin MH, Bégaud B, Haramburu F, Albin H. Sensibilité et spécificité des critères d'imputabilité: étude et comparaison de ces indices d'efficacité pour sept méthodes. [in French] Therapie. 1985;40:307–312
    • View In Article
    • MEDLINE
12. Danan G, Benichou C. Causality assessment of adverse reactions to drugs: I. A novel method based on the conclusions of international consensus meetings: application to drug-induced liver injuries. J Clin Epidemiol. 1993;46:1323–1330
    • View In Article
    • MEDLINE
    • CrossRef
13. Benichou C, Danan G, Flahault A. Causality assessment of adverse reactions to drugs: II. An original model for validation of drug causality assessment methods: case reports with positive rechallenge. J Clin Epidemiol. 1993;46:1331–1336
    • View In Article
    • MEDLINE
    • CrossRef
14. Karch FE, Lasagna L. Toward the operational identification of adverse drug reactions. Clin Pharmacol Ther. 1977;21:247–254
    • View In Article
    • MEDLINE
15. Emanueli A, Sacchetti G. An algorithm for the classification of untoward events in large scale clinical trials. Agents Actions Suppl. 1980;7:318–322
    • View In Article
    • MEDLINE
16. Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981;30:239–245
    • View In Article
    • MEDLINE
    • CrossRef
17. Edwards IR, Biriell C. Harmonisation in pharmacovigilance. Drug Saf. 1994;10:93–102
    • View In Article
    • MEDLINE
    • CrossRef
18. Irey NS. Teaching monograph: tissue reactions to drugs. Am J Pathol. 1976;82:613–647
    • View In Article
    • MEDLINE
19. Dangoumau J, Evreux JC, Jouglard J. Méthode d'imputabilité des effets indésirables des médicaments. [in French] Therapie. 1978;33:373–381
    • View In Article
    • MEDLINE
20. Bégaud B, Evreux JC, Jouglard J, Lagier G. Imputabilité des effets inattendus ou toxiques des médicaments. [in French] Therapie. 1985;40:111–118
    • View In Article
    • MEDLINE
21. Salamon R, Peytour P. Recherche du poids des critères dans un algorithme d'imputabilité. [in French] Therapie. 1981;36:229–231
    • View In Article
    • MEDLINE
22. Rawlins MD, Thompson JW. Pathogenesis of adverse drug reactions. In:  Davies DM editors. Textbook of adverse drug reactions. Oxford: Oxford University Press; 1977;p. 10–31
    • View In Article