Scientific hypotheses can be tested by comparing the effects of one treatment over many diseases in a systematic review

Journal Club

Scientific hypotheses can be tested by comparing the effects of one treatment over many diseases in a systematic review

21 January 2015

Facilitated by Research Fellow Malene Plejdrup Hansen


Background Systematic reviews of randomised controlled trials (RCT) underpin the practice of evidence-based medicine and have become the gold standard for synthesizing evidence to inform clinical and policy decisions. However, it is not uncommon that systematic reviews produce inconclusive findings because of insufficient evidence. For example, many adverse events are too uncommon or too long term to be observed within individual RCTs. For these reasons, a typical systematic review of controlled trials focusing on a specific indication may not provide sufficient evidence on the adverse effects profile of an intervention. Therefore it might be useful to examine the effect or adverse effects of a given treatment across different indications (Gillies, 2015). There is no standardised terminology for reviews of studies across multiple indications but it might be named a “multiple-indication review” (Chen, 2014).

Paper presented (Chen, 2014)

Objectives: Chen et al. aimed to describe the use of multiple-indication reviews and highlight the potential contributions and methodological considerations of their use.

Study design: The Cochrane Database of Systematic Reviews and MEDLINE (2003-2013) were searched for reviews and reviews of systematic reviews (overviews) – focusing on synthesis of RCT evidence on the effectiveness and harms of allopathic medicine.

Results: A total of 52 multiple-indication reviews were retrieved. Three broad nonexclusive uses of multiple-indication reviews were identified:

  • To detect unintended effects
  • To improve estimates of effectiveness
  • To examine heterogeneity of effect across disease groups

Chen et al found that the use of the method has increased over the last decade and two-thirds of the multiple-indication reviews are systematic reviews of primary studies. More than 60% of the reviews focused on pharmacologic interventions and most of them focused specifically on the effectiveness (37%) rather than on unintended effects (25%). The majority of studies (65%) presented results in a graphic format such as forest plots. Quantitative data were presented and analysed broadly in three different ways:

1) Results presented separately for each indication in a narrative approach 2) Pooling of results across indications 3) Examining the variation in effect sizes between different indications with or without pooling across indications.

Authors discussion and conclusion: The authors conclude that with due attention to methodological caveats, much can be learned  by comparing the effects of a given treatment across many related indications. They argue that when examining the effectiveness of an intervention, diseases should be included on the ground that they are or may be linked by a theoretical construct. When looking for unexpected effects they urge to include all the conditions for which the treatment has been used, as adverse effects are not disease specific. Importantly, it is highlighted that attention needs to be given to heterogeneity, potential confounding factors and various biases at both trial and review level when undertaking a multiple-indication review.

Journal Club Commentary

This is a very informative and interesting paper, which highlights the utilities and caveats in the use of multiple-indication reviews. The coherent methodology of this article is admirable and we acknowledge that there is a lack of coherent terminology. The term “multiple-indication review” is unambiguous and should be used in future studies. We agree that producers of systematic reviews should consider using this kind of reviews instead of, or in addition to, reviews focusing on a single indication. Important information that we cannot get with traditional methods (single-indication reviews) can be achieved if this methodology is followed. For example, multiple-indication reviews are very useful in the fight against antibiotic resistance. Providing Health Care Professionals and patients with comprehensive information about the risk of adverse effects and the benefit-harm trade-off may reduce their desire for use of antibiotics. Chen et al identified three uses of multiple-indication reviews. However, we propose that the use of this kind of review can be broadly categorised as either:

  • To get a better estimate of the effectiveness or harms E.g. What are the adverse effects of amoxicillin? (P* I C H1, H2, …)
  • To examine heterogeneity across indications or interventions E.g., When are prophylactic antibiotics effective? (P1, P2, P3, … I C O1),

E.g., What is the optimum timing of prophylactic antibiotics before any surgery? (P* I1 I2 I3, … C O1)

(PICO notation: P* = any disease; P1 P2 P3 = set of disease, I1 I2 I3 = set of interventions, C = comparison, O1 = outcome, H = harm)

When undertaking a multiple-indication review much attention has to be paid to the methodologically caveats such as ‘overlapping’ of included reviews; the extra level of complexity (potential heterogeneity in the contributing systematic reviews, in addition to heterogeneity in the primary trials); potential confounders (e.g. methodological quality of included studies and reviews) and risk of bias (e.g. different duration or dose of tested treatment or different control groups). Some of these methodological challenges can be dealt with in the designing of the review and some should be taken into account in the analytical approach. ‘Overlapping’ can be circumvented if only the data from the originals trials are included in the multiple-indication review. However, if the multiple-indication review is based on results from systematic reviews one need to take account of the potential ‘overlapping’ of included studies and we would be very interested in software routines to conduct these reviews – especially in a 2-step frequentist approach.

  1. Gillies M, Ranakusuma A, Hoffmann T, et al. Common harms from amoxicillin: a systematic review and meta-analysis of randomized placebo-controlled trials for any indication. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne 2015; 187(1): E21-31.
  2. Chen YF, Hemming K, Chilton PJ, Gupta KK, Altman DG, Lilford RJ. Scientific hypotheses can be tested by comparing the effects of one treatment over many diseases in a systematic review. Journal of clinical epidemiology 2014; 67(12): 1309-19.