The two main outcomes for therapeutic trials are to influence clinical practice and, where appropriate, to make a successful claim for a drug with the regulatory authorities. Investigators are eternally optimistic and frequently plan their trials to look for large effects. Reality is different. The results of a planned (or unplanned) series of clinical trials may
26Whitehead J 1992 The Design Analysis of Sequential Clinical Trials, 2nd Edition. Ellis Horwood, Chester.
vary considerably for several reasons but most significantly because the studies are too small to detect a treatment effect. In common but serious diseases such as cancer or heart disease, however, even small treatment effects can be important in terms of their total impact on public health. It may be unreasonable to expect dramatic advances in these diseases; we should be looking for small effects. Drug developers too should be interested not only in whether a treatment works, but also how well and for whom.
The collecting together of a number of trials with the same objective in a systematic review27 and analysing the accumulated results using appropriate statistical methods is termed meta-analysis. The principles of a meta-analysis are that
• It should be comprehensive, i.e. include data from all trials, published and unpublished,
• Only randomised controlled trials should be analysed, with patients entered on the basis of 'intention to treat',28
• The results should be determined using clearly defined, disease-specific endpoints (this may involve a re-analysis of original trials).
There are strong advocates and critics of the concept, its execution and interpretation. Arguments that have been advanced against meta-analysis are:
• An effect of reasonable size ought to be demonstrable in a single trial,
• Different study designs cannot be pooled,
• Lack of accessibility of all relevant studies,
• Publication bias ('positive' trials are more likely to be published).
In practice, the analysis involves calculating an 'odds ratio' for each trial included in the meta-
27A review that strives comprehensively to identify and synthesise all the literature on a given subject (sometimes called an overview). The unit of analysis is the primary study and the same scientific principles and rigour apply as for any study. If a review does not state clearly whether and how all relevant studies were identified and synthesised it is not a systematic review (The Cochrane Library, 1998).
28Reports of therapeutic trials should contain an analysis of all patients entered, regardless of whether they dropped out or failed to complete, or even started the treatment for any reason. Omission of these subjects can lead to serious bias (Laurence D R, Carpenter J 1998 A dictionary of pharmacological and allied topics. Elsevier, Amsterdam).
analysis. This is the ratio of the number of patients experiencing a particular endpoint, e.g. death, and the number who do not, compared with the equivalent figures for the control group. The number of deaths observed in the treatment group is then compared with the number to be expected if it is assumed that the treatment is ineffective, to give the 'observed minus expected' statistic. The treatment effects for all trials in the analysis are then obtained by summing all the 'observed minus expected' values of the individual trials to obtain the overall odds ratio. An odds ratio of 1.0 indicates that the treatment has no effect, an odds ratio of 0.5 indicates a halving and an odds ratio of 2.0 indicates a doubling of the risk that patients will experience the chosen endpoint.
From the position of drug development, the general requirement that scientific results have to be repeatable has been interpreted in the past by the Food and Drug Administration (the regulatory agency in the USA) to mean that two well-controlled studies are required to support a claim. But this requirement is itself controversial and its relation to a meta-analysis in the context of drug development is unclear.
In clinical practice, and in the era of cost-effectiveness, the use of meta-analysis as a tool to aid medical decision making and underpinning 'evidence-based medicine' is here to stay.
Figure 4.3 shows detailed results from 11 trials in which antiplatelet therapy after myocardial infarction was compared with a control group. The number of vascular events per treatment group is shown in the figures in the second and third columns and the odds ratios, with the point estimates (the value most likely to have resulted from the study) represented by black squares and their 95% confidence intervals (CI), in the fourth column.
The size of the square is proportional to the number of events. The diamond gives the point estimate and CI for overall effect.
Was this article helpful?