Please use this identifier to cite or link to this item:
Title: Evaluating treatments in health care: The instability of a one-legged stool
Authors: Kaplan, Bonnie
Giesbrecht, Gerald
Shannon, Scott
McLeod, Kevin
Issue Date: 2011
Publisher: BMC Medical Research Methodology
Citation: Kaplan, B., Giesbrecht, G.F., Shannon, S., & McLeod, K. (2011). Evaluating treatments in health care: The instability of a one-legged stool. BMC Medical Research Methodology, 11:65
Abstract: Background: Both scientists and the public routinely refer to randomized controlled trials (RCTs) as being the 'gold standard' of scientific evidence. Although there is no question that placebo-controlled RCTs play a significant role in the evaluation of new pharmaceutical treatments, especially when it is important to rule out placebo effects, they have many inherent limitations which constrain their ability to inform medical decision making. The purpose of this paper is to raise questions about over-reliance on RCTs and to point out an additional perspective for evaluating healthcare evidence, as embodied in the Hill criteria. The arguments presented here are generally relevant to all areas of health care, though mental health applications provide the primary context for this essay. Discussion: This article first traces the history of RCTs, and then evaluates five of their major limitations: they often lack external validity, they have the potential for increasing health risk in the general population, they are no less likely to overestimate treatment effects than many other methods, they make a relatively weak contribution to clinical practice, and they are excessively expensive (leading to several additional vulnerabilities in the quality of evidence produced). Next, the nine Hill criteria are presented and discussed as a richer approach to the evaluation of health care treatments. Reliance on these multi-faceted criteria requires more analytical thinking than simply examining RCT data, but will also enhance confidence in the evaluation of novel treatments. Summary: Excessive reliance on RCTs tends to stifle funding of other types of research, and publication of other forms of evidence. We call upon our research and clinical colleagues to consider additional methods of evaluating data, such as the Hill criteria. Over-reliance on RCTs is similar to resting all of health care evidence on a one-legged stool.
Description: This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Appears in Collections:Giesbrecht, Gerald

Files in This Item:
File Description SizeFormat 
Evaluating treatments in health care....pdfArticle PDF183.13 kBAdobe PDFView/Open

This item is licensed under a Creative Commons License Creative Commons