Proponents of evidence-based medicine (EBM) provide the “hierarchy of evidence” as a criterion for judging the reliability of therapeutic decisions. EBM's hierarchy places randomized interventional studies (and systematic reviews of such studies) higher in the hierarchy than observational studies, unsystematic clinical experience, and basic science. Recent philosophical work has questioned whether EBM's special emphasis on evidence from randomized interventional studies can be justified. Following the critical literature, and in particular the work of John Worrall, I agree that many of the arguments put forward by advocates of EBM do not justify the ambitious claims that are often made on behalf of randomization. However, in contrast to the recent philosophical work, I argue that a justification for EBM's hierarchy of evidence can be provided. The hierarchy should be viewed as a hierarchy of comparative internal validity. Although this justification is defensible, the claims that EBM's hierarchy substantiates when viewed in this way are considerably more circumscribed than some claims found in the EBM literature.