Meta-analysis Under the Microscope

Extract

There was a time when the application of statistics to medicine was an innovative idea. In 19th century England, an early advocate of this idea was William A. Guy, Professor of Forensic Medicine at King's College, London ( 1 ). Guy's scholarly essay on the topic appeared in an 1839 volume of the Journal of the Statistical Society of London and was sandwiched between a tally of the daily vital statistics of the 300 000 residents of New York City and a description of the Belgian railway system ( 2 ). In his essay, Guy laid down the philosophical and scientific rationale for applying statistical methods to medicine. He described the advantages of measuring disease rates and numerical averages of large numbers of observations and proposed that causal theories could be tested statistically.

More than a century and a half later, there are those who say that the application of statistics to medicine has gone too far ( 3 – 5 ). The target of their vitriolic humor is meta-analysis—a method for contrasting and combining results of studies ( 6 ). Although these critics are not champions of observational studies, they make it clear that meta-analysis of nonexperimental research is the problem, calling it a “bad concept” ( 3 ), an “absurdity” ( 5 ), “numerological abracadabra” ( 5 ), and “statistical alchemy” ( 4 ) more akin to “acts of politics than science” ( 4 ).