Response: Re: Promoting Healthy Skepticism in the News: Helping Journalists Get It Right

Extract

Dr Pharoah may be right that there is no user-friendly definition for the P value. But that does not mean there cannot be one. And there needs to be one because medical journals report P values all the time. A user-friendly definition is crucial to help doctors, patients, and journalists understand the results of research.

In response to Dr Pharoah's comments, we have edited the P value entry in the glossary posted on the Journal website to read as follows:

The P value is the probability that the observed results could arise by chance alone.

By convention, if P > .05 we say “consistent with chance,” “not statistically significant,” and if P < .05 we say “not consistent with chance alone,” “statistically significant.”

Remember, even with a very low P value (“highly statistically significant”), results can still be very wrong: The study may be biased or confounded.

For example, the observed difference in the 5-year risk of death between the drug and placebo groups is not consistent with chance alone (ie, P = .0004, there is only a 4 in 10 000 chance of seeing differences this big or bigger if drug and placebo were equally effective).