Why representativeness should be avoided

Extract

The essence of knowledge is generalisation. That rubbing wood in a certain way can produce fire is a knowledge derived by generalisation from individual experiences; the statement means that rubbing wood in this way will always produce fire. The art of discovery is therefore the art of correct generalisation. What is irrelevant, such as the particular shape or size of the piece of wood used, is to be excluded from the generalisation; what is relevant, for example, the dryness of the wood, is to be included in it. The meaning of the term relevant can thus be defined: that is relevant which must be mentioned for the generalisation to be valid. The separation of relevant from irrelevant factors is the beginning of knowledge.

—Hans Reichenbach¹

Why do so many believe that selecting representative study populations is a fundamental research aim for scientific studies? This view is widely held: representativeness is exalted along with motherhood, apple pie and statistical significance. For some researchers this goal can be so important that they would deem a study not worth undertaking if representativeness cannot be achieved. That was the case for two advisors to the U.S. National Children’s Health Study, who resigned when the study design was changed so that representativeness was threatened.² We admire people who take a stand for principle over expediency, but what exactly is the principle that representativeness embodies? Here we suggest that representativeness may be essential for conducting opinion polls, or for public-health applications, but it is not a reasonable aim for a scientific study.