Effects of Filter Questions in Public Opinion Surveys

Abstract

Extending previous work, the authors find that the wording of a filter question can make a significant difference in the percentage of “don't know” (DK) responses elicited by an item, especially with topics that are more abstract or less familiar to survey respondents. They also find, however, that the content of an item can have a substantial, independent effect on DK or “no opinion” responses, regardless of how the filter question is worded. In general, it appears that the less familiar the issue or topic, the greater the increase in DK responses produced by adding a filter. Even more important, the analysis shows that filtering can in some instances dramatically affect the conclusions a pollster would draw about the distribution of public opinion on an issue. Indeed, such effects may occur more often than has previously been suspected, though the circumstances under which they emerge remain elusive. The authors suggest that such effects may become amenable to analysis by probing respondents about “what they had in mind” as they answered the question.