Question-Order Effects on Presidential Popularity

Abstract

A methodological experiment is conducted on the same question that is used virtually every month in the Gallup Poll to measure presidential popularity. The point of the experiment is to determine whether presidential popularity is affected by the placement of the question within the survey, in response to a recent charge that alternations in the order in which this question has been asked invalidate time-series analyses of presidential popularity. The primary finding is that the order in which the question is presented does not significantly affect the direction of response (the balance between approval and disapproval), but it does affect opinionation (the willingness to make a directional response, whether positive or negative). This effect is found to be particularly pronounced for less educated respondents. The implications of these findings for time series analysis of presidential popularity are spelled out.