Data mining reconsidered: encompassing and the general‐to‐specific approach to specification search

Summary

This paper examines the efficacy of the general‐to‐specific modeling approach associated with the LSE school of econometrics using a simulation framework. A mechanical algorithm is developed which mimics some aspects of the search procedures used by LSE practitioners. The algorithm is tested using 1000 replications of each of nine regression models and a data set patterned after Lovell’s (1983) study of data mining. The algorithm is assessed for its ability to recover the data‐generating process. Monte Carlo estimates of the size and power of exclusion tests based on t‐statistics for individual variables in the specification are also provided. The roles of alternative sizes for specification tests in the algorithm, the consequences of different signal‐to‐noise ratios, and strategies for reducing overparameterization are also investigated. The results are largely favorable to the general‐to‐specific approach. In particular, the size of exclusion tests remains close to the nominal size used in the algorithm despite extensive search.