The physiological properties of neurons in inferior temporal (IT) cortex of the macaque monkey suggest that this cortical area plays a major role in visual pattern recognition. Based on the properties of IT, and one of its major sources of input V4, a model is proposed that can account for some of the shape recognition properties of IT neurons including selectivity for complex visual stimuli and tolerance to the size and location of the stimuli. The model is composed of three components. First stimulus location tolerance is modeled after the complex-cell-like properties observed in some V4 neurons. The second component of the model is an attentionally controlled scaling mechanism that facilitates size-invariant shape recognition. The transition from edge orientation-selective neurons in V4 to neurons with more complicated stimulus preference in IT is explained by the third component of the model, a competitive learning mechanism. Single-unit analysis of receptive field properties, stimulus selectivity, and stimulus size and position tolerance was performed on “neurons” from the simulation. Comparison of results from the simulation and a study of actual IT neurons shows that the set of mechanisms incorporated into the simulation is sufficient to emulate the physiological data.