Using fMRI techniques sensitive to blood oxygen-level dependent (BOLD) contrast, we measured brain activity in participants (n = 8) as they viewed images of faces presented periodically within a continuously changing montage of common objects. Consistent with prior studies, we identified regions of ventral extrastriate cortex, primarily in the fusiform and inferior temporal gyri and nearby cortex, that were activated by faces as measured by an increase in BOLD signal. In addition, we made the novel observation that faces deactivated other areas of ventral extrastriate cortex, primarily in the lingual and parahippocampal gyri and medial to activations. These deactivated regions, identified by a decrease in BOLD signal, may reflect populations of neurons that decrease their activity when faces appear, possibly as a consequence of category-specific inhibition.