It is hypothesized that eye movements are used to coordinate elements of a mental model with elements of the visual field. In two experiments, eye movements were recorded while observers imagined or recalled objects that were not present in the visual display. In both cases, observers spontaneously looked at particular blank regions of space in a systematic fashion, to manipulate and organize spatial relationships between mental and/or retinal images. These results contribute to evidence that interpreting a linguistic description of a visual scene requires a spatial (mental model) representation, and they support claims regarding the allocation of position markers in visual space for the manipulation of visual attention. More broadly, our results point to a concrete embodiment of cognition, in that a construction of a mental image is almost "acted out" by the eye movements, and a mental search of internal memory is accompanied by an ocolumotor search of external space.