Gaze Contingent Displays and the Ideal Searcher

Dr. William Geisler
University of Texas

Although visual search is perhaps the most important and ubiquitous visual task humans perform, it is a complex task involving attention, pattern recognition, and motor planning. To perform visual search, humans encode a large field of view with a visual system having variable spatial resolution, and then use high-speed eye movements (and slower head and body movements) to direct the highest resolution region, the fovea, at potential target locations in the environment. Two major problems have held back progress in understanding this kind of visual search: the difficulty of controlling the stimulus on the retina, and the lack of a formal theory of optimal eye movement strategies. To address the first problem we have developed a method for creating real-time displays where either spatial resolution or chromatic content can be varied arbitrarily across the visual field, contingent on the current gaze direction. To address the second problem we have derived the ideal searcher for tasks where a known target is embedded at an unknown location within a random background that has the spectral characteristics of natural scenes. We find that many qualitative properties of human fixation patterns are consistent with an optimal eye movement strategy. Remarkably, we also find that the number of fixations humans require to find the target is very similar to that of the ideal searcher, even though humans cannot integrate information perfectly across fixations. By analyzing the performance of the ideal searcher we show that, in fact, there is only a small benefit from integrating information perfectly across fixations—much more important is efficient parallel processing of the information on each fixation. Apparently, evolution has exploited these facts to achieve efficient visual search and eye movement strategies with minimal neural resources devoted to memory and information integration.